Multi-target tracking method and device, computer equipment and storage medium
By pausing tracking trajectories with match failures greater than the preset number in the multi-objective tracking technology, the problem of inefficiency in the existing technology is solved, and more efficient multi-objective tracking is achieved.
Patent Information
- Application Number
- CN202411505415.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-28
- Publication Date
- 2025-02-25
AI Technical Summary
The existing multi-objective tracking technology has a large amount of calculation during the tracking process, resulting in inefficiency.
By obtaining the detection box and prediction box in the video frame, matching and updating the number of match failures, pausing tracking of tracking of tracks with match failures greater than the preset number, thereby reducing the amount of calculation.
Reduces the calculation amount of the tracking process and improves the efficiency of multi-objective tracking.
Smart Images
Figure CN119515916A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image recognition technology, and in particular to a multi-target tracking method, device, computer equipment and storage medium. Background Art
[0002] Multi-object tracking technology is to locate multiple objects of interest in a given video image sequence, maintain individual ID information and record their trajectories between consecutive frames. Multi-object tracking technology is one of the most important and complex tasks in the field of computer vision, and is applied to fields such as autonomous driving, security inspection, and smart cities.
[0003] Compared with target detection technology, which only outputs the positioning information of the target at the current stationary moment, multi-target tracking technology adds the individual ID information of the one-dimensional target. This ID information can be used to build the connection between frames, thereby identifying the same object in adjacent frames. From the perspective of application scenarios, the difference between the two tasks can be better understood. Object detection scenarios such as steel bar counting, industrial quality inspection, power inspection, wheat ear detection, etc., all require only the detection of the state of the object at a certain moment, a certain point, and a stationary moment. Target tracking scenarios such as smart transportation, medical analysis, livestock inventory, military surveys, etc., all require continuous tracking of the continuous motion state of the object, so these tasks cannot be replaced by target detection, and thus need to be achieved through multi-target tracking technology. Summary of the invention
[0004] Based on this, it is necessary to provide a multi-target tracking method, apparatus, computer equipment, computer-readable storage medium and computer program product that can reduce the computational complexity of the tracking process in order to address the above technical problems.
[0005] In a first aspect, the present application provides a multi-target tracking method, comprising:
[0006] For the current video frame, obtain the detection frame in the current video frame;
[0007] For the trajectory obtained based on the video frame before the current video frame, predict the prediction box of the trajectory in the current video frame;
[0008] Match all detection boxes with all prediction boxes to obtain matching prediction boxes and detection boxes;
[0009] According to the matched prediction box and detection box, update the number of matching failures of the corresponding trajectory;
[0010] The trajectories with more than a preset number of matching failures are regarded as overlapping hidden targets, and the tracking of the trajectories with more than a preset number of matching failures is suspended.
[0011] In one embodiment, obtaining a detection frame in a current video frame includes:
[0012] Get the initial feature map of the current video frame;
[0013] Performing feature extraction on the initial feature map through the first network to obtain a first feature map representing a heat map of the target center point, performing feature extraction on the initial feature map through the second network to obtain a second feature map representing a width and a height of a detection frame, and performing feature extraction on the initial feature map through the third network to obtain a third feature map representing a position of the target center point;
[0014] Based on the first feature map, the second feature map and the third feature map, a detection box in the current video frame is obtained.
[0015] In one embodiment, all detection boxes are matched with all prediction boxes to obtain prediction boxes and detection boxes that match each other, including:
[0016] Extract the appearance features in each detection frame and the appearance features in each prediction frame;
[0017] Calculate the appearance similarity between the appearance features of each detection frame and the appearance features in each prediction frame, match them based on the appearance similarity, and obtain the prediction frame and detection frame on which they are preliminarily matched;
[0018] The center distance between each initially matched prediction box and each initially matched detection box is calculated, and matching is performed based on the center distance to obtain the re-matched prediction box and detection box.
[0019] In one of the embodiments, the appearance features are extracted by a re-identification feature network branch, and the corresponding loss function of the re-identification feature network is an L-softmax loss function.
[0020] In one embodiment, after matching based on the center distance to obtain the re-matched prediction box and detection box, the method further includes:
[0021] The intersection-over-union ratio between each re-matched prediction box and each re-matched detection box is calculated, and matching is performed based on the intersection-over-union ratio to obtain the final mutually matched prediction box and detection box.
[0022] In one embodiment, after stopping tracking of a trajectory with a matching failure number greater than a preset number, the method further includes:
[0023] After a preset number of frames, the tracking of the trajectories whose matching failure times are greater than the preset number is resumed.
[0024] In a second aspect, the present application also provides a multi-target tracking device, comprising:
[0025] An acquisition module is used to acquire a detection frame in a current video frame;
[0026] A prediction module, for predicting a prediction box of a trajectory in a current video frame based on a trajectory obtained based on a video frame before the current video frame;
[0027] The matching module is used to match all detection boxes with all prediction boxes to obtain prediction boxes and detection boxes that match each other;
[0028] An update module is used to update the number of matching failures of the corresponding trajectory according to the prediction box and the detection box that match each other;
[0029] The pause module is used to treat the trajectory with a matching failure number greater than a preset number as an overlapping hidden target, and pause tracking the trajectory with a matching failure number greater than a preset number.
[0030] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0031] For the current video frame, obtain the detection frame in the current video frame;
[0032] For the trajectory obtained based on the video frame before the current video frame, predict the prediction box of the trajectory in the current video frame;
[0033] Match all detection boxes with all prediction boxes to obtain matching prediction boxes and detection boxes;
[0034] According to the matched prediction box and detection box, update the number of matching failures of the corresponding trajectory;
[0035] The trajectories with more than a preset number of matching failures are regarded as overlapping hidden targets, and the tracking of the trajectories with more than a preset number of matching failures is suspended.
[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:
[0037] For the current video frame, obtain the detection frame in the current video frame;
[0038] For the trajectory obtained based on the video frame before the current video frame, predict the prediction box of the trajectory in the current video frame;
[0039] Match all detection boxes with all prediction boxes to obtain matching prediction boxes and detection boxes;
[0040] According to the matched prediction box and detection box, update the number of matching failures of the corresponding trajectory;
[0041] The trajectories with more than a preset number of matching failures are regarded as overlapping hidden targets, and the tracking of the trajectories with more than a preset number of matching failures is suspended.
[0042] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:
[0043] For the current video frame, obtain the detection frame in the current video frame;
[0044] For the trajectory obtained based on the video frame before the current video frame, predict the prediction box of the trajectory in the current video frame;
[0045] Match all detection boxes with all prediction boxes to obtain matching prediction boxes and detection boxes;
[0046] According to the matched prediction box and detection box, update the number of matching failures of the corresponding trajectory;
[0047] The trajectories with more than a preset number of matching failures are regarded as overlapping hidden targets, and the tracking of the trajectories with more than a preset number of matching failures is suspended.
[0048] The above-mentioned multi-target tracking method, apparatus, computer equipment, storage medium and computer program product pause the tracking of trajectories with a cumulative number of matching failures during the tracking process. Compared with involving all trajectories in the target tracking processing process corresponding to each image frame, the tracking targets hidden due to overlap will not appear temporarily. Pausing tracking can reduce the overall calculation amount, thereby improving tracking efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0050] FIG1 is a diagram of an application environment of a multi-target tracking method according to an embodiment;
[0051] FIG2 is a schematic diagram of a flow chart of a multi-target tracking method in one embodiment;
[0052] FIG3 is a schematic diagram of a flow chart of a multi-target tracking method in another embodiment;
[0053] FIG4 is a block diagram of a multi-target tracking device according to an embodiment;
[0054] FIG. 5 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0056] The multi-target tracking method provided in the embodiment of the present application can be applied in the application environment shown in FIG. 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 can be a collection device, which collects video data and uploads the video data to the server 104. The server 104 obtains the video data and obtains the detection frame in the current video frame for the current video frame. For the trajectory obtained based on the video frame before the current video frame, the prediction frame of the trajectory in the current video frame is predicted. All detection frames are matched with all prediction frames to obtain mutually matching prediction frames and detection frames. According to the mutually matching prediction frames and detection frames, the number of matching failures of the corresponding trajectory is updated. The trajectory with a matching failure number greater than the preset number is regarded as an overlapping hidden target, and the tracking of the trajectory with a matching failure number greater than the preset number is suspended.
[0057] The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.
[0058] In an exemplary embodiment, as shown in FIG2 , a multi-target tracking method is provided, which is described by taking the method applied to the server 102 in FIG1 as an example, and includes the following steps 202 to 210. Among them:
[0059] Step 202: for the current video frame, obtain a detection frame in the current video frame.
[0060] Specifically, the method provided in the embodiment of the present application can be applied to video multi-target tracking, and can be applied in the business hall of an institution such as a bank to track different pedestrian targets, and the embodiment of the present application does not specifically limit this. In this step, the video frame can be first input into the encoder-decoder, and the encoder-decoder can first extract the initial feature map by downsampling layer by layer, and then upsampling layer by layer, and then obtain the detection box position in the current video frame based on the initial feature map.
[0061] Step 204: predict a prediction box of the trajectory in the current video frame based on the trajectory obtained based on the video frame before the current video frame.
[0062] For the current video frame, the trajectory refers to the sequence of detection frames formed by associating the detection frames in the previous video frame of the current video frame, which can be called the trajectory. Among them, the corresponding order of the detection frame sequence is determined according to the corresponding order of the video frame where the detection frame is located. In this step, the prediction process can be implemented by Kalman filtering, which can be implemented in two stages. The first stage is to predict the position of the trajectory in the current video frame. The second stage can update the position of the preliminary prediction frame obtained in the first stage according to the detection frame in the current video frame, thereby obtaining the prediction frame of the trajectory in the current video frame.
[0063] Step 206: Match all detection frames with all prediction frames to obtain prediction frames and detection frames that match each other.
[0064] Among them, the matching process mainly matches the detection box with the prediction box, and the matching can be based on the re-identification feature, the intersection and union ratio, and the center distance between the boxes. Specifically, taking the re-identification feature (appearance feature) as an example, the appearance features between all detection boxes and all prediction boxes can form a matrix. It can be clearly seen that as a matching result, the closer the similarity between the appearance features between the matched detection box and the prediction box, the better. Therefore, the above content can be used as a matching target, and the matching process becomes how to determine the optimal matching result based on the matrix. In actual implementation, the Hungarian algorithm can be used to obtain prediction boxes and detection boxes that match each other.
[0065] Step 208: Update the number of matching failures of the corresponding trajectory according to the matched prediction box and detection box.
[0066] It is understandable that when matching the detection frame with the prediction frame, there may be a situation where the prediction frame cannot find a matching detection frame. In this case, for the prediction frame that cannot find a matching detection frame, the corresponding trajectory of the prediction frame is the trajectory that failed to match in the current video frame, and the number of matching failures can be counted as 1. The trajectory that failed to match participates in the matching process corresponding to the next video frame. If the matching fails again according to the above process, the number of matching failures can be increased by 1.
[0067] Step 210: The trajectory with a matching failure number greater than a preset number is regarded as an overlapping hidden target, and the tracking of the trajectory with a matching failure number greater than the preset number is suspended.
[0068] Specifically, if the number of failed matches accumulated for a certain track in the current video frame is greater than a preset number, it means that the target corresponding to the track may have been hidden due to target overlap. Therefore, in the subsequent tracking process, the tracking of the track with more than the preset number of failed matches can be suspended.
[0069] In the above-mentioned multi-target tracking method, by pausing the tracking of the trajectories with cumulative matching failures during the tracking process, compared with involving all trajectories in the target tracking processing process corresponding to each image frame, the tracking targets hidden due to overlap will not appear for a short time. Pausing tracking can reduce the overall calculation amount, thereby improving tracking efficiency.
[0070] In an exemplary embodiment, obtaining a detection frame in a current video frame includes:
[0071] Get the initial feature map of the current video frame;
[0072] Performing feature extraction on the initial feature map through the first network to obtain a first feature map representing a heat map of the target center point, performing feature extraction on the initial feature map through the second network to obtain a second feature map representing a width and a height of a detection frame, and performing feature extraction on the initial feature map through the third network to obtain a third feature map representing a position of the target center point;
[0073] Based on the first feature map, the second feature map and the third feature map, a detection box in the current video frame is obtained.
[0074] Among them, the initial feature map can be processed by an encoder-decoder, and the encoder-decoder can be based on an improved DLA (Deep Layer Aggregation) network as its backbone network. The improved DLA network can be different from the original DLA network. The improved DLA network can use jump connections similar to FPN (feature pyramid networks) to associate low-level feature information with high-level feature information.
[0075] In addition, the deformable convolutional layer can be used to replace all the convolutional layers of the upsampling module in the original DLA network, so that the convolutional layer can adaptively set the receptive field range according to the spatial size and motion state of the target object. In addition, the target feature map extracted by the backbone network can be obtained at 4 times the sampling rate of the original input image, that is, the target feature map is gathered from the entire image to accurately track the spatial position of the target object.
[0076] In an embodiment of the present application, the network structure of the Centerent target detection framework can be used to convert target detection into a bounding box regression task based on the center point on a high-resolution feature information map. On the above-mentioned improved DLA backbone network, three parallel regression head layers can be added to independently predict the heat map, the target object space center offset, and the anchor box size range. The three parallel regression layers can correspond to the first network, the second network, and the third network in the embodiment of the present application, the heat map can correspond to the first feature map, the target object space center offset can correspond to the position of the target center point, and the anchor box size range can correspond to the detection box width and height. Through the three networks, the first feature map, the second feature map, and the third feature map can be obtained respectively. The three feature maps are input into a 3×3 convolutional layer containing 256 channels, and then continue to pass through the 1×1 convolutional layer to obtain the detection box in the current video.
[0077] In the above embodiment, since three parallel regression features can be integrated to obtain the detection frame, the detection result is more accurate.
[0078] In an exemplary embodiment, all detection boxes are matched with all prediction boxes to obtain prediction boxes and detection boxes that match each other, including:
[0079] Extract the appearance features in each detection frame and the appearance features in each prediction frame;
[0080] Calculate the appearance similarity between the appearance features of each detection frame and the appearance features in each prediction frame, match them based on the appearance similarity, and obtain the prediction frame and detection frame on which they are preliminarily matched;
[0081] The center distance between each initially matched prediction box and each initially matched detection box is calculated, and matching is performed based on the center distance to obtain the re-matched prediction box and detection box.
[0082] Among them, the appearance features can be extracted by re-identification feature network branches, and the embodiments of the present application do not specifically limit this. After obtaining the appearance features of the detection frame and the prediction frame, the appearance similarity between different detection frames and different prediction frames can be calculated, which can be specifically cosine similarity, so that a cosine similarity matrix can be obtained. The rows in the matrix can be used to represent different detection frames, and the columns in the matrix can be used to represent different prediction frames. Through the Hungarian algorithm, the prediction frame and detection frame on the preliminary match can be obtained.
[0083] The above preliminary matching process is equivalent to screening the detection frame and the prediction frame, and then the center distances between different detection frames and prediction frames can be calculated. Similarly, the center distances between different detection frames and prediction frames can be represented by the center distance matrix. Through the Hungarian algorithm, the prediction frame and detection frame that are matched again can be obtained.
[0084] In the above embodiment, secondary matching of appearance similarity and center distance can be performed, so that the matching accuracy can be improved, and thus the target tracking accuracy can be improved.
[0085] In an exemplary embodiment, the appearance features are extracted by a re-identification feature network branch, and the corresponding loss function of the re-identification feature network is an L-softmax loss function.
[0086] Among them, the loss function is used to evaluate the similarity between the model's predicted value f(x) and the true value Y. The smaller the loss function, the better the robustness of the model. The loss function guides model learning. The purpose of machine learning is to learn a set of parameters so that the predicted value is infinitely close to the true value by backpropagating according to the loss function. Softmax is a kind of loss function, which is a loss function obtained based on a set of output results of each fully connected layer in the neural network, as well as the cross entropy loss function. The L-softmax loss function proposes a judgment interval based on the softmax loss function, so that the large angle interval characteristics between different classes can be learned.
[0087] The function of the re-identification feature network branch is to output label information that defines different categories of objects. Generally, the similarity between different categories of objects is much greater than the similarity between objects of the same category. The re-identification feature network branch can input the feature map output by the backbone network into a convolutional layer containing 128 kernels to obtain appearance features.
[0088] In the above embodiment, since the Softmax loss function can be extended to the L-softmax loss function from the perspective of phase aggregation, there is a greater phase difference between the features of different training categories, that is, a more obvious classifiable difference, which facilitates feature distinction and makes the recognition result more accurate.
[0089] In an exemplary embodiment, after matching based on the center distance to obtain the re-matched prediction box and detection box, the method further includes:
[0090] The intersection-over-union ratio between each re-matched prediction box and each re-matched detection box is calculated, and matching is performed based on the intersection-over-union ratio to obtain the final mutually matched prediction box and detection box.
[0091] Among them, when matching is performed based on the intersection-and-union ratio, the Hungarian algorithm can be used, and the embodiment of the present application does not make any specific limitation to this.
[0092] In the above embodiment, the matching accuracy can be improved by performing three matches of appearance similarity, center distance and intersection-over-union ratio, thereby improving the accuracy of target tracking.
[0093] In an exemplary embodiment, after stopping tracking of a trajectory with a matching failure number greater than a preset number, the method further includes:
[0094] After a preset number of frames, the tracking of the trajectories whose matching failure times are greater than the preset number is resumed.
[0095] Specifically, in the process of tracking each video frame, a target trajectory set can be maintained, and the trajectories in the target trajectory set can be gradually formed from the first video frame. When a trajectory with a matching failure number greater than a preset number appears, the trajectory can be deleted from the target trajectory set and retained in the paused tracking trajectory set. After a preset number of frames, the trajectory can be moved from the paused tracking trajectory set to the target trajectory set, thereby resuming tracking. Among them, the preset number of frames can be summarized according to actual conditions.
[0096] In the above embodiment, pausing tracking can reduce the overall computational effort to improve tracking efficiency. However, considering that overlapping hidden targets may reappear, the corresponding trajectory can be resumed after a preset number of frames, thereby ensuring the comprehensiveness of tracking.
[0097] In an exemplary embodiment, as shown in FIG3 , in combination with the above content, a multi-target tracking method is provided, which is specifically as follows:
[0098] Step 302: input a video frame, perform multi-target detection on the video frame, obtain a detection frame, and filter the detection frame;
[0099] Step 304: based on the formed trajectory, obtain a prediction box corresponding to the trajectory in the video frame;
[0100] Step 306: Perform cascade matching on the prediction box and the detection box to obtain a cascade matching result;
[0101] Step 308: Perform intersection-and-union ratio matching based on the cascade matching result to obtain an intersection-and-union ratio matching result;
[0102] Step 310: Perform overlapping mechanism judgment based on the intersection-over-union matching result to obtain the target tracking result corresponding to the video frame.
[0103] In this embodiment, by pausing the tracking of the track with the accumulated number of matching failures during the tracking process, compared with involving all tracks in the target tracking process corresponding to each image frame, the tracking target hidden due to overlap will not appear for a short time, and pausing the tracking can reduce the overall calculation amount, thereby improving the tracking efficiency. In addition, since the three matches of appearance similarity, center distance and intersection-over-union ratio can be performed, the matching accuracy can be improved, and then the accuracy of target tracking can be improved.
[0104] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0105] Based on the same inventive concept, the embodiment of the present application also provides a multi-target tracking device for implementing the multi-target tracking method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more multi-target tracking device embodiments provided below can refer to the limitations on the multi-target tracking method above, and will not be repeated here.
[0106] In an exemplary embodiment, as shown in FIG4 , a multi-target tracking method is provided, including: an acquisition module 402, a prediction module 404, a matching module 406, an update module 408 and a pause module 410, wherein:
[0107] The acquisition module 402 is used to acquire a detection frame in the current video frame;
[0108] A prediction module 404, configured to predict a prediction box of a trajectory in a current video frame based on a trajectory obtained based on a video frame before the current video frame;
[0109] A matching module 406 is used to match all detection frames with all prediction frames to obtain prediction frames and detection frames that match each other;
[0110] An updating module 408, configured to update the number of matching failures of the corresponding trajectory according to the matched prediction box and detection box;
[0111] The pause module 410 is used to treat the trajectory with a matching failure number greater than a preset number as an overlapping hidden target, and pause tracking the trajectory with a matching failure number greater than a preset number.
[0112] In some embodiments, the acquisition module 402 is used to perform feature extraction on the initial feature map through a first network to obtain a first feature map representing a heat map of the target center point, perform feature extraction on the initial feature map through a second network to obtain a second feature map representing the width and height of the detection frame, perform feature extraction on the initial feature map through a third network to obtain a third feature map representing the position of the target center point; based on the first feature map, the second feature map and the third feature map, obtain the detection frame in the current video frame.
[0113] In some embodiments, the matching module 406 is used to extract the appearance features within each detection frame and the appearance features within each prediction frame; calculate the appearance similarity between the appearance features of each detection frame and the appearance features within each prediction frame, perform matching based on the appearance similarity, and obtain the prediction frame and detection frame on the preliminary match; calculate the center distance between each prediction frame on the preliminary match and each detection frame on the preliminary match, perform matching based on the center distance, and obtain the prediction frame and detection frame on the re-match.
[0114] In some embodiments, the appearance features are extracted by a re-identification feature network branch, and the corresponding loss function of the re-identification feature network is an L-softmax loss function.
[0115] In some embodiments, the matching module 406 is further used to calculate the intersection-and-union ratio between each re-matched prediction box and each re-matched detection box, and perform matching based on the intersection-and-union ratio to obtain the final mutually matched prediction box and detection box.
[0116] In some embodiments, the pause module 410 is further configured to resume tracking of trajectories with a matching failure count greater than a preset number of times after a preset number of frames have passed.
[0117] Each module in the above-mentioned multi-target tracking device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.
[0118] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG5. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a multi-target tracking method is implemented.
[0119] Those skilled in the art will understand that the structure shown in FIG. 5 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.
[0120] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:
[0121] For the current video frame, obtain the detection frame in the current video frame;
[0122] For the trajectory obtained based on the video frame before the current video frame, predict the prediction box of the trajectory in the current video frame;
[0123] Match all detection boxes with all prediction boxes to obtain matching prediction boxes and detection boxes;
[0124] According to the matched prediction box and detection box, update the number of matching failures of the corresponding trajectory;
[0125] The trajectories with more than a preset number of matching failures are regarded as overlapping hidden targets, and the tracking of the trajectories with more than a preset number of matching failures is suspended.
[0126] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
[0127] For the current video frame, obtain the detection frame in the current video frame;
[0128] For the trajectory obtained based on the video frame before the current video frame, predict the prediction box of the trajectory in the current video frame;
[0129] Match all detection boxes with all prediction boxes to obtain matching prediction boxes and detection boxes;
[0130] According to the matched prediction box and detection box, update the number of matching failures of the corresponding trajectory;
[0131] The trajectories with more than a preset number of matching failures are regarded as overlapping hidden targets, and the tracking of the trajectories with more than a preset number of matching failures is suspended.
[0132] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:
[0133] For the current video frame, obtain the detection frame in the current video frame;
[0134] For the trajectory obtained based on the video frame before the current video frame, predict the prediction box of the trajectory in the current video frame;
[0135] Match all detection boxes with all prediction boxes to obtain matching prediction boxes and detection boxes;
[0136] According to the matched prediction box and detection box, update the number of matching failures of the corresponding trajectory;
[0137] The trajectories with more than a preset number of matching failures are regarded as overlapping hidden targets, and the tracking of the trajectories with more than a preset number of matching failures is suspended.
[0138] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.
[0139] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0140] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A multi-target tracking method, characterized in that: The method comprises: For the current video frame, obtain the detection frame in the current video frame; For a trajectory obtained based on a video frame before the current video frame, predict a prediction box of the trajectory in the current video frame; Match all detection boxes with all prediction boxes to obtain matching prediction boxes and detection boxes; According to the matched prediction box and detection box, update the number of matching failures of the corresponding trajectory; The trajectories with more than a preset number of matching failures are regarded as overlapping hidden targets, and the tracking of the trajectories with more than a preset number of matching failures is suspended.
2. The method according to claim 1, characterized in that The obtaining of the detection frame in the current video frame includes: Obtaining an initial feature map of the current video frame; Performing feature extraction on the initial feature map through the first network to obtain a first feature map representing a heat map of the target center point, performing feature extraction on the initial feature map through the second network to obtain a second feature map representing a width and a height of a detection frame, and performing feature extraction on the initial feature map through the third network to obtain a third feature map representing a position of the target center point; Based on the first feature map, the second feature map and the third feature map, a detection box in the current video frame is obtained.
3. The method according to claim 1, characterized in that The step of matching all detection frames with all prediction frames to obtain prediction frames and detection frames that match each other includes: Extract the appearance features in each detection frame and the appearance features in each prediction frame; Calculate the appearance similarity between the appearance features of each detection frame and the appearance features in each prediction frame, match them based on the appearance similarity, and obtain the prediction frame and detection frame on which they are preliminarily matched; The center distance between each initially matched prediction box and each initially matched detection box is calculated, and matching is performed based on the center distance to obtain the re-matched prediction box and detection box.
4. The method according to claim 3, characterized in that The appearance features are extracted through a re-identification feature network branch, and the corresponding loss function of the re-identification feature network is an L-softmax loss function.
5. The method according to claim 3, characterized in that: After matching based on the center distance to obtain the predicted box and the detection box that are matched again, the method further includes: The intersection-over-union ratio between each re-matched prediction box and each re-matched detection box is calculated, and matching is performed based on the intersection-over-union ratio to obtain the final mutually matched prediction box and detection box.
6. The method according to claim 1, characterized in that After stopping tracking the trajectory with a matching failure number greater than a preset number, the method further includes: After a preset number of frames, the tracking of the trajectories whose matching failure times are greater than the preset number is resumed.
7. A multi-target tracking device, characterized in that: The device comprises: An acquisition module is used to acquire a detection frame in a current video frame; A prediction module, configured to predict a prediction box of a trajectory in a current video frame based on a trajectory obtained based on a video frame before the current video frame; The matching module is used to match all detection boxes with all prediction boxes to obtain prediction boxes and detection boxes that match each other; An update module is used to update the number of matching failures of the corresponding trajectory according to the prediction box and the detection box that match each other; The pause module is used to treat the trajectory with a matching failure number greater than a preset number as an overlapping hidden target, and pause tracking the trajectory with a matching failure number greater than a preset number.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.