A target tracking method, device, equipment and storage medium

Abstract

The present invention discloses a target tracking method, device, equipment and storage medium. The method includes: performing target detection on the current frame image to obtain the detection box position information of each target; predicting the trajectory box position information corresponding to each trajectory identification number through Kalman filtering based on the trajectory information including the trajectory identification number of the previous frame image; performing trajectory matching on the target based on the detection box position information and the trajectory box position information; if the matching is successful, assigning the successfully matched trajectory identification number to the target, otherwise creating a new trajectory identification number for the target; updating the consecutive missing frame number of the trajectory identification number based on the trajectory matching result; updating the current frame missing threshold of the trajectory identification number based on the trajectory box position information; if the consecutive missing frame number is greater than the current frame missing threshold, modifying the trajectory state of the trajectory identification number to the missing state. It solves the technical problem of how to achieve precise tracking of targets in specific application scenarios based on the importance of target tracking at different positions.

Description

Technical Field

[0001] The present invention relates to the fields of image processing and computer vision, and particularly relates to an object tracking method, device, equipment, and storage medium. Background Art

[0002] Tracking algorithms are a technology for computer vision to track objects, which can identify and track objects of interest, such as people, vehicles, animals, etc. in a complex environment. Common application scenarios include video surveillance systems, autonomous vehicle driving, motion analysis, etc. In the prior art, tracking algorithm papers focus on solving the situation of occlusion of multiple moving objects, but ignore the consideration of the importance of tracking objects at different positions.

[0003] Therefore, how to achieve precise tracking of objects in specific application scenarios based on the importance of object tracking at different positions has become an urgent problem to be solved currently. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide an object tracking method, device, equipment, and storage medium, which solves the technical problem of how to achieve precise tracking of objects in specific application scenarios based on the importance of object tracking at different positions in the prior art.

[0005] To solve the above technical problem, the present invention proposes a tracking algorithm with an attention field, which uses the same trajectory matching logic as the ByteTrack tracking algorithm, adds an attention function to fit the attention scores of the tracking algorithm at different positions, and dynamically updates the tracking algorithm parameters according to the attention scores, so as to solve the technical problem of how to achieve precise tracking of objects in specific application scenarios based on the importance of object tracking at different positions in the prior art.

[0006] The present invention provides an object tracking method, including:

[0007] Performing object detection on the current frame image to obtain the detection box position information of each object;

[0008] Based on the trajectory information including the trajectory identification number of the previous frame image, predicting the trajectory box position information corresponding to each trajectory identification number through Kalman filtering, where the trajectory information further includes the consecutive missing frame numbers of the trajectory identification number, the current frame missing threshold, and the trajectory state;

[0009] Performing trajectory matching on the object based on the detection box position information and the trajectory box position information; if the matching is successful, assigning the successfully matched trajectory identification number to the object, otherwise creating a new trajectory identification number for the object;

[0010] Updating the consecutive missing frame numbers of the trajectory identification number based on the trajectory matching result;

[0011] Update the current frame loss threshold of the trajectory identification number based on the trajectory box position information;

[0012] If the number of consecutive lost frames is greater than the current frame loss threshold, modify the trajectory status of the trajectory identification number to the lost status.

[0013] Optionally, the updating the current frame loss threshold of the trajectory identification number based on the trajectory box position information includes:

[0014] Based on the coordinate position of the center point of the trajectory box in the current frame, calculate the probability distribution values of each direction of the trajectory identification number in the multi-dimensional Gaussian distribution as the attention weights, and update the current frame loss threshold of the trajectory identification number according to the attention weights. The attention weight value is the product of the probability distribution values of each direction.

[0015] Optionally, when the Gaussian distribution is two-dimensional, the attention weight value is the product of the probability distribution values of the x direction and the y direction of the trajectory identification number in the two-dimensional Gaussian distribution.

[0016] Optionally, the current frame loss threshold is the integer result of the product of the attention weight and a first preset value.

[0017] Optionally, the trajectory information further includes the consecutive matching frames number and the consecutive matching threshold of the trajectory identification number. After performing trajectory matching on the target based on the detection box position information and the trajectory box position information, it further includes:

[0018] Update the consecutive matching frames number of the trajectory identification number based on the trajectory matching result;

[0019] If the number of consecutive matching frames is greater than the consecutive matching threshold, modify the trajectory status of the trajectory identification number to the confirmed status.

[0020] Optionally, the performing trajectory matching on the target based on the detection box position information and the trajectory box position information includes:

[0021] Obtain the detection box position coordinates and the trajectory box position coordinates, and use the Hungarian algorithm to perform trajectory matching on the target.

[0022] Optionally, the initial trajectory status of the trajectory identification number is the doubtful status.

[0023] The present invention also provides a target tracking device, including:

[0024] A target detection module, configured to perform target detection on the current frame image to obtain the detection box position information of each target;

[0025] A trajectory prediction module, configured to predict, through Kalman filtering, the position information of the trajectory box corresponding to each trajectory identification number based on the trajectory information including the trajectory identification number in the previous frame image, where the trajectory information further includes the consecutive lost frame number of the trajectory identification number, the current frame loss threshold, and the trajectory state;

[0026] A trajectory matching module, configured to perform trajectory matching on the target based on the detection box position information and the trajectory box position information; if the matching is successful, assign the trajectory identification number with successful matching to the target, otherwise create a new trajectory identification number for the target;

[0027] The trajectory matching module is further configured to update the consecutive lost frame number of the trajectory identification number based on the trajectory matching result;

[0028] The trajectory matching module is further configured to update the current frame loss threshold of the trajectory identification number based on the trajectory box position information;

[0029] The trajectory matching module is further configured to modify the trajectory state of the trajectory identification number to a lost state if the consecutive lost frame number is greater than the current frame loss threshold.

[0030] The present invention further provides a target tracking device, including:

[0031] A memory, configured to store a computer program;

[0032] A processor, configured to implement the steps of any embodiment of the above target tracking method when executing the computer program.

[0033] The present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any embodiment of the above target tracking method are implemented.

[0034] It can be seen that in the present invention, the position information of the detection box of each target is obtained by performing target detection on the current frame image; based on the trajectory information including the trajectory identification number in the previous frame image, the position information of the trajectory box corresponding to each trajectory identification number is predicted by Kalman filtering. The trajectory information further includes the continuous missing frame number of the trajectory identification number, the current frame missing threshold, and the trajectory state; the trajectories of the targets are matched based on the detection box position information and the trajectory box position information; if the matching is successful, the successfully matched trajectory identification number is assigned to the target, otherwise a new trajectory identification number is created for the target; the continuous missing frame number of the trajectory identification number is updated based on the trajectory matching result; the current frame missing threshold of the trajectory identification number is updated based on the trajectory box position information; if the continuous missing frame number is greater than the current frame missing threshold, the trajectory state of the trajectory identification number is modified to the missing state. The present invention matches the trajectories in the image, updates the continuous missing frame number of each trajectory based on the trajectory matching result, updates the current frame missing threshold of each trajectory based on the trajectory box position information, and determines the current trajectory state by comparing the continuous missing frame number and the current frame missing threshold of the trajectory. A method for trajectory matching and target tracking based on frame-by-frame detection is provided. Among them, updating the current frame missing threshold of each trajectory based on the trajectory box position information is an important step in applying the sensitivity factor of the trajectory position into the tracking algorithm, effectively optimizing the effect of target tracking.

[0035] In addition, the present invention also provides a target tracking method, device, equipment and computer-readable storage medium, which also have the above beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0037] Figure 1 It is one of the flowcharts of a target tracking method provided by an embodiment of the present invention;

[0038] Figure 2 It is another flowchart of a target tracking method provided by an embodiment of the present invention;

[0039] Figure 3 It is the third flowchart of a target tracking method provided by an embodiment of the present invention;

[0040] Figure 4 It is one of the structural diagrams of a target tracking device provided by an embodiment of the present invention;

[0041] Figure 5One of the structural schematic diagrams of a target tracking device provided by an embodiment of the present invention. Detailed implementation manners

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are only a part rather than all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0043] Tracking algorithm: The object detection algorithm detects the position of the target object in a single-frame image. The tracking algorithm uses the temporal information of the front and back frames of the video to track the motion trajectory of the target object.

[0044] In specific application scenarios, the importance of different-position targets to the tracking algorithm is not the same. For example, in the scenario of counting the passenger flow in a shopping mall, compared with the pedestrians passing by the store entrance, the motion trajectories of the pedestrians entering and leaving the store are more worthy of attention, and ensuring the accuracy of tracking these pedestrians often has greater value.

[0045] In the prior art, the papers on tracking algorithms focus on solving the situation of occlusion of multiple moving targets, but ignore the consideration of the importance of tracking different-position targets.

[0046] To solve the above technical problems, the present invention proposes a tracking algorithm with an attention field, which uses the same trajectory matching logic as the ByteTrack tracking algorithm, adds an attention function to fit the attention scores of the tracking algorithm at different positions, and dynamically updates the parameters of the tracking algorithm according to the attention scores, so as to solve the technical problem of how to accurately track the target in a specific application scenario based on the importance of tracking different-position targets in the prior art.

[0047] Please refer to Figure 1 , Figure 1 One of the flowcharts of a target tracking method provided by an embodiment of the present invention; the method may include:

[0048] Step S101: Perform object detection on the current frame image to obtain the detection box position information of each target.

[0049] It should be further noted that the method of performing object detection on the current frame image herein includes but is not limited to using an object detection model. And the object detection model can be a general and mature model with good detection effects, or an object detection model whose model parameters are adjusted after pre-training the object detection model based on data containing the prior knowledge of user requirements.

[0050] In addition, it can be understood that in some other embodiments of the present invention, the output of the object detection model includes the detection box position information of each object, and may also include the category information of each object. For example, the object categories in the current frame image are identified, such as people, dogs, cash register devices, etc., and the detection box position information of each category is output. Based on the object tracking method of the present invention, the tracking process of objects of the same category in the same scene can be realized, and the tracking process of objects of multiple categories in the same scene can also be carried out. In this embodiment, the object tracking based on a typical human object is described.

[0051] Step S102: Based on the trajectory information including the trajectory identification number in the previous frame image, the trajectory box position information corresponding to each trajectory identification number is predicted through Kalman filtering. The trajectory information further includes the consecutive lost frame number of the trajectory identification number, the current frame loss threshold, and the trajectory state.

[0052] It should be further noted that the Kalman filtering algorithm is a special form of the Bayesian filtering algorithm, which is used to predict the Bayesian optimal value in an uncertain system when there are both inaccurate observation values and mathematical formula prediction values. The mathematical formula of the Kalman filtering algorithm follows the uniform linear motion mathematical model in the scenario of predicting the position of a moving box. It has the position and speed of the trajectory box at time T-1, and assumes that the speed remains unchanged until time T, so as to predict the position of the trajectory box at time T. The input of the Kalman filtering is the trajectory information of the previous frame image, and the trajectory information includes the trajectory identification number of each trajectory, its consecutive lost frame number, the current frame loss threshold, and the trajectory state. The output of the Kalman filtering is the predicted trajectory box position information corresponding to each trajectory identification number.

[0053] It can be understood that the entire object tracking process can be optimized by setting the initial trajectory state of the trajectory identification number to a doubtful state.

[0054] Furthermore, in a specific embodiment, the trajectory information further includes the consecutive matching frame number of the trajectory identification number and the consecutive matching threshold. After the trajectory matching of the object based on the detection box position information and the trajectory box position information, it further includes:

[0055] Updating the consecutive matching frame number of the trajectory identification number based on the trajectory matching result;

[0056] If the consecutive matching frame number is greater than the consecutive matching threshold, modify the trajectory state of the trajectory identification number to a confirmed state.

[0057] In the embodiments of the present invention, the number of consecutive matching frames or the number of consecutive missing frames is modified based on the matching result of each frame, and the trajectory state is further determined based on the comparison result between the number of consecutive matching frames and the consecutive matching threshold, realizing the refined tracking of the target. The manner of obtaining the consecutive matching threshold is not limited herein, and the consecutive matching threshold may be a value predefined by the user.

[0058] Step S103: Perform trajectory matching on the target based on the detected box position information and the trajectory box position information; if the matching is successful, assign the trajectory identification number of the successful match to the target, otherwise create a new trajectory identification number for the target.

[0059] In a specific implementation manner, optionally, the performing trajectory matching on the target based on the detected box position information and the trajectory box position information includes:

[0060] Obtain the position coordinates of the detected box and the position coordinates of the trajectory box, and perform trajectory matching on the target using the Hungarian algorithm.

[0061] It should be further noted that the method of performing trajectory matching on the target based on the detected box position information and the trajectory box position information herein may be the Hungarian algorithm or other methods, which are not limited herein. In the embodiments of the present invention, the trajectory matching result and the trajectory identification number of each target in the current frame image are determined through the detected box position information and the trajectory box position information, and are used for the subsequent target tracking process.

[0062] Step S104: Update the number of consecutive missing frames of the trajectory identification number based on the trajectory matching result.

[0063] It should be further noted that for the trajectory with an unsuccessful matching result in the trajectory matching, its trajectory information is updated, specifically, the number of consecutive missing frames in the trajectory information may be updated.

[0064] Step S105: Update the current frame missing threshold of the trajectory identification number based on the trajectory box position information.

[0065] In a specific implementation manner, optionally, the updating the current frame missing threshold of the trajectory identification number based on the trajectory box position information includes:

[0066] Based on the coordinate position of the center point of the trajectory box in the current frame, calculate the probability distribution values of the trajectory identification number in each direction in the multi-dimensional Gaussian distribution as attention weights, and update the current frame loss threshold of the trajectory identification number according to the attention weights. The attention weight value is the product of the probability distribution values in each direction. It can be understood that the current frame loss threshold can be the integer result of the product of the attention weight and the first preset value. Taking the Gaussian distribution as a three-dimensional Gaussian distribution as an example, the attention weight value is the product of the probability distribution values of the trajectory identification number in the x direction, y direction, and z direction of the three-dimensional Gaussian distribution.

[0067] Further, taking the Gaussian distribution as a two-dimensional Gaussian distribution as an example, the attention weight value is the product of the probability distribution values of the trajectory identification number in the x direction and y direction of the two-dimensional Gaussian distribution.

[0068] In a specific implementation manner, after predicting the position of the trajectory box at time T, the current frame loss threshold of the current trajectory can be updated according to the center point coordinates of the predicted box position according to the following formula:

[0069]

[0070]

[0071] where dx is the probability distribution value of the trajectory identification number in the x direction of the two-dimensional Gaussian distribution; dy is the probability distribution value of the trajectory identification number in the y direction of the two-dimensional Gaussian distribution. x, y are the two-dimensional coordinate representations (x, y) of the center point of the predicted box position; w, h represent the width and height of the current frame image; μ is the mathematical expectation, and σ 2 is the variance. dx * dy is the product of the probability distribution values of the trajectory identification number in the x direction and y direction of the two-dimensional Gaussian distribution, that is, the attention weight value.

[0072] Further, the current frame loss threshold of the trajectory identification number can be updated according to the attention weight. One calculation method of the current frame loss threshold can be the integer result of the product of the attention weight value and the first set value. It can be understood that when the center point coordinates of the trajectory are closer to the center in the graph, the calculated current frame loss threshold is larger, and the algorithm can tolerate the existence of the trajectory even when it has not been matched with the detection box for many frames; when the trajectory center point is closer to the edge in the graph, the calculated current frame loss threshold is smaller, and if the trajectory is not matched with the detection box, the trajectory will be quickly marked as a lost state.

[0073] In an embodiment of the present invention, a tracking algorithm with an attention field is proposed. The main idea is that this algorithm assumes that the target will not disappear in the middle position of the graph but only after passing through the edge position. By calculating the probability distribution values of the trajectory identification number in each direction in the multi-dimensional Gaussian distribution as the attention weights, the current frame loss threshold of the trajectory identification number is updated according to the attention weights. Among them, the attention weight value is the product of the probability distribution values in each direction, effectively realizing the incorporation of the sensitivity factor of the position into the tracking algorithm and effectively optimizing the target tracking effect.

[0074] It should be noted that when the target tracking method of the present invention is applied to verify the tracking feasibility in the takeaway detection scenario, the tracking effect is good, and the tracking algorithm can better meet the needs of the scenario and users.

[0075] Step S106: If the number of consecutive lost frames is greater than the current frame loss threshold, modify the trajectory state of the trajectory identification number to the lost state.

[0076] It should be further noted that if the number of consecutive lost frames is greater than the current frame loss threshold, modify the trajectory state of the trajectory identification number to the lost state. For the current frame, when the position of the detection frame changes, both the trajectory matching result and the current frame loss threshold may be updated. By comparing the number of consecutive lost frames and the current frame loss threshold, the trajectory state will be updated in various changing situations of the detection frame position, realizing the accurate tracking of the target.

[0077] It can be seen that the present invention obtains the detection frame position information of each target through target detection of the current frame image; based on the trajectory information including the trajectory identification number of the previous frame image, predicts the trajectory frame position information corresponding to each trajectory identification number through Kalman filtering. The trajectory information also includes the number of consecutive lost frames, the current frame loss threshold, and the trajectory state of the trajectory identification number; performs trajectory matching on the target based on the detection frame position information and the trajectory frame position information; if the matching is successful, assigns the successfully matched trajectory identification number to the target, otherwise creates a new trajectory identification number for the target; updates the number of consecutive lost frames of the trajectory identification number based on the trajectory matching result; updates the current frame loss threshold of the trajectory identification number based on the trajectory frame position information; if the number of consecutive lost frames is greater than the current frame loss threshold, modify the trajectory state of the trajectory identification number to the lost state. The present invention provides a method for realizing trajectory matching and target tracking based on frame-by-frame detection. Among them, updating the current frame loss threshold of each trajectory based on the trajectory frame position information is an important step in applying the sensitivity factor of the trajectory position into the tracking algorithm, effectively optimizing the target tracking effect.

[0078] For a better understanding of the present invention, please specifically refer to Figure 2 , Figure 2 which is the second flowchart of a target tracking method provided by an embodiment of the present invention; the method includes:

[0079] Step S21: Define a trajectory, initialize a target detection algorithm and a Kalman filtering algorithm, and initialize a Gaussian distribution.

[0080] It should be noted that the following attributes are defined for the trajectory:

[0081] mean: The position information of the trajectory box, including the center point position of the trajectory box and the width and height of the box;

[0082] covariance: The speeds of the trajectory box in the X and Y directions, so as to be used to input the Kalman filter to obtain predicted position information;

[0083] state: The trajectory state; there are three states in total: [confirmed, doubtful, lost];

[0084] Continuous matching frame number: Record how many consecutive frames the trajectory has been matched with the detection box;

[0085] Continuous matching threshold: When the trajectory state is [doubtful], only when the [continuous matching frame number] is greater than or equal to the continuous matching threshold, the trajectory state changes to [confirmed];

[0086] Continuous lost frame number: Record how many consecutive frames the trajectory has not been successfully matched with the detection box;

[0087] Current frame lost threshold: When the trajectory state is [doubtful], only when the [continuous lost frame number] is greater than or equal to the current frame lost threshold, the trajectory state changes to [lost].

[0088] Trajectory identification number, i.e., trajectory id: an integer of type int, used to uniquely identify the trajectory.

[0089] It can be understood that the initial trajectory information is obtained through the observations of the camera in the first 3 frames.

[0090] Step S22: Execute the target detection algorithm to obtain the position information of the detection box in the current frame.

[0091] Please refer to Figure 3 , Figure 3It is the third flowchart of a target tracking method provided by an embodiment of the present invention. It should be noted that, by using an object detection algorithm, the position and category of the object in the image are output by inputting the image captured by the camera at time T. The present invention uses YOLOX as the object detection algorithm. The object detection set is represented by the letter D. Based on the trajectory information at time T-1 and the execution logic at time T, the trajectory information of the current frame image is updated, and the updated trajectory information is also used for the target tracking process at time T+1.

[0092] Step S23: Predict the position of the trajectory target based on the Kalman filter algorithm to obtain the position information of the prediction box.

[0093] It can be understood that the Kalman filter algorithm follows the mathematical model of uniform linear motion in the scenario of predicting the position of the moving box, has the position and speed of the trajectory box at time T-1, and assumes that the speed remains unchanged until time T, thereby predicting the position of the trajectory box at time T. The set of prediction box positions is represented by the letter P.

[0094] Step S24: Use the Hungarian algorithm to match the detection box set and the prediction box set.

[0095] It can be understood that the Hungarian algorithm is used to match the detection box set D and the prediction box set P to obtain the matching result of each object in the current frame image. If the matching is successful, the trajectory identification number of the successful match is assigned to the object, otherwise a new trajectory identification number is created for the object.

[0096] Step S25: Update the trajectory according to the matching result.

[0097] It can be understood that the trajectory update here includes updating the trajectory information. Based on the above definition of the trajectory, the trajectory information includes the trajectory identification number of each trajectory, its consecutive lost frame number, the current frame lost threshold, the consecutive matching frame number, the consecutive matching threshold, and the trajectory state. Based on the matching result, the trajectory identification number, the consecutive lost frame number, or the consecutive matching frame number is updated, and then the current frame lost threshold is updated. According to the comparison result of the latest consecutive lost frame number and the current frame lost threshold, and the comparison result of the latest consecutive matching frame number and the consecutive matching threshold, the trajectory state is updated. The trajectory state is one of the three states: [confirmed, doubtful, lost].

[0098] In an embodiment of the present invention, the position information of the detection frame of each target is obtained by performing target detection on the current frame image; based on the trajectory information including the trajectory identification number in the previous frame image, the position information of the trajectory frame corresponding to each trajectory identification number is predicted through Kalman filtering, and the trajectory information further includes the consecutive missing frame number of the trajectory identification number, the current frame missing threshold, and the trajectory state; the trajectory matching of the target is performed based on the detection frame position information and the trajectory frame position information; if the matching is successful, the successfully matched trajectory identification number is assigned to the target, otherwise a new trajectory identification number is created for the target; the consecutive missing frame number of the trajectory identification number is updated based on the trajectory matching result; the current frame missing threshold of the trajectory identification number is updated based on the trajectory frame position information; if the consecutive missing frame number is greater than the current frame missing threshold, the trajectory state of the trajectory identification number is modified to the missing state. The present invention matches the trajectories in the image, updates the consecutive missing frame number of each trajectory based on the trajectory matching result, updates the current frame missing threshold of each trajectory based on the trajectory frame position information, and determines the current trajectory state by comparing the consecutive missing frame number and the current frame missing threshold of the trajectory. A method for trajectory matching and target tracking based on frame-by-frame detection is provided. Among them, updating the current frame missing threshold of each trajectory based on the trajectory frame position information is an important step in applying the sensitivity factor of the trajectory position to the tracking algorithm, effectively optimizing the target tracking effect.

[0099] Next, a target tracking device provided by an embodiment of the present invention will be introduced. A target tracking device described below can be correspondingly referred to the target tracking method described above.

[0100] Specifically, please refer to Figure 4 , Figure 4 which is one of the structural schematic diagrams of a target tracking device provided by an embodiment of the present invention. The target tracking device 200 may include:

[0101] A target detection module 21, configured to perform target detection on the current frame image to obtain the position information of the detection frame of each target;

[0102] A trajectory prediction module 22, configured to predict, through Kalman filtering, the position information of the trajectory frame corresponding to each trajectory identification number based on the trajectory information including the trajectory identification number in the previous frame image, where the trajectory information further includes the consecutive missing frame number of the trajectory identification number, the current frame missing threshold, and the trajectory state;

[0103] A trajectory matching module 23, configured to perform trajectory matching on the target based on the detection frame position information and the trajectory frame position information; if the matching is successful, assign the successfully matched trajectory identification number to the target, otherwise create a new trajectory identification number for the target;

[0104] The trajectory matching module 23 is further configured to update the consecutive frame loss count of the trajectory identification number based on the trajectory matching result;

[0105] The trajectory matching module 23 is further configured to update the current frame loss threshold of the trajectory identification number based on the trajectory box position information;

[0106] The trajectory matching module 23 is further configured to modify the trajectory state of the trajectory identification number to a lost state if the consecutive frame loss count is greater than the current frame loss threshold.

[0107] The target tracking device in this embodiment can implement each process of the target tracking method embodiment described above and achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0108] It should be noted that the order of the modules and units in the above-mentioned target tracking device can be changed before and after without affecting the logic.

[0109] Next, a target tracking device provided by an embodiment of the present invention will be introduced. The target tracking device described below can be correspondingly referred to the target tracking method described above.

[0110] Please refer to Figure 5 , Figure 5 which is one of the structural schematic diagrams of a target tracking device provided by an embodiment of the present invention, and may include:

[0111] A memory 10 for storing computer programs;

[0112] A processor 20 for executing the computer program to implement the above-mentioned target tracking method.

[0113] The memory 10, the processor 20, and the communication interface 30 complete the communication with each other through a communication bus 40.

[0114] In the embodiment of the present invention, the memory 10 is used to store one or more programs. The program may include program codes, and the program codes include computer operation instructions. In the embodiment of the present invention, the memory 10 may store programs for implementing the following functions:

[0115] Performing target detection on the current frame image to obtain the detection box position information of each target;

[0116] Based on the trajectory information including the trajectory identification number of the previous frame image, predicting the trajectory box position information corresponding to each trajectory identification number through Kalman filtering. The trajectory information further includes the consecutive frame loss count, the current frame loss threshold, and the trajectory state of the trajectory identification number;

[0117] Perform trajectory matching on the target based on the detected box position information and the trajectory box position information; if the matching is successful, assign the trajectory identification number of the successful match to the target, otherwise create a new trajectory identification number for the target;

[0118] Update the consecutive lost frame count of the trajectory identification number based on the trajectory matching result;

[0119] Update the current frame loss threshold of the trajectory identification number based on the trajectory box position information;

[0120] If the consecutive lost frame count is greater than the current frame loss threshold, modify the trajectory status of the trajectory identification number to the lost status.

[0121] In a possible implementation, the memory 10 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function, etc.; the data storage area may store data created during use.

[0122] In addition, the memory 10 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A part of the memory may also include NVRAM. The memory stores an operating system and operation instructions, executable modules or data structures, or subsets thereof, or extended sets thereof. Among them, the operation instructions may include various operation instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and processing hardware-based tasks.

[0123] The processor 20 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices. The processor 20 may be a microprocessor or any conventional processor, etc. The processor 20 may call the program stored in the memory 10.

[0124] The communication interface 30 may be an interface of a communication module for connecting to other devices or systems.

[0125] Of course, it should be noted that Figure 4 The structure shown does not constitute a limitation on a target tracking device in an embodiment of the present invention. In practical applications, a target tracking device may include more or fewer components than Figure 4 shown, or combine some components.

[0126] Next, a computer-readable storage medium provided by an embodiment of the present invention will be introduced. The computer-readable storage medium described below can be mutually referred to with the target tracking method described above.

[0127] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned target tracking method are implemented.

[0128] The computer-readable storage medium may include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs.

[0129] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description in the method section.

[0130] Those skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to their functions in the above description. Whether these functions are executed in the form of hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0131] Finally, it should also be noted that in this article, relationships such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variant is intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article, or device.

[0132] The above has introduced in detail a target tracking method, device, equipment and computer-readable storage medium provided by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation on the present invention.

Claims

1. A target tracking method, characterized in that, Including: Performing object detection on the current frame image to obtain the position information of the detection box for each object; Based on the trajectory information including the trajectory identification number in the previous frame image, predicting the position information of the trajectory box corresponding to each trajectory identification number through Kalman filtering, where the trajectory information further includes the consecutive lost frame number of the trajectory identification number, the current frame loss threshold, and the trajectory state; Performing trajectory matching on the object based on the detection box position information and the trajectory box position information; If the matching is successful, assigning the successfully matched trajectory identification number to the object, otherwise creating a new trajectory identification number for the object; Updating the consecutive lost frame number of the trajectory identification number based on the trajectory matching result; Updating the current frame loss threshold of the trajectory identification number based on the trajectory box position information; Specifically, based on the coordinate position of the center point of the trajectory box in the current frame, calculating the probability distribution values in each direction of the trajectory identification number in the multi-dimensional Gaussian distribution as the attention weights, and updating the current frame loss threshold of the trajectory identification number according to the attention weights, where the attention weight value is the product of the probability distribution values in each direction; If the consecutive lost frame number is greater than the current frame loss threshold, modifying the trajectory state of the trajectory identification number to the lost state.

2. The target tracking method according to claim 1, characterized in that The updating the current frame loss threshold of the trajectory identification number based on the trajectory box position information includes: Based on the coordinate position of the center point of the trajectory box in the current frame, calculating the probability distribution values in each direction of the trajectory identification number in the multi-dimensional Gaussian distribution as the attention weights, and updating the current frame loss threshold of the trajectory identification number according to the attention weights, where the attention weight value is the product of the probability distribution values in each direction.

3. The target tracking method according to claim 2, wherein When the Gaussian distribution is two-dimensional, the attention weight value is the product of the probability distribution values in the x direction and the y direction of the two-dimensional Gaussian distribution of the trajectory identification number.

4. The target tracking method according to claim 2, wherein The current frame loss threshold is the integer result of the product of the attention weight and the first preset value.

5. The target tracking method according to claim 1, wherein The trajectory information further includes the consecutive matching frame number and the consecutive matching threshold of the trajectory identification number. After performing trajectory matching on the object based on the detection box position information and the trajectory box position information, it further includes: Updating the consecutive matching frame number of the trajectory identification number based on the trajectory matching result; If the consecutive matching frame number is greater than the consecutive matching threshold, modifying the trajectory state of the trajectory identification number to the confirmed state.

6. The target tracking method according to claim 1, wherein The performing trajectory matching on the object based on the detection box position information and the trajectory box position information includes: Obtaining the detection box position coordinates and the trajectory box position coordinates, and using the Hungarian algorithm to perform trajectory matching on the object.

7. The target tracking method according to any one of claims 1-6, characterized in that, The initial trajectory state of the trajectory identification number is the doubtful state.

8. A target tracking device, characterized in that, Including: An object detection module for performing object detection on the current frame image to obtain the position information of the detection box for each object; A trajectory prediction module for predicting the position information of the trajectory box corresponding to each trajectory identification number through Kalman filtering based on the trajectory information including the trajectory identification number in the previous frame image, where the trajectory information further includes the consecutive lost frame number of the trajectory identification number, the current frame loss threshold, and the trajectory state; A trajectory matching module, configured to perform trajectory matching on the target based on the detected box position information and the trajectory box position information; If the matching is successful, assign the trajectory identification number of the successful match to the target; otherwise, create a new trajectory identification number for the target; The trajectory matching module is further configured to update the consecutive lost frame number of the trajectory identification number based on the trajectory matching result; The trajectory matching module is further configured to update the current frame loss threshold of the trajectory identification number based on the trajectory box position information; specifically, based on the coordinate position of the center point of the trajectory box in the current frame, calculate the probability distribution values in each direction of the trajectory identification number in the multi-dimensional Gaussian distribution as attention weights, and update the current frame loss threshold of the trajectory identification number according to the attention weights, where the attention weight value is the product of the probability distribution values in each direction; The trajectory matching module is further configured to, if the consecutive lost frame number is greater than the current frame loss threshold, modify the trajectory state of the trajectory identification number to the lost state.

9. A target tracking device, characterized in that, Comprising: A memory, configured to store a computer program; A processor, configured to implement the steps of the target tracking method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the target tracking method according to any one of claims 1 to 7 are implemented.

Images