Multi-object tracking method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a multi-object tracking method and device, equipment and a storage medium, and relates to the field of financial science and technology or other related fields. The method comprises the following steps: if a target object is not tracked at two consecutive moments, acquiring actual measurement information of the target object at a historical moment; wherein the actual measurement information represents actual position information and speed information of the object; the historical moment is the previous moment of two continuous moments; according to the actual measurement information of the target object at the historical moment, determining the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in the two continuous moments; and determining a tracking trajectory of the target object according to the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in the two continuous moments. According to the method provided by the invention, the precision and accuracy of multi-target tracking are improved.

Description

Technical Field

[0001] The present application relates to the field of financial technology or other related fields, and in particular to a multi-object tracking method, device, equipment and storage medium. Background Art

[0002] Intelligent robots have become an important part of financial services, playing an important role in patrolling and monitoring key areas, assisting customers in completing various business needs, and realizing human-computer interaction through intelligent navigation. When tracking a target, the robot may encounter various obstacles (including fixed objects, pedestrians, etc.), so it is necessary to correctly bypass various obstacles and determine the optimal path.

[0003] The process of tracking a target is affected by many factors. The target may be blocked or disappear during movement. When using existing technologies for tracking, the target tracking frame cannot correctly circle the target due to insufficient target information in the tracking frame, resulting in target tracking failure.

[0004] Therefore, how to improve the precision and accuracy of target tracking when the target is blocked or disappears is a problem that needs to be solved urgently. Summary of the invention

[0005] The present application provides a multi-object tracking method, device, equipment and storage medium to improve the precision and accuracy of target tracking.

[0006] In a first aspect, the present application provides a multi-object tracking method, comprising:

[0007] If the target object is not tracked at two consecutive moments, then the actual measurement information of the target object at the historical moment is obtained; wherein the actual measurement information represents the actual position information and speed information of the object; and the historical moment is the previous moment between the two consecutive moments;

[0008] According to the actual measurement information of the target object at the historical moment, determining the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in the two consecutive moments;

[0009] The tracking trajectory of the target object is determined according to the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment between the two consecutive moments; wherein the tracking trajectory is the movement trajectory of the target object between the historical moment and the next moment between the two consecutive moments.

[0010] In a second aspect, the present application provides a multi-object tracking device, comprising:

[0011] An acquisition module, for acquiring actual measurement information of the target object at a historical moment if the target object is not tracked at two consecutive moments; wherein the actual measurement information represents the actual position information and speed information of the object; and the historical moment is the previous moment between the two consecutive moments;

[0012] A first determination module is used to determine the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in the two consecutive moments according to the actual measurement information of the target object at the historical moment;

[0013] The second determination module is used to determine the tracking trajectory of the target object based on the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment between the two consecutive moments; wherein the tracking trajectory is the movement trajectory of the target object between the historical moment and the next moment between the two consecutive moments.

[0014] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0015] The memory stores computer-executable instructions;

[0016] The processor executes the computer-executable instructions stored in the memory to implement the method according to the first aspect.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect.

[0018] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.

[0019] The present application provides a multi-object tracking method, device, equipment and storage medium. If the target object is not tracked at two consecutive moments, the actual measurement information of the target object at the historical moment is obtained; wherein the actual measurement information represents the actual position information and speed information of the object; the historical moment is the previous moment of the two consecutive moments; according to the actual measurement information of the target object at the historical moment, the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in the two consecutive moments are determined; according to the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in the two consecutive moments, the tracking trajectory of the target object is determined; wherein the tracking trajectory is the movement trajectory of the target object between the historical moment and the next moment in the two consecutive moments. In this scheme, according to the actual measurement information of the target object at the historical moment, the measurement information of the target object at two consecutive moments after the historical moment is predicted, and then the tracking trajectory of the target object is determined according to the measurement information of these three moments, so as to realize the tracking of the target object when the target object is blocked, thereby improving the precision and accuracy of target tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0021] Figure 1 A schematic diagram of multi-hypothesis tracking of a multi-object tracking method provided in this application;

[0022] Figure 2 A flowchart of a multi-object tracking method provided in this application;

[0023] Figure 3 A flowchart of a multi-object tracking method provided in this application;

[0024] Figure 4 A complete flow chart of a multi-object tracking method provided in this application;

[0025] Figure 5 A structural block diagram of a multi-object tracking device provided in an embodiment of the present application;

[0026] Figure 6 A structural block diagram of a multi-object tracking device provided in an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the structure of a multi-object tracking device provided in this application.

[0028] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0029] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0030] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] In addition, this application involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and the use of artificial intelligence technology for automated decision-making, and a technical solution for making decisions that have a significant impact on personal rights and interests based on the results of automated decision-making. The application provides users with corresponding operation entrances for them to choose to agree or reject the results of automated decision-making; if the user chooses to reject, the expert decision-making process will be entered.

[0032] It should be noted that the multi-object tracking method, device, equipment and storage medium provided in the present application can be used in the field of financial technology, and can also be used in any field other than the field of financial technology. The application field of the multi-object tracking method, device, equipment and storage medium in the present application is not limited.

[0033] First, the terms involved in this application are explained:

[0034] Data association: Data association is a key step in the multi-target tracking task, and its main purpose is to match multiple targets between frames.

[0035] Kalman filtering: Kalman filtering is an algorithm that uses the linear system state equation to optimally estimate the system state through the system input and output observation data. Since the observation data includes the influence of noise and interference in the system, the optimal estimation can also be regarded as a filtering process.

[0036] Dual decomposition method: It is a solution method that decomposes the optimization variables and objective functions of the original problem into multiple sub-problems. It is an iterative optimization method and a type of analytical method in optimization theory. It decomposes the original problem into several sub-problems and simplifies complex problems in order to obtain the optimal solution faster and more efficiently.

[0037] Multi-Hypothesis Tracking (MHT): is an algorithm for target tracking. The core idea is to consider multiple possibilities when tracking a target. Its theoretical basis is Bayesian estimation and optimization theory, and it can handle complex situations such as target appearance, disappearance, overlap, and occlusion.

[0038] Intelligent robots have become an important part of financial services. They play an important role in patrolling and monitoring key areas, assisting customers in completing various business needs, and realizing human-computer interaction through intelligent navigation. Based on this, it is particularly important to improve the tracking accuracy of multiple targets so that they can do a good job of path planning. In its path planning, the environment it faces is relatively complex. In the process of moving towards the destination, it may encounter various obstacles (including fixed objects, pedestrians, etc.). The robot needs to correctly bypass various moving targets and make the best path planning.

[0039] Usually, the Kalman filter algorithm is used to predict and track the target. The general process is: input the initial frame and specify the target to be tracked, generate a large number of candidate frames in the next frame and extract key features, the observation model scores these candidate frames, and finally find the candidate frame with the highest score among these scores as the predicted target, or fuse multiple prediction values ​​to obtain a better predicted target. Based on the above algorithm, the prediction of the first frame to the second frame is realized, and the subsequent frames are analogous, and the model is updated according to the specified rules. However, target tracking is affected by many factors. When the posture of the moving target changes, resulting in changes in the feature and appearance model, the change in the target scale causes the background to be mixed, and the target is blocked and disappears during the movement, it may cause the target tracking frame to not correctly circle the target, resulting in insufficient target information in the tracking frame, and the background and other target information are mixed, resulting in interference frames, and target tracking failure.

[0040] Therefore, for multi-target tracking, when the target is occluded or disappears, how to improve the precision and accuracy of target tracking is an urgent problem to be solved.

[0041] The present application provides a multi-object tracking method, which relates to the field of financial technology or other related fields, and in particular to a multi-object tracking method, device, equipment and storage medium, aiming to solve the above technical problems of the prior art.

[0042] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0043] Figure 1 A schematic diagram of multi-hypothesis tracking of a multi-object tracking method provided in this application, such as Figure 1 As shown in the figure, from time t-3 to time t, there are multiple target trajectories, namely T1-T9. Assume that there are currently two target objects. At time t-3, T1 is an existing trajectory of one of the target objects. There are two actual measurement information at time t-2, that is, two trajectories can be formed, namely T1 and T2. T3 is a new trajectory, that is, the object corresponding to the trajectory is not the specified target object. T1 is an existing trajectory of one of the target objects, then there are two possible trajectories from time t-3 to time t-2. For trajectory T2, there are also two possible trajectories from time t-2 to time t-1, and they are divided in sequence until time t. Calculate the weight of each trajectory and delete the trajectory with the smallest weight. For example, if the weight of T7 trajectory is the smallest, delete its corresponding trajectory and its branches, that is, delete the T1-T1-T4-T7 trajectory. If the target corresponding to the new trajectory is not the specified target object, then the new trajectory is also deleted, where the new trajectory refers to the trajectory that is not the specified target object. Convert the trajectory tree to Figure 1The following figure in the figure is described. Take a target as an example. At time t-3, the actual measurement information of the target object is obtained, where the actual measurement information includes the position information and speed information of the target object, and the target object is located at point A. A predicted position at time t-2, that is, point a, can be determined based on the actual measurement information. With point a as the center, a range is determined. There may be multiple objects in the range. Based on point a again, the predicted position b at time t-1 is determined. And so on, until time t, there is a defined range at each time, and there may be multiple objects in each range. Therefore, multiple trajectories can be determined from the position at time t-3 to the position at time t, that is, A-1-4-8 or A-1-4-7. The trajectory with the highest weight in the trajectory formed from point A to the position at time t is determined as the tracking trajectory of the target object.

[0044] Figure 2 A flowchart of a multi-object tracking method provided in this application is shown in FIG. Figure 2 As shown, the method includes:

[0045] S201. If the target object is not tracked at two consecutive moments, obtain actual measurement information of the target object at a historical moment; wherein the actual measurement information represents the actual position information and speed information of the object; and the historical moment is the previous moment between the two consecutive moments.

[0046] Exemplarily, the actual measurement information includes the actual position information and speed information of the object. The position information refers to the specific position coordinates of the target object in space, and the speed information refers to the movement speed and direction of the target object. The target object is the object that the robot needs to track, and the robot can track multiple target objects. The historical moment refers to the previous moment between two consecutive moments. For example, the current moment is moment t+1, and the consecutive moments are moment t and moment t+1, then the historical moment represents moment t-1. If the target object cannot be detected or identified at two consecutive moments, this may be due to reasons such as the target object being blocked or environmental interference. The system will go back to an earlier time point, namely the historical moment, to obtain the actual measurement information of the target object. Therefore, at moment t-1, if the target object already has an existing track, the actual measurement information of the target object at moment t-1 is obtained. The mathematical model for establishing the actual measurement information is:

[0047] Assuming that there are multiple objects at time τ, that is, there are multiple actual measurement information, the target trajectory at the current time τ is:

[0048] i∈X τ-1 ={1,2,…,n τ}

[0049] Where X represents the set of trajectories of all objects up to time τ, i is the index of each target trajectory, that is, the unique label of each trajectory, and n τ is the number of trajectories of all existing objects.

[0050] The actual measurement information of the candidate box selected at the current moment is:

[0051]

[0052] in, Indicates the actual measurement information obtained by scanning at a certain moment, such as time t, m t is the total number of actual measurement information scanned at time t, τ represents the total number of times the actual measurement information is scanned, and time t is a moment from the historical moment to time τ, and the actual measurement information is scanned once at each moment. For example, there are 3 targets at time t. The total number of actual measurement information scanned at time t is 3, and there are a total of 4 moments from time t to time t+3, so the total number of times the actual measurement information is scanned is 4. For another example, the total number of actual measurement information at time t+2 means that the total number of actual measurement information is 3 as of time t+2.

[0053] The beneficial effect of such a setting is that the actual measurement information of the target at the historical moment is obtained, which is convenient for improving the accuracy of target tracking.

[0054] S202: Determine the actual measurement information collected at a previous moment and the actual measurement information collected at a later moment in two consecutive moments according to the actual measurement information of the target object at the historical moment.

[0055] Exemplarily, the actual measurement information of the target object at the historical moment includes the position information and the speed information. For example, if the historical moment is moment t-1, then all the actual measurement information collected at the previous moment and all the actual measurement information collected at the next moment in two consecutive moments are all the actual measurement information collected at moment t and all the actual measurement information collected at moment t+1, respectively. For example, if the target object to be tracked is pre-specified, a predicted position can be determined at moment t and moment t+1 respectively according to the actual measurement information of the target object at the historical moment using the Kalman filter algorithm, with the predicted position as the center of the circle and the preset value as the radius, the framed range corresponding to moment t and moment t+1 is determined respectively, and the framed range is scanned at moment t and moment t+1 respectively, and all the actual measurement information collected within the framed range at moment t and all the actual measurement information collected within the framed range at moment t+1 are collected.

[0056] The beneficial effect of such a setting is that actual measurement information at two consecutive moments can be determined respectively based on historical actual measurement information, thereby facilitating improving the accuracy of target tracking.

[0057] In this embodiment, the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment between two consecutive moments are determined based on the actual measurement information of the target object at the historical moment, including: determining the predicted measurement information of the target object at the previous moment between two consecutive moments based on the actual measurement information of the target object at the historical moment; wherein the predicted measurement information represents the predicted position information and speed information of the target object; determining the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment between two consecutive moments based on the predicted measurement information of the target object at the previous moment between two consecutive moments.

[0058] Exemplarily, the predicted measurement information represents the predicted position information and speed information of the target object, and the actual measurement information of the target object includes the actual position information and speed information of the target object. For example, the historical moment is the t-1 moment, and the two consecutive moments are the t moment and the t+1 moment. First, according to the actual measurement information of the target object at the historical moment, the predicted measurement information of the target object at the t moment is determined by using the Kalman filter algorithm. The position in the predicted measurement information can be used as the center of the circle and the preset value as the radius to define the scanning range of a system. The robot system determines all objects that can be scanned within the scanning range and collects the actual measurement information of all scanned objects, wherein a radar or sensor can be installed in the robot to scan the objects within the scanning range. Then, according to the predicted measurement information of the target object at the t moment and the Kalman filter algorithm, the new predicted measurement information of the target object at the t+1 moment is determined, and the predicted position in the new predicted measurement information is used as the center of the circle and the preset value as the radius to determine a new scanning range. The system scans all existing objects in the new scanning range and collects the actual measurement information of all scanned objects. Among them, the scanning range determined at time t is the same size as the new scanning range determined at time t+1, but the positions in the predicted measurement information are different, that is, the positions of the center of the circle are different. In addition, the scanning range can also be a square centered on the predicted position in the predicted measurement information. In this embodiment, the preset Kalman filter algorithm is not specifically limited. The actual measurement information of the target object at the historical moment can be input into the Kalman filter algorithm as a variable, and the output data is the predicted measurement information of the target object at time t; the predicted measurement information of the target object at time t is input into the Kalman filter algorithm, and the output data is the predicted measurement information of the target object at time t+1.

[0059] The beneficial effect of such a setting is that the predicted measurement information is determined based on the actual measurement information of the target object at a historical moment, which makes it easier to use the historical position of the target as a reference to make the predicted measurement information more accurate, thereby facilitating improving the accuracy and precision of target tracking.

[0060] In this embodiment, the predicted measurement information of the target object at the previous moment between two consecutive moments is determined based on the actual measurement information of the target object at the historical moment, including: based on the actual measurement information of the target object at the historical moment, based on a preset Kalman filtering algorithm, determining the predicted measurement information of the target object at the previous moment between two consecutive moments.

[0061] Exemplarily, the actual measurement information of the target object at the historical moment includes position information and speed information. For example, if the target object is not tracked at two consecutive moments, that is, the target may be obscured, it is assumed that the two consecutive moments are the current moment and the moment before the current moment, that is, the current moment is the t+1 moment, the moment before the current moment is the t moment, and the historical moment is the t-1 moment. For example, if the target object can be tracked at the t-1 moment, that is, the actual trajectory of the target object and the position information of the target object at the t-1 moment are already stored at the t-1 moment, then the actual measurement information of the target object at the t-1 moment is first obtained, that is, the actual position information and the actual speed information, wherein the actual position information and the actual speed information of the target object at the t-1 moment can be obtained by the sensor or radar in the robot, and the actual position information and the actual speed information of the target object at the t-1 moment can reflect the real state of the target object at that moment. According to the actual position information and actual speed information of the target object at time t-1, based on the preset Kalman filter algorithm, the predicted measurement information of the target object at time t can be determined, that is, the input data is the actual measurement information of the target object at time t-1, and the output data is the predicted measurement information of the target object at time t. The predicted measurement information includes the predicted position information and predicted speed information of the target object at time t.

[0062] The beneficial effect of this setting is that the Kalman filter algorithm is used to determine the predicted measurement information based on the actual measurement information of the target object at the historical moment, which is convenient for referring to the historical position of the target object, quickly determining the predicted position, and improving the accuracy and efficiency of target tracking.

[0063] In this embodiment, the predicted measurement information includes predicted position information; based on the predicted measurement information of the target object at the previous moment between two consecutive moments, the actual measurement information collected at the previous moment between two consecutive moments and the actual measurement information collected at the next moment between two consecutive moments are determined, including: determining a first monitoring area of ​​a preset range size centered on the predicted position information of the target object at the previous moment between two consecutive moments; based on the first monitoring area, obtaining the actual measurement information collected at the previous moment between two consecutive moments; based on the predicted position information of the target object at the previous moment between two consecutive moments, determining the actual measurement information collected at the next moment between two consecutive moments.

[0064] Exemplarily, the predicted measurement information includes predicted position information. For example, assuming that the historical moment is moment t-1, and the two consecutive moments are moment t and moment t+1, if the target is not tracked at two consecutive moments, the system first obtains the actual measurement information of the target object at the historical moment, that is, the position information and speed information at moment t-1. According to the position information and speed information at moment t-1, based on the preset Kalman filter algorithm, the predicted measurement information of the target object at moment t can be obtained, wherein the predicted measurement information includes position information and speed information. A circular scanning area, i.e., the first monitoring area, can be determined with the position in the predicted measurement information as the center and the preset value as the radius. For another example, a square scanning area can be determined with the position in the predicted measurement information as the center. Among them, the shape of the first monitoring area can include multiple types, which can be circular or square. The system scans all objects existing in the first monitoring area and collects the actual measurement information of all objects existing in the area, i.e., the actual measurement information collected at the previous moment in two consecutive moments. Among them, the scanning can be performed by using the radar or sensor in the robot. Based on the predicted measurement information at time t and the preset Kalman filtering algorithm, new predicted measurement information at time t+1 is obtained. A new scanning area can be determined with the new predicted measurement information as the center. The system scans all objects in the new scanning area and collects actual measurement information of all objects in the new scanning area, that is, the actual measurement information collected at the latter moment between two consecutive moments.

[0065] The beneficial effect of such a setting is that the scanning area is determined according to the predicted measurement information, which facilitates the reduction of the scanning area and improves the accuracy of target tracking.

[0066] In this embodiment, actual measurement information collected at a later moment between two consecutive moments is determined based on the predicted position information of the target object at the previous moment between two consecutive moments, including: determining new predicted measurement information of the target object at the later moment between two consecutive moments based on a preset Kalman filtering algorithm based on the predicted position information of the target object at the previous moment between two consecutive moments; determining a second monitoring area of ​​a preset range size centered on the new predicted position information of the target object at the later moment between two consecutive moments; and obtaining the actual measurement information collected at the later moment between two consecutive moments based on the second monitoring area.

[0067] Exemplarily, the predicted measurement information includes position information and speed information. For example, the historical moment is moment t-1, and the two consecutive moments are moment t and moment t+1. After determining the predicted measurement information of the target object at moment t based on the actual measurement information of the historical moment of the target object, the Kalman filter algorithm is used again, and the predicted measurement information of the target object at moment t is used as input data to obtain the new predicted measurement information of the target object at moment t+1, that is, the output data is the predicted measurement information of the target object at moment t+1. With the position in the new predicted measurement information as the center, a new scanning range is determined, that is, the second monitoring area, wherein the scanning range can be a circular area with the position in the predicted measurement information as the center and a preset value as the radius, or a square area with the position in the predicted measurement information as the center. Based on the second monitoring area, the system scans all objects existing in the second monitoring area and collects the actual measurement information of all objects in the second monitoring area.

[0068] The beneficial effect of such a setting is that the actual measurement information at the next moment in the continuous moments is determined based on the predicted measurement information at the previous moment in the continuous moments, thereby determining the second monitoring area, which facilitates narrowing the scanning area and improving the precision and accuracy of target tracking.

[0069] S203, determining a tracking trajectory of the target object based on actual measurement information collected at a previous moment and actual measurement information collected at a later moment between two consecutive moments; wherein the tracking trajectory is a movement trajectory of the target object between a historical moment and a later moment between two consecutive moments.

[0070] Exemplarily, the tracking trajectory is the trajectory between the position of the target object at the historical moment and the position of the target object at the latter moment of two consecutive moments. The actual measurement information includes position information and speed information. For example, assuming that the historical moment is moment t-1, and the two consecutive moments are moment t and moment t+1, after determining the actual measurement information of the previous moment of the two consecutive moments according to the actual measurement information of the target at the historical moment, that is, the actual measurement information at moment t. According to the predicted measurement information of the target object at moment t and the Kalman filter algorithm, a new predicted measurement information can be determined, and the actual measurement information at the latter moment, that is, the actual measurement information at moment t+1, can be determined according to the new predicted measurement information. According to the actual measurement information of the target object at moment t-1 and the actual measurement information at moment t, an associated variable of the actual measurement information of the target object at moment t-1 and the actual measurement information at moment t can be established, wherein the associated variable of the actual measurement information of the target object at moment t-1 and the actual measurement information at moment t represents the association relationship between the actual measurement information of the target object at moment t-1 and the actual measurement information at moment t. Then, based on the actual measurement information at time t and the actual measurement information at time t+1, the associated variables of the actual measurement information at time t and the actual measurement information at time t+1 are established. The associated variables of the actual measurement information at time t and the actual measurement information at time t+1 represent the association relationship between the actual measurement information at time t and the actual measurement information at time t+1. Based on the above two associated variables, an associated variable between the actual measurement information of the target object at time t-1 and the actual measurement information at time t+1 can be obtained. Among them, there can be multiple associated variables between the actual measurement information of the target object at time t-1 and the actual measurement information at time t+1. For each trajectory corresponding to the associated variable between the actual measurement information of the target object at time t-1 and the actual measurement information at time t+1, the cost value is calculated, and the one with the smallest cost value is found, that is, the objective function of the optimal trajectory from time t-1 to time t+1 is established. The cost value refers to the importance of each trajectory. The process of establishing the objective function is as follows:

[0071] The current time is τ, and each existing trajectory i∈X τ , X τ is the trajectory of all objects that exist up to the current time. The associated variable from the actual measurement information of the target object at time t to the actual measurement information at the current time or the missed detection is Among them, time t is a historical moment before the current time. If a missed detection occurs and the target is not detected, the associated variable of the target at this time is is zero. The single trajectory association hypothesis set of the i-th target object trajectory at the current time τ is a i τ=(a i 1,…a i τ )∈A τ , where a single trajectory refers to each trajectory from the location of the target object at a historical moment to the location of all objects at the current moment. The global assumption is A τ Represents the global hypothesis of all target objects. One target object corresponds to one global hypothesis, and the global hypothesis of each target object contains multiple sets of single trajectory association hypotheses.

[0072] Assume that the weight of a single trajectory is Then the cost of each trajectory is:

[0073]

[0074] in, is the weight of a single trajectory, a i τ Associate a set of hypotheses for a single trajectory.

[0075] Therefore, solving the data association problem is to find the set of each single trajectory hypothesis that minimizes the total cost under the constraints of the classical hypothesis, that is, the optimal global hypothesis. Therefore, the objective function is:

[0076]

[0077] Among them, A * is the optimal global assumption, is the cost of a single trajectory, i is the index of each trajectory, that is, the unique label of each trajectory, X t is the set of trajectories of all objects up to time t, A τ Represents the global hypothesis for all target objects, where one target corresponds to one global hypothesis.

[0078] The beneficial effect of such a setting is that the tracking trajectory of the target object is determined based on the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in two consecutive moments, and the multi-hypothesis method is used to make the target object have multiple possible trajectories, which is convenient for improving the accuracy of target tracking.

[0079] The present application provides a multi-object tracking method, device, equipment and storage medium. If the target object is not tracked at two consecutive moments, the actual measurement information of the target object at the historical moment is obtained; wherein the actual measurement information represents the actual position information and speed information of the object; the historical moment is the previous moment of the two consecutive moments; according to the actual measurement information of the target object at the historical moment, the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in the two consecutive moments are determined; according to the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in the two consecutive moments, the tracking trajectory of the target object is determined; wherein the tracking trajectory is the movement trajectory of the target object between the historical moment and the next moment in the two consecutive moments. In this scheme, according to the actual measurement information of the target object at the historical moment, the measurement information of the target object at two consecutive moments after the historical moment is predicted, and then the tracking trajectory of the target object is determined according to the measurement information of these three moments, so as to realize the tracking of the target object when the target object is blocked, thereby improving the precision and accuracy of target tracking.

[0080] Figure 3 A flowchart of a multi-object tracking method provided in this application is shown in FIG. Figure 3 As shown, the method includes:

[0081] S301. If the target object is not tracked at two consecutive moments, then obtain actual measurement information of the target object at a historical moment; wherein the actual measurement information represents the actual position information and speed information of the object; and the historical moment is the previous moment between the two consecutive moments.

[0082] Exemplarily, this step may refer to the above-mentioned step S201 and will not be described in detail.

[0083] S302: Determine the actual measurement information collected at a previous moment and the actual measurement information collected at a later moment in two consecutive moments according to the actual measurement information of the target object at the historical moment.

[0084] Exemplarily, this step may refer to the above-mentioned step S202 and will not be described in detail.

[0085] S303, determining a candidate trajectory of the target object according to the historical location information, the first location information, and the second location information; wherein the candidate trajectory represents a trajectory from the historical location information, through any first location information, to any second location information.

[0086] Exemplarily, the candidate trajectory represents a trajectory from the historical position information, through any first position information, to any second position information. The first position information belongs to the actual measurement information collected at the previous moment between two consecutive moments, and there can be multiple first position information. The second position information belongs to the actual measurement information collected at the next moment between two consecutive moments, and there can also be multiple second position information. For example, the historical moment is moment t-1, and the two consecutive moments are moment t and moment t+1 respectively. After collecting the actual measurement information at the previous moment between the two consecutive moments and the actual measurement information at the next moment between the two consecutive moments, the actual measurement information of the target object at the historical moment is associated with the actual measurement information at the previous moment between the two consecutive moments to form a first segment of the trajectory, that is, the trajectory formed from the target position at moment t-1 to the object position at moment t, and there can be multiple objects at moment t. For example, the position of the target at time t-1 is point A. For example, the positions of the objects scanned at time t are 1, 2, and 3, i.e., the first position information. Then, three first-segment trajectories are formed from point A to 1, 2, and 3, respectively. The actual measurement information of the previous moment in two consecutive moments is associated with the actual measurement information of the latter moment in two consecutive moments to form a second-segment trajectory. For example, the positions of the objects scanned at time t+1 are 4, 5, and 6, i.e., the second position information. Then, multiple second-segment trajectories are formed from the positions of the objects at time t to the positions of the objects at time t+1, which can be 1-4 or 2-4. At this time, multiple candidate trajectories can be obtained according to the position of the target at time t-1, i.e., point A, the first segment trajectory, and the second segment trajectory, i.e., A-1-4 or A-2-6. Therefore, according to the historical position information, i.e., point A, the first position information, and the second position information, multiple trajectories from point A to the second position information, i.e., candidate trajectories, can be determined.

[0087] The beneficial effect of such a setting is that a plurality of candidate trajectories are formed, which facilitates tracking of the occluded target and improves the accuracy of target tracking.

[0088] S304. For each candidate trajectory, determine a first trajectory weight between the historical location information and the first location information according to the historical location information and the first location information; wherein the first trajectory weight represents the importance of the trajectory between the historical location information and the first location information.

[0089] Exemplarily, the first trajectory weight characterizes the importance of the trajectory between the historical position information and the first position information. For example, assuming that the historical moment is moment t-1, and the two consecutive moments are moment t and moment t+1, the historical position information represents the location of the target object at moment t-1, and the first position information is the location information of all objects at moment t, that is, there can be multiple first location information, and each object corresponds to one first location information. Assuming that the target location at moment t-1 is point A, and there are multiple objects at moment t, for example, the positions of each object at moment t are 1, 2, and 3, respectively, then the first trajectory weight refers to the weight of the trajectory formed from point A to each position at moment t, characterizing the importance of each trajectory formed from point A to each position at moment t. Among them, there are multiple first trajectory weights, that is, it can be the weight of the trajectory formed from point A to 1, or it can be the weight of the trajectory formed from point A to 2.

[0090] The beneficial effect of such a setting is that, based on the historical position information and the first position information, the first trajectory weight between the historical position information and the first position information is determined, so that the importance of the trajectory is determined according to the weight of each trajectory, thereby deleting the trajectory with low weight, finding the optimal trajectory, and improving the accuracy of target tracking.

[0091] In this embodiment, determining a first trajectory weight between the historical location information and the first location information according to the historical location information and the first location information includes: determining the first trajectory weight between the historical location information and the first location information based on a preset first joint probability density function according to the historical location information and the first location information.

[0092] Exemplarily, the first trajectory weight represents the importance of the trajectory formed from the historical position to the first position information. Assuming that the historical moment is moment t-1, and the two consecutive moments are moment t and moment t+1, the historical position information is the location of the target object at moment t-1, and the first position information is the location information of all objects in the set area at moment t, that is, there can be multiple first position information, and each object has a first position information. According to the location information and the first location information at the historical moment, based on the preset first joint probability density function, the weight of the trajectory formed between the historical position and the location of the first location information can be determined, that is, the first trajectory weight. Among them, there can be multiple first trajectory weights, and multiple trajectories can be formed between the historical position and the location of the first location information, and each trajectory corresponds to a first trajectory weight. The joint probability density function is used to describe the probability of two random variables (here, the historical location information and the first location information) appearing at the same time.

[0093] The beneficial effect of such a setting is that the first track weight is determined according to the historical position and the first position information, so that the importance of the track can be determined according to the weight of each track, thereby screening the tracks and improving the accuracy of target tracking.

[0094] S305 , determining a second trajectory weight between the first location information and the second location information according to the first location information, the second location information, and the first trajectory weight; wherein the second trajectory weight represents the importance of the candidate trajectory.

[0095] Exemplarily, assuming that the historical moment is moment t-1, and the two consecutive moments are moment t and moment t+1, the historical location information is the location of the target object at moment t-1, the first location information is the location information of all objects in the set area at moment t, and the second location information is the location information of all objects in the set area at moment t+1, that is, there are multiple first location information and second location information. Multiple trajectories can be formed from the historical position to the location where the first location information is located, and multiple trajectories can also be formed from the location where each first location information is located to the location where each second location information is located. For example, after determining the first trajectory weight according to the historical position and the first location information, the second trajectory weight can be determined according to the first location information, the second location information and the first trajectory weight, that is, the second trajectory weight represents the weight of the trajectory formed from the historical position to the location where the second location information is located, wherein there are multiple second trajectory weights, and the second trajectory weight is the weight of the candidate trajectory. For example, the historical position at moment t-1 is point A, the first location information has 1, 2, and 3, and the second location information has 4, 5, and 6. Then the second track weight can be the weight of the A-1-4 track or the weight of the A-2-6 track.

[0096] The beneficial effect of such a setting is that the weights of the second trajectories are determined according to the first position information, the second position information, and the first trajectory weight, so that the best trajectory can be determined according to the second trajectory weights of each trajectory, thereby improving the accuracy of target tracking.

[0097] In this embodiment, determining the second trajectory weight between the first position information and the second position information according to the first position information, the second position information, and the first trajectory weight includes: determining the second trajectory weight between the first position information and the second position information based on a preset second joint probability density function according to the first position information, the second position information, and the first trajectory weight.

[0098] Exemplarily, after obtaining the first trajectory weight using the preset first joint probability density function according to the historical position and the first position information, the weight of the trajectory formed from the location of the first position information to the location of the second position information, i.e., the second trajectory weight, can be determined according to the first position information, the second position information, and the first trajectory weight. Among them, the first trajectory weight refers to the weight of the trajectory formed from the historical position to the location of the first position information, and there are multiple first trajectory weights and second trajectory weights. The second joint probability density function is used to describe the probability of two random variables (here, the first position information and the second position information) appearing at the same time. For example, assuming that the historical position is point A, the first position information is a, b, and c, respectively, and the second position information is d, e, and f, respectively, then the trajectory from the location of the first position to the location of the second position information can be ad or bf. According to the obtained first trajectory weight, the first position information, and the second position information, the second trajectory weight can be obtained using the preset second joint probability density function. Among them, the second trajectory weight refers to the weight of the trajectory formed from the historical position to the location of the second position information, i.e., the second trajectory weight can be the weight of the Aad trajectory or the weight of the Abf trajectory.

[0099] The beneficial effect of such a setting is that the second track weight is determined by using the first track weight, which facilitates the rapid determination of the second track weight of each track, makes the second track weight more accurate, and improves the accuracy and efficiency of target tracking.

[0100] S306 : Determine a tracking trajectory of the target object according to the second trajectory weight of each candidate trajectory.

[0101] Exemplarily, the second trajectory weight refers to the weight of the trajectory formed from the historical position to the location of the second position information. The second trajectory weight is the weight of the candidate trajectory, and there can be multiple candidate trajectories. For example, after associating the historical position, the location of the first position information, and the location of the second information, multiple associated variables are obtained. Each associated variable corresponds to a trajectory from the historical position to the location of the second information. Establish an objective function based on multiple associated variables. Before solving the objective function, first determine two constraints: First, each trajectory can and can only be used once, which means that for each global hypothesis there is only one single trajectory hypothesis, that is, there can only be one optimal trajectory; second, each actual measurement information can only be used once, that is, each actual measurement information belongs to only one target. Use a binary variable ρ(a i )∈{0,1} to indicate that each of the above trajectories and each actual measurement information can only be used once, then the above constraint can be expressed as:

[0102] Constraint 1: Among them, i is the index of each trajectory, that is, the unique label of each trajectory, X is the set of all object trajectories, and a i Represents the associated variable of a trajectory, A i It represents the global hypothesis of the target corresponding to a certain trajectory. One target corresponds to one global hypothesis. One global hypothesis contains multiple single trajectory sets, where a single trajectory refers to the trajectory formed from the historical position to the position of the object at a certain moment.

[0103] Constraint 2: Where i is the index of each trajectory, X is the set of all object trajectories, j is the index of each actual measurement information, and R t is the collection of all actual measurement information, It represents the set of association hypotheses between the jth actual measurement information and the i-th trajectory at time t.

[0104] in, is the set of all hypotheses, A i is the global hypothesis for the ith trajectory.

[0105] Based on the above two constraints, the objective function is split and refined to obtain:

[0106]

[0107] Among them, ρ is the set of two constraints, i is the index of each trajectory, X is the set of all trajectories, and a i is the associated variable for each trajectory, c(a i ) is the cost of each target object, that is, the weight w(a i ), ρ(a i ) is a binary variable.

[0108] The above function is solved using the dual decomposition method, which decomposes the original problem into subproblems containing copies of the original variables and introduces a constraint, which is performed through Lagrangian relaxation.

[0109] By introducing the Lagrange multiplier, the solution of the original objective function is transformed into the solution of the sub-problem, and the t-th sub-problem is:

[0110]

[0111] in, R is the constraint set of the preset Lagrange multiplier, δ t (a i ) is the Lagrange multiplier, ρ t (a i ) is a set of two constraints, c(a i) is the cost of each trajectory, τ is the current time, a i is the associated variable of each trajectory, and Ai is the hypothesis set of all trajectories.

[0112] Using the gradient update to solve the subproblem, for any assumption, the original problem is:

[0113]

[0114] Among them, a i is the associated variable of each trajectory, Ai is the hypothesis set of all trajectories, c(a i ) is the cost of each trajectory, τ is the current moment, δ t (a i ) is the Lagrange multiplier.

[0115] The update of Lagrange multipliers is;

[0116] δ t (a i )=δ t (a i )+α k g t (a i )

[0117] Among them, g t (a i ) is the subgradient of the projection, g t = {g t (a i )∈R|a i ∈A i , i∈X},α k is the iteration step size.

[0118] The subgradient update is:

[0119]

[0120] Among them, ρ t (a i ) is a set of two constraints, ρ t′ (a i ) is the constraint condition at t'.

[0121] Solving the dual cost is:

[0122]

[0123] The iterative step size is:

[0124]

[0125] Among them, L kis the original cost, D k is the dual cost, g t is a sub-gradient.

[0126] The condition for stopping iteration is when the difference gap is less than the preset threshold:

[0127]

[0128] When the difference gap is below the preset threshold, the iteration stops, that is, L k is the best (minimum) feasible primal cost obtained so far, D k is the dual cost calculated after k iterations. The trajectory corresponding to the minimum primitive cost value is the tracking trajectory of each target. For example, there are 3 target objects, whose historical positions are A, B, and C, whose first position information is 1, 3, and 5, and whose second position information is 2, 4, and 6. The trajectory of each target is A-1-2, B-3-4, and C-5-6. If the sum of the cost values ​​of the trajectory of each target object is the minimum primitive cost, then the above three trajectories are determined to be the tracking trajectories of the three target objects. Among them, each target object can have multiple trajectories, and the primitive cost means selecting any one trajectory from the multiple trajectories of each target object to minimize its total cost value.

[0129] The beneficial effect of such a setting is that the best tracking trajectory is determined according to the second trajectory weight, which facilitates the deletion of trajectories with low weights, thereby quickly finding the best trajectory and improving the accuracy of target tracking.

[0130] In this embodiment, determining the tracking trajectory of the target object according to the second trajectory weight of each candidate trajectory includes:

[0131] Determine the negative logarithm of the second track weight of the candidate track; sort the negative logarithms corresponding to the candidate tracks, and determine the tracking track of the target object from the candidate tracks according to the sorting result.

[0132] Exemplarily, a candidate trajectory refers to a trajectory formed from the historical moment position to the position where the second position information is located. There are multiple candidate trajectories. The second trajectory weight is the weight of the trajectory formed from the historical moment position to the position where the second position information is located. First, the cost of each second trajectory weight is determined. The cost of the second trajectory weight is the negative logarithm of the second trajectory weight. The cost value of each candidate trajectory is sorted to obtain a sorting result, wherein the lower the cost value, the greater the possibility of the trajectory corresponding to the cost value. A minimum cost value is determined from the cost value sorting result of the candidate trajectory, that is, the trajectory corresponding to the cost value is the tracking trajectory of the target object. For example, the historical position is point A, the first position information is 1, 2, 3, and the second position information is 4, 5, 6. There are multiple candidate trajectories, for example, the candidate trajectory can be A-1-4 or A-3-5. The cost value is calculated for each candidate trajectory, and a trajectory with the lowest cost value is selected from multiple candidate trajectories. For example, the cost value of A-3-6 is the lowest, and this trajectory is the best trajectory of the target object.

[0133] The beneficial effect of such a setting is that the cost value of the trajectory is determined according to the weight, and the cost values ​​are sorted to determine the tracking trajectory of the target object, which facilitates the selection of the best trajectory of the target object and improves the accuracy of target tracking.

[0134] In this embodiment, a plurality of target objects are pre-set; determining the tracking trajectory of the target object according to the second trajectory weight of each candidate trajectory includes: determining a candidate trajectory from the candidate trajectory of each target object; wherein the candidate trajectory belongs to the candidate trajectory; determining a trajectory cost value according to the negative logarithm corresponding to the candidate trajectories of all target objects; wherein the trajectory cost value represents the possibility that the tracking trajectory of each target object is the corresponding candidate trajectory; and determining the tracking trajectory of each target object according to the trajectory cost value.

[0135] Exemplarily, the trajectory cost value represents the possibility that the tracking trajectory of each target object is the corresponding candidate trajectory. For the tracking of multiple targets, for example, there are three targets, and the historical positions of the three targets at historical moments are point A, point B, and point C, respectively. The first position information has the position information of four targets 1, 2, 3, and 4, and the second position information has the position information of four targets 5, 6, 7, and 8. For target A, there are multiple candidate trajectories of A, for example, it can be A-1-5 or A-1-6, there are multiple candidate trajectories of target B, for example, it can be B-1-5 or B-2-7, and target C also has multiple candidate trajectories, which can be C-3-6 or C-4-8. Select one trajectory from the multiple candidate trajectories of target A as the first candidate trajectory, select one trajectory from the multiple candidate trajectories of target B as the second candidate trajectory, and select one trajectory from the multiple candidate trajectories of target C as the third candidate trajectory. According to the negative logarithm value corresponding to the weight of each candidate trajectory, that is, the cost of each target candidate trajectory, the cost value of each target trajectory is determined.

[0136] The beneficial effect of such a setting is that the trajectory cost value is determined according to the negative logarithm of the candidate trajectory of each target, so that the trajectory with the minimum cost value is determined according to the cost value of each trajectory, thereby determining the optimal trajectory and improving the accuracy of target tracking.

[0137] In this embodiment, the trajectory cost value is determined according to the negative logarithms corresponding to the candidate trajectories of all target objects, including: adding the negative logarithms corresponding to the candidate trajectories of each target object to obtain the trajectory cost value.

[0138] Exemplarily, the candidate trajectory refers to the trajectory formed from the historical position of the target logarithm to the position where the second position information is located. For example, for the three targets A, B, and C, each target corresponds to multiple candidate trajectories, and each candidate trajectory corresponds to a weight, that is, each candidate trajectory has a cost value. Select one trajectory from the multiple candidate trajectories of target A as the first candidate trajectory, select one from the multiple candidate trajectories of target B as the second candidate trajectory, and select one from the multiple candidate trajectories of target C as the third candidate trajectory, and calculate the negative logarithm of the weight of the candidate trajectory of each target, that is, the cost value. Add the cost value of the first candidate trajectory, the cost value of the second candidate trajectory, and the cost value of the third candidate trajectory to obtain the trajectory cost value. Among them, there are multiple trajectory cost values, and the obtained trajectory cost values ​​are sorted to determine the minimum value of the trajectory cost value. Then the candidate trajectory of each target corresponding to the minimum value is the optimal tracking trajectory for each target.

[0139] The beneficial effect of such a setting is that the cost values ​​of the candidate trajectories of each target are added together to determine the minimum trajectory cost value, which facilitates the determination of the best tracking trajectory for each target and improves the accuracy of target tracking.

[0140] The present application provides a multi-object tracking method, device, equipment and storage medium. If the target object is not tracked at two consecutive moments, the actual measurement information of the target object at the historical moment is obtained; wherein the actual measurement information represents the actual position information and speed information of the object; the historical moment is the previous moment of the two consecutive moments; according to the actual measurement information of the target object at the historical moment, the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in the two consecutive moments are determined; according to the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in the two consecutive moments, the tracking trajectory of the target object is determined; wherein the tracking trajectory is the movement trajectory of the target object between the historical moment and the next moment in the two consecutive moments. In this scheme, according to the actual measurement information of the target object at the historical moment, the measurement information of the target object at two consecutive moments after the historical moment is predicted, and then the tracking trajectory of the target object is determined according to the measurement information of these three moments, so as to realize the tracking of the target object when the target object is blocked, thereby improving the precision and accuracy of target tracking.

[0141] Figure 4 A complete flow chart of a multi-object tracking method provided in this application, such as Figure 4 As shown, including:

[0142] S401, input candidate box target point information.

[0143] S402, Kalman filtering, predicting multi-target point information.

[0144] Exemplarily, assuming that the historical moment is moment t, the target point trace information of each target at moment t is obtained, wherein the target point trace information includes the position information and speed information of the target. The Kalman filter algorithm and the point trace information at moment t are used to predict the multi-target point trace information, that is, to determine the predicted position of each target at moment t+1. According to the predicted position and the preset range size, the scanning area is determined with the predicted position as the center, and in the determined area, all objects in the area are scanned to determine the point trace information of each object.

[0145] The beneficial effect of such a setting is that the point information of each target is determined according to the historical position of the target object and the Kalman filter algorithm, so as to improve the accuracy of target tracking.

[0146] S403: Establish a matching model and optimization problem, and find the optimal solution.

[0147] S404, obtaining the optimal solution, selecting the next step of associated points, and updating the trajectory.

[0148] Exemplarily, after determining the scanning area at time t+1 according to the predicted position, collect the point trace information of all objects in the area, and then determine the predicted position of the new target at time t+2 according to the predicted position and Kalman filter algorithm, determine a new scanning area according to the predicted position of the new target, and collect the point trace information of all objects in the new scanning area. According to the historical position, the point trace information of the target at time t+1, and the point trace information of the target at time t+2, establish a matching model and optimization problem, determine an objective function, and solve the objective function using the dual solution method to obtain the optimal solution of the objective function. The trajectory corresponding to the optimal solution is the best trajectory corresponding to each target.

[0149] The beneficial effect of such a setting is that a matching model is established for the target object and the optimal solution is solved, which facilitates the determination of the best trajectory for each target and improves the accuracy of target tracking.

[0150] S405: Is it finished?

[0151] S406: Output multi-target trajectories.

[0152] For example, after solving the objective function and obtaining an optimal solution, after determining the tracking trajectory of each target, it is determined whether the target object is still in the system scanning area within a preset time period. If it is, the next point of the target object is tracked. When the target object does not appear in the system scanning area within the preset time period, it is determined that the target is no longer in the area, and the tracking of the target is terminated at this time to obtain the tracking trajectory of each target.

[0153] The beneficial effect of such a setting is that it facilitates accurate tracking of the target even when the target is blocked, thereby improving the accuracy of target tracking.

[0154] Figure 5 A structural block diagram of a multi-object tracking device provided in an embodiment of the present application, such as Figure 5 As shown, a data transmission device 500 provided in this embodiment includes:

[0155] The acquisition module 501 is used to acquire the actual measurement information of the target object at a historical moment if the target object is not tracked at two consecutive moments; wherein the actual measurement information represents the actual position information and speed information of the object; and the historical moment is the previous moment between the two consecutive moments;

[0156] A first determination module 502 is used to determine the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in the two consecutive moments according to the actual measurement information of the target object at the historical moment;

[0157] The second determination module 503 is used to determine the tracking trajectory of the target object based on the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment between the two consecutive moments; wherein the tracking trajectory is the movement trajectory of the target object between the historical moment and the next moment between the two consecutive moments.

[0158] Figure 6 A structural block diagram of a multi-object tracking device provided in an embodiment of the present application, such as Figure 6 As shown, the device 600 includes: an acquisition module 601, a first determination module 602, and a second determination module 603, wherein the first determination module 602 includes a first determination unit 6021 and a second determination unit 6022, and the second determination module 603 includes a third determination unit 6031, a fourth determination unit 6032, a fifth determination unit 6033, and a sixth determination unit 6034.

[0159] In one example, the first determining module 602 includes:

[0160] The first determination unit 6021 is used to determine the predicted measurement information of the target object at the previous moment of the two consecutive moments according to the actual measurement information of the target object at the historical moment; wherein the predicted measurement information represents the predicted position information and speed information of the target object.

[0161] The second determination unit 6022 is used to determine the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in the two consecutive moments according to the predicted measurement information of the target object at the previous moment in the two consecutive moments.

[0162] In one example, the first determining unit 6021 includes:

[0163] The first determination subunit is used to determine the predicted measurement information of the target object at the previous moment of the two consecutive moments according to the actual measurement information of the target object at the historical moment and based on a preset Kalman filter algorithm.

[0164] In one example, the predicted measurement information includes predicted position information, and the second determining unit 6022 includes:

[0165] A second determining subunit is used to determine a first monitoring area of ​​a preset size with the predicted position information of the target object at the previous moment of the two consecutive moments as the center;

[0166] A third determining subunit is configured to obtain, based on the first monitoring area, actual measurement information collected at a previous moment in the two consecutive moments;

[0167] The fourth determining subunit is used to determine the actual measurement information collected at the latter moment between the two consecutive moments according to the predicted position information of the target object at the former moment between the two consecutive moments.

[0168] In one example, the fourth determination subunit is specifically used to determine new predicted measurement information of the target object at the later moment between the two consecutive moments based on the predicted position information of the target object at the previous moment between the two consecutive moments and based on a preset Kalman filtering algorithm; determine a second monitoring area of ​​a preset range size centered on the new predicted position information of the target object at the later moment between the two consecutive moments; and obtain actual measurement information collected at the later moment between the two consecutive moments based on the second monitoring area.

[0169] In one example, the actual measurement information of the target object at a historical moment includes historical location information, the actual measurement information collected at a previous moment in the two consecutive moments includes at least one first location information, and the actual measurement information collected at a later moment in the two consecutive moments includes at least one second location information, and the second determination module 603 includes:

[0170] The third determining unit 6031 is used to determine a candidate trajectory of the target object according to the historical location information, the first location information, and the second location information; wherein the candidate trajectory represents a trajectory from the historical location information, through any first location information, to any second location information;

[0171] The fourth determining unit 6032 is configured to determine, for each candidate trajectory, a first trajectory weight between the historical location information and the first location information according to the historical location information and the first location information; wherein the first trajectory weight represents the importance of the trajectory between the historical location information and the first location information;

[0172] a fifth determining unit 6033, configured to determine a second trajectory weight between the first location information and the second location information according to the first location information, the second location information, and the first trajectory weight; wherein the second trajectory weight represents the importance of the candidate trajectory;

[0173] The sixth determining unit 6034 is configured to determine the tracking trajectory of the target object according to the second trajectory weight of each candidate trajectory.

[0174] In one example, the fourth determining unit 6032 includes:

[0175] The fifth determining subunit is configured to determine, according to the historical location information and the first location information, a first trajectory weight between the historical location information and the first location information based on a preset first joint probability density function.

[0176] In one example, the fifth determining unit 6033 includes:

[0177] The sixth determining subunit is configured to determine, according to the first position information, the second position information, and the first trajectory weight, a second trajectory weight between the first position information and the second position information based on a preset second joint probability density function.

[0178] In one example, the sixth determining unit 6034 includes:

[0179] a seventh determination subunit, configured to determine a negative logarithm of a second trajectory weight of the candidate trajectory;

[0180] The eighth determination subunit is configured to sort the negative logarithms corresponding to the candidate trajectories, and determine the tracking trajectory of the target object from the candidate trajectories according to the sorting result.

[0181] In one example, multiple target objects are preset, and the sixth determining unit 6034 includes:

[0182] a ninth determination subunit, configured to determine a candidate trajectory from the candidate trajectories of each target object; wherein the candidate trajectory belongs to the candidate trajectory;

[0183] a tenth determination subunit, configured to determine a trajectory cost value according to negative logarithms corresponding to candidate trajectories of all target objects; wherein the trajectory cost value represents the possibility that the tracking trajectory of each target object is the corresponding candidate trajectory;

[0184] The eleventh determining subunit is used to determine the tracking trajectory of each target object according to the trajectory cost value.

[0185] In one example, the tenth determining subunit is specifically configured to add the negative logarithms corresponding to the candidate trajectories of the target objects to obtain the trajectory cost value.

[0186] Figure 7 This is a schematic diagram of the structure of a multi-object tracking device provided by this application. Figure 7 As shown, the electronic device 700 provided in this embodiment includes: at least one processor 701 and a memory 702. Optionally, the device 700 also includes a communication component 703. The processor 701, the memory 702 and the communication component 703 are connected via a bus.

[0187] In a specific implementation process, at least one processor 701 executes the computer-executable instructions stored in the memory 702, so that at least one processor 701 executes the above method.

[0188] The specific implementation process of the processor 701 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0189] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0190] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0191] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0192] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0193] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. If not specifically stated, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. If not specifically stated, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc.

[0194] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0195] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0196] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0197] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A multi-object tracking method, characterized in that: include: If the target object is not tracked at two consecutive moments, then the actual measurement information of the target object at the historical moment is obtained; wherein the actual measurement information represents the actual position information and speed information of the object; and the historical moment is the previous moment between the two consecutive moments; According to the actual measurement information of the target object at the historical moment, determining the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in the two consecutive moments; The tracking trajectory of the target object is determined according to the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment between the two consecutive moments; wherein the tracking trajectory is the movement trajectory of the target object between the historical moment and the next moment between the two consecutive moments.

2. The method according to claim 1, characterized in that Determining the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in the two consecutive moments according to the actual measurement information of the target object at the historical moment includes: Determine the predicted measurement information of the target object at the previous moment in the two consecutive moments according to the actual measurement information of the target object at the historical moment; wherein the predicted measurement information represents the predicted position information and speed information of the target object; According to the predicted measurement information of the target object at the previous moment in the two consecutive moments, the actual measurement information collected at the previous moment in the two consecutive moments and the actual measurement information collected at the next moment in the two consecutive moments are determined.

3. The method according to claim 2, characterized in that Determining predicted measurement information of the target object at a previous moment in the two consecutive moments according to actual measurement information of the target object at the historical moment includes: According to the actual measurement information of the target object at the historical moment, based on a preset Kalman filter algorithm, the predicted measurement information of the target object at the previous moment of the two consecutive moments is determined.

4. The method according to claim 2, characterized in that The predicted measurement information includes predicted position information; determining the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in the two consecutive moments according to the predicted measurement information of the target object at the previous moment in the two consecutive moments, including: Determine a first monitoring area of ​​a preset size with the predicted position information of the target object at the previous moment of the two consecutive moments as the center; Based on the first monitoring area, obtaining actual measurement information collected at a previous moment in the two consecutive moments; According to the predicted position information of the target object at the previous moment in the two consecutive moments, the actual measurement information collected at the latter moment in the two consecutive moments is determined.

5. The method according to claim 4, characterized in that Determining actual measurement information collected at a later moment in the two consecutive moments according to the predicted position information of the target object at a previous moment in the two consecutive moments, including: Determine new predicted measurement information of the target object at a later moment in the two consecutive moments based on the predicted position information of the target object at a previous moment in the two consecutive moments and based on a preset Kalman filter algorithm; Determine a second monitoring area of ​​a preset size with the new predicted position information of the target object at the latter of the two consecutive moments as the center; Based on the second monitoring area, actual measurement information collected at a later moment of the two consecutive moments is obtained.

6. The method according to claim 1, characterized in that The actual measurement information of the target object at the historical moment includes the historical position information, the actual measurement information collected at the previous moment of the two consecutive moments includes at least one first position information, and the actual measurement information collected at the next moment of the two consecutive moments includes at least one second position information; determining the tracking trajectory of the target object according to the actual measurement information collected at the previous moment of the two consecutive moments and the actual measurement information collected at the next moment, including: Determine a candidate trajectory of the target object according to the historical location information, the first location information, and the second location information; wherein the candidate trajectory represents a trajectory from the historical location information, through any first location information, to any second location information; For each candidate trajectory, determining a first trajectory weight between the historical location information and the first location information according to the historical location information and the first location information; wherein the first trajectory weight represents the importance of the trajectory between the historical location information and the first location information; Determine a second trajectory weight between the first location information and the second location information according to the first location information, the second location information, and the first trajectory weight; wherein the second trajectory weight represents the importance of the candidate trajectory; A tracking trajectory of the target object is determined according to the second trajectory weight of each candidate trajectory.

7. The method according to claim 6, characterized in that Determining a first track weight between the historical location information and the first location information according to the historical location information and the first location information includes: According to the historical location information and the first location information, based on a preset first joint probability density function, a first trajectory weight between the historical location information and the first location information is determined.

8. The method according to claim 6, characterized in that Determining a second trajectory weight between the first location information and the second location information according to the first location information, the second location information, and the first trajectory weight includes: According to the first position information, the second position information, and the first trajectory weight, based on a preset second joint probability density function, a second trajectory weight between the first position information and the second position information is determined.

9. The method according to claim 6, characterized in that Determining the tracking trajectory of the target object according to the second trajectory weight of each candidate trajectory includes: Determining a negative logarithm of a second trajectory weight of the candidate trajectory; The negative logarithms corresponding to the candidate trajectories are sorted, and according to the sorting result, the tracking trajectory of the target object is determined from the candidate trajectories.

10. The method according to claim 9, characterized in that A plurality of target objects are preset; and according to the second trajectory weight of each candidate trajectory, the tracking trajectory of the target object is determined, including: Determine a candidate trajectory from the candidate trajectories of each target object; wherein the candidate trajectory belongs to the candidate trajectory; Determine a trajectory cost value according to the negative logarithms corresponding to the candidate trajectories of all target objects; wherein the trajectory cost value represents the possibility that the tracking trajectory of each target object is the corresponding candidate trajectory; The tracking trajectory of each target object is determined according to the trajectory cost value.

11. The method according to claim 10, characterized in that According to the negative logarithms corresponding to the candidate trajectories of all target objects, the trajectory cost value is determined, including: The negative logarithms corresponding to the candidate trajectories of each target object are added together to obtain the trajectory cost value.

12. A multi-object tracking device, comprising: An acquisition module, for acquiring actual measurement information of the target object at a historical moment if the target object is not tracked at two consecutive moments; wherein the actual measurement information represents the actual position information and speed information of the object; and the historical moment is the previous moment between the two consecutive moments; A first determination module is used to determine the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment in the two consecutive moments according to the actual measurement information of the target object at the historical moment; The second determination module is used to determine the tracking trajectory of the target object based on the actual measurement information collected at the previous moment and the actual measurement information collected at the next moment between the two consecutive moments; wherein the tracking trajectory is the movement trajectory of the target object between the historical moment and the next moment between the two consecutive moments.

13. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 11 when executed by a processor.

15. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 11 when being executed by a processor.

Images