Data association method, electronic device and storage medium for target tracking

Abstract

The present application provides a data association method, electronic device, and storage medium for target tracking, relating to the field of computer vision. The method comprises: obtaining n measurement data to be associated in a current frame of an image; obtaining m target sources determined in a preceding frame of the current frame, where m and n are both positive integers; determining the current association gate range value of each of the m target sources after data association is completed in the preceding frame based on the data change between consecutive adjacent frames preceding the current frame for the same target source; obtaining similarity distances; constructing a cost matrix between the m target sources and the n measurement data to be associated based on the similarity distances; and obtaining data association results between the n measurement data to be associated and the m target sources based on the cost matrix. The present application improves data association accuracy.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a data association method, electronic device, and storage medium for target tracking. Background Art

[0002] In existing multi-target tracking application scenarios, it is necessary to associate measurement data of uncertain sources with a target source through data association processing based on target detection. For example, the measurement data between two frames is associated with the corresponding target source to determine whether the two frames of measurement data belong to the same target object.

[0003] To reduce computational complexity in multi-target tracking scenarios, the K-Nearest Neighbor (KNN) algorithm is typically used to associate measurement data with corresponding target sources. However, the existing KNN algorithm relies on empirical experience to set the association gate (i.e., the search subspace consisting of the target source and its surrounding area), and its data association accuracy needs to be further improved. Summary of the Invention

[0004] In a first aspect, an embodiment of the present application provides a data association method for target tracking, comprising:

[0005] Obtain n measurement data to be associated in a current frame of an image;

[0006] Obtain m target sources determined in a frame preceding the current frame, where m and n are both positive integers;

[0007] Determine, based on a data change amount between consecutive adjacent frames preceding the current frame for the same target source, a current association gate range value for each of the m target sources after data association is completed in the preceding frame;

[0008] Obtaining a similarity distance between each target source in the m target sources and each to-be-associated measurement data within a current association gate range value;

[0009] Assigning a preset value to the similarity distance between each target source in the m target sources and each to-be-associated measurement data outside its current association gate range value;

[0010] Constructing a cost matrix between the m target sources and the n measurement data to be associated according to the similarity distance;

[0011] According to the cost matrix, data association results between the n measurement data to be associated and the m target sources are obtained.

[0012] In some embodiments, determining the association gate range value of each of the m target sources after data association is completed in the preceding frame based on the data change amount between consecutive adjacent frames of the same target source before the current frame includes:

[0013] By correlating the target source and measurement data of multiple consecutive frames, the data change of each target source between every two consecutive adjacent frames is obtained;

[0014] For each target source, based on the data changes of consecutive adjacent frames, respectively obtain the maximum data change of each target source;

[0015] According to the maximum data variation of each target source and the initial correlation gate range value, a current correlation gate range value of each target source in the current frame after the data correlation of the previous frame is completed is obtained.

[0016] In some embodiments, the method further comprises:

[0017] According to the maximum data variation of each target source and the initial correlation gate range value, the current correlation gate range value of each target source after the data association of the preceding frame is completed is obtained by using the correlation gate range value formula. The correlation gate range value formula is:

[0018] r k =e -αk R0+(1-e -αk +c)Δs max ;

[0019] Δs max =max{Δs1, ..., Δs k};

[0020] Among them, r k It represents the current association gate range value of a target source after completing the association calculation at the kth frame. R0 represents the initial association gate range value and is larger than the preset normal association gate radius. a is a parameter greater than 0. c represents the compensation parameter. Δs max Indicates the maximum data change, Δs k Indicates the data change of the target source from the k-1th frame to the kth frame. k is a positive integer marking the number of frames the target source is tracked. Each target source has its own k value.

[0021] In some embodiments, the similarity distance between each target source in the m target sources and each to-be-associated measurement data within its current association gate range value is the Mahalanobis distance between each target source and each to-be-associated measurement data within its current association gate range value.

[0022] In some embodiments, the preset value is a constant that is much larger than a preset normal distance.

[0023] In some embodiments, obtaining data association results between the n to-be-associated measurement data and the m target sources according to the cost matrix includes:

[0024] Solving the cost matrix to obtain optimal association pairs between the m target sources and the n measurement data to be associated;

[0025] The similarity distance of each optimal association pair is compared with a preset cost threshold to obtain a data association result between each measurement data to be associated and each target source, wherein the preset cost threshold is half of the constant.

[0026] In some embodiments, comparing the similarity distance of each optimal association pair with a preset cost threshold to obtain a data association result between each measurement data to be associated and each target source includes:

[0027] Determining that the similarity distance of the first optimal association pair is less than or equal to the preset cost threshold, determining that the measurement data to be associated in the first optimal association pair is successfully associated with the target source, updating the successfully associated measurement data to be associated to the successfully associated target source, and updating the data change amount and the association gate range value of the successfully associated target source;

[0028] Determining that the similarity distance of the second optimal association pair is greater than the preset cost threshold, determining that the second optimal association pair is not successfully associated, and disconnecting the association relationship between the to-be-associated measurement data in the second optimal association pair and the unsuccessfully associated target source.

[0029] In some embodiments, the method further comprises:

[0030] Mark the target source that has not been successfully associated as a data loss source, and re-track the data loss source. If the data loss state remains within a preset time period, delete the data loss source;

[0031] The measurement data that has not been successfully associated is initialized into a new target source, and data association is performed on the new target source in the next frame according to the initial association gate range value of the new target source.

[0032] In a second aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the methods described above is implemented.

[0033] In a third aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements any of the methods described above when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0035] Figure 1 A diagram of an application environment in which the data association method provided in one embodiment of the present application can be run;

[0036] Figure 2 A flowchart of a data association method for tracking a target in an image provided by one embodiment of the present application;

[0037] Figure 3 A schematic diagram of the process of associating a target source with data to be associated provided in one embodiment of the present application;

[0038] Figure 4 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0039] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0040] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first" and "second" are generally of the same type, and do not limit the number of objects. For example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0041] In the data association process under the existing multi-target tracking application scenario, the KNN algorithm is generally used to achieve the association between the target and the data. However, the setting of the range value size of the association gate in the KNN algorithm is mainly based on user experience and is set manually, resulting in a strong subjective factor in the maximum search radius near each target source. If the range value is set to a small value, it will cause missed associations, that is, a certain measurement data should belong to a certain target source, but the target source is not associated with this measurement data; if the range value is set to too large, it is easy to cause misassociation, that is, a certain measurement data is associated with the wrong target source. In this application, the source or target source refers to the object to which the measurement data belongs. For example, in the multi-target tracking application scenario, the source refers to the target that is successfully tracked.

[0042] It should be noted that the data association method for target tracking provided in the embodiments of the present application can be applied to various scenarios of association between targets and data. For example, in an unmanned driving scenario, multiple sensors are used to track targets in front of the vehicle. At this time, the target can be a pedestrian, a building, an obstacle or a lane line, etc. The measurement data includes but is not limited to target speed, target position information and target acceleration information.

[0043] An embodiment of the present application provides a nearest neighbor data association method based on online learning. By changing the association gate range in the nearest neighbor association algorithm from manually designed to automatic online learning, it minimizes missed associations while minimizing the occurrence of false associations, thereby improving data association accuracy.

[0044] The target tracking data association method provided by an embodiment of the present application can be applied to Figure 1 In the application environment shown. Figure 1 This is a diagram of an application environment in which the target tracking data association method provided by one embodiment of the present application can be run. Figure 1 As shown, the application environment includes a terminal 110 and a server 120. Terminal 110 and server 120 communicate with each other via a network. The communication network can be a wireless communication network or a wired communication network. The number of terminals and servers is not limited. The wireless communication network can include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity) and Bluetooth. The wired communication network can include, but is not limited to, at least one of the following: a wide area network, a metropolitan area network, and a local area network.

[0045] In some embodiments, the terminal 110 (terminal device) includes various handheld devices with wireless communication capabilities, in-vehicle devices, wearable devices, computing devices, or other processing devices connected to a wireless modem, such as mobile phones, tablets, desktop notebooks, and smart devices that can run applications, including the central console of a smart car. Specifically, it can refer to user equipment (UE), access terminal, subscriber unit, subscriber station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent, or user device. The terminal device may also be a satellite phone, a cellular phone, a smart phone, a wireless data card, a wireless modem, a machine type communication device, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device or a wearable device, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, a terminal device in a 5G network or a future communication network, etc. The terminal can be powered by a battery or can be attached to and powered by a vehicle or vessel's power system, which can also charge the terminal's battery to extend the terminal's communication time.

[0046] The server 120 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0047] It should be noted that the implementation of the method in the present application can be completed directly on the terminal 110 or on the server 120 , or it can be completed on the server 120 and then sent to the terminal 110 by the server 120 .

[0048] Figure 2A flow chart of a data association method for target tracking provided in one embodiment of the present application is shown as follows: Figure 2 As shown, the embodiment of the present application provides a data association method for tracking a target in an image. The process steps of this method are only a possible implementation of the present application, including:

[0049] Step 201: Obtain n measurement data to be associated in a current frame of an image;

[0050] Step 202: Obtain m target sources determined in a frame preceding the current frame, where m and n are both positive integers.

[0051] Step 203 : determining a current association gate range value of each of the m target sources after data association is completed in the preceding frame based on the data variation between consecutive adjacent frames of the same target source before the current frame.

[0052] In the embodiment of the present application, an unmanned driving multi-target tracking application scenario is used for illustration. Through the video acquisition device and sensor modules in the vehicle-mounted equipment, data is collected on the target in front of the vehicle to obtain measurement data, and then the measurement data obtained in each frame is associated with the target source to formulate a corresponding unmanned driving operation plan.

[0053] Specifically, the embodiment of the present application improves the selection method of the correlation gate range of the KNN algorithm by introducing a maximum change statistic Δs of adjacent frame data. max =max{Δs1,...,Δs k}, dynamically adjust the initial correlation gate range value of each target source to avoid the selection process of human experience and realize the automatic learning and determination of the correlation gate range. In the embodiment of the present application, the specific value of the correlation gate range value is the maximum data change statistic Δs between the consecutive adjacent frames (i.e., the previous frames) before the current frame of each target source. max It is certain that it can be understood that in the entire multi-target tracking task, the selection of the association gate in different time periods is dynamically changed, so that the association gate in the improved KNN algorithm is updated simultaneously with the change in the data change amount through the association gate range value.

[0054] Step 204: obtaining a similarity distance between each target source in the m target sources and each measurement data to be associated within a current association gate range value;

[0055] Step 205 : assigning a preset value to the similarity distance between each of the m target sources and each to-be-associated measurement data outside the current association gate range.

[0056] In this embodiment, data in a multi-target tracking scenario is correlated by acquiring multiple frames of measurement data within a time window. As the correlation gate range value corresponding to each target source changes, a similarity distance is calculated for each target source from the source itself to all measurement data within its current correlation gate. This similarity distance measures the similarity between the source and the measurement data, with larger values ​​indicating lower similarity.

[0057] Furthermore, the similarity distance between each target source and all measurement data outside its currently associated gate can be assigned a preset value. This preset value is a constant, and the constant only needs to be significantly greater than the preset normal distance. It should be noted that in this embodiment of the present application, the similarity distance between each target source itself and all measurement data within its currently associated gate, as well as the similarity distance between each target source and all measurement data outside its currently associated gate, can be collectively referred to as the similarity distance, and the cost matrix can be constructed based on this similarity distance.

[0058] Step 206 : constructing a cost matrix between the m target sources and the n measurement data to be associated based on the similarity distance.

[0059] In the embodiment of the present application, for each frame of measurement data in a time window, a cost matrix from all sources to all measurement data is obtained, wherein the constructed cost matrix can be referred to Table 1:

[0060] Table 1

[0061] Measurement data Z1 … Measurement data Zn Source Data S1 d11 … d1n … … … … Source data Sm dm1 … dmn

[0062] In Table 1, Si represents the i-th target source, i∈m; Zj represents the j-th measurement data, j∈n; dij represents the similarity distance from the i-th source to the j-th measurement data, that is, the cost value. Assuming there are m sources, then the value range of i is 1 to m. Assuming there are n sources, then the value range of j is 1 to n. dij can represent any of these m×n distances, but it is not the closest distance from the i-th target source to all measurement data.

[0063] Figure 3 The schematic diagram of the association process between the target source and the data to be associated is provided in one embodiment of the present application. Figure 3 As shown, in the embodiment of the present application, the associated door can be rectangular or in other shapes, such as square or circular, etc., and the embodiment of the present application does not specifically limit this. Figure 3In (a), at time t0, since the starting time of different sources may be different, each source may be tracked to its own k-th frame. That is, the frame at time t0 is the X1-th frame for target 301 to be tracked, the X2-th frame for target 302 to be tracked, and the X3-th frame for target 303 to be tracked. It should be noted that in the embodiment of the present application, k is for each source, and k can be used as a measure of the survival time of a source. Different sources may have different k. For example, the above-mentioned X1-th frame, X2-th frame, and X3-th frame may have the same k value or different k values. In some embodiments, each target 301, 302, and 303 may be a different person, and each target source may be a rectangle, circle, or square marked on the person's head. The specific shape is not limited (see Figure 3 Furthermore, the data change is the similarity distance between the same source in the previous frame (i.e., time t0) and the current frame (i.e., time t1). It is a one-dimensional value with size but no direction. Figure 3 (b), at the current frame t1, the distance can be obtained based on the size change of each rectangular box used to mark the target source and the displacement of the center point of the rectangular box. It should be noted that in one embodiment of the present application, the target source may have other attributes, and a comprehensive calculation can be performed based on these attributes; in addition, in order to reduce the amount of calculation, it can also be assumed that the shape of the target source remains unchanged (i.e. Figure 3 The rectangular box marking the target source remains unchanged. At this point, the data change for each target source is equal to the position change of that target source. Of course, the data change for each target source can only be obtained after the to-be-associated data at time t1 is associated with the target source at time t0.

[0064] Furthermore, in some embodiments, the cost value represents the positional relationship with the target source itself (i.e. Figure 3 The embodiment of the present application uses the Hungarian algorithm to calculate the cost of each association pair and obtain the min(m, n) optimal association pairs from m×n possible association pairs, ensuring that the total cost of the obtained min(m, n) optimal association pairs is less than the total cost of any min(m, n) other association pairs. The optimal cost dij is selected from the m×n dij, and the number of optimal costs dij is min(m, n), that is, the minimum value of m and n.

[0065] In the association process based on the KNN algorithm, there may be multiple groups of nearest neighbor groups (a group of nearest neighbor groups indicates that the association gate range of these sources forms a connected domain). The general solution method is to solve each group separately. However, in one embodiment of the present application, a larger threshold is set for the paths that cannot be successfully associated (the threshold can be set according to the type of measurement data. For example, when tracking pedestrians, it is generated by multiplying a larger multiplier based on the normal moving speed of the same pedestrian target in each frame). This realizes the completion of the data association solution process through a unified cost matrix, thereby improving the efficiency of data association.

[0066] Step 207: Obtain data association results between the n measurement data to be associated and the m target sources according to the cost matrix.

[0067] In an embodiment of the present application, the cost matrix is ​​obtained in the above embodiment, and the Hungarian algorithm is used to solve and obtain the optimal association pairs, that is, under the condition of satisfying the minimum path, the optimal combination between the source and the measurement data is obtained, where an association pair can be expressed as (Si, Zj), and its cost value is dij.

[0068] Furthermore, the cost value of each optimal association pair is compared with the preset cost threshold, and the optimal association pair that is less than or equal to the preset cost threshold is retained, indicating that the optimal association pair is effectively associated, and its measurement data is updated to the associated target source, that is, the measurement data in the effectively associated optimal association pair is associated with the target source, and the measurement data in the optimal association pair becomes the corresponding target source of the target source in the previous frame in the optimal association pair in the current frame. For sources or measurement data that are not successfully associated, the next round of data association process is performed again to ensure that each measurement data can be accurately associated with the target source. Assuming there are 8 sources, 10 test data are tracked at this time. After Hungarian matching, if the matching process is perfect, there are still 2 test data that will not be paired. For example, you can refer to Figure 3 (a), at time t0, there are 3 targets, and Figure 3 In (b), data association is performed at time t1. The association result may be that the data to be associated 304 is associated with the target 301, the data to be associated 305 is associated with the target 302, and the data to be associated 306 is associated with the target 303. The data to be measured 307 is assigned a new target source. In some cases, there may be more than two unpaired test data. In this case, the next round of data association is performed for the source or measurement data that has not been successfully associated.

[0069] The target tracking data association method provided in an embodiment of the present application determines the association gate range value of each target source by using the data change between the measurement data of adjacent frames. Then, based on the nearest neighbor association algorithm, the similarity distance between each target source and each measurement data to be associated is obtained according to the association gate range value. Thus, based on the cost matrix constructed by the similarity distance, the data association result between each measurement data to be associated and each target source is obtained, thereby completing the data association between the target source and the measurement data. Compared with the existing technology, the accuracy of data association is improved.

[0070] It should be noted that each implementation method of the present application can be freely combined, the order can be changed, or it can be executed separately, and does not need to rely on or depend on a fixed execution order.

[0071] In some embodiments, determining the association gate range value of each of the m target sources after data association is completed in the preceding frame based on the data change amount between consecutive adjacent frames of the same target source before the current frame includes:

[0072] By associating the target source and measurement data of multiple consecutive frames, the data change of each target source between every two consecutive adjacent frames is obtained;

[0073] For each target source, based on the data changes of consecutive adjacent frames, respectively obtain the maximum data change of each target source;

[0074] According to the maximum data variation of each target source and the initial correlation gate range value, a current correlation gate range value of each target source in the current frame after the data correlation of the previous frame is completed is obtained.

[0075] In an embodiment of the present application, the measurement data is obtained within a preset time window. At the beginning (such as the measurement data obtained in the first 5 frames or the first 10 frames), the association gate corresponding to each target source can be initialized to a larger value. At this time, almost every target source is within the association gate range of other sources, which is similar to the global nearest neighbor association. As the frame data is updated, the association gate gradually converges to the optimal local value, which is similar to the local nearest neighbor association, but its effectiveness is better than the simple local nearest neighbor association. This is because the association gate radius converges to the optimal value close to each target source, which reduces the amount of calculation and also reduces the occurrence of association errors.

[0076] Specifically, in one embodiment, the correlation gate is set to a circular area. This is because no assumption is made about the data change direction of each target source, that is, it is assumed that the change direction of the measurement data between adjacent frames can be arbitrary, and the radius of the correlation gate is r k , the formula is:

[0077]

[0078] Among them, k is the frame number of a target source, Δt k is the duration from the k-1th frame to the kth frame, Δs k is the data change from the k-1th frame to the kth frame.

[0079] In the embodiment of the present application, if there is no source when the measurement data arrives, a new source is assigned to each measurement data and initialized accordingly. At this time, k=0 for each new source; Δs max = 0. Specifically, the embodiment of the present application further needs to determine an initial association gate range value R0. This initialized initial association gate range value R0 is a value significantly larger than the normal association gate radius and can be obtained based on the data type of the measurement data. For example, when tracking a pedestrian target, the initial association gate range value R0 can be determined by multiplying the displacement data of the pedestrian target in two adjacent frames of measurement data and the time difference between the two frames of measurement data by the product of the displacement data and the time difference.

[0080] In some embodiments, if the measurement data between adjacent frames are of equal time intervals (generally, in a time window, the time intervals between adjacent frames are the same), that is, Δt1=Δt2=...=Δt k , which can be set to 1, thus reducing the radius r k The formula is simplified to obtain the associated gate range value formula, specifically:

[0081] According to the maximum data variation of each target source and the initial correlation gate range value, the current correlation gate range value of each target source after the data association of the preceding frame is completed is obtained using the correlation gate range value formula. The correlation gate range value formula is:

[0082] r k =e -αk R0+(1-e -αk +c)Δs max ;

[0083] Δs max =max{Δs1, ..., Δs k};

[0084] Among them, r k represents the current association gate range value of a target source after the association calculation is completed at the kth frame, R0 represents the initial association gate range value and is larger than the preset normal association gate radius; a is a parameter greater than 0, and in the embodiment of the present application, a can be set to 1; c represents the compensation item parameter, and in the embodiment of the present application, c can be set to 0.1; Δs max Indicates the maximum data change, Δs kIndicates the data change of the target source from the k-1th frame to the kth frame. k is a positive integer marking the number of frames the target source is tracked. Each target source has its own k value.

[0085] In one embodiment of the present application, a corresponding initial association gate range value is determined for each target source, and then the association gate range value of each target source is adjusted based on the data change of the measured data within this initial association gate range value, so that the search radius corresponding to the association gate in the KNN algorithm is no longer fixed and does not need to be manually set, but is generated in real time based on the data change between adjacent frames in each target source.

[0086] In one embodiment of the present application, for the measurement data within a preset time window, after determining the maximum data change, the target association gate range value is obtained (which is the optimal search radius of each target source at the current moment), and all measurement data before the current frame are data-associated with each target source. As time goes on, for example, in an unmanned driving scenario, data association needs to be performed again based on new measurement data acquired in real time. In the embodiment of the present application, the association gate range value of each target source realizes the function of online automatic learning as the tracking target changes and new measurement data is acquired. The update of the association gate range value is not stopped until the entire multi-target tracking task is completed, so that the accuracy of the data association process continues to be maintained at a high precision.

[0087] In some embodiments, the similarity distance between each target source in the m target sources and each to-be-associated measurement data within its current association gate range value is the Mahalanobis distance between each target source and each to-be-associated measurement data within its current association gate range value.

[0088] In the embodiment of the present application, for a certain target source, assuming that the current frame is the kth frame, the data difference Δs between the current frame k and the previous frame k-1 of the target source is calculated first. k , and update Δs max , if the data change Δs k is the maximum data change, then the data change Δs is used k , calculate the associated door radius r through the associated door range value formula k ; If the data change Δs k If it is not the maximum data change, then according to the updated Δsm ax Select the maximum data change and calculate the associated gate radius r by the associated gate range value formula k .

[0089] Furthermore, after determining the current associated gate radius r of each target source k(i.e., the association gate range value of the target source in the current frame), first obtain multiple measurement data (which can be recorded as first measurement data) of each target source within the radius of the current association gate, and then calculate the similarity distance between each target source and the first measurement data corresponding to each target source. Preferably, in an embodiment of the present application, the measurement method of the similarity distance can be determined according to the dimensional information of the measurement data. For a multi-dimensional multi-target tracking and detection process, since data association needs to be performed from multiple dimensions, the embodiment of the present application can use the Mahalanobis distance to measure the similarity distance between the target source and each first measurement data; and for some application scenarios where the measurement data is relatively single, such as only needing to perform data association with the target source based on speed information or coordinate information, the Euclidean distance can be used to measure the similarity distance between the target source and the first measurement data.

[0090] In one embodiment of the present application, a single cost matrix is ​​used to complete multiple groups of local nearest neighbor association processes. In an actual multi-target tracking scenario, as time goes by, the association gate range value and similarity distance calculation are performed again through the acquired measurement data and new target sources (refer to the process of the above embodiment), thereby updating the cost matrix, so that in the entire multi-target tracking task, a unified cost matrix is ​​used to complete the association solution process.

[0091] In some embodiments, obtaining data association results between the n to-be-associated measurement data and the m target sources according to the cost matrix includes:

[0092] Solving the cost matrix to obtain optimal association pairs between the m target sources and the n measurement data to be associated;

[0093] The similarity distance of each optimal association pair is compared with a preset cost threshold to obtain a data association result between each measurement data to be associated and each target source, wherein the preset cost threshold is half of the constant.

[0094] In the embodiment of the present application, the shortest path is taken as the goal, and the cost matrix is ​​solved by the Hungarian algorithm to obtain multiple optimal association pairs, where the optimal association pair can be expressed as (Si, Zj), and its cost value (i.e., similarity distance) is dij. The specific solution process is as follows:

[0095] S1, first convert the cost matrix into a square matrix of size n×n (assuming m is less than n, fill the missing elements with 0);

[0096] S2, subtract the row minimum value from each row of the converted cost matrix;

[0097] S3, subtract the minimum value of each column of the converted cost matrix;

[0098] S4, cover all zeros with the least horizontal / vertical lines. If the number of horizontal / vertical lines is equal to n, the algorithm stops; if the number of horizontal / vertical lines is less than n, go to S5;

[0099] S5, creates additional 0s based on the elements not covered by the horizontal / vertical lines in S4, and then enters S4 again, where the method of creating additional 0s is: all uncovered elements are subtracted from the minimum value of the uncovered elements, and the intersection element of the horizontal line and the vertical line is added to the minimum value of the uncovered elements.

[0100] In some embodiments, for example, it is assumed that there is a cost matrix of size 3×4 as shown in Table 2 below:

[0101] Table 2

[0102] 30 40 50 60 65 55 45 35 33 99 66 77

[0103] First, add rows with 0 elements to the cost matrix in Table 2 to make the cost matrix a square matrix, as shown in Table 3:

[0104] Table 3

[0105] 30 40 50 60 65 55 45 35 33 99 66 77 0 0 0 0

[0106] Then, subtract the minimum value of each row in the matrix in Table 3, that is, subtract 30 from the first row, 35 from the second row, and 33 from the third row. See Table 4 for details:

[0107] Table 4

[0108] 0 10 20 30 30 20 10 0 0 66 33 44 0 0 0 0

[0109] Next, subtract the minimum value of each column of the square matrix in Table 4. Since the minimum value of each column is 0 in the embodiment of the present application, the results before and after processing are the same and are not shown here. Further, all the 0s in Table 4 are covered with the least horizontal / vertical lines. As shown in Table 4, all 0s can be covered by 2 horizontal lines and 1 vertical line, that is, the values ​​covered by the first horizontal line are 30, 20, 10 and 0, the values ​​covered by the second horizontal line are 0, 0, 0 and 0, and the values ​​covered by the vertical line are 0, 30, 0 and 0. It can be seen that all the 0s in Table 4 can be covered by 2 horizontal lines and 1 vertical line, and the number of horizontal and vertical lines is the least. Since the total number of horizontal and vertical lines is less than 4, it is necessary to create additional 0s based on the elements not covered by the horizontal and vertical lines.

[0110] Furthermore, as shown in Table 4, among the elements not covered by the horizontal / vertical lines, the minimum value is 10. It is necessary to subtract the minimum value 10 from the uncovered elements in Table 4, and add 10 to the elements where the horizontal and vertical lines intersect, thereby obtaining an additional 0. For details, see Table 5:

[0111] Table 5

[0112] 0 0 10 20 40 20 10 0 0 56 23 34 10 0 0 0

[0113] Finally, use the minimum number of horizontal / vertical lines (i.e., 4 horizontal lines or 4 vertical lines) to cover all 0 elements in Table 5. At this time, the number of lines is equal to 4, and the optimal path is solved. The 0 elements in Table 5 correspond to the best matching pairs. Refer to Table 2. The best associated pairs of the original cost matrix are: (S1, Z2), with a corresponding cost value of 40; (S2, Z4), with a corresponding cost value of 35; (S3, Z1), with a corresponding cost value of 33.

[0114] In some embodiments, comparing the similarity distance of each optimal association pair with a preset cost threshold to obtain a data association result between each measurement data to be associated and each target source includes:

[0115] Determining that the similarity distance of the first optimal association pair is less than or equal to the preset cost threshold, determining that the measurement data to be associated in the first optimal association pair is successfully associated with the target source, updating the successfully associated measurement data to be associated to the successfully associated target source, and updating the data change amount and the association gate range value of the successfully associated target source;

[0116] Determining that the similarity distance of the second optimal association pair is greater than the preset cost threshold, determining that the second optimal association pair is not successfully associated, and disconnecting the association relationship between the to-be-associated measurement data in the second optimal association pair and the unsuccessfully associated target source.

[0117] In the embodiment of the present application, the validity of each optimal association pair is determined based on the judgment result between the cost value of the optimal association pair and the preset cost threshold. Specifically, if dij is greater than the preset cost threshold, then the association pair (Si, Zj) is determined to be invalid, and the association between the target source and the measurement data in the association pair is disconnected; if dij is less than or equal to the preset cost threshold, then the association pair (Si, Zj) is determined to be valid, and the association pair is retained, and the measurement data in the association pair is updated to the target source of the association pair, and the change amount Δs of the target source is updated at the same time. k , and update Δs max .

[0118] It should be noted that in the embodiment of the present application, the distance between each target source and the measurement data outside the associated gate is assigned to a constant. The setting of this constant is not strictly limited and only needs to be much larger than the normal distance. That is, the preset value is a constant much larger than the preset normal distance. Preferably, in some embodiments, the data type of the measurement data is first obtained, such as coordinate data, velocity data, or angular velocity, and then the maximum variation range of the measurement data in adjacent frames is multiplied by a larger coefficient to obtain a constant much larger than the preset normal distance. Then, the preset cost threshold is obtained based on this constant. For example, in the tracking scenario of a pedestrian target, the measurement data obtained from adjacent frames is the pedestrian speed. In general, the maximum variation range of the pedestrian speed between adjacent frames (the time interval between two frames has been confirmed) is known. At this time, this maximum variation range value (a value can be selected from the maximum variation range value) is multiplied by 10 (or 100, which is not specifically limited in the embodiment of the present application) to obtain a constant. Half of this constant can be defined as the preset cost threshold.

[0119] In some embodiments, the method further comprises:

[0120] Mark the target source that has not been successfully associated as a data loss source, and re-track the data loss source. If the data loss state remains within a preset time period, delete the data loss source;

[0121] The measurement data that has not been successfully associated is initialized into a new target source, and data association is performed on the new target source in the next frame according to the initial association gate range value of the new target source.

[0122] In this embodiment of the present application, the target source that failed to be successfully associated in the current round is marked as data lost, and its k value is initialized to 1, which is equivalent to enlarging the radius of its association gate for the next round of data association. If the target source is marked as lost for a long time, it is determined to be invalid and does not require maintenance, and is released.

[0123] For the measurement data that is not successfully associated, a new target source is initialized through the measurement data, and the next round of data association is performed again using the improved KNN algorithm.

[0124] For the successfully associated target source and measurement data, update the measurement data to the target source with which it is successfully associated, and update the data change Δs of the target source k and Δs max .

[0125] Figure 4 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application is shown in FIG. Figure 4As shown, the electronic device may include: a processor (Processor) 401, a communication interface (Communications Interface) 402, a memory (Memory) 403 and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404. The processor 401 can call logic instructions in the memory 403 to execute a data association method for target tracking, the method comprising: obtaining n measurement data to be associated in a current frame of an image; obtaining m target sources determined in a previous frame of the current frame, where m and n are both positive integers; determining a current association gate range value of each of the m target sources after data association is completed in the previous frame based on a data change between consecutive adjacent frames preceding the current frame for the same target source; obtaining a similarity distance between each of the m target sources and each measurement data to be associated within its current association gate range value; assigning a preset value to the similarity distance between each of the m target sources and each measurement data to be associated outside its current association gate range value; constructing a cost matrix between the m target sources and the n measurement data to be associated based on the similarity distance; and obtaining a data association result between the n measurement data to be associated and the m target sources based on the cost matrix.

[0126] In addition, the logic instructions in the above-mentioned memory 403 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0127] On the other hand, the present application also provides a computer program product, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions. When the program instructions are executed by a computer, the computer can perform the data association method for target tracking provided by the above methods, the method including: obtaining n measurement data to be associated in a current frame of an image; obtaining m target sources determined in a previous frame of the current frame, where m and n are both positive integers; determining a current association gate range value for each of the m target sources after data association is completed in the previous frame based on a data change between consecutive adjacent frames preceding the current frame for the same target source; obtaining a similarity distance between each of the m target sources and each measurement data to be associated within its current association gate range value; assigning a preset value to the similarity distance between each of the m target sources and each measurement data to be associated outside its current association gate range value; constructing a cost matrix between the m target sources and the n measurement data to be associated based on the similarity distance; and obtaining a data association result between the n measurement data to be associated and the m target sources based on the cost matrix.

[0128] In another aspect, the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data association method for target tracking provided in the above-mentioned embodiments, the method comprising: obtaining n measurement data to be associated in a current frame of an image; obtaining m target sources determined in a frame preceding the current frame, where m and n are both positive integers; determining a current association gate range value for each of the m target sources after data association is completed in the preceding frame based on a data change between consecutive adjacent frames preceding the current frame for the same target source; obtaining a similarity distance between each of the m target sources and each measurement data to be associated within its current association gate range value; assigning a preset value to the similarity distance between each of the m target sources and each measurement data to be associated outside its current association gate range value; constructing a cost matrix between the m target sources and the n measurement data to be associated based on the similarity distance; and obtaining a data association result between the n measurement data to be associated and the m target sources based on the cost matrix.

[0129] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0130] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data association method for target tracking, characterized in that: include: Obtain n measurement data to be associated in a current frame of an image; Obtain m target sources determined in a frame preceding the current frame, where m and n are both positive integers; Determine, based on a data change amount between consecutive adjacent frames preceding the current frame for the same target source, a current association gate range value for each of the m target sources after data association is completed in the preceding frame; Obtaining a similarity distance between each target source in the m target sources and each to-be-associated measurement data within a current association gate range value; Assigning a preset value to the similarity distance between each target source in the m target sources and each to-be-associated measurement data outside its current association gate range value; Constructing a cost matrix between the m target sources and the n measurement data to be associated according to the similarity distance; Obtaining data association results between the n measurement data to be associated and the m target sources according to the cost matrix; The determining, based on the data change amount between consecutive adjacent frames before the current frame for the same target source, an association gate range value of each target source after the m target sources complete data association in the preceding frame includes: By correlating the target source and measurement data of multiple consecutive frames, the data change of each target source between every two consecutive adjacent frames is obtained; For each target source, based on the data changes of consecutive adjacent frames, respectively obtain the maximum data change of each target source; According to the maximum data variation of each target source and the initial correlation gate range value, a current correlation gate range value of each target source in the current frame after the data correlation of the previous frame is completed is obtained.

2. The target tracking data association method according to claim 1, characterized in that: The method further comprises: According to the maximum data variation of each target source and the initial correlation gate range value, the current correlation gate range value of each target source after the data association of the preceding frame is completed is obtained by using the correlation gate range value formula. The correlation gate range value formula is: r k s.e -αk R0+(1-e -αk +c)Δs max 100. Δs max =max{Δs1,...,Δs k }; Among them, r k It represents the current association gate range value of a target source after completing the association calculation at the kth frame. R0 represents the initial association gate range value and is larger than the preset normal association gate radius. a is a parameter greater than 0. c represents the compensation parameter. Δs max Indicates the maximum data change, Δs k Indicates the data change of the target source from the k-1th frame to the kth frame. k is a positive integer marking the number of frames the target source is tracked. Each target source has its own k value.

3. The target tracking data association method according to claim 1, characterized in that: The similarity distance between each target source in the m target sources and each to-be-associated measurement data within the current association gate range value is the Mahalanobis distance between each target source and each to-be-associated measurement data within the current association gate range value.

4. The target tracking data association method according to claim 1, characterized in that: The preset value is a constant that is much larger than the preset normal distance.

5. The target tracking data association method according to claim 4, characterized in that: The obtaining, according to the cost matrix, data association results between the n to-be-associated measurement data and the m target sources includes: Solving the cost matrix to obtain optimal association pairs between the m target sources and the n measurement data to be associated; The similarity distance of each optimal association pair is compared with a preset cost threshold to obtain a data association result between each measurement data to be associated and each target source, wherein the preset cost threshold is half of the constant.

6. The target tracking data association method according to claim 5, characterized in that: The step of comparing the similarity distance of each optimal association pair with a preset cost threshold to obtain a data association result between each measurement data to be associated and each target source includes: Determining that the similarity distance of the first optimal association pair is less than or equal to the preset cost threshold, determining that the measurement data to be associated in the first optimal association pair is successfully associated with the target source, updating the successfully associated measurement data to be associated to the successfully associated target source, and updating the data change amount and the association gate range value of the successfully associated target source; Determining that the similarity distance of the second optimal association pair is greater than the preset cost threshold, determining that the second optimal association pair is not successfully associated, and disconnecting the association relationship between the to-be-associated measurement data in the second optimal association pair and the unsuccessfully associated target source.

7. The target tracking data association method according to claim 6, characterized in that: The method further comprises: Mark the target source that has not been successfully associated as a data loss source, and re-track the data loss source. If the data loss state remains within a preset time period, delete the data loss source; The unassociated measurement data is initialized into a new target source, and data association is performed on the new target source in the next frame according to the initial association gate range value of the new target source.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the data association method for target tracking according to any one of claims 1 to 7 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data association method for target tracking according to any one of claims 1 to 7 is implemented.

Images