A multi-target data tracking method and system

Abstract

The present invention relates to a multi-target data tracking method and system. The method includes: obtaining measurement data of multiple sensors and multiple prediction data, and matching each measurement target and each prediction target according to the vector correlation consistency between the measurement data of each sensor and the multiple prediction data; counting the number of successful matches between each measurement target and each prediction target, and constructing an association matrix based on the counts; and managing the measurement targets and prediction targets according to the Kalman filtering method based on a kinematic model and the association matrix. The present invention solves the problem of errors caused by the maximum matching algorithm through a matching method based on vector correlation consistency, and realizes real-time multi-sensor multi-target data tracking and prediction.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving and sensor data processing, and particularly relates to a multi-target data tracking method and system. Background Art

[0002] Sensor fusion in the field of autonomous driving is a relatively important field. In the multi-sensor target fusion stage, it is necessary to perform data association on the target outputs of different sensors to determine whether multiple data belong to the same target.

[0003] In the current field of autonomous driving perception fusion, the Hungarian matching algorithm is usually used for data association. However, the Hungarian algorithm is a maximum matching algorithm. In special cases, two unrelated targets can also be matched, resulting in errors in the fusion result. Summary of the Invention

[0004] To solve the problem of errors in multi-sensor data fusion caused by the maximum matching algorithm, in the first aspect of the present invention, a multi-target data tracking method is provided, including the following steps:

[0005] Obtain measurement data of multiple sensors and multiple prediction data, and match each measurement target and each prediction target according to the vector correlation consistency between the measurement data of each sensor and the multiple prediction data.

[0006] Count the number of successful matches between each measurement target and each prediction target, and construct an association matrix based on it.

[0007] Manage the measurement targets and prediction targets according to the Kalman filtering method based on the kinematic model and the association matrix.

[0008] In some embodiments of the present invention, the vector correlation consistency is calculated by the following method:

[0009]

[0010] Where ρ(.) represents the vector correlation consistency index of the association matrix; S represents the prediction target data, T represents the measurement target data, E(.) represents the corresponding mean; s represents the ordinal number of the prediction target data, t represents the ordinal number of the measurement target data, and M and N respectively represent the total number of measurement targets and the total number of prediction targets.

[0011] Furthermore, if there is a measurement target with equal vector correlation consistency indices with multiple prediction targets, then: select a set of prediction target data with the smallest Euclidean distance from the measurement target from the data of the multiple prediction targets as the matching data.

[0012] In some embodiments of the present invention, the management of the measurement target and the prediction target according to the Kalman filtering method based on the kinematic model and the association matrix includes:

[0013] Updating the data of the measurement target and the prediction target according to the Kalman filtering method based on the kinematic model; counting the matching times of the updated prediction target and the measurement target, and managing the data of the measurement target and the prediction target according to the matching times.

[0014] Further, the counting the matching times of the updated prediction target and the measurement target, and managing the data of the measurement target and the prediction target according to the matching times includes: if there are measurement target and prediction target data whose matching times within a preset number of frames are not less than the threshold, retaining the measurement target and its data; otherwise, removing the measurement target from the association matrix.

[0015] In the above embodiment, the association matrix is constructed by the following method: taking the ordinals of the measurement data of each sensor and the prediction data of each as the row data and column data of the association matrix respectively; performing a vector correlation consistency calculation on the measurement data corresponding to the row data and the prediction data corresponding to the column data; determining the value of the element in the association matrix corresponding to the row data and the column data according to the result of the vector correlation consistency calculation.

[0016] In the second aspect of the present invention, a multi-target data tracking system is provided, including an acquisition module, a matching module, and a management module. The acquisition module is used to acquire the measurement data of multiple sensors and multiple prediction data, and match each measurement target and each prediction target according to the vector correlation consistency of the measurement data of each sensor and the multiple prediction data; the matching module is used to count the number of times of successful matching of each measurement target and each prediction target, and construct an association matrix according to the number of times of successful matching; the management module is used to manage the measurement target and the prediction target according to the Kalman filtering method based on the kinematic model and the association matrix.

[0017] Further, the management module includes an update unit and a management unit.

[0018] The update unit is used to update the data of the measurement target and the prediction target according to the Kalman filtering method based on the kinematic model.

[0019] The management unit counts the matching times of the updated prediction target and the measurement target, and manages the data of the measurement target and the prediction target according to the matching times.

[0020] In a third aspect of the present invention, there is provided an electronic device, comprising: one or more processors; a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the multi-target data tracking method provided in the first aspect of the present invention.

[0021] In a fourth aspect of the present invention, there is provided a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the multi-target data tracking method provided in the first aspect of the present invention.

[0022] The beneficial effects of the present invention are as follows:

[0023] 1. By means of the matching method based on vector correlation consistency, the present invention solves the problem of errors caused by the maximum matching algorithm and improves the matching accuracy.

[0024] 2. The multi-sensor and multi-target data are effectively managed through the association matrix.

[0025] 3. By means of the Kalman filtering method, the data is updated and the number of matching times is statistically counted, so as to realize real-time multi-sensor multi-target data tracking and prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the basic flow of the multi-target data tracking method in some embodiments of the present invention;

[0027] Figure 2 It is a schematic diagram of the specific flow of the multi-target data tracking method in some embodiments of the present invention;

[0028] Figure 3 It is a schematic diagram of the representation of the association matrix in the multi-target data tracking method in some embodiments of the present invention;

[0029] Figure 4 It is a schematic diagram of the structure of the multi-target data tracking system in some embodiments of the present invention;

[0030] Figure 5 It is a schematic diagram of the structure of the electronic device in some embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The principles and features of the present invention will be described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0032] Reference Figure 1 And Figure 2, in the first aspect of the present invention, a multi-target data tracking method is provided, including the following steps: S100. Obtain measurement data of multiple sensors and multiple prediction data, and match each measurement target and each prediction target according to the vector correlation consistency between the measurement data of each sensor and the multiple prediction data;

[0033] S200. Count the number of successful matches between each measurement target and each prediction target, and construct an association matrix based on it;

[0034] S300. Manage the measurement targets and prediction targets according to the Kalman filtering method based on the kinematic model and the association matrix.

[0035] In steps S100 to S300 of some embodiments of the present invention, the vector correlation consistency is calculated by the following method:

[0036]

[0037] where ρ(.) represents the vector correlation consistency index of the association matrix; S represents the prediction target data, T represents the measurement target data, E(.) represents the corresponding mean value; s represents the ordinal number of the prediction target data, t represents the ordinal number of the measurement target data, and M and N respectively represent the total number of measurement targets and the total number of prediction targets (corresponding to one frame of data or multiple frames of data).

[0038] It can be understood that E(.) represents the average value, and its calculation methods include but are not limited to arithmetic mean, geometric mean, variance, etc. The measurement target data or prediction target data includes one variable or multiple variables. When the data is multiple variables, S or T corresponds to multi-dimensional data types such as matrices or sequences. The vector correlation consistency index of the association matrix can also be calculated by the method of matrix similarity; the ordinal numbers of the prediction target data or measurement target data represented by the above s or t can be replaced by the ordinal numbers of the spatial frame or time frame respectively.

[0039] Further, if there is a measurement target with equal vector correlation consistency indices with multiple prediction targets, then: select a set of prediction target data with the smallest Euclidean distance from the measurement target from the data of the multiple prediction targets as the matching data. For example, in the data of a certain prediction target, both the inertial navigation sensor and the satellite positioning sensor can be used to measure or predict the pose information of the target, so it is necessary to match the data generated by the two.

[0040] In step S300 of some embodiments of the present invention, the management of the measurement targets and prediction targets according to the Kalman filtering method based on the kinematic model and the association matrix includes:

[0041] S301. Update the data of the measurement target and the prediction target according to the Kalman filtering method based on the kinematic model; S302. Count the matching times of the updated prediction target and the measurement target, and manage the data of the measurement target and the prediction target according to the matching times. The kinematic models include: Constant Velocity (CV), Constant Acceleration (CA), Constant Turn Rate and Velocity (CTRV), Constant Turn Rate and Acceleration (CTRA), Constant Steering Angle and Velocity (CSAV), and Constant Curvature and Acceleration (CCA).

[0042] Specifically, in step S301, the steps of updating the data of the measurement target and the prediction target based on the Kalman filtering method are as follows:

[0043] The Kalman process based on the CV model is as follows:

[0044] Prediction equation:

[0045] (Linear CV motion model)

[0046] (State covariance matrix)

[0047] Update equation:

[0048] (Calculate the Kalman gain)

[0049] (Update the estimate through measurement)

[0050] P t =(I - K t H)P t - ⑤(Update the error covariance)

[0051] Among them, in equation ①, x(t)=(P x , P y , V x , V y ) T , x is the state vector (state vector), p represents the position information of the target (x and y are the components on their corresponding coordinate axes), v represents the relative velocity of the target; the superscript ^ represents the measured value of the corresponding variable, and the superscript - represents the predicted value of the corresponding variable; and:

[0052]

[0053] v t = v t-1 + u t ×Δt;

[0054] Arrange the above into matrix form:

[0055]

[0056] Among them, P in Equation ② is the state covariance matrix, Q is the noise of the prediction model, and F represents the state transition equation, which describes the uncertainty of target prediction; in Equation ③, K is the calculated Kalman gain, where H is the transition matrix, R is the covariance matrix of the measurement (sensor observation input) noise, t is the time or number of frames, Δt is the time interval, and u t is the acceleration or rate of change of velocity.

[0057] Further, in step S302 of some embodiments, the method of counting the matching times of the updated prediction target and the measurement target and managing the data of the measurement target and the prediction target according to the matching times includes: if there are data of the measurement target and the prediction target whose matching times within the preset number of frames are not less than the threshold, then retain the measurement target and its data; otherwise, remove the measurement target from the association matrix.

[0058] Specifically, count the matching times of the prediction target and the measurement target in multiple frames of data. If they match 3 times in 5 frames of data, it is considered valid data. If they do not match 3 times in 5 frames, it is considered invalid data, and the invalid data needs to be removed from the association matrix.

[0059] Reference Figure 3 , in the above embodiments, the association matrix is constructed by the following method: Take the ordinal numbers of the measurement data of each sensor and the prediction data of each as the row data and column data of the association matrix respectively; perform vector correlation consistency calculation on the measurement data corresponding to the row data and the prediction data corresponding to the column data; according to the result of the vector correlation consistency calculation, determine the value of the element in the association matrix corresponding to the row data and the column data. The value of the corresponding element in the association matrix for the successfully matched row data and column data is 1, otherwise it is 0.

[0060] Embodiment 2

[0061] Reference Figure 4, the second aspect of the present invention provides a multi-target data tracking system 1, including an acquisition module 11, a matching module 12, and a management module 13. The acquisition module is used to acquire the measurement data of multiple sensors and multiple prediction data, and match each measurement target and each prediction target according to the vector correlation consistency between the measurement data of each sensor and the multiple prediction data. The matching module is used to count the number of successful matches between each measurement target and each prediction target, and construct an association matrix based on it. The management module is used to manage the measurement targets and prediction targets according to the Kalman filtering method based on the kinematic model and the association matrix.

[0062] Further, the management module 13 includes an update unit and a management unit. The update unit is used to update the data of the measurement target and the prediction target according to the Kalman filtering method based on the kinematic model.

[0063] The management unit counts the number of matches between the updated prediction targets and measurement targets, and manages the data of the measurement targets and prediction targets based on it.

[0064] Embodiment 3

[0065] Reference Figure 5 , the third aspect of the present invention provides an electronic device, including: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enabling the one or more processors to implement the method provided in the first aspect of the present invention. Optionally, the electronic device can be used in in-vehicle devices, on-board devices, or the electronic control unit ECU in an aircraft and the control unit for realizing target prediction management.

[0066] The electronic device 500 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage device 508 into the random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. The input / output (I / O) interface 505 is also connected to the bus 504.

[0067] Typically, the following devices can be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a hard disk, etc.; and a communication device 509. The communication device 509 can allow the electronic device 500 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 5 the electronic device 500 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices can be implemented or had. Figure 5 Each block shown in

[0068] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by a processing device 501, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed. It should be noted that the computer-readable medium described in the embodiments of the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiments of the present disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the embodiments of the present disclosure, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable signal medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0069] The above-mentioned computer-readable medium can be included in the above-mentioned electronic device; or it can exist separately and not be assembled into the electronic device. The above-mentioned computer-readable medium carries one or more computer programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device is caused to:

[0070] Computer program code for performing the operations of the embodiments of the present disclosure may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, Python, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0071] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0072] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A multi-target data tracking method, characterized in that, it includes the following steps: Obtain the measurement data of multiple sensors and multiple prediction data, and match each measurement target and each prediction target according to the vector correlation consistency between the measurement data of each sensor and the multiple prediction data; the vector correlation consistency is calculated by the following method: ； Among them, ρ (.) represents the vector correlation consistency index of the correlation matrix; S represents the predicted target data, T represents the measured target data, E (.) represents the corresponding mean value; s represents the ordinal number of the predicted target data, t represents the ordinal number of the measured target data, M and N respectively represent the total number of measured targets and the total number of predicted targets; Count the number of successful matches between each measurement target and each prediction target, and construct an association matrix according to it; the association matrix is constructed by the following method: Use the ordinal numbers of the measurement data and each prediction data of each sensor as the row data and column data of the association matrix respectively; perform vector correlation consistency calculation on the measurement data corresponding to the row data and the prediction data corresponding to the column data; according to the result of the vector correlation consistency calculation, determine the value of the element in the association matrix corresponding to the row data and the column data; Manage the measurement target and the prediction target according to the Kalman filtering method based on the kinematic model and the association matrix: Update the data of the measurement target and the prediction target according to the Kalman filtering method based on the kinematic model; count the number of matches between the updated prediction target and the measurement target, and manage the data of the measurement target and the prediction target according to it; The counting of the number of matches between the updated prediction target and the measurement target, and managing the data of the measurement target and the prediction target according to it includes: If there is a measurement target and the number of matches of the prediction target data within a preset number of frames is not less than the threshold, then retain the measurement target and its data; otherwise, remove the measurement target from the association matrix.

2. The multi-target data tracking method according to claim 1, characterized in that, If there is a measurement target with equal vector correlation consistency indices with multiple prediction targets, then: Select a set of prediction target data with the smallest Euclidean distance from the measurement target from the data of the multiple prediction targets as the matching data.

3. A multi-target data tracking system, characterized in that, it includes an acquisition module, a matching module, and a management module, The acquisition module is used to obtain the measurement data of multiple sensors and multiple prediction data, and match each measurement target and each prediction target according to the vector correlation consistency between the measurement data of each sensor and the multiple prediction data; the vector correlation consistency is calculated by the following method: ； Among them, ρ (.) represents the vector correlation consistency index of the correlation matrix; S represents the predicted target data, T represents the measured target data, E (.) represents the corresponding mean value; s represents the ordinal number of the predicted target data, t represents the ordinal number of the measured target data, M and N respectively represent the total number of measured targets and the total number of predicted targets; The matching module is used to count the number of successful matches between each measurement target and each prediction target, and construct an association matrix according to it; the association matrix is constructed by the following method: Use the ordinal numbers of the measurement data and each prediction data of each sensor as the row data and column data of the association matrix respectively; perform vector correlation consistency calculation on the measurement data corresponding to the row data and the prediction data corresponding to the column data; according to the result of the vector correlation consistency calculation, determine the value of the element in the association matrix corresponding to the row data and the column data; A management module, configured to manage the measurement target and the prediction target according to the Kalman filtering method based on the kinematic model and the association matrix: update the data of the measurement target and the prediction target according to the Kalman filtering method based on the kinematic model; count the matching times of the updated prediction target and the measurement target, and manage the data of the measurement target and the prediction target according to the matching times; the step of counting the matching times of the updated prediction target and the measurement target and managing the data of the measurement target and the prediction target according to the matching times includes: if there are measurement target and prediction target data whose matching times within a preset number of frames are not less than the threshold, retain the measurement target and its data; otherwise, remove the measurement target from the association matrix.

4. An electronic device, comprising: one or more processors; a storage device configured to store one or more programs, which when executed by the one or more processors cause the one or more processors to implement the multi-target data tracking method according to any one of claims 1 to 2.

5. A computer-readable medium having a computer program stored thereon, wherein, the computer program, when executed by a processor, implements the multi-target data tracking method according to any one of claims 1 to 2.

Images