Multi-sensor joint space-time deviation calibration and multi-target association fusion method and device

Abstract

Embodiments of the present invention relate to the field of data processing technology, and in particular to a multi-sensor joint space-time deviation calibration and multi-target association fusion method and device. The method includes: based on the track list of the previous moment, determining the expanded dimension state estimation sampling point set and weight set corresponding to each track in the track list, combining the observation data list of the current moment, calculating the association cost of each pair of track-observation data combination, to associate the track with the observation data, and obtain an association result list, an unassociated track list and an unassociated observation data list; respectively updating the tracks in the unassociated track list and the association result list to obtain the unassociated updated track list and the associated track list at the current moment; performing feedback fusion processing of the space-time deviation on the associated track list to obtain the fused associated track list, so as to obtain the track list at the current moment. The present invention can realize effective multi-sensor multi-target tracking.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of data processing technology, and in particular to a multi-sensor joint space-time deviation calibration and multi-target association fusion method and device. Background Art

[0002] Multi-sensor data fusion methods can enhance the overall performance of the system, so they have been widely used in many fields. Bias compensation and data association are two important prerequisites for obtaining accurate data fusion results, and the two are highly coupled. Bias compensation relies on the multi-sensor observation data set from the same target generated by data association, and data association can correctly solve the problem of observation source uncertainty after bias compensation. Therefore, in order to obtain accurate multi-sensor data fusion results, it is necessary to process the two simultaneously.

[0003] In the related art, the existing multi-sensor data fusion methods are mostly iterative optimization processing of bias compensation and data association. This processing method is time-consuming and does not meet the requirements of real-time processing of the actual system. In addition, the existing related technologies only consider the situation where the sensor has spatial bias and observation source uncertainty, and do not consider the impact of time bias on fusion performance. Therefore, when space-time bias and observation source uncertainty exist at the same time, the data fusion results produced by the existing related technologies are not accurate, and it is difficult to achieve effective multi-sensor multi-target tracking.

[0004] Therefore, a new method for multi-sensor space-time bias calibration and correlation fusion is urgently needed. Summary of the invention

[0005] In order to solve the problem that the fusion result of the original multi-sensor data fusion method has low accuracy, the embodiment of the present invention provides a multi-sensor joint space-time deviation calibration and multi-target association fusion method and device.

[0006] In a first aspect, an embodiment of the present invention provides a multi-sensor joint space-time deviation calibration and multi-target association fusion method, including:

[0007] Based on the track list of the previous moment, determine the extended-dimensional state estimation sampling point set and weight set corresponding to each track in the track list; the track list includes the extended-dimensional state estimation and the extended-dimensional state estimation covariance of multiple tracks corresponding to several targets; the extended-dimensional state estimation includes the basic state estimation and the space-time deviation estimation;

[0008] Based on the expanded dimension state estimation sampling point set and the weight set corresponding to each track at the previous moment and the observation data list at the current moment, the association cost of each pair of track-observation data combination is calculated; the observation data list includes multiple observation data measured by the sensor on the target or clutter;

[0009] Based on the association cost and the one-to-one constraint relationship between the track and the observation data, the track and the observation data are associated to obtain an association result list, an unassociated track list and an unassociated observation data list;

[0010] Respectively updating the tracks in the unassociated track list and the associated result list to obtain the unassociated updated track list at the current moment and the associated track list at the current moment;

[0011] Performing feedback fusion processing of space-time deviation on the associated track list to obtain a fused associated track list;

[0012] Merging the unassociated updated track list and the fused associated track list, and managing the tracks in the merged list to obtain a surviving track list at the current moment;

[0013] Each observation data in the unassociated observation data list is unbiasedly transformed to construct the expanded-dimensional state estimate and the expanded-dimensional state estimate covariance corresponding to the new track to obtain a new track list, and the new track list is merged with the surviving track list to obtain the track list at the current moment.

[0014] Preferably, the calculation of the association cost of each pair of track-observation data combination based on the expanded dimension state estimation sampling point set and the weight set corresponding to each track at the previous moment and the observation data list at the current moment includes:

[0015] For each track-observation data pair, execute:

[0016] Determine an observation function of the sensor based on the track and the observation data in the pair of track-observation data combinations and the sensor corresponding to the observation data list;

[0017] Determine the observation prediction error and the observation prediction covariance of the pair of track-observation data combinations according to the observation function, the expanded dimension state estimation sampling point set, the weight set and the observation data in the pair of track-observation data combinations;

[0018] Calculate the Mahalanobis distance of the pair of track-observation data combinations according to the observation prediction error and the observation prediction covariance of the pair of track-observation data combinations;

[0019] The association cost of the track-observation data combination is calculated according to the Mahalanobis distance.

[0020] Preferably, the observation function is determined according to the following formula:

[0021]

[0022]

[0023] Where p is the track number, s is the sensor number, and X p is the expanded-dimensional state estimate of the track, k is the time, (x s,p (k),y s,p (k)) is the actual position of the track at the time of the observation data, (x s ,y s ) is the position of sensor s, Δr s (k) and Δθ s (k) is the distance deviation and angle deviation of sensor s, x p (k) and y p (k) are the positions of track p in the x and y directions, and are the velocities of track p in the x and y directions, Δt s,l (k) is the time deviation of sensor s relative to reference sensor l, where s = 1, ..., S and s ≠ l, S is the number of sensors;

[0024] The association cost is determined according to the following formula:

[0025]

[0026] In the formula, is the association cost of the track-observation data combination, i s is the observation data number of sensor s, u(i s ) is a binary variable, when i s =0, u(i s )=0, when i s When ≠0, u(i s )=1, is the detection probability of sensor s, ψ s is the area of ​​the sensor s observation area; is the Mahalanobis distance of the track-observation data pair, is the observation-prediction covariance for the track-observation data combination.

[0027] Preferably, the associating the track with the observation data based on the association cost and the one-to-one constraint relationship between the track and the observation data to obtain an association result list, an unassociated track list and an unassociated observation data list includes:

[0028] According to the association cost of each pair of track-observation data combination and the one-to-one constraint relationship between track and observation data, a two-dimensional allocation model of track-observation data association is established;

[0029] A generalized auction algorithm is used to determine the associated result list, the unassociated track list and the unassociated observation data list.

[0030] Preferably, the updating of the unassociated track list and the tracks in the associated result list respectively to obtain the unassociated updated track list and the associated track list at the current moment comprises:

[0031] Calculate the expanded-dimension state prediction and the expanded-dimension state prediction covariance of each track in the unassociated track list, use the expanded-dimension state prediction and the expanded-dimension state prediction covariance of each track as the expanded-dimension state estimation and the expanded-dimension state estimation covariance of each track at the current moment, and obtain the unassociated updated track list at the current moment; the unassociated updated track list includes the expanded-dimension state estimation and the expanded-dimension state estimation covariance of each track at the current moment;

[0032] An unscented Kalman filtering method is used to filter each observation data in the association result list to obtain the expanded-dimensional state estimate and the expanded-dimensional state estimate covariance of each track associated with each observation data at the current moment, so as to obtain the associated track list at the current moment; the associated track list includes the expanded-dimensional state estimate and the expanded-dimensional state estimate covariance of each track at the current moment.

[0033] Preferably, performing feedback fusion processing of space-time deviation on the associated track list to obtain a fused associated track list includes:

[0034] For each track in the associated track list whose update number is greater than the first set value, execute:

[0035] Extracting the space-time deviation estimate and the space-time deviation estimate covariance of each track whose update times are greater than the first set value at the current moment, so as to calculate the space-time deviation estimate after fusion of the track and the space-time deviation estimate covariance after fusion;

[0036] The fused space-time deviation estimate of the track replaces the original space-time deviation estimate in the expanded-dimensional state estimate at the current moment to obtain the fused expanded-dimensional state estimate of the track;

[0037] The unscented transformation method is used to calculate the covariance between the space-time deviation estimate after the track fusion and the basic state estimate of the track, so as to obtain the covariance of the expanded-dimensional state estimate after the track fusion;

[0038] Obtaining the fused track; the fused track includes the fused extended-dimensional state estimate and the fused extended-dimensional state estimate covariance;

[0039] Each fused track whose update times are greater than the first set value replaces the corresponding track in the associated track list to obtain a fused associated track list.

[0040] Preferably, the managing of the tracks in the merged list to obtain a list of surviving tracks at the current moment includes:

[0041] Determine the update number of each track in the merged list, and if the update number is equal to a second set value and there is no update number associated with the observation data greater than the specified number, then remove the track from the merged list; the specified number is less than the second set value;

[0042] If the update number is greater than the second set value, and the track is not associated with the observation data in the most recent set number of times, the track is revoked from the merged list;

[0043] Get the list of surviving tracks at the current moment.

[0044] In a second aspect, an embodiment of the present invention further provides a multi-sensor joint space-time deviation calibration and multi-target association fusion device, including:

[0045] A determination unit, for determining, based on a track list at a previous moment, an expanded-dimensional state estimation sampling point set and a weight set corresponding to each track in the track list; the track list includes expanded-dimensional state estimation and expanded-dimensional state estimation covariance of multiple tracks corresponding to a number of targets; the expanded-dimensional state estimation includes a basic state estimation and a space-time deviation estimation;

[0046] A calculation unit, configured to calculate the association cost of each pair of track-observation data combinations based on the set of expanded-dimensional state estimation sampling points and the set of weights corresponding to each track at the previous moment and a list of observation data at the current moment; the list of observation data includes a plurality of observation data measured by the sensor on the target or clutter;

[0047] An association unit, used for associating the track with the observation data based on the association cost and the one-to-one constraint relationship between the track and the observation data, to obtain an association result list, an unassociated track list and an unassociated observation data list;

[0048] An updating unit, used for updating the tracks in the unassociated track list and the associated result list respectively, to obtain the unassociated updated track list and the associated track list at the current moment;

[0049] A fusion unit, used for performing feedback fusion processing of space-time deviation on the associated track list to obtain a fused associated track list;

[0050] A management unit, used to merge the unassociated updated track list with the merged associated track list, and manage the tracks in the merged list to obtain a surviving track list at the current moment;

[0051] A generating unit is used to perform an unbiased transformation on each observation data in the unassociated observation data list to construct an expanded-dimensional state estimate and an expanded-dimensional state estimate covariance corresponding to a new track to obtain a new track list, and merge the new track list with the surviving track list to obtain a track list at the current moment.

[0052] In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any embodiment of this specification is implemented.

[0053] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute the method described in any embodiment of this specification.

[0054] The embodiment of the present invention provides a multi-sensor joint space-time deviation calibration and multi-target association fusion method and device. First, based on the expanded-dimensional state estimation of each track in the track list at the previous moment, the expanded-dimensional state estimation sampling point set and weight set corresponding to each track are determined, wherein the expanded-dimensional state estimation includes the space-time deviation estimation and the target basic state estimation; then, in combination with the observation data list reported by the sensor at the current moment, the association cost of each pair of track-observation data combination is calculated, and then the track and the observation data are associated according to the association cost to generate an association result list, an unassociated track list and an unassociated observation data list, and the tracks in the association result list and the unassociated track list are updated, and then the updated associated track list is subjected to feedback fusion processing of the space-time deviation to further improve the effect of space-time deviation compensation, and finally, a new track is generated by managing the track and using each observation data in the unassociated observation data list to obtain the track list at the current moment. This scheme expands the space-time deviation into the basic state vector of the target and establishes a state space model under the condition of space-time deviation. Based on this model, the joint data association and deviation calibration problem can be converted into a classical data association and filtering problem under a unified Bayesian framework, without the need for iterative optimization processing between deviation calibration and data association. The compensation performance of space-time deviation is further improved through feedback fusion processing, thereby achieving effective multi-sensor multi-target tracking in the presence of space-time deviation. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 It is a flow chart of a multi-sensor joint space-time deviation calibration and multi-target association fusion method provided by one embodiment of the present invention;

[0057] Figure 2 This is a graph showing the RMSEs comparison results of the time deviation estimation of three methods provided in one embodiment of the present invention;

[0058] Figure 3 This is a graph comparing the RMSEs of distance deviation estimation of three methods provided in one embodiment of the present invention;

[0059] Figure 4 This is a comparison result diagram of RMSEs of angle deviation estimation of three methods provided by an embodiment of the present invention;

[0060] Figure 5 This is a graph comparing the RMSEs of target position estimation using three methods provided in one embodiment of the present invention;

[0061] Figure 6 This is a comparison result diagram of RMSEs of target speed estimation of three methods provided by an embodiment of the present invention;

[0062] Figure 7 This is a comparison result diagram of correct association probabilities of three methods provided by an embodiment of the present invention;

[0063] Figure 8 is a hardware architecture diagram of an electronic device provided by an embodiment of the present invention;

[0064] Fig. 9 It is a structural diagram of a multi-sensor joint space-time deviation calibration and multi-target association fusion device provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0066] As mentioned above, when solving the problem of simultaneous existence of observation source uncertainty and sensor bias, the relevant technologies generally perform iterative optimization processing of data association and bias estimation, but face the problem of time-consuming algorithm and not meeting the real-time processing needs of the actual system, and do not consider the time bias problem of multiple sensors; when the relevant technologies process multi-sensor observation data containing space-time bias and observation source uncertainty, they will face the problem of severe reduction in filtering fusion accuracy, making it difficult to achieve effective multi-sensor multi-target tracking.

[0067] In order to solve the above technical problems, the inventors can consider expanding the space-time deviation into the basic state vector of the target, and establish a state space model in the presence of space-time deviation. Based on the above state space model, the joint data association and deviation compensation problem can be converted into a data association and filtering problem under a unified Bayesian framework, without the need for iterative optimization processing between data association and deviation estimation; the inventors sequentially associate the observation data reported by each sensor with the track, that is, based on the expanded state estimation sampling point set and weight set corresponding to each track at the previous moment and the observation data list reported by the sensor at the current moment, the association cost of each track at the previous moment and each observation data in the observation data list at the current moment is calculated, and the generalized auction algorithm is used to complete the data association, generate an association result list, an unassociated track list and an unassociated observation data list, and update the tracks in the association result list and the unassociated track list, and then, the updated associated track list is subjected to feedback fusion processing of space-time deviation to further improve the effect of space-time deviation compensation, and finally, a new track is generated by managing the track and using each observation data in the unassociated observation data list to obtain the track list at the current moment. Therefore, this scheme can complete accurate multi-sensor joint space-time deviation calibration and multi-target association fusion, and can realize effective multi-sensor multi-target tracking in the presence of space-time deviation.

[0068] The specific implementation of the above concept is described below.

[0069] Please refer to Figure 1 The embodiment of the present invention provides a multi-sensor joint space-time deviation calibration and multi-target association fusion method, the method comprising:

[0070] Step 100: Based on the track list at the previous moment, determine the extended-dimensional state estimation sampling point set and weight set corresponding to each track in the track list; the track list includes the extended-dimensional state estimation and the extended-dimensional state estimation covariance of multiple tracks corresponding to several targets; the extended-dimensional state estimation includes the basic state estimation and the space-time deviation estimation;

[0071] Step 102: Calculate the association cost of each pair of track-observation data combination based on the expanded-dimensional state estimation sampling point set and weight set corresponding to each track at the previous moment and the observation data list at the current moment; the observation data list includes multiple observation data measured by the sensor on the target or clutter;

[0072] Step 104: Based on the association cost and the one-to-one constraint relationship between the track and the observation data, the track and the observation data are associated to obtain an association result list, an unassociated track list, and an unassociated observation data list;

[0073] Step 106: updating the tracks in the unassociated track list and the associated result list respectively, to obtain the unassociated updated track list at the current moment and the associated track list at the current moment;

[0074] Step 108: performing feedback fusion processing of space-time deviation on the associated track list to obtain a fused associated track list;

[0075] Step 110: merging the unassociated updated track list and the fused associated track list, and managing the tracks in the merged list to obtain a surviving track list at the current moment;

[0076] Step 112: Perform an unbiased transformation on each observation data in the unassociated observation data list to construct the expanded-dimensional state estimate and the expanded-dimensional state estimate covariance corresponding to the new track to obtain a new track list, and merge the new track list with the surviving track list to obtain the track list at the current moment.

[0077] In an embodiment of the present invention, firstly, based on the expanded-dimensional state estimation of each track in the track list at the previous moment, the expanded-dimensional state estimation sampling point set and the weight set corresponding to each track are determined, wherein the expanded-dimensional state estimation includes the space-time deviation estimation and the target basic state estimation; then, in combination with the observation data list reported by the sensor at the current moment, the association cost of each pair of track-observation data combination is calculated, and then the track and the observation data are associated according to the association cost to generate an association result list, an unassociated track list and an unassociated observation data list, and the tracks in the association result list and the unassociated track list are updated, and then, the updated associated track list is subjected to feedback fusion processing of the space-time deviation to further improve the effect of space-time deviation compensation, and finally, a new track is generated by managing the track and using each observation data in the unassociated observation data list to obtain the track list at the current moment. This scheme expands the space-time deviation into the basic state vector of the target and establishes a state space model under the condition of space-time deviation. Based on this model, the joint data association and deviation calibration problem can be converted into a classical data association and filtering problem under a unified Bayesian framework, without the need for iterative optimization processing between deviation calibration and data association. The compensation performance of space-time deviation is further improved through feedback fusion processing, thereby achieving effective multi-sensor multi-target tracking in the presence of space-time deviation.

[0078] Described below Figure 1 How the various steps are performed.

[0079] For step 100:

[0080] It should be noted that this scheme performs data association, space-time deviation compensation and fusion processing from steps 100 to 112 based on the track list at the previous moment and the observation data reported by the sensor at the current moment to obtain the track list at the current moment, and executes steps 100 to 112 in a loop to realize multi-sensor joint space-time deviation calibration and multi-target association fusion at moments k = 1, 2, ...

[0081] For example, based on the track list at time k-1 The unscented transformation method is used to calculate the extended-dimensional state estimation sampling point set δ corresponding to each track p p (k-1|k-1) and the corresponding weight set W p , the specific formula is:

[0082]

[0083] Where p is the track number, n t (k-1) is the number of tracks at the k-1th moment, and is the i-th extended-dimensional state estimation sampling point and its corresponding weight, where i = 0,…,2m x ;m x is the dimension of the expanded state estimation vector, and κ is used to determine the expanded state estimation at the k-1th moment. The scale parameter of the distribution state of the surrounding sampling points, and satisfy (m x +κ)≠0; for The i-th row or i-th column of and P p (k-1|k-1) are the expanded-dimensional state vector X of the track p at the k-1th moment. p (k-1) The corresponding expanded-dimensional state estimate and the expanded-dimensional state estimate covariance.

[0084] In an embodiment of the present invention, multiple sensors are used to track multiple targets in an area, and each sensor provides distance and angle observation data of multiple targets in a polar coordinate system. However, the observation data of each sensor has a fixed distance deviation and angle deviation, and is affected by the asynchronous clock, data transmission and / or the time required for signal processing. There is a fixed unknown delay between the timestamp of the observation data and the actual observation time. Therefore, the difference between the actual observation interval and the timestamp interval of the observation data from different sensors is used as the time deviation, and the time deviation and the spatial deviation are simultaneously expanded to the basic state vector of the target to construct an expanded state vector.

[0085] Specifically, the expanded-dimensional state vector X of the track p at the kth time p The specific form of (k) is:

[0086]

[0087] in,

[0088]

[0089]

[0090] Where, X p (k) is the basic state vector of track p, x p (k) and y p (k) are the positions of track p in the x and y directions, and are the velocities of track p in the x and y directions, is the space-time bias vector of the S sensors, B(k) is the spatial bias vector of the S sensors, b s (k) = [Δr s (k),Δθ s(k)]′ is the spatial deviation of sensor s=1,...,S, where Δr s (k) and Δθ s (k) is the distance deviation and angle deviation of sensor s, Ψ(k) is the time deviation vector of S-1 sensors relative to the reference sensor l, where Δt s,l (k) is the time deviation of sensor s relative to sensor l, where s=1,...,S and s≠l.

[0091] It should be noted that the embodiment of the present invention adopts the unscented transformation method to obtain the expanded-dimensional state estimation sampling point set and the weight set corresponding to each track, and other methods may also be adopted, so it is not specifically limited.

[0092] Regarding step 102:

[0093] It should be noted that the embodiment of the present invention uses multiple sensors to track multiple targets in an area. The sampling period of each sensor is different. Therefore, the observation data list obtained by each sensor measuring the multiple targets is a list reported by one of the sensors at regular intervals, and each reported observation data list corresponds to a moment. Therefore, the embodiment of the present invention sequentially associates the observation data in the observation data list reported by each sensor with the track of the previous moment corresponding to each observation data list.

[0094] In some implementations, step 102 may include the following steps S1-S4:

[0095] Step S1, for each pair of track-observation data combination, the following are performed: according to the track and observation data in the pair of track-observation data combination, and the sensor corresponding to the measurement observation data list, determining the observation function of the sensor;

[0096] Step S2, determining the observation prediction error and the observation prediction covariance of the pair of track-observation data combinations according to the observation function, the expanded dimension state estimation sampling point set, the weight set and the observation data in the pair of track-observation data combinations;

[0097] Step S3, calculating the Mahalanobis distance of the pair of track-observation data combinations according to the observation prediction error and the observation prediction covariance of the pair of track-observation data combinations;

[0098] Step S4, calculating the association cost of the track-observation data combination according to the Mahalanobis distance.

[0099] In step S1, in order to align the observation data with the basic state of the target by using the time deviation, the observation data needs to be expressed as a function of the basic state and the space-time deviation, that is, the observation function.

[0100] The observation function is determined according to the following formula:

[0101]

[0102]

[0103] Where p is the track number, s is the sensor number, and X p is the expanded-dimensional state estimate of the track, k is the time, (x s,p (k),y s,p (k)) is the actual position of the track at the time of observation data, (x s ,y s ) is the position of sensor s, Δr s (k) and Δθ s (k) is the distance deviation and angle deviation of sensor s, x p (k) and y p (k) are the positions of track p in the x and y directions, and are the velocities of track p in the x and y directions, Δt s,l (k) is the time deviation of sensor s relative to the reference sensor l, where s=1,...,S and s≠l, and S is the number of sensors.

[0104] In step S2, the observation prediction error and observation prediction covariance of the track-observation data combination are determined by the following formula:

[0105]

[0106]

[0107] in,

[0108]

[0109]

[0110]

[0111]

[0112]

[0113]

[0114] In the formula, is the track-observation data combination (p,i s ), is the track-observation data combination (p,is ), the observation data list can be expressed as i s is the observation data number of sensor s, n s (k) is the number of observations reported by sensor s at the current time, is the i-th reported by sensor s s Observation data, is the observed prediction of track p, is the i-th observation prediction sampling point of track p and observational prediction The difference, is the observation noise covariance of sensor s, σ r and σ θ are the distance and angle observation noise standard deviations, respectively; is the i-th dimensional expansion state prediction sampling point of track p, h s (·) is the observation function of sensor s, F(k-1) is the extended state transfer matrix, F(k-1) is the transfer matrix corresponding to the basic state, 0 4,(3S-1) is a zero matrix of dimension 4·(3S-1), 0 (3S-1),4 is a zero matrix of dimension (3S-1)·4, I 3S-1 is the identity matrix with dimension 3·S-1; ΔT(k-1) is the timestamp of the observation data provided by sensor s at the kth moment and the timestamp of the observation data provided by sensor l at the k-1th moment The difference between .

[0115] The track-observation data pair (p, i) in equations (7) and (8) can be expressed as s ) is calculated using the observation-forecast error and observation-forecast covariance.

[0116] In step S3, the Mahalanobis distance of the track-observation data combination is calculated by the following formula:

[0117]

[0118] In the formula, is the track-observation data combination (p,i s ), is the track-observation data combination (p,i s ), is the track-observation data combination (p,i s ) is the observed-predicted covariance of .

[0119] In step S4, the association cost is determined according to the following formula:

[0120]

[0121] In the formula, is the association cost of the track-observation data combination, i s is the observation data number of sensor s, u(i s ) is a binary variable, when i s =0, u(i s )=0, when i s When ≠0, u(i s )=1, is the detection probability of sensor s, ψ s is the area of ​​the sensor s observation area; is the Mahalanobis distance of the track-observation data pair, is the observation-prediction covariance for the track-observation data combination.

[0122] Regarding step 104:

[0123] In some implementations, step 104 may include the following steps:

[0124] According to the association cost of each pair of track-observation data combination and the one-to-one constraint relationship between track and observation data, a two-dimensional allocation model of track-observation data association is established;

[0125] A generalized auction algorithm is used to determine the associated result list, the unassociated track list and the unassociated observation data list.

[0126] In this step, the two-dimensional distribution model of track-observation data association is established as follows:

[0127]

[0128] in, is a binary variable, when (p,i s ) in the result of the track-observation data association, otherwise

[0129] List of track-observation data association results obtained using the generalized auction algorithm Unassociated track list and a list of unassociated observations They can be expressed as:

[0130]

[0131]

[0132]

[0133] In the formula, is the number of track-observation combinations in the association result list, is the serial number of the track-observation data combination in the association result list, (m(a),n(a)) means that the track and observation data in the a-th association combination are numbered m(a) and n(a) respectively, and m(·) is a to {1,...,n t (k-1)}, n(·) is a one-way mapping from to {1,...,n s (k)}, indicating that the track in the a-th association combination may be the track list T k-1 Any of the observations in the a-th associated combination may be the observation list z s (k), and the track-observation data in each associated combination are numbered differently from those in other combinations.

[0134] In addition, the track list is not associated is the total track list T k-1 and the difference of the associated tracks, similarly, the list of unassociated observations is the total observation data list z s (k) and the difference between the associated observations.

[0135] Regarding step 106:

[0136] In some implementations, step 106 may include the following steps B1-B2:

[0137] Step B1, calculate the expanded-dimension state prediction and the expanded-dimension state prediction covariance of each track in the unassociated track list, use the expanded-dimension state prediction and the expanded-dimension state prediction covariance of each track as the expanded-dimension state estimation and the expanded-dimension state estimation covariance of each track at the current moment, and obtain the unassociated updated track list at the current moment; the unassociated updated track list includes the expanded-dimension state estimation and the expanded-dimension state estimation covariance of each track at the current moment;

[0138] Step B2, use the unscented Kalman filtering method to filter each observation data in the association result list to obtain the expanded-dimensional state estimate and the expanded-dimensional state estimate covariance of each track associated with each observation data at the current moment, to obtain the associated track list at the current moment; the associated track list includes the expanded-dimensional state estimate and the expanded-dimensional state estimate covariance of each track at the current moment.

[0139] In step B1, the unassociated track list can be calculated according to the following formula: The expanded-dimensional state prediction and expanded-dimensional state prediction covariance of each track in:

[0140]

[0141] P p (k|k-1)=F(k-1)·P p (k-1|k-1)·F(k-1)′+Q p (k-1) (22)

[0142] in,

[0143] Q p (k-1) = Γ(k-1)Q p (k-1)Γ(k-1)′ (23)

[0144]

[0145]

[0146] In the formula, is the expanded-dimensional state prediction of trajectory p, P p (k|k-1) is the expanded-dimensional state prediction covariance of track p, Q p (k-1) is the noise covariance of the expanded process of track p, Γ(k-1) is the noise gain matrix of the expanded process, is the process noise covariance corresponding to the target’s base state, v x and v y are the standard deviations of the process noise in the x and y directions, Γ(k-1) is the process noise gain matrix corresponding to the basic state of the track, and 0 (3S-1),2 is a zero matrix of dimension (3S-1)·2, and F(k-1) is the expanded state transfer matrix.

[0147] The expanded-dimensional state prediction and expanded-dimensional state prediction covariance of each track are used as the expanded-dimensional state estimation and expanded-dimensional state estimation covariance of each track at the current moment, that is, P p (k|k)=P p (k|k-1), where and P p (k|k) are the expanded-dimensional state estimate and the expanded-dimensional state estimate covariance of the current track p, respectively. Then, the unassociated updated track list at the current moment can be obtained. The specific expression is:

[0148]

[0149] In step B2, the unscented Kalman filter method is used to filter the association result list The observation data associated with each track in the filter process is filtered to obtain the expanded-dimensional state estimate and expanded-dimensional state estimate covariance of each track associated with each observation data at the current moment. The specific formula is as follows:

[0150]

[0151]

[0152] In the formula, and They are the observation prediction covariance and observation prediction error of the track-observation data association combination; and P p (k|k-1) are the expanded-dimensional state prediction and the expanded-dimensional state prediction covariance of the track p, respectively. is the cross-covariance between the expanded state and the observed data, and the three are expressed as:

[0153]

[0154] P p (k|k-1)=F(k-1)·P p (k-1|k-1)·F(k-1)′+Q p (k-1) (29)

[0155]

[0156] In the formula, Q p (k-1) is the noise covariance of the expanded process of track p, m x is the dimension of the expanded state vector, is the weight corresponding to the i-th extended-dimensional state estimation sampling point, F(k-1) is the extended-dimensional state transfer matrix, is the i-th observation prediction sampling point of track p and observational prediction The specific expression is the formula (10) in step 102, is the i-th extended-dimension state prediction sampling point and extended-dimension state prediction of track p The specific formula is:

[0157]

[0158] According to formulas (28)-(31), the expanded-dimensional state estimation and expanded-dimensional state estimation covariance of each track in the association result list at the current moment can be calculated by formulas (26) and (27), and the associated track list at the current moment can be obtained. The specific expression of the associated track list is:

[0159]

[0160] Regarding step 108:

[0161] In some implementations, step 108 may include the following steps:

[0162] For each track in the list of associated tracks with an update count greater than the first set value, execute:

[0163] Extracting the space-time deviation estimate and the space-time deviation estimate covariance of each track whose update times are greater than the first set value at the current moment, so as to calculate the space-time deviation estimate after fusion of the track and the space-time deviation estimate covariance after fusion;

[0164] The fused space-time deviation estimate of the track replaces the original space-time deviation estimate in the expanded-dimensional state estimate at the current moment to obtain the fused expanded-dimensional state estimate of the track;

[0165] The unscented transformation method is used to calculate the covariance between the space-time deviation estimate after the track fusion and the basic state estimate of the track, so as to obtain the covariance of the expanded-dimensional state estimate after the track fusion;

[0166] The fused track is obtained; the fused track includes the fused expanded-dimensional state estimation and the fused expanded-dimensional state estimation covariance;

[0167] Each fused track whose update times are greater than the first set value replaces the corresponding track in the associated track list to obtain a fused associated track list.

[0168] In this step, first, the associated track list obtained in step 106 is determined. Whether the update times of each track p is greater than 4, extract the space-time deviation estimate and space-time deviation estimate covariance of all tracks with update times greater than 4 at the current moment.

[0169] Since the relationship between the extended-dimensional state estimation and the space-time deviation estimation of the track is:

[0170]

[0171] in, and They are the basic state estimation and space-time deviation estimation of track p, is the basic state estimate covariance of track p, is the cross-covariance of the basic state estimate and the space-time deviation estimate of track p, is the estimated covariance of the space-time deviation of track p.

[0172] Extract the space-time deviation estimate at the current moment for all tracks with update times greater than 4 and space-time bias estimation covariance After fusion, the space-time deviation estimate after fusion of each track and the covariance of the space-time deviation estimate after fusion are calculated according to the following formula:

[0173]

[0174]

[0175] in,

[0176]

[0177]

[0178] In the formula, and P b,f (k|k) are the space-time deviation estimate after fusion of the track and the covariance of the space-time deviation estimate after fusion, respectively, and w p is the weight of the space-time deviation estimate of track p, I p is the inverse matrix of the space-time deviation estimate covariance of track p, and I is the sum of the inverse matrices of the space-time deviation estimate covariance of all tracks in the list.

[0179] Then, the fused space-time deviation estimate of each track with an update number greater than 4 replaces the original space-time deviation estimate in the expanded-dimensional state estimate at the current moment to obtain the expanded-dimensional state estimate after each track fusion. The specific expression is:

[0180]

[0181] Next, the unscented transformation method is used to calculate the covariance between the space-time deviation estimate after each track fusion and the corresponding basic state estimate to obtain the covariance of the expanded-dimensional state estimate after each track fusion, which is calculated specifically according to the following formula:

[0182]

[0183]

[0184] in, and They are the basic state estimates corresponding to the track p. and the basic state estimation covariance The basic state estimation sampling points and weights, is the dimension of the basic state estimate, and is the space-time deviation estimate corresponding to the fusion And the fused space-time deviation estimation covariance P b,fThe fused spatial-temporal deviation estimation sampling points and weights of (k|k), m b is the dimension of the space-time bias estimate; and The calculation method is the same as in step 100 and The calculation method of is similar and will not be repeated here; similarly, and The calculation method will not be described in detail.

[0185] Then, we can obtain each fused track with an update number greater than 4. Formulas (36) and (38) are the expressions of the fused expanded-dimensional state estimate and the covariance of the fused expanded-dimensional state estimate for each track, respectively.

[0186] Finally, each fused track with an update number greater than 4 is replaced with the corresponding track in the associated track list obtained in step 106 to obtain a fused associated track list. At this time, the expression of the fused associated track list is:

[0187]

[0188] Regarding step 110:

[0189] The unassociated updated track list at the current moment obtained in step 106 and the list of associated tracks obtained in step 108 After merging, the expression of the merged list is:

[0190]

[0191] In some implementations, managing the tracks in the merged list to obtain a list of surviving tracks at the current moment may include the following steps:

[0192] Determine the update number of each track in the merged list, if the update number is equal to the second set value, and there is no update number associated with the observation data greater than the specified number, then remove the track from the merged list; the specified number is less than the second set value;

[0193] If the update number is greater than the second set value, and the track has not been associated with the observation data in the most recent set number of times, the track is removed from the merged list;

[0194] Get the list of surviving tracks at the current moment.

[0195] In the embodiment of the present invention, the update times of each track in the merged list is judged. If the update times are not greater than 4, it is a temporary track. For a temporary track whose update times are exactly equal to 4, the 3 / 4 logic method is used for judgment, that is, if the number of times that the track is not assigned observation data in 4 updates is greater than 1, the track is removed from the merged list, and the track number n is changed to t (k-1) minus 1; if the track is assigned observation data at least 3 times in 4 updates, the temporary track is successfully started and converted into a confirmed track.

[0196] If the number of updates is greater than 4, then the track is a confirmed track. Then check whether the track has been assigned observation data at least once in the last 5 updates. If so, continue to maintain the track as a confirmed track for subsequent processing. If not, it means that the track is no longer maintained or has left the observation area of ​​the sensor. Then cancel the track from the merged list and change the track number n. t (k-1) minus 1.

[0197] Assume that after the above processing, a total of Tracks; filter the merged list to get the list of surviving tracks at the current moment The form is:

[0198]

[0199] in,

[0200] Regarding step 112:

[0201] In the embodiment of the present invention, based on the unassociated observation data list in step 104 For each observation data, a new track is generated using the single-point initialization method, and a new track list is established And the list of surviving tracks obtained in step 100 Merge to generate the final complete track list T at the current moment k , the specific process is:

[0202] Since the observation data are distance and angle observation data in polar coordinates, and the basic state of the track is position and velocity data in the Cartesian coordinate system, it is necessary to perform an unbiased transformation on each observation data that is not associated with the observation data list to construct a new track.

[0203] First, for Each observation in First, the unbiased transformation method is used to calculate the unbiased transformation position measurement of the data in the Cartesian coordinate system and the unbiased transformed position measurement covariance The specific formula is:

[0204]

[0205]

[0206] in,

[0207]

[0208]

[0209]

[0210]

[0211] In the formula, and The observation data are The distance and angle observation data in and are the unbiased transformed position observation data in the x and y directions, respectively, is the error compensation factor, μ u is the mean of the transformed position errors; is the position observation variance in the x direction, is the position observation variance in the y direction, and is the cross-covariance of the position observations in the x-direction and the y-direction; is the compensation factor.

[0212] Subsequently, the unbiased transformed position observation data and position observation covariance in formulas (39)-(44) are used to establish the expanded-dimensional state estimate and expanded-dimensional state estimate covariance of track p, where the uncorrelated observation data list is used. The number of the first observation data to establish the track is set to That is, the maximum track number in the surviving track list is increased by 1, and the number of the newly generated tracks is processed as p=p+1. The specific formulas of the expanded-dimensional state estimation and the expanded-dimensional state estimation covariance of track p are:

[0213]

[0214]

[0215] in, is the basic state estimate of track p, is the space-time deviation estimate of track p, P p (k|k) is the basic state estimate covariance corresponding to track p, is the estimated covariance of the space-time deviation of track p; the forms of these four variables are:

[0216]

[0217]

[0218]

[0219]

[0220]

[0221] Among them, v max is the maximum possible speed, is the covariance of the spatial bias of a single sensor, is the variance of the time deviation of a single sensor, Δr max , Δθ max and Δt max are the maximum possible distance deviation, angle deviation and time deviation respectively.

[0222] From the above, it can be seen that the four variables of formulas (47)-(50) are formed according to the unbiased transformed position observation data and position observation covariance in formulas (39)-(44). Substituting the four variables into formulas (45) and (46), the expanded-dimensional state estimate and the expanded-dimensional state estimate covariance of the new track established using the observation data in the unassociated observation data list can be obtained.

[0223] According to the above process, the unassociated observation data list is used Each observation data creates a new track, and finally obtains a list of new tracks Its specific expression is:

[0224]

[0225] Finally, the list of surviving tracks and a list of new tracks Merge to generate the final complete track list T at the current moment k , the specific expression is:

[0226]

[0227] in, is the number of tracks in the track list at the current moment.

[0228] In an embodiment of the present invention, when asynchronous multi-sensors have space-time deviations and observation source uncertainties, this scheme can simultaneously realize data association, space-time deviation compensation and multi-target state estimation fusion processing; repeating the above processing steps can realize effective multi-sensor multi-target tracking while compensating for multi-sensor space-time deviations at time k=1, 2,...

[0229] In order to verify the effectiveness of the multi-sensor joint space-time deviation calibration and multi-target association fusion (JDABCF) method proposed in this paper, Monte Carlo experiments were carried out using simulated data, the performance of the algorithm was evaluated according to the root mean square error (RMSEs), and the posterior Cramer-Rao lower bound (PCRLB) was used to measure the best performance that the method can achieve. The methods used for comparison are the EM-UKF method and the JDABCF method without feedback fusion processing (JDABCF-NFF), in which the EM-UKF method does not consider the time deviation, and the JDABCF-NFF method does not perform feedback fusion processing. The purpose of comparing with these two methods is to illustrate the necessity of the JDABCF method to simultaneously process space-time deviations and perform feedback fusion steps.

[0230] In the simulation experiment, four asynchronous sensors are used to track multiple targets. The four sensors are located at (-5000m, 5000m), (8000m, -6000m), (8000m, 5000m) and (-5000m, -6000m). The sampling periods of the four sensors are 5s, 2s, 2s and 1s, and the initial sampling times are 0s, 4s, 6s and 4s. Sensor 1 is set as the reference sensor. The observation noise covariance of each sensor is R s (k) = diag((20m) 2 ,(0.01rad) 2 ). The sensors all have a distance deviation Δr s =50m and angle deviation Δθ s =0.03rad. The rest of the parameters are set as follows.

[0231] The detection probability of the four sensors is the same, which is 0.95. The timestamp delays of the four sensors are 3s, 1s, 1s, and 0.5s respectively, so the time deviations of sensors 2, 3, and 4 relative to sensor 1 are Δt 2,1 =2s, Δt 3,1 =2s and Δt 4,1=2.5s. The four targets appear in the observation area from t=0s, and their initial positions are (-3000m, 8m / s, 1000m, -12m / s), (0m, 15m / s, 1000m, 8m / s), (1000m, 12m / s, 500m, 10m / s) and (5000m, -13m / s, -1500m, 8m / s). The standard deviation of the process noise of the four targets is 0.001m / s. 2 In addition, the clutter is evenly distributed in the observation area, and the number of clutter at each moment is determined by a Poisson distribution variable with a mean of 30.

[0232] Considering that the estimation performance of multi-sensor space-time deviations is similar under the same filtering framework, the present invention provides the average result of multi-sensor space-time deviation estimation as the final space-time deviation estimation result. Similarly, the present invention provides the average result of multi-target state estimation as the final target state estimation result.

[0233] like Figure 2-Figure 7 As shown in the figure, the results of the three methods are compared. Figure 2 The RMSEs of temporal bias estimation for the three methods are shown; Figure 3 The distance bias estimation RMSEs of the three methods are shown; Figure 4 The RMSEs of angle deviation estimation for the three methods are shown; Figure 5 The target position estimation RMSEs of the three methods are shown; Figure 6 The target velocity estimation RMSEs of the three methods are shown; Figure 7 The correct association probabilities for the three methods are shown.

[0234] Depend on Figure 2-Figure 6 It can be seen that the RMSEs result of the EM-UKF method is greater than the RMSEs result of the JDABCF method. This is because the EM-UKF method does not consider and compensate for the time deviation, resulting in an estimation performance inferior to that of the JDABCF method. The JDABCF-NFF method has a certain performance improvement, but is still inferior to the JDABCF method. In contrast, the JDABCF method proposed in the present invention can effectively compensate for the space-time deviation to achieve accurate fusion estimation of the target state. Figure 7 The correct association probability is given, among which the JDABCF method still shows the best effect. These results illustrate the effectiveness of the JDABCF method in compensating for the space-time bias and performing the feedback fusion step. It is worth noting that the EM-UKF method performs multiple EM iterations and smoothing to update the initial state estimation results, which leads to faster convergence of the RMSEs of this method in the initial stage and a smoother RMSEs curve.

[0235] like Figure 8 , Fig. 9 As shown, the embodiment of the present invention provides a multi-sensor joint space-time deviation calibration and multi-target association fusion device. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. From the hardware level, Figure 8 As shown, it is a hardware architecture diagram of an electronic device in which a multi-sensor joint space-time deviation calibration and multi-target association fusion device provided by an embodiment of the present invention is located. Figure 8 In addition to the processor, memory, network interface, and non-volatile memory shown, the electronic device in the embodiment may also include other hardware, such as a forwarding chip responsible for processing messages, etc. Taking software implementation as an example, Fig. 9 As shown, as a device in a logical sense, the CPU of the electronic device in which it is located reads the corresponding computer program in the non-volatile memory into the internal memory and runs it.

[0236] like Fig. 9 As shown, this embodiment provides a multi-sensor joint space-time deviation calibration and multi-target association fusion device, including:

[0237] The determination unit 901 is used to determine, based on the track list at the previous moment, the extended-dimensional state estimation sampling point set and the weight set corresponding to each track in the track list; the track list includes the extended-dimensional state estimation and the extended-dimensional state estimation covariance of multiple tracks corresponding to several targets; the extended-dimensional state estimation includes the basic state estimation and the space-time deviation estimation;

[0238] A calculation unit 902 is used to calculate the association cost of each pair of track-observation data combination based on the extended dimension state estimation sampling point set and weight set corresponding to each track at the previous moment and the observation data list at the current moment; the observation data list includes multiple observation data measured by the sensor on the target or clutter;

[0239] An association unit 903 is used to associate the track with the observation data based on the association cost and the one-to-one constraint relationship between the track and the observation data, and obtain an association result list, an unassociated track list and an unassociated observation data list;

[0240] An updating unit 904 is used to update the tracks in the unassociated track list and the associated result list respectively, to obtain the unassociated updated track list and the associated track list at the current moment;

[0241] A fusion unit 905 is used to perform feedback fusion processing of space-time deviation on the associated track list to obtain a fused associated track list;

[0242] A management unit 906 is used to merge the unassociated updated track list and the merged associated track list, and manage the tracks in the merged list to obtain a surviving track list at the current moment;

[0243] The generation unit 907 is used to perform an unbiased transformation on each observation data in the unassociated observation data list to construct an expanded-dimensional state estimate and an expanded-dimensional state estimate covariance corresponding to a new track to obtain a new track list, and merge the new track list with the surviving track list to obtain the track list at the current moment.

[0244] In one embodiment of the present invention, the computing unit 902 is configured to perform the following operations:

[0245] For each track-observation data pair, execute:

[0246] Determine the observation function of the sensor according to the track and the observation data in the pair of track-observation data combinations and the sensor corresponding to the measurement observation data list;

[0247] Determine the observation prediction error and the observation prediction covariance of the pair of track-observation data combinations according to the observation function, the expanded-dimensional state estimation sampling point set, the weight set, and the observation data in the pair of track-observation data combinations;

[0248] Calculate the Mahalanobis distance of the pair of track-observation data combinations according to the observation prediction error and the observation prediction covariance of the pair of track-observation data combinations;

[0249] The association cost of the track-observation data pair is calculated based on the Mahalanobis distance.

[0250] In one embodiment of the present invention, in the calculation unit 902, the observation function is determined according to the following formula:

[0251]

[0252]

[0253] Where p is the track number, s is the sensor number, and X p is the expanded-dimensional state estimate of the track, k is the time, (x s,p (k),y s,p (k)) is the actual position of the track at the time of observation data, (x s ,y s ) is the position of sensor s, Δr s (k) and Δθ s (k) is the distance deviation and angle deviation of sensor s, x p (k) and y p(k) are the positions of track p in the x and y directions, and are the velocities of track p in the x and y directions, Δt s,l (k) is the time deviation of sensor s relative to reference sensor l, where s = 1, ..., S and s ≠ l, S is the number of sensors;

[0254] The association cost is determined according to the following formula:

[0255]

[0256] In the formula, is the association cost of the track-observation data combination, i s is the observation data number of sensor s, u(i s ) is a binary variable, when i s =0, u(i s )=0, when i s When ≠0, u(i s )=1, is the detection probability of sensor s, ψ s is the area of ​​the sensor s observation area; is the Mahalanobis distance of the track-observation data pair, is the observation-prediction covariance for the track-observation data combination.

[0257] In one embodiment of the present invention, the association unit 903 is configured to perform the following operations:

[0258] According to the association cost of each pair of track-observation data combination and the one-to-one constraint relationship between track and observation data, a two-dimensional allocation model of track-observation data association is established;

[0259] A generalized auction algorithm is used to determine the associated result list, the unassociated track list and the unassociated observation data list.

[0260] In one embodiment of the present invention, the updating unit 904 is configured to perform the following operations:

[0261] Calculate the expanded-dimensional state prediction and the expanded-dimensional state prediction covariance of each track in the unassociated track list, use the expanded-dimensional state prediction and the expanded-dimensional state prediction covariance of each track as the expanded-dimensional state estimate and the expanded-dimensional state estimate covariance of each track at the current moment, and obtain the unassociated updated track list at the current moment; the unassociated updated track list includes the expanded-dimensional state estimate and the expanded-dimensional state estimate covariance of each track at the current moment;

[0262] The unscented Kalman filtering method is used to filter each observation data in the association result list to obtain the expanded-dimensional state estimate and the expanded-dimensional state estimate covariance of each track associated with each observation data at the current moment, so as to obtain the associated track list at the current moment; the associated track list includes the expanded-dimensional state estimate and the expanded-dimensional state estimate covariance of each track at the current moment.

[0263] In one embodiment of the present invention, the fusion unit 905 is configured to perform the following operations:

[0264] For each track in the list of associated tracks with an update count greater than the first set value, execute:

[0265] Extracting the space-time deviation estimate and the space-time deviation estimate covariance of each track whose update times are greater than the first set value at the current moment, so as to calculate the space-time deviation estimate after fusion of the track and the space-time deviation estimate covariance after fusion;

[0266] The fused space-time deviation estimate of the track replaces the original space-time deviation estimate in the expanded-dimensional state estimate at the current moment to obtain the fused expanded-dimensional state estimate of the track;

[0267] The unscented transformation method is used to calculate the covariance between the space-time deviation estimate after the track fusion and the basic state estimate of the track, so as to obtain the covariance of the expanded-dimensional state estimate after the track fusion;

[0268] The fused track is obtained; the fused track includes the fused expanded-dimensional state estimation and the fused expanded-dimensional state estimation covariance;

[0269] Each fused track whose update times are greater than the first set value replaces the corresponding track in the associated track list to obtain a fused associated track list.

[0270] In one embodiment of the present invention, the management unit 906, when managing the tracks in the merged list to obtain the surviving track list at the current moment, is used to perform the following operations:

[0271] Determine the update number of each track in the merged list, if the update number is equal to the second set value, and there is no update number associated with the observation data greater than the specified number, then remove the track from the merged list; the specified number is less than the second set value;

[0272] If the update number is greater than the second set value, and the track has not been associated with the observation data in the most recent set number of times, the track is removed from the merged list;

[0273] Get the list of surviving tracks at the current moment.

[0274] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on a multi-sensor joint space-time deviation calibration and multi-target association fusion device. In other embodiments of the present invention, a multi-sensor joint space-time deviation calibration and multi-target association fusion device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0275] The information interaction, execution process and other contents between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For the specific contents, please refer to the description in the embodiment of the method of the present invention, and no further description is given here.

[0276] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, a multi-sensor joint space-time deviation calibration and multi-target association fusion method in any embodiment of the present invention is implemented.

[0277] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor executes a multi-sensor joint space-time deviation calibration and multi-target association fusion method in any embodiment of the present invention.

[0278] Specifically, a system or device equipped with a storage medium can be provided, on which software program code that implements the functions of any of the above-mentioned embodiments is stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program code stored in the storage medium.

[0279] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0280] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0281] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0282] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion module connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0283] It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical factors in the process, method, article or device including the elements.

[0284] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-sensor joint space-time deviation calibration and multi-target association fusion method, characterized in that: include: Based on the track list at the previous moment, determine the expanded dimension state estimation sampling point set and weight set corresponding to each track in the track list; The track list includes extended-dimensional state estimates and extended-dimensional state estimate covariances of multiple tracks corresponding to a plurality of targets; the extended-dimensional state estimates include basic state estimates and space-time deviation estimates; Based on the expanded dimension state estimation sampling point set and the weight set corresponding to each track at the previous moment and the observation data list at the current moment, the association cost of each pair of track-observation data combination is calculated; the observation data list includes multiple observation data measured by the sensor on the target or clutter; Based on the association cost and the one-to-one constraint relationship between the track and the observation data, the track and the observation data are associated to obtain an association result list, an unassociated track list and an unassociated observation data list; Respectively updating the tracks in the unassociated track list and the associated result list to obtain the unassociated updated track list at the current moment and the associated track list at the current moment; Performing feedback fusion processing of space-time deviation on the associated track list to obtain a fused associated track list; Merging the unassociated updated track list and the fused associated track list, and managing the tracks in the merged list to obtain a surviving track list at the current moment; Each observation data in the unassociated observation data list is unbiasedly transformed to construct the expanded-dimensional state estimate and the expanded-dimensional state estimate covariance corresponding to the new track to obtain a new track list, and the new track list is merged with the surviving track list to obtain the track list at the current moment.

2. The method according to claim 1, characterized in that The step of calculating the associated cost of each pair of track-observation data combinations based on the expanded dimension state estimation sampling point set and the weight set corresponding to each track at the previous moment and the observation data list at the current moment includes: For each track-observation data pair, execute: Determine an observation function of the sensor based on the track and the observation data in the pair of track-observation data combinations and the sensor corresponding to the observation data list; Determine the observation prediction error and the observation prediction covariance of the pair of track-observation data combinations according to the observation function, the expanded dimension state estimation sampling point set, the weight set and the observation data in the pair of track-observation data combinations; Calculate the Mahalanobis distance of the pair of track-observation data combinations according to the observation prediction error and the observation prediction covariance of the pair of track-observation data combinations; The association cost of the track-observation data combination is calculated according to the Mahalanobis distance.

3. The method according to claim 2, characterized in that The observation function is determined according to the following formula: Where p is the track number, s is the sensor number, and X p is the expanded-dimensional state estimate of the track, k is the time, (x s,p (k),y s,p (k)) is the actual position of the track at the time of the observation data, (x s ,y s ) is the position of sensor s, Δr s (k) and Δθ s (k) is the distance deviation and angle deviation of sensor s, x p (k) and y p (k) are the positions of track p in the x and y directions, and are the velocities of track p in the x and y directions, Δt s,l (k) is the time deviation of sensor s relative to reference sensor l, where s = 1, ..., S and s ≠ l, S is the number of sensors; The association cost is determined according to the following formula: In the formula, is the association cost of the track-observation data combination, i s is the observation data number of sensor s, u(i s ) is a binary variable, when i s =0, u(i s )=0, when i s When ≠0, u(i s )=1, is the detection probability of sensor s, ψ s is the area of ​​the sensor s observation area; is the Mahalanobis distance of the track-observation data pair, is the observation-prediction covariance for the track-observation data combination.

4. The method according to claim 1, characterized in that: Based on the association cost and the one-to-one constraint relationship between the track and the observation data, the track and the observation data are associated to obtain an association result list, an unassociated track list and an unassociated observation data list, including: According to the association cost of each pair of track-observation data combination and the one-to-one constraint relationship between track and observation data, a two-dimensional allocation model of track-observation data association is established; A generalized auction algorithm is used to determine the associated result list, the unassociated track list and the unassociated observation data list.

5. The method according to claim 1, characterized in that The updating process is performed on the unassociated track list and the tracks in the associated result list respectively to obtain the unassociated updated track list and the associated track list at the current moment, including: Calculate the expanded-dimension state prediction and the expanded-dimension state prediction covariance of each track in the unassociated track list, use the expanded-dimension state prediction and the expanded-dimension state prediction covariance of each track as the expanded-dimension state estimation and the expanded-dimension state estimation covariance of each track at the current moment, and obtain the unassociated updated track list at the current moment; the unassociated updated track list includes the expanded-dimension state estimation and the expanded-dimension state estimation covariance of each track at the current moment; An unscented Kalman filtering method is used to filter each observation data in the association result list to obtain the expanded-dimensional state estimate and the expanded-dimensional state estimate covariance of each track associated with each observation data at the current moment, so as to obtain the associated track list at the current moment; the associated track list includes the expanded-dimensional state estimate and the expanded-dimensional state estimate covariance of each track at the current moment.

6. The method according to claim 1, characterized in that The step of performing feedback fusion processing of space-time deviation on the associated track list to obtain a fused associated track list includes: For each track in the associated track list whose update number is greater than the first set value, execute: Extracting the space-time deviation estimate and the space-time deviation estimate covariance of each track whose update times are greater than the first set value at the current moment, so as to calculate the space-time deviation estimate after fusion of the track and the space-time deviation estimate covariance after fusion; The fused space-time deviation estimate of the track replaces the original space-time deviation estimate in the expanded-dimensional state estimate at the current moment to obtain the fused expanded-dimensional state estimate of the track; The unscented transformation method is used to calculate the covariance between the space-time deviation estimate after the track fusion and the basic state estimate of the track, so as to obtain the covariance of the expanded-dimensional state estimate after the track fusion; Obtaining the fused track; the fused track includes the fused extended-dimensional state estimate and the fused extended-dimensional state estimate covariance; Each fused track whose update times are greater than the first set value replaces the corresponding track in the associated track list to obtain a fused associated track list.

7. The method according to any one of claims 1 to 6, characterized in that: The track in the merged list is managed to obtain a list of surviving tracks at the current moment, including: Determine the update number of each track in the merged list, and if the update number is equal to a second set value and there is no update number associated with the observation data greater than the specified number, then remove the track from the merged list; the specified number is less than the second set value; If the update number is greater than the second set value, and the track is not associated with the observation data in the most recent set number of times, the track is revoked from the merged list; Get the list of surviving tracks at the current moment.

8. A multi-sensor joint space-time deviation calibration and multi-target association fusion device, characterized in that: include: A determination unit, for determining, based on a track list at a previous moment, an expanded-dimensional state estimation sampling point set and a weight set corresponding to each track in the track list; the track list includes expanded-dimensional state estimation and expanded-dimensional state estimation covariance of multiple tracks corresponding to a number of targets; the expanded-dimensional state estimation includes a basic state estimation and a space-time deviation estimation; A calculation unit, configured to calculate the association cost of each pair of track-observation data combinations based on the set of expanded-dimensional state estimation sampling points and the set of weights corresponding to each track at the previous moment and a list of observation data at the current moment; the list of observation data includes a plurality of observation data measured by the sensor on the target or clutter; An association unit, used for associating the track with the observation data based on the association cost and the one-to-one constraint relationship between the track and the observation data, to obtain an association result list, an unassociated track list and an unassociated observation data list; An updating unit, used for updating the tracks in the unassociated track list and the associated result list respectively, to obtain the unassociated updated track list and the associated track list at the current moment; A fusion unit, used for performing feedback fusion processing of space-time deviation on the associated track list to obtain a fused associated track list; A management unit, used to merge the unassociated updated track list with the merged associated track list, and manage the tracks in the merged list to obtain a surviving track list at the current moment; A generating unit is used to perform an unbiased transformation on each observation data in the unassociated observation data list to construct an expanded-dimensional state estimate and an expanded-dimensional state estimate covariance corresponding to a new track to obtain a new track list, and merge the new track list with the surviving track list to obtain a track list at the current moment.

9. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 7.

Images