Multi-sensor-based monitoring target fusion method and device and electronic device

Abstract

The application discloses a kind of based on multi-sensor monitoring target fusion method and its device, electronic equipment, wherein the fusion method includes: the state estimation value of the target state of each monitoring target in the state estimation value of the target state of each monitoring target obtained in previous moment and the outline estimation value of target outline are particle sampling, obtain the weighted particle of the predicted measurement value of monitoring target in current moment, then the measurement data of each sensor is processed, obtain the current state estimation value and current outline estimation value of all monitoring targets corresponding to sensor in current moment, again screening target sensor set corresponding to each monitoring target, fuse the current state estimation value and current outline estimation value corresponding to each target sensor in target sensor set, obtain the target state value and target outline value of monitoring target in current moment.The application solves the technical problem that the state value and the outline value of monitoring target are calculated in the related art, and the timeliness is poor and the accuracy is low.

Description

Technical Field

[0001] This invention relates to the field of multi-sensor information fusion technology, and more specifically, to a multi-sensor-based monitoring target fusion method, apparatus, and electronic device. Background Technology

[0002] With the development of "smart cities," an increasing number of sensors are being applied in various scenarios. To avoid the privacy risks associated with visual sensors, sensors such as millimeter-wave radar can be used, enabling fast and accurate localization and tracking while avoiding privacy violations. Furthermore, due to the high resolution of these sensors, a single monitored target (e.g., a car, a pedestrian) will generate multiple measurements. Since identifying the shape of the monitored target is beneficial for further analysis, it is considered to treat it as an extended target.

[0003] In related technologies, the localization and tracking of extended targets often requires preprocessing operations such as clustering before tracking. This method is too computationally burdensome for embedded platforms with limited computing power and cannot meet real-time requirements.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a multi-sensor-based monitoring target fusion method, apparatus, and electronic device to at least solve the technical problems of poor timeliness and low accuracy in calculating the state value and contour value of the monitoring target in related technologies.

[0006] According to one aspect of the present invention, a multi-sensor-based target fusion method is provided, comprising: acquiring state estimates of the target state and contour estimates of the target contours of each monitored target at a previous time; performing particle sampling on the state estimates and contour estimates of the monitored targets to obtain weighted particles of the predicted measurements of the monitored targets at the current time; processing the measurement data of each sensor based on the weighted particles of the predicted measurements of the monitored targets at the current time to obtain current state estimates and current contour estimates of all monitored targets corresponding to the sensors at the current time; filtering a set of target sensors corresponding to each monitored target based on the current state estimates and current contour estimates; and fusing the current state estimates and current contour estimates corresponding to each target sensor in the set of target sensors to obtain the target state value and target contour value of the monitored targets at the current time.

[0007] Optionally, after obtaining the state estimate of the target state and the contour estimate of the target contour of each monitored target at the previous time, the method further includes: determining the state prediction value of the monitored target at the current time based on the state estimate of the monitored target at the previous time; determining the contour prediction value of the monitored target at the current time based on the contour estimate of the monitored target at the previous time; and obtaining the predicted measurement value of the monitored target at the current time based on the state prediction value and the contour prediction value, wherein the predicted measurement value is the measurement value corresponding to the predicted state of the monitored target at the current time.

[0008] Optionally, the step of performing particle sampling on the state estimate and contour estimate of the monitored target to obtain weighted particles of the predicted measurement value of the monitored target at the current time includes: performing particle sampling on the state estimate and contour estimate to generate multiple particles; calculating the initial particle weight of each particle based on preset parameters; normalizing each initial particle weight based on all the initial particle weights to obtain the target particle weight of each particle, wherein the particle carrying the target particle weight is represented as a weighted particle; updating the weighted particles to obtain the weighted particles of the monitored target associated with the predicted measurement value at the current time.

[0009] Optionally, before processing the measurement data of each sensor based on the weighted particles of the predicted measurement value of the monitored target at the current time, the method further includes: acquiring the measurement data of each sensor, wherein the measurement data includes at least: a set of measurement vectors and a total number of measurements; constructing a target-oriented correlation variable based on the measurement data, wherein the target-oriented correlation variable is used to represent a measurement index of a first preset number of measurement bits generated by the preset monitoring target at the current time; and constructing a measurement correlation variable oriented towards vector measurements based on the measurement data, wherein the measurement correlation variable is used to represent a second preset number of measurement bits generated by the monitoring target mapped by the measurement index at the current time.

[0010] Optionally, the step of processing the measurement data of each sensor based on the weighted particles of the predicted measurement values ​​of the monitored targets at the current time to obtain the current state estimate and current profile estimate of all monitored targets corresponding to the sensor at the current time includes: evaluating the target-related variables based on the weighted particles of the predicted measurement values ​​of the monitored targets at the current time and the measurement vector set to obtain a likelihood probability; constructing a correlation factor graph based on the target-related variables and the measurement-related variables when the likelihood probability falls within a preset probability threshold range; performing iterative calculations based on the correlation factor graph to obtain a first parameter value and a second parameter value; updating the measurement data of the monitored targets of the sensor based on the first parameter value and the second parameter value to obtain an update result; obtaining the current profile estimate of the monitored targets corresponding to the sensor at the current time based on the update result, and performing weighted calculations on the weighted particles based on the update result to obtain the current state estimate of the monitored targets corresponding to the sensor.

[0011] Optionally, the step of selecting the set of target sensors corresponding to each monitoring target based on the current state estimate and the current contour estimate includes: representing the current state estimate as the mean of a preset Gaussian distribution and representing the current contour estimate as the covariance matrix of the preset Gaussian distribution; calculating the estimated probability of each sensor for the monitoring target based on the mean and the covariance matrix; normalizing all the estimated probabilities to obtain the target estimated probability; and adding the sensor corresponding to the target estimated probability to the set of target sensors corresponding to the monitoring target if the target estimated probability is greater than a preset probability threshold.

[0012] Optionally, the step of fusing the current state estimate and the current contour estimate corresponding to each target sensor in the target sensor set to obtain the target state value and target contour value of the monitored target at the current time includes: calculating the average of all the current state estimates corresponding to each target sensor to obtain the target state value; determining the first semi-axis length, the second semi-axis length, and the deflection angle value of the current target contour of the monitored target based on the current contour estimate value; determining the target covariance matrix based on the covariance matrix of all the target sensors; determining the target deflection angle value based on the target covariance matrix; calculating the target first semi-axis length and the target second semi-axis length for each target sensor based on the target deflection angle value; obtaining the target semi-axis length value based on all the target first semi-axis length values ​​and the target second semi-axis length values; and obtaining the target contour value based on the target semi-axis length value and the target deflection angle value.

[0013] According to another aspect of the present invention, a multi-sensor-based target fusion device is also provided, comprising: an acquisition unit, configured to acquire state estimates of the target states and contour estimates of the target contours of each monitored target at a previous time; a sampling unit, configured to perform particle sampling on the state estimates and contour estimates of the monitored targets to obtain weighted particles of the predicted measurements of the monitored targets at the current time; a processing unit, configured to process the measurement data of each sensor based on the weighted particles of the predicted measurements of the monitored targets at the current time to obtain current state estimates and current contour estimates of all monitored targets corresponding to the sensors at the current time; a filtering unit, configured to filter a set of target sensors corresponding to each monitored target based on the current state estimates and current contour estimates; and a fusion unit, configured to fuse the current state estimates and current contour estimates corresponding to each target sensor in the set of target sensors to obtain the target state value and target contour value of the monitored targets at the current time.

[0014] Optionally, the fusion device further includes: a first determining module, configured to, after acquiring the state estimate of the target state and the contour estimate of the target contour of each monitored target at the previous time, determine the state prediction value of the monitored target at the current time based on the state estimate of the monitored target at the previous time; a second determining module, configured to determine the contour prediction value of the monitored target at the current time based on the contour estimate of the monitored target at the previous time; and a first output module, configured to obtain the predicted measurement value of the monitored target at the current time based on the state prediction value and the contour prediction value, wherein the predicted measurement value is the measurement value corresponding to the predicted state of the monitored target at the current time.

[0015] Optionally, the sampling unit includes: a first generation module, used to perform particle sampling on the state estimate and the contour estimate to generate multiple particles; a first calculation module, used to calculate the initial particle weight of each particle based on preset parameters; a first processing module, used to normalize each initial particle weight based on all the initial particle weights to obtain the target particle weight of each particle, wherein the particle carrying the target particle weight is represented as a weighted particle; and a first update module, used to update the weighted particles to obtain the weighted particles of the monitoring target associated with the predicted measurement value at the current time.

[0016] Optionally, the fusion device further includes: a first acquisition module, configured to acquire the measurement data of each sensor before processing the measurement data of each sensor based on weighted particles of the predicted measurement value of the monitoring target at the current time, wherein the measurement data includes at least: a set of measurement vectors and a total number of measurements; a first construction module, configured to construct a target-oriented correlation variable based on the measurement data, wherein the target-oriented correlation variable is used to represent a measurement index of a first preset number of measurement bits generated by the preset monitoring target at the current time; and a second construction module, configured to construct a measurement correlation variable oriented towards vector measurements based on the measurement data, wherein the measurement correlation variable is used to represent a second preset number of measurement bits generated by the monitoring target mapped by the measurement index at the current time.

[0017] Optionally, the processing unit includes: a first evaluation module, configured to evaluate the target-related variables based on the weighted particles of the predicted measurement values ​​of the monitored target at the current time and the measurement vector set, to obtain a likelihood probability; a third construction module, configured to construct a correlation factor graph based on the target-related variables and the measurement-related variables, provided that the likelihood probability falls within a preset probability threshold range; a second calculation module, configured to perform iterative calculations based on the correlation factor graph to obtain a first parameter value and a second parameter value; a second update module, configured to update the measurement data of the monitored target of the sensor based on the first parameter value and the second parameter value, to obtain an update result; and a third calculation module, configured to obtain the current contour estimate of the monitored target corresponding to the sensor at the current time based on the update result, and to perform weighted calculations on the weighted particles based on the update result to obtain the current state estimate of the monitored target corresponding to the sensor.

[0018] Optionally, the filtering unit includes: a first characterization module, configured to characterize the current state estimate as the mean of a preset Gaussian distribution and the current contour estimate as the covariance matrix of the preset Gaussian distribution; a fourth calculation module, configured to calculate the estimated probability of each sensor for the monitored target based on the mean and the covariance matrix; a second processing module, configured to normalize all the estimated probabilities to obtain the target estimated probability; and a first addition module, configured to add the sensor corresponding to the target estimated probability to the target sensor set corresponding to the monitored target if the target estimated probability is greater than a preset probability threshold.

[0019] Optionally, the fusion unit includes: a fifth calculation module, used to calculate the average of all current state estimates corresponding to each of the target sensors to obtain the target state value; a third determination module, used to determine the first semi-axis length, the second semi-axis length, and the deflection angle value of the current target contour of the monitored target based on the current contour estimate; a fourth determination module, used to determine the target covariance matrix based on the covariance matrix of all the target sensors; a fifth determination module, used to determine the target deflection angle value based on the target covariance matrix; a sixth calculation module, used to calculate the target first semi-axis length and the target second semi-axis length for each of the target sensors based on the target deflection angle value; a second output module, used to obtain the target semi-axis length value based on all the target first semi-axis length values ​​and the target second semi-axis length values; and a third output module, used to obtain the target contour value based on the target semi-axis length value and the target deflection angle value.

[0020] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the above-described multi-sensor-based monitoring target fusion method.

[0021] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the multi-sensor-based monitoring target fusion method described above.

[0022] In this disclosure, the state estimates of the target states and the contour estimates of the target contours of each monitored target at the previous time are obtained. Particle sampling is performed on the state estimates and contour estimates of the monitored targets to obtain weighted particles of the predicted measurements of the monitored targets at the current time. Based on the weighted particles of the predicted measurements of the monitored targets at the current time, the measurement data of each sensor is processed to obtain the current state estimates and current contour estimates of all monitored targets corresponding to the sensors at the current time. Based on the current state estimates and current contour estimates, the target sensor set corresponding to each monitored target is selected. The current state estimates and current contour estimates corresponding to each target sensor in the target sensor set are fused to obtain the target state value and target contour value of the monitored targets at the current time. In this disclosure, particle sampling is first performed on the state estimates and contour estimates of the target states of each monitored target at the previous time step to obtain weighted particles of the predicted measurement values ​​of the monitored targets at the current time step. Then, the measurement data of each sensor is processed based on the weighted particles to obtain the current state estimates and current contour estimates of all monitored targets corresponding to each sensor at the current time step. Next, the target sensor set of each monitored target is filtered, and then the current state estimates and current contour estimates corresponding to each target sensor are fused to obtain the target state value and target contour value of the monitored target at the current time step. This method can calculate the state value and contour value of the monitored target in real time, and can efficiently obtain accurate estimates of the state value and contour value of each monitored target by using the filtering results. This solves the technical problems of poor timeliness and low accuracy in calculating the state value and contour value of the monitored target in related technologies. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0024] Figure 1 This is a flowchart of an optional multi-sensor-based target fusion method according to an embodiment of the present invention;

[0025] Figure 2 This is a flowchart illustrating the principle of an optional multi-sensor, multi-extended target fusion algorithm for lifeline safety according to an embodiment of the present invention.

[0026] Figure 3 This is a schematic diagram of an optional multi-sensor-based target fusion monitoring device according to an embodiment of the present invention;

[0027] Figure 4 This is a hardware structure block diagram of an electronic device (or mobile device) for a multi-sensor-based target fusion method according to an embodiment of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving consent information from the aforementioned user or organization.

[0031] This invention proposes a multi-sensor and multi-extended target fusion algorithm for urban lifeline safety. Based on factor graph and sum-product algorithms, this algorithm uses a voting mechanism to optimize sensor estimation results, achieving rapid fusion of multiple sensors and multiple extended targets. First, particle sampling is performed on the state and contour estimates of each monitored target from the previous time step. Then, these weighted particles are updated to obtain the predicted state value at the current time step. Next, each sensor performs iterative data association in parallel, obtaining its own estimates of the state and contour of each target. Then, a voting mechanism is used to optimize the estimation results, and finally, fusion is performed to obtain the estimated state and contour of each target. This algorithm has the advantages of low computational complexity and high computational speed. Furthermore, by utilizing the optimized estimation results, rapid fusion can be achieved, efficiently obtaining accurate estimates of the state and contour of each target.

[0032] The present invention will now be described in detail with reference to various embodiments.

[0033] Example 1

[0034] According to an embodiment of the present invention, an embodiment of a multi-sensor-based target fusion method for monitoring is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] Figure 1 This is a flowchart of an optional multi-sensor-based target fusion method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0036] Step S101: Obtain the state estimate of the target state and the contour estimate of the target contour of each monitored target at the previous moment.

[0037] Step S102: Perform particle sampling on the state estimate and contour estimate of the monitored target to obtain weighted particles of the predicted quantity measurement of the monitored target at the current time.

[0038] Step S103: Based on the weighted particles of the predicted measurement values ​​of the monitored targets at the current time, process the measurement data of each sensor to obtain the current state estimate and current contour estimate of all monitored targets corresponding to the sensor at the current time.

[0039] Step S104: Based on the current state estimate and the current profile estimate, filter the set of target sensors corresponding to each monitoring target.

[0040] Step S105: Fuse the current state estimate and the current contour estimate corresponding to each target sensor in the target sensor set to obtain the target state value and target contour value of the monitored target at the current moment.

[0041] Through the above steps, the state estimates and contour estimates of the target states of each monitored target at the previous moment can be obtained. Particle sampling is performed on the state estimates and contour estimates of the monitored targets to obtain weighted particles of the predicted measurements of the monitored targets at the current moment. Based on the weighted particles of the predicted measurements of the monitored targets at the current moment, the measurement data of each sensor is processed to obtain the current state estimates and current contour estimates of all monitored targets corresponding to the sensors at the current moment. Based on the current state estimates and current contour estimates, the target sensor set corresponding to each monitored target is selected. The current state estimates and current contour estimates corresponding to each target sensor in the target sensor set are fused to obtain the target state value and target contour value of the monitored targets at the current moment. In this embodiment of the invention, particle sampling can be performed on the state estimate and contour estimate of each monitored target at the previous moment to obtain weighted particles of the predicted measurement value of the monitored target at the current moment. Then, the measurement data of each sensor is processed according to the weighted particles to obtain the current state estimate and current contour estimate of all monitored targets corresponding to each sensor at the current moment. Then, the target sensor set of each monitored target is filtered, and then the current state estimate and current contour estimate of each target sensor are fused to obtain the target state value and target contour value of the monitored target at the current moment. This can calculate the state value and contour value of the monitored target in real time, and the filtering results can be used to efficiently obtain accurate estimates of the state value and contour value of each monitored target, thereby solving the technical problems of poor timeliness and low accuracy in calculating the state value and contour value of the monitored target in related technologies.

[0042] The embodiments of the present invention will now be described in detail with reference to the steps described above.

[0043] In this embodiment, it is assumed that S sensors are used to monitor K targets, where S and K are fixed and known. Using x... k (t) represents the state of the monitored target k∈{1,2,…,K} at time t.

[0044] First, the model for the monitoring target is set as follows:

[0045] xk (t)=Fx k (t-1)+v k (t-1)(1);

[0046] Among them, v k (t-1) is Gaussian white noise with zero mean, i.e., v k (t-1)~N(0,Q k (t-1)), Q k (t-1) represents the covariance, F is the state transition matrix, and x k (t-1) represents the state of the monitored target at time t-1.

[0047] Sensor s∈{1,2,…,S} generates at time t The first measurement will be the first Each measurement is recorded as Define measurement vector

[0048] The target contour is described using the lengths of its major and minor semi-axis and the deflection angles (l1, l2, θ). In this embodiment, it is assumed that the deflection angle of the target contour is consistent with the direction of target movement.

[0049] The observation model for the target at sensor s is set as follows:

[0050]

[0051] in, μ represents a measurement generated by target k on sensor s at time t. k (t) is Gaussian white noise with zero mean, i.e., μ k (t)~N(0,R k (t)), R k (t) represents the covariance, X k (t) represents the profile of the monitored target at time t, and h(·) is a nonlinear function:

[0052]

[0053] Where h1 and h2 are random variables with a mean of 0 that follow a uniform distribution in the interval [-1, 1].

[0054] In this embodiment, the initial target state x of all monitored targets k (0) and the initial target contour X k (0) is known.

[0055] Step S101: Obtain the state estimate of the target state and the contour estimate of the target contour of each monitored target at the previous moment.

[0056] In this embodiment of the invention, the state and contour parameters of each monitored target at time t-1 (i.e. the previous time) can be estimated first (i.e., the state estimate of the target state and the contour estimate of the target contour at the previous time).

[0057] Optionally, after obtaining the state estimate of the target state and the contour estimate of the target contour of each monitored target at the previous moment, the method further includes: determining the state prediction value of the monitored target at the current moment based on the state estimate of the monitored target at the previous moment; determining the contour prediction value of the monitored target at the current moment based on the contour estimate of the monitored target at the previous moment; and obtaining the predicted measurement value of the monitored target at the current moment based on the state prediction value and the contour prediction value, wherein the predicted measurement value is the measurement value corresponding to the predicted state of the monitored target at the current moment.

[0058] In this embodiment of the invention, for each monitoring target k, we have:

[0059]

[0060]

[0061] in, These represent the state of the monitored target k and the estimated profile at time t-1, respectively.

[0062] In this embodiment, the state estimate of the monitored target at the previous moment can be used as a basis. and contour estimates Determine the state prediction value and profile prediction value of the monitoring target at the current moment (that is, based on the estimation of the state and profile of the monitoring target at the previous moment, the state prediction value and profile prediction value of the monitoring target at the current moment can be obtained through formula (4), formula (5) and formula (1).

[0063] In this embodiment, the predicted measurement value is the measurement value corresponding to the predicted state of the monitored target at the current time (i.e., the measurement value corresponding to the predicted state of the monitored target at time t).

[0064] Step S102: Perform particle sampling on the state estimate and contour estimate of the monitored target to obtain weighted particles of the predicted quantity measurement of the monitored target at the current time.

[0065] Optionally, the step of performing particle sampling on the state estimate and contour estimate of the monitored target to obtain weighted particles of the predicted measurement value of the monitored target at the current time includes: performing particle sampling on the state estimate and contour estimate to generate multiple particles; calculating the initial particle weight of each particle based on preset parameters; normalizing each initial particle weight based on all initial particle weights to obtain the target particle weight of each particle, wherein the particle carrying the target particle weight is represented as a weighted particle; and updating the weighted particles to obtain the weighted particles of the associated predicted measurement value of the monitored target at the current time.

[0066] In this embodiment of the invention, particle sampling can be performed on the state estimate and contour estimate of each monitored target to obtain weighted particles of the predicted quantity measurement of the monitored target at the current time. Specifically:

[0067] For each set of target states and target contours (x) k (t),X k Particle sampling is performed on (t) to generate J particles (i.e., particle sampling is performed on the state estimate and the contour estimate to generate multiple particles), resulting in a set of weighted particles. Among them, w k,j This represents the particle weight of particle j.

[0068] The particle weights are calculated as follows:

[0069] w k,j =f(x) k,j |x k (t),cX k (t))(6);

[0070] Where c > 1 is the amplification factor (i.e., the preset parameter), and after all weights are calculated, normalization is performed (i.e., based on the preset parameter, the initial particle weight of each particle is calculated, and then based on all the initial particle weights, the initial particle weights are normalized to obtain the target particle weight of each particle):

[0071]

[0072] In this embodiment, particles carrying target particle weights can be represented as weighted particles. The weighted particles are then updated to obtain the weighted particles associated with the predicted measurements of the monitored targets at the current time (i.e., updating the sampled weighted particles to obtain the weighted particles corresponding to the predicted states of each target at time t). The formula for updating the states in the weighted particles is:

[0073] x k,j =Fx k,j (8);

[0074] Where F represents the state transition matrix.

[0075] Optionally, before processing the measurement data of each sensor based on the weighted particles of the predicted measurement values ​​of the monitored target at the current time, the method further includes: acquiring the measurement data of each sensor, wherein the measurement data includes at least: a set of measurement vectors and a total number of measurements; constructing a target-oriented correlation variable based on the measurement data, wherein the target-oriented correlation variable is used to represent a measurement index of a first preset number of measurement bits generated by the preset monitored target at the current time; and constructing a measurement correlation variable for surface-to-vector measurements based on the measurement data, wherein the measurement correlation variable is used to represent a second preset number of measurement bits generated by the monitored target mapped by the measurement index at the current time.

[0076] In this embodiment of the invention, for any sensor s, the monitored target k generates at most [number missing] [units missing]. To distinguish which measurements are generated by the monitored target k, the sensor s uses target-oriented correlation variables for description:

[0077]

[0078] To indicate whether the measurement originates from the target or from clutter, sensor s defines the following correlation variable for surface measurement:

[0079]

[0080] In this embodiment, the sensor index can be defined as s∈{1,2,…,S}, where S is the total number of sensors; the target index can be defined as k∈{1,2,…,K}, where K is the total number of targets, and these are assumed to be known. z s represents the set of measurements taken by sensor s. This represents the total number of measurements generated by sensor s. Assume the maximum number of measurements generated by each target is... Correlation variable α oriented towards target k kq This represents the measurement index corresponding to the q-th measurement generated by target k, and its value range is... Where 0 indicates that no corresponding measurement was generated; the correlation variable β of the surface measurement m = (k, q), which uses a tuple to represent that the measurement with index m corresponds to the qth measurement generated by the target k.

[0081] In this embodiment, measurement data for each sensor can be acquired first. This measurement data includes at least: a set of measurement vectors zs and a total number of measurements. Then, based on the measurement data, a target-related variable α is constructed for the monitoring target. kqThis target correlation variable is used to represent the measurement index of the first preset number of measurement indications (e.g., the qth) generated by the preset monitoring target at the current time, and based on the measurement data, the measurement correlation variable β of the surface vector measurement is constructed. m = (k, q), this measurement-related variable is used to represent the second preset number of bits (e.g., the qth bit) of measurement generated by the monitoring target mapped by the measurement index at the current time.

[0082] Step S103: Based on the weighted particles of the predicted measurement values ​​of the monitored targets at the current time, process the measurement data of each sensor to obtain the current state estimate and current contour estimate of all monitored targets corresponding to the sensor at the current time.

[0083] Optionally, the step of processing the measurement data of each sensor based on the weighted particles of the predicted measurement values ​​of the monitored targets at the current time to obtain the current state estimate and current profile estimate of all monitored targets corresponding to the sensor at the current time includes: measuring and evaluating the target-related variables based on the weighted particles and measurement vector set of the predicted measurement values ​​of the monitored targets at the current time to obtain the likelihood probability; constructing a correlation factor graph based on the target-related variables and measurement-related variables under the profile estimate where the likelihood probability is within a preset probability threshold range; performing iterative calculations based on the correlation factor graph to obtain the first parameter value and the second parameter value; updating the measurement data of the monitored targets of the sensor based on the first parameter value and the second parameter value to obtain the update result; obtaining the current profile estimate of the monitored targets corresponding to the sensor at the current time based on the update result, and performing weighted calculations on the weighted particles based on the update result to obtain the current state estimate of the monitored targets corresponding to the sensor.

[0084] In this embodiment of the invention, all sensors s can perform internal data association iterations in parallel, thereby obtaining estimates of the state and contour parameters of each target at time t (i.e., based on weighted particles of the predicted measurement values ​​of the monitored targets at the current time, the measurement data of each sensor can be processed to obtain the current state estimate and current contour estimate of all monitored targets corresponding to the sensor at the current time). Specifically:

[0085] (1) Measurement evaluation: The measurement data is evaluated by using weighted particles corresponding to the predicted state of the target at time t to calculate the corresponding likelihood probability (i.e., based on the weighted particles and measurement vector set of the predicted measurement value of the monitored target at the current time, the target-related variables are evaluated to obtain the likelihood probability):

[0086] use For each objective k, the correlation variable α kqThe measurement and evaluation are conducted using the following specific evaluation formula:

[0087]

[0088] in, This represents a weighted particle, where J is the total number of particles; j kq (x k,j ,X k ,α kq ;z s The calculation formula is as follows:

[0089]

[0090] Among them, X k The target contour of target k is represented by a random matrix in this embodiment; for In x k,j X is the mean. k is the probability under the Gaussian distribution of the covariance matrix; μfa is the mean number of false alarm measurements in the measurement scenario. In this embodiment, it is assumed that the mean number follows a Poisson distribution and a uniform distribution within the measurement range. This represents the total number of measurements generated by sensor s. This represents the m-th measurement generated by sensor s. This indicates the false alarm probability of the corresponding measurement.

[0091] (2) Iterative data association: By constructing a data association factor graph for the association variables of the monitoring target and the measurement, the probability corresponding to each association situation is obtained through iterative calculation (i.e., under the case where the likelihood probability is within the preset probability threshold range, the profile estimation value is used to construct an association factor graph based on the target association variables and the measurement association variables, and iterative calculation is performed based on the association factor graph to obtain the first parameter value and the second parameter value):

[0092] All sensors can execute in parallel, each using its own measurement data for iterative data correlation. The iterative calculation index is defined as... Where nit represents the total number of iterations, where each iteration calculates for all feasible pairs. and all measurements Perform the following calculations:

[0093]

[0094]

[0095] in, This indicates that the sum-product algorithm is performed on the selected pair (k,q);

[0096]

[0097] α k The number of valid measurements included. This represents the probability corresponding to the number of valid measurements.

[0098] for Perform the following calculations:

[0099]

[0100] And the following initial calculations are performed for the iterative calculations:

[0101]

[0102] By leveraging the characteristics of factor graphs to optimize the correlation calculation of this iterative data, the above calculation formula can be replaced as follows:

[0103]

[0104]

[0105]

[0106] in,

[0107] (3) Measurement update, using the results obtained from iterative data correlation (i.e., the first parameter value) and the second parameter value The system updates the weighted particles by measurement and simultaneously estimates the target's contour by selecting a suitable subset of measurements based on the correlation probability (i.e., based on the first and second parameter values, it updates the sensor's measurement data of the monitored target, obtains the update result, and based on the update result, obtains the current contour estimate of the monitored target corresponding to the sensor at the current moment).

[0108] After iterative data association calculation, we obtain as well as The following measurement updates were performed:

[0109]

[0110]

[0111]

[0112] For each target, a measurement is selected for updating the contour. And the following updates will be made:

[0113]

[0114]

[0115] Where γ represents the association probability, Z k τ represents the covariance matrix corresponding to the selected measurement. k and ν k These are the parameters used in the random matrix update.

[0116] (4) Target estimation: Weighted calculations are performed on the weighted particles to obtain an estimate of the target state (i.e., based on the update results, weighted calculations are performed on the weighted particles to obtain an estimate of the current state of the monitored target corresponding to the sensor):

[0117] Using the obtained measurement update results, and combining them with each group of weighted particle pairs, the state of each target is estimated:

[0118]

[0119] Step S104: Based on the current state estimate and the current profile estimate, filter the set of target sensors corresponding to each monitoring target.

[0120] Optionally, the step of selecting the target sensor set corresponding to each monitoring target based on the current state estimate and the current contour estimate includes: representing the current state estimate as the mean of a preset Gaussian distribution and representing the current contour estimate as the covariance matrix of a preset Gaussian distribution; calculating the estimated probability of each sensor for the monitoring target based on the mean and the covariance matrix; normalizing all estimated probabilities to obtain the target estimated probability; and adding the sensor corresponding to the target estimated probability to the target sensor set corresponding to the monitoring target if the target estimated probability is greater than a preset probability threshold.

[0121] In this embodiment of the invention, the fusion center can select the estimate with higher accuracy based on the estimates obtained from each sensor using a voting mechanism (i.e., based on the current state estimate and the current contour estimate, filter the set of target sensors corresponding to each monitoring target). Specifically:

[0122] For the estimation of the target state and target contour obtained from each sensor The fusion center selects from the estimates as follows:

[0123] For each target k and sensor s, the estimates of the state and profile from the other sensors are treated as the mean and covariance matrices of a Gaussian distribution. The corresponding probabilities are calculated, and cross-validation voting is performed (i.e., the current state estimate is represented as the mean of a preset Gaussian distribution, and the current profile estimate is represented as the covariance matrix of a preset Gaussian distribution; based on the mean and covariance matrix, the estimated probability of each sensor for the monitored target is calculated). ):

[0124]

[0125] After calculation, the probabilities of all sensors for the same target k are normalized (i.e., all estimated probabilities are normalized to obtain the target estimated probability), and sensor estimates greater than the threshold ε are selected for subsequent fusion. The set of sensor indices selected for target k is denoted as S. k (That is, if the estimated probability of the target is greater than the preset probability threshold ε, the sensor corresponding to the estimated probability of the target will be added to the target sensor set corresponding to the monitored target).

[0126] Step S105: Fuse the current state estimate and the current contour estimate corresponding to each target sensor in the target sensor set to obtain the target state value and target contour value of the monitored target at the current moment.

[0127] Optionally, the step of fusing the current state estimate and current contour estimate corresponding to each target sensor in the target sensor set to obtain the target state value and target contour value of the monitored target at the current moment includes: calculating the average of all current state estimates corresponding to each target sensor to obtain the target state value; determining the first semi-axis length, the second semi-axis length, and the deflection angle value of the current target contour of the monitored target based on the current contour estimate; determining the target covariance matrix based on the covariance matrix of all target sensors; determining the target deflection angle value based on the target covariance matrix; calculating the target first semi-axis length and the target second semi-axis length for each target sensor based on the target deflection angle value; obtaining the target semi-axis length based on all target first semi-axis length and target second semi-axis length values; and obtaining the target contour value based on the target semi-axis length and the target deflection angle value.

[0128] In this embodiment of the invention, the estimates in the preferred set of target sensors are fused to finally obtain the state and contour parameters of each target at time t (i.e., the current state estimate and the current contour estimate corresponding to each target sensor in the set of target sensors are fused to obtain the target state value and target contour value of the monitored target at the current time).

[0129] For target k, based on the determined sensor index set S kPerform state and profile estimation.

[0130] For the target state, the arithmetic mean method is used for calculation (i.e., the average of all current state estimates corresponding to each target sensor is calculated to obtain the target state value):

[0131]

[0132] For the target's outline, it can first be converted into the form of semi-axis length plus deflection angle (that is, based on the current outline estimate, determine the first semi-axis length, second semi-axis length, and deflection angle of the current target outline of the monitored target, where the first semi-axis length is the long semi-axis length, and the second semi-axis length is the short semi-axis length):

[0133]

[0134] Among them, l s1 ,l s2 ,θ s These are the length of the major semi-axis, the length of the minor semi-axis, and the deflection angle, respectively.

[0135] For the contour estimates from each sensor, we treat them as covariance matrices of a Gaussian distribution, and then apply these Gaussian distributions... Multiplying them yields the final covariance matrix σ. 2 (That is, based on the covariance matrices of all target sensors, the target covariance matrix is ​​determined). The formula for the covariance matrix of Gaussian distributions multiplied together is as follows:

[0136]

[0137] The target deflection angle is estimated using the obtained covariance matrix σ². (That is, based on the target covariance matrix, the target deflection angle value is determined), from which the estimates of the target's major and minor semi-axes corresponding to each sensor can be calculated (that is, based on the target deflection angle value, the length of the first semi-axis of the target for each sensor is calculated). and the target second half-axis length value ):

[0138]

[0139]

[0140] Finally, the semi-major and semi-minor axis lengths are estimated using the arithmetic mean method (i.e., based on all the target's first and second semi-axis length values, the target's semi-axis length value is obtained). Then, based on the target semi-axis length and the target deflection angle, the target contour value is obtained:

[0141]

[0142] Figure 2 This is a flowchart illustrating the principle of an optional multi-sensor, multi-extended target fusion algorithm for lifeline safety according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following:

[0143] Step (1): Estimation of the target at time t-1, that is, obtaining the state of each target and the estimated contour parameters at time t-1;

[0144] Step (2): Particle sampling, that is, particle sampling is performed on the estimates of each target;

[0145] Step (3): State update, that is, update the weighted particles obtained by sampling in step (2) to obtain the weighted particles of the measurement values ​​corresponding to the predicted state of each target at time t.

[0146] Step (4): Parallel data association iteration of sensors, that is, all sensors 1, ..., sensor s, etc. perform internal data association iteration in parallel, and each obtains the state of each target and the estimate of the contour parameters at time t.

[0147] Step (5): Optimize the estimate, that is, the fusion center selects the estimate with higher accuracy based on the estimates obtained from each sensor and a voting mechanism;

[0148] Step (6): Estimation fusion, that is, the selected estimates in step (5) are fused to finally obtain the estimate of the target at time t (i.e. the state of each target and the estimate of the contour parameters at time t).

[0149] In this embodiment of the invention, multi-extended target tracking of sensors can be realized based on factor graph optimization, and after the sensor estimation results are optimized by using a voting mechanism, the extended target estimation fusion of multiple sensors is realized, achieving the following beneficial effects: (1) Multi-extended target tracking based on factor graph optimization has a fast calculation speed and does not require preprocessing operations such as clustering; (2) The voting mechanism optimization and multi-sensor extended target estimation fusion, using cross-validation voting can ensure the optimization effect and effectively improve the accuracy of the estimation results obtained by using the measurement data of multiple sensors.

[0150] The following is a detailed description with reference to another embodiment.

[0151] Example 2

[0152] The multi-sensor-based target fusion device provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in Embodiment 1 above.

[0153] Figure 3This is a schematic diagram of an optional multi-sensor-based target fusion monitoring device according to an embodiment of the present invention, such as... Figure 3 As shown, the fusion device may include: an acquisition unit 30, a sampling unit 31, a processing unit 32, a filtering unit 33, and a fusion unit 34, wherein...

[0154] The acquisition unit 30 is used to acquire the state estimate of the target state of each monitored target and the contour estimate of the target contour at the previous moment.

[0155] The sampling unit 31 is used to perform particle sampling on the state estimate and contour estimate of the monitored target to obtain the weighted particles of the predicted quantity measurement of the monitored target at the current time.

[0156] The processing unit 32 is used to process the measurement data of each sensor based on the weighted particles of the predicted measurement values ​​of the monitored targets at the current time, so as to obtain the current state estimate and the current contour estimate of all monitored targets corresponding to the sensors at the current time.

[0157] The filtering unit 33 is used to filter the set of target sensors corresponding to each monitoring target based on the current state estimate and the current profile estimate.

[0158] The fusion unit 34 is used to fuse the current state estimate and the current contour estimate corresponding to each target sensor in the target sensor set to obtain the target state value and target contour value of the monitored target at the current moment.

[0159] The aforementioned fusion device can acquire the state estimate and contour estimate of the target state of each monitored target at the previous moment through the acquisition unit 30, sample the state estimate and contour estimate of the monitored target through the sampling unit 31 to obtain the weighted particles of the predicted measurement value of the monitored target at the current moment, process the measurement data of each sensor based on the weighted particles of the predicted measurement value of the monitored target at the current moment through the processing unit 32 to obtain the current state estimate and current contour estimate of all monitored targets corresponding to the sensor at the current moment, filter the target sensor set corresponding to each monitored target through the filtering unit 33 based on the current state estimate and current contour estimate, and fuse the current state estimate and current contour estimate of each target sensor in the target sensor set through the fusion unit 34 to obtain the target state value and target contour value of the monitored target at the current moment. In this embodiment of the invention, particle sampling can be performed on the state estimate and contour estimate of each monitored target at the previous moment to obtain weighted particles of the predicted measurement value of the monitored target at the current moment. Then, the measurement data of each sensor is processed according to the weighted particles to obtain the current state estimate and current contour estimate of all monitored targets corresponding to each sensor at the current moment. Then, the target sensor set of each monitored target is filtered, and then the current state estimate and current contour estimate of each target sensor are fused to obtain the target state value and target contour value of the monitored target at the current moment. This can calculate the state value and contour value of the monitored target in real time, and the filtering results can be used to efficiently obtain accurate estimates of the state value and contour value of each monitored target, thereby solving the technical problems of poor timeliness and low accuracy in calculating the state value and contour value of the monitored target in related technologies.

[0160] Optionally, the fusion device further includes: a first determining module, configured to determine the state prediction value of the monitored target at the current time based on the state estimation value of the monitored target at the previous time, after acquiring the state estimation value of the target state and the contour estimation value of the target contour of each monitored target at the previous time; a second determining module, configured to determine the contour prediction value of the monitored target at the current time based on the contour estimation value of the monitored target at the previous time; and a first output module, configured to obtain the predicted measurement value of the monitored target at the current time based on the state prediction value and the contour prediction value, wherein the predicted measurement value is the measurement value corresponding to the predicted state of the monitored target at the current time.

[0161] Optionally, the sampling unit includes: a first generation module for performing particle sampling on the state estimate and the contour estimate to generate multiple particles; a first calculation module for calculating the initial particle weight of each particle based on preset parameters; a first processing module for normalizing each initial particle weight based on all initial particle weights to obtain the target particle weight of each particle, wherein particles carrying the target particle weight are represented as weighted particles; and a first update module for updating the weighted particles to obtain the weighted particles of the associated predicted measurement of the monitored target at the current time.

[0162] Optionally, the fusion device further includes: a first acquisition module, configured to acquire measurement data of each sensor before processing the measurement data of each sensor based on weighted particles of the predicted measurement values ​​of the monitored target at the current time, wherein the measurement data includes at least: a set of measurement vectors and a total number of measurements; a first construction module, configured to construct a target-oriented correlation variable based on the measurement data, wherein the target-oriented correlation variable is used to represent a measurement index of a first preset number of measurement bits generated by the preset monitored target at the current time; and a second construction module, configured to construct a measurement correlation variable oriented towards vector measurements based on the measurement data, wherein the measurement correlation variable is used to represent a second preset number of measurement bits generated by the monitored target mapped by the measurement index at the current time.

[0163] Optionally, the processing unit includes: a first evaluation module, used to evaluate the target-related variables based on the weighted particles and measurement vector set of the predicted measurement values ​​of the monitored target at the current time, and obtain the likelihood probability; a third construction module, used to construct a correlation factor graph based on the target-related variables and measurement-related variables, provided that the likelihood probability falls within a preset probability threshold range and the contour estimate is within the range; a second calculation module, used to perform iterative calculations based on the correlation factor graph to obtain a first parameter value and a second parameter value; a second update module, used to update the measurement data of the monitored target of the sensor based on the first parameter value and the second parameter value, and obtain an update result; and a third calculation module, used to obtain the current contour estimate of the monitored target corresponding to the sensor at the current time based on the update result, and to perform weighted calculations on the weighted particles based on the update result to obtain the current state estimate of the monitored target corresponding to the sensor.

[0164] Optionally, the filtering unit includes: a first characterization module, used to characterize the current state estimate as the mean of a preset Gaussian distribution and the current contour estimate as the covariance matrix of a preset Gaussian distribution; a fourth calculation module, used to calculate the estimated probability of each sensor for the monitored target based on the mean and the covariance matrix; a second processing module, used to normalize all estimated probabilities to obtain the target estimated probability; and a first addition module, used to add the sensor corresponding to the target estimated probability to the target sensor set corresponding to the monitored target if the target estimated probability is greater than a preset probability threshold.

[0165] Optionally, the fusion unit includes: a fifth calculation module for calculating the average of all current state estimates corresponding to each target sensor to obtain a target state value; a third determination module for determining the first semi-axis length, the second semi-axis length, and the deflection angle value of the current target contour of the monitored target based on the current contour estimate; a fourth determination module for determining the target covariance matrix based on the covariance matrix of all target sensors; a fifth determination module for determining the target deflection angle value based on the target covariance matrix; a sixth calculation module for calculating the first semi-axis length and the second semi-axis length of each target sensor based on the target deflection angle value; a second output module for obtaining the target semi-axis length value based on all the first and second semi-axis length values ​​of the target; and a third output module for obtaining the target contour value based on the target semi-axis length value and the target deflection angle value.

[0166] The aforementioned fusion device may also include a processor and a memory. The aforementioned acquisition unit 30, sampling unit 31, processing unit 32, filtering unit 33, fusion unit 34, etc., are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0167] The aforementioned processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and by adjusting kernel parameters, the current state estimate and current contour estimate corresponding to each target sensor in the target sensor set are fused to obtain the target state value and target contour value of the monitored target at the current moment.

[0168] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0169] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program with the following initialization steps: obtaining the state estimate of the target state and the contour estimate of the target contour of each monitored target at the previous time; performing particle sampling on the state estimate and contour estimate of the monitored target to obtain weighted particles of the predicted measurement value of the monitored target at the current time; processing the measurement data of each sensor based on the weighted particles of the predicted measurement value of the monitored target at the current time to obtain the current state estimate and the current contour estimate of all monitored targets corresponding to the sensor at the current time; filtering the target sensor set corresponding to each monitored target based on the current state estimate and the current contour estimate; fusing the current state estimate and the current contour estimate corresponding to each target sensor in the target sensor set to obtain the target state value and the target contour value of the monitored target at the current time.

[0170] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute the above-described multi-sensor-based monitoring target fusion method.

[0171] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the above-described multi-sensor-based monitoring target fusion method.

[0172] Figure 4 This is a hardware structure block diagram of an electronic device (or mobile device) for a multi-sensor-based target fusion method according to an embodiment of the present invention. Figure 4 As shown, an electronic device may include one or more ( Figure 4 The processor 402 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and the memory 404 for storing data may also be included. In addition, it may include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 4 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device may also include components that are more... Figure 4 The more or fewer components shown, or having the same Figure 4 The different configurations shown.

[0173] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0174] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0175] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0176] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0177] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0178] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0179] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A multi-sensor based monitoring target fusion method, characterized in that, The method comprises the following steps: obtaining state estimation values of target states of each monitoring target and contour estimation values of target contours at a previous time point; performing particle sampling on the state estimation values of the monitoring targets and the contour estimation values to obtain weighted particles of predicted measurement values of the monitoring targets at a current time point; processing measurement data of each sensor based on the weighted particles of the predicted measurement values of the monitoring targets at the current time point to obtain current state estimation values and current contour estimation values of all the monitoring targets corresponding to the sensor at the current time point; screening a target sensor set corresponding to each monitoring target based on the current state estimation values and the current contour estimation values; fusing the current state estimation values and the current contour estimation values corresponding to each target sensor in the target sensor set to obtain target state values and target contour values of the monitoring targets at the current time point; wherein the step of screening the target sensor set corresponding to each monitoring target based on the current state estimation values and the current contour estimation values comprises: characterizing the current state estimation values as mean values of a preset Gaussian distribution and characterizing the current contour estimation values as covariance matrices of the preset Gaussian distribution; the step of fusing the current state estimation values and the current contour estimation values corresponding to each target sensor in the target sensor set to obtain target state values and target contour values of the monitoring targets at the current time point comprises: calculating the average values of the current state estimation values corresponding to each target sensor to obtain the target state values; determining first semi-axis length values, second semi-axis length values and deflection angle values of the current target contours of the monitoring targets based on the current contour estimation values; determining a target covariance matrix based on the covariance matrices of all the target sensors; determining a target deflection angle value based on the target covariance matrix; calculating target first semi-axis length values and target second semi-axis length values of each target sensor based on the target deflection angle value; obtaining target semi-axis length values based on all the target first semi-axis length values and the target second semi-axis length values; obtaining the target contour values based on the target semi-axis length values and the target deflection angle value.

2. The fusion method of claim 1, wherein, After obtaining the state estimation values of target states of each monitoring target and the contour estimation values of target contours at a previous time point, the method further comprises the following steps: determining state prediction values of the monitoring targets at the current time point based on the state estimation values of the monitoring targets at the previous time point; determining contour prediction values of the monitoring targets at the current time point based on the contour estimation values of the monitoring targets at the previous time point; obtaining predicted measurement values of the monitoring targets at the current time point based on the state prediction values and the contour prediction values, wherein the predicted measurement values are measurement values corresponding to predicted states of the monitoring targets at the current time point.

3. The fusion method of claim 2, wherein, The step of performing particle sampling on the state estimation value and the contour estimation value to obtain a weighted particle of a predicted measurement value of the monitoring target at a current time comprises: performing particle sampling on the state estimation value and the contour estimation value to generate a plurality of particles; calculating an initial particle weight of each particle based on a preset parameter; normalizing each initial particle weight based on all initial particle weights to obtain a target particle weight of each particle, wherein the particle carrying the target particle weight is characterized as a weighted particle; updating the weighted particle to obtain a weighted particle of the monitoring target associated with the predicted measurement value at the current time.

4. The fusion method of claim 1, wherein Before processing the measurement data of each sensor based on the weighted particle of the predicted measurement value of the monitoring target at the current time, it further comprises: obtaining the measurement data of each sensor, wherein the measurement data at least includes a measurement vector set and a total number of measurements; constructing a target association variable for the monitoring target based on the measurement data, wherein the target association variable is used to represent a first preset number of measurement indicators of measurement indexes generated by a preset monitoring target at the current time; constructing a measurement association variable for the measurement based on the measurement data, wherein the measurement association variable is used to represent a second preset number of measurements generated by the monitoring target mapped by the measurement index at the current time.

5. The fusion method of claim 4, wherein, The step of processing the measurement data of each sensor based on the weighted particle of the predicted measurement value of the monitoring target at the current time to obtain the current state estimation value and the current contour estimation value of all monitoring targets corresponding to the sensor at the current time comprises: performing measurement evaluation on the target association variable based on the weighted particle of the predicted measurement value of the monitoring target at the current time and the measurement vector set to obtain a likelihood probability; in the case where the likelihood probability belongs to a preset probability threshold range, constructing an association factor graph based on the target association variable and the measurement association variable; performing iterative calculation based on the association factor graph to obtain a first parameter value and a second parameter value; updating the measurement data of the monitoring target of the sensor based on the first parameter value and the second parameter value to obtain an update result; based on the update result, obtaining the current contour estimation value of the monitoring target corresponding to the sensor at the current time, and based on the update result, performing weighted calculation on the weighted particle to obtain the current state estimation value of the monitoring target corresponding to the sensor.

6. The fusion method of claim 1, wherein The step of screening a target sensor set corresponding to each monitoring target based on the current state estimation value and the current contour estimation value comprises: calculating an estimation probability of each sensor for the monitoring target based on the mean and the covariance matrix; performing normalization processing on all estimation probabilities to obtain a target estimation probability; In a case where the target estimated probability is greater than a preset probability threshold, the sensor corresponding to the target estimated probability is added to the target sensor set corresponding to the monitoring target.

7. A multi-sensor based monitoring target fusion apparatus characterized by comprising: Comprise: An acquisition unit is configured to acquire state estimation values of target states of each monitoring target and contour estimation values of target contours at a previous time point; A sampling unit is configured to perform particle sampling on the state estimation values and the contour estimation values of the monitoring target to obtain weighted particles of predicted measurement values of the monitoring target at a current time point; A processing unit is configured to process measurement data of each sensor based on the weighted particles of the predicted measurement values of the monitoring target at the current time point to obtain current state estimation values and current contour estimation values of all the monitoring targets corresponding to the sensor at the current time point; A screening unit is configured to screen a target sensor set corresponding to each monitoring target based on the current state estimation values and the current contour estimation values; A fusion unit is configured to fuse the current state estimation values and the current contour estimation values of each target sensor in the target sensor set to obtain target state values and target contour values of the monitoring target at the current time point; The screening unit further comprises a first representation module configured to represent the current state estimation values as mean values of a preset Gaussian distribution and represent the current contour estimation values as covariance matrices of the preset Gaussian distribution. The fusion unit further comprises a fifth calculation module configured to calculate mean values of all the current state estimation values corresponding to each target sensor to obtain the target state values; a third determination module configured to determine first semi-axis length values, second semi-axis length values, and deflection angle values of the current target contour of the monitoring target based on the current contour estimation values; a fourth determination module configured to determine a target covariance matrix based on the covariance matrices of all the target sensors; a fifth determination module configured to determine a target deflection angle value based on the target covariance matrix; a sixth calculation module configured to calculate target first semi-axis length values and target second semi-axis length values of each target sensor based on the target deflection angle value; a second output module configured to obtain target semi-axis length values based on all the target first semi-axis length values and the target second semi-axis length values; and a third output module configured to obtain the target contour values based on the target semi-axis length values and the target deflection angle value.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium comprises a stored computer program, wherein the computer-readable storage medium controls a device in which the computer-readable storage medium is located to execute the multi-sensor-based monitoring target fusion method of any one of claims 1 to 6 when the computer program is running.

9. An electronic device, comprising: The device comprises one or more processors and a memory configured to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the multi-sensor-based monitoring target fusion method of any one of claims 1 to 6.

Images