A distributed multi-sensor fusion tracking method and system

Abstract

The application relates to a distributed multi-sensor fusion tracking method and system, which comprises the following steps: acquiring original measurement information of each sensor, and adopting a pre-established extended label multi-Bernoulli filter to obtain multi-sensor local posterior probability density; determining a distributed multi-sensor fusion network topology structure and a relative weight matrix, and fusing the multi-sensor local posterior probability density based on the relative weight matrix to obtain a fusion probability posterior density; and updating a target state and quantity according to the fusion probability posterior density to obtain a distributed multi-sensor fusion tracking result. The application realizes simultaneous estimation of multi-target states and quantities based on the extended label multi-Bernoulli filter, does not need a complex data association algorithm, improves calculation efficiency, and realizes real-time multi-target tracking. Therefore, the application can be widely applied to the environment perception field of intelligent automobiles.

Description

Technical Field

[0001] This invention relates to the field of environmental perception for intelligent vehicles, and in particular to a distributed multi-sensor fusion tracking method and system based on Extended Multi-Bernoulli (LMB) filtering. Background Technology

[0002] The driving environment of intelligent vehicles, especially urban road environments, is highly complex and dynamic, placing high demands on the accuracy, stability, and real-time performance of vehicle perception systems. Utilizing multi-sensor data fusion perception, based on the data characteristics of different sensors and a wider spatiotemporal perception range, can improve target tracking accuracy, reduce false alarm rates, and maintain perception functionality even when a single sensor fails, ensuring the robustness of perception results.

[0003] Multi-sensor fusion tracking research is mainly divided into centralized fusion and distributed fusion. Centralized fusion directly transmits the raw measurement information from each sensor to the fusion center, which then processes the raw measurement information uniformly to achieve multi-target tracking. Although it can make full use of the raw measurement information from each sensor, the large amount of information from vehicle perception sensors necessitates high communication bandwidth requirements and a heavy computational load on the fusion center, posing challenges to the real-time performance of perception. Furthermore, if the fusion center fails, the information fusion function of the entire sensor network will degrade or even fail, resulting in poor system stability.

[0004] In a distributed fusion architecture, each sensor processes its measurement information separately to obtain preliminary detection results, which are then fused to obtain the final tracking trajectory. Compared to centralized fusion, this method has lower system bandwidth requirements and makes the system more stable. However, there are inconsistencies in the preliminary detection results of each sensor, and the fusion center has a low utilization rate of sensor information with low confidence.

[0005] Furthermore, traditional distributed fusion primarily utilizes a "measurement-track" association framework. This involves first associating the perception results of various sensors for the same target, then using data association algorithms to associate the track with the measurement, and finally estimating the state based on the association results. Within this framework, data association becomes the core component of the fusion algorithm. Commonly used data association algorithms, such as Global Nearest Neighbor Association (GNN), Joint Probability Density Association (JPDA), and Multiple Hypothesis Tracking (MHT), each have their own advantages and are widely applied. However, a common problem with these algorithms is that, due to their NP-hard (non-deterministic polynomial hard) computational complexity (requiring super-polynomial time to solve), they experience a "combinatorial explosion" phenomenon in scenarios with high target or false alarm density, leading to computational bottlenecks and failing to meet practical requirements in terms of computational efficiency and tracking performance. Simultaneously, how to measure the observations of the same target by different sensor data with different characteristics has also become a major technical challenge for distributed fusion. Summary of the Invention

[0006] To address the aforementioned problems, the present invention aims to provide a distributed multi-sensor fusion tracking method and system. This method utilizes multi-target tracking technology with random finite sets, directly extending the single-target Bayesian filtering framework to the multi-target scenario, directly estimating the number and state of multiple targets, and avoiding complex data association processes. A distributed multi-sensor information fusion structure is adopted, where each sensor is considered an independent node, including its own measurement information processing module and a fusion information processing module. The self-measurement information processing module uses extended label multi-Bernoulli filtering to achieve local target trajectory estimation, obtaining the probability posterior density of the target state set. The fusion information processing module receives the posterior estimation results from local and other nodes, uses the GCI rule to achieve fusion, and sends the updated results to other nodes, realizing unified observation of the same target by multiple sensors.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides a distributed multi-sensor fusion tracking method, comprising the following steps:

[0009] The raw measurement information of each sensor is obtained, and the local posterior probability density of the multi-sensor sensor is obtained by using a pre-established extended label-based multi-Bernoulli filter.

[0010] The topology and relative weight matrix of the distributed multi-sensor fusion network are determined, and the local posterior probability densities of the multi-sensor networks are fused based on the relative weight matrix to obtain the fused probability posterior density.

[0011] Based on the posterior density of the fusion probability, the target state and quantity are updated to obtain the distributed multi-sensor fusion tracking result.

[0012] Furthermore, the acquisition of the raw measurement information from each sensor, and the use of a pre-established extended label-based multi-Bernoulli filter to obtain the multi-sensor local posterior probability density, includes:

[0013] The target of interest in the environment at the current time k is taken as the target set, the measurement information of each sensor is taken as the observation set, and the observation set is filtered based on the extended label multi-Bernoulli filter to obtain the local posterior probability density estimation result of the state set at the current time.

[0014] Based on the local posterior probability density estimate of the current k-state set and the motion model, the local posterior probability density estimate of the next k+ state set is predicted.

[0015] Based on the observation set acquired by the sensor at time k+1 and the local posterior probability density estimation result of the state set at time k+1, the posterior density of the state set at time k+1 is updated.

[0016] Repeat the above steps to obtain the local posterior probability density of the state set of the multi-sensor system at each time step.

[0017] Furthermore, the process of determining the topology and relative weight matrix of the distributed multi-sensor fusion network, and fusing the local posterior probability densities of the multi-sensors based on the relative weight matrix to obtain the fused probability posterior density, includes:

[0018] Determine the topology of the distributed multi-sensor fusion network;

[0019] Determine the relative weight matrix of the distributed multi-sensor fusion network;

[0020] The fused probability posterior density is obtained by fusing the local posterior probability densities of multiple sensors based on the relative weight matrix.

[0021] Furthermore, the topology of the distributed multi-sensor fusion network is as follows:

[0022] Each type of sensor is equipped with a local center sensor and surrounding sensors. When there are many surrounding sensors, they are fused in pairs according to a set order, and the last-level surrounding sensors transmit the fusion result to the center sensor.

[0023] For different types of center sensors, the two center sensors are fused in pairs according to a set order, and the final center sensor outputs the final fusion result.

[0024] Furthermore, determining the relative weight matrix of the distributed multi-sensor fusion network includes:

[0025] The trust matrix of the distributed multi-sensor fusion network is obtained by adopting the DeGroot model.

[0026] Based on the topology of the distributed multi-sensor fusion network, the adjacency matrix of the distributed multi-sensor fusion network is obtained;

[0027] The relative weight matrix of the distributed multi-sensor fusion network is determined based on the trust matrix and the adjacency matrix.

[0028] Furthermore, the process of fusing the local posterior probability densities of multiple sensors based on the relative weight matrix to obtain the fused probability posterior density includes:

[0029] At time k, the local filter of sensor s obtains a local posterior probability density estimate based on its original measurement information and sends the information to its fusion processing module;

[0030] The fusion processing module sends the local probability posterior density estimate to the receiving node p corresponding to the node, and at the same time receives the local probability posterior density estimate of the incoming node q.

[0031] After obtaining the local posterior probability density of each sensor, under the GCI fusion principle, suboptimal distribution fusion is performed based on the local posterior probability density of each sensor to obtain the fused probability posterior density.

[0032] Furthermore, the step of updating the target state and quantity based on the posterior density of the fusion probability to obtain the distributed multi-sensor fusion tracking result includes:

[0033] The obtained fusion probability posterior density is pruned, truncated, and merged to remove branches with weights less than a set threshold, resulting in a simplified fusion probability posterior density.

[0034] The number of targets N is estimated based on the simplified fusion probability posterior density.

[0035] The states corresponding to the first N branches are selected as the estimated multi-target tracking results after fusion.

[0036] Secondly, the present invention provides a distributed multi-sensor fusion tracking system, comprising:

[0037] The local filtering module is used to acquire the raw measurement information of each sensor and to obtain the local posterior probability density of the multi-sensor system using a pre-established extended label-based multi-Bernoulli filter.

[0038] The fusion module is used to determine the topology and relative weight matrix of the distributed multi-sensor fusion network, and to fuse the local posterior probability densities of the multi-sensor networks based on the relative weight matrix to obtain the fused probability posterior density.

[0039] The tracking module is used to update the target state and quantity based on the posterior density of the fusion probability using the maximum likelihood estimation method, thereby obtaining the distributed multi-sensor fusion tracking results.

[0040] Thirdly, the present invention provides a processing device, the processing device including at least a processor and a memory, the memory storing a computer program, and the processor executing the steps to implement the distributed multi-sensor fusion tracking method when running the computer program.

[0041] Fourthly, the present invention provides a computer storage medium storing computer-readable instructions thereon, which can be executed by a processor to implement the steps of the distributed multi-sensor fusion tracking method.

[0042] The present invention has the following advantages due to the adoption of the above technical solutions:

[0043] 1. This invention achieves simultaneous estimation of the state and quantity of multiple targets based on an extended-label multi-Bernoulli filter, eliminating the need for complex data association algorithms, thus improving computational efficiency and enabling real-time multi-target tracking. Furthermore, it utilizes introduced extended label information for trajectory management and adds other sensor-perceived information, such as category information and target size information, as extended labels to the LMB's label space. This allows for the acquisition of other effective information while tracking the target trajectory, meeting the needs of advanced autonomous driving.

[0044] 2. This invention proposes a decentralized distributed multi-sensor fusion general framework structure. In view of the characteristics of the large number and multiple types of vehicle-mounted perception sensors, a general connection structure and weight allocation rule are proposed. The corresponding nodes of each sensor can process multi-source perception information in parallel, thereby realizing multi-sensor information fusion.

[0045] 3. This invention can process information features from different vehicle-mounted sensing sensors, including point cloud data and video data, under a unified tracking framework, making it widely adaptable and highly applicable.

[0046] Therefore, this invention can be widely applied to the field of environmental perception in intelligent vehicles. Attached Figure Description

[0047] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. In the drawings:

[0048] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0049] Figure 2 This is the sensor network fusion result used in this invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0051] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0052] In some embodiments of the present invention, a distributed multi-sensor fusion tracking method is provided. This method utilizes data from various vehicle-mounted perception sensors, including lidar, millimeter-wave radar, and video data. Through a decentralized distributed information fusion structure, it obtains an approximate posterior probability density result of a random finite set of multi-target states, thereby estimating the number and state of the multi-targets. Simultaneously, it utilizes a tag space to achieve track management.

[0053] Correspondingly, in other embodiments of the present invention, a distributed multi-sensor fusion tracking system, device, and storage medium are provided.

[0054] Example 1

[0055] like Figure 1 As shown, this embodiment provides a distributed multi-sensor fusion tracking method, which specifically includes the following steps:

[0056] 1) Obtain the raw measurement information of each sensor and use a pre-established extended label multi-Bernoulli filter (LMB filter) to obtain the local posterior probability density of the multi-sensor system.

[0057] 1.1) Take the target of interest in the environment at the current time k as the target set X(t), and the measurement information of each sensor as the observation set Z(t). Filter the observation set Z(t) based on the extended label multi-Bernoulli filter to obtain the local posterior probability density estimation result of the state set X at the current time.

[0058] Suppose that at any time t, the set of targets in the environment is X(t), and the set of observations of targets in the environment by the sensor system is Z(t), and the number of variables in the target set and the observation set are M(t) and N(t) respectively, then it can be expressed as:

[0059]

[0060]

[0061] in, For state space, The observation space is defined as follows: The target set X(t) and the observation set Z(t) are respectively considered as the state of multiple targets and the observations of multiple targets, transforming the multi-target tracking problem into a (multi-target) state space. and observation space The filtering problem on the set is addressed by using Bayesian estimation through mathematical modeling to solve the uncertainty problem. Therefore, uncertainty is represented by a random finite set. and Let x and z be random finite sets consisting of the target state and the target observation, respectively, and let z be random variables in the state set and the observation set, representing arbitrary state variables and observation variables.

[0062] The extended-label multi-Bernoulli filter approximates the probability distribution of a random finite set of multi-objective states in the target state space through a multi-Bernoulli distribution with extended labels. and extended tag space Above, define a multi-objective stateful finite set with extended labels. ( It is the target state space. It is an extended label space, and its posterior probability density π(X) can be derived from the multiple Bernoulli distribution parameters. To represent, that is:

[0063]

[0064]

[0065]

[0066]

[0067] Where, r (l) It is the existence probability of a single Bernoulli random finite set; in actual updates, it is used as... To characterize r (l) The continuously updated value serves as the weight of each single Bernoulli random set within the entire multi-Bernoulli distribution, hereinafter abbreviated as w(L); p(x, l) is the corresponding probability density function, the specific implementation of which is given by p (l)(x) is determined, and will be abbreviated as p in the following text; It is the target state space X and the extended label space The projection function between them Δ(X) is a unique label indicator function, which is only valid if the target state space X and the label space are identical. It is only when the number of variables in the distribution is the same that it can be proven that each state variable has a unique label. This is the premise of the labeled multi-Bernoulli distribution mentioned above.

[0068] Where w(L) and p(x, l) satisfy:

[0069]

[0070] The extended label space defined in the extended label multi-Bernoulli filter will be explained next: For each target, let l t = (k, i), where k is the initial time of target trajectory establishment, and i is the index of the target at the time of its creation. The target trajectory can be extracted by tracing back the trajectory label information. Simultaneously, if the current sensor can detect other relevant information about the target, it is added to the semantic label l. o Thus, a generalized label vector for a target x∈X is obtained as l=l t ∪l o .

[0071] The indicator function 1 used in formula (2) M (N) satisfies that, for any variables M and N:

[0072]

[0073] The Kronecker function δ used in formula (4) M (N) satisfies that, for any variables M and N, we have:

[0074]

[0075] The iterative process of a local LMB filter over time mainly consists of two steps: prediction and update.

[0076] 1.2) Estimation based on the local posterior probability density of the current time k-state set X And the motion model, for estimating the local posterior probability density of the state set X at the next time step k+1. Make predictions.

[0077] The motion model can be specified based on the motion characteristics of the target in the environment, such as a uniform linear motion model or a uniformly accelerated linear motion model, to define the specific implementation of f(x|·,l). The local posterior probability density estimate of the state set X at the next time step k+1 is expressed as follows:

[0078]

[0079]

[0080]

[0081] In this context, variables with subscript k+1|k represent posterior estimates obtained at time k+1 using Bayesian filtering and random finite set theory, based on the observations and state variables at time k; variables with subscript B represent the correlation parameters of the random finite set related to the newborn target under a multi-Bernoulli distribution, w B It is the probability weight of the new target, p B (x, l) is the probability density of the newly formed target; variables with the subscript S represent the relevant parameters of the random set of target states that continue to exist under a multi-Bernoulli distribution; w S It is the probability weight of the continued existence of the target, p S (x, l) is the probability that target x will continue to exist in the next moment; It is the predicted posterior probability density of the continuing target state; f(x|·,l) represents the state transition function of the continuing target state random set from time k to time k+1.

[0082] Because it is assumed that the multi-objective state random set is at time k. One implementation is X k Then, at time k+1, the multi-objective state random set is composed of the continuing objective state random set. And the new target B from time k to time k+1 k+1|k (x k )composition:

[0083]

[0084] 1.3) Based on the observation set Z acquired by the sensor at time k+1 and the local posterior probability density estimation results of the state set X at time k+1, the posterior density of the state set X at time k+1 is... Update:

[0085] π k+1 (X k+1 |Z)=Δ(X k+1 (15)

[0086]

[0087]

[0088]

[0089] In the formula, θ∈Θ, θ represents the label space The correlation functions mapped to the observation space Z, where Θ represents the set of correlation functions: θ(l) = 0 indicates that the observed target is empty; variables with the subscript z are all related to the observation. Since the observation includes measurements of a finite number of targets and measurements of clutter, specifically: Represents the probability weights of the observed variables; p represents the composition function of the probability of existence. k+1 (θ) (x, l|Z): represents the posterior estimated probability of the label variable l and the state variable x at time k+1, obtained based on the observation set Z; ψ z (x, l; θ) represents the augmented probability likelihood function, including two cases: the observed target is empty and the observed target has a valid target; p D (x, l) represents the probability that the target is observed; κ(z) θ(l) ) represents the probability intensity function of Poisson clutter; g(z) θ(l) |x,l) represents the likelihood function for estimating the observed variables based on the state variables.

[0090] 1.4) Repeat steps 1.2) to 1.3) to obtain the local posterior probability density of the state set X of the multi-sensor at each time step.

[0091] Specifically, in practical applications, multi-target tracking tasks typically set the state variable X(k) = [p x p y v x v y ] T , representing the target's position and velocity in the radial and lateral directions, respectively. For the LMB filter, separate models can be created for each vehicle-mounted perception sensor:

[0092] (1) Millimeter-wave radar

[0093] Measurement variables The observation matrix is That is, the spatial transformation matrix from the reference coordinate system to the millimeter-wave radar coordinate system can be determined by the calibration matrix. Where x... radar y radar These are the radial and lateral positions of the target as observed by millimeter-wave radar. It refers to the radial and lateral relative Doppler velocities of a target as observed by millimeter-wave radar.

[0094] (2) LiDAR

[0095] Measurement variable Z2(k)=[x lidar y lidar , z lidar ] T The observation matrix is That is, the spatial transformation matrix from the reference coordinate system to the lidar coordinate system.

[0096] (3) Image

[0097] Measurement variables Z3(k) = [u, v, w, h] T u and v are the pixel positions of the center point of the target bounding box in the image detection; w and h are the length and width of the target bounding box in the image detection.

[0098] The observation matrix is ​​H3(k)=[p 11 p 12 0 0; p 21 p 22 0 0] T , parameter (p 11 p 12 p 21 p 22 The value is determined by the projection of the calibration matrix from the target spatial position to the image pixel position in the following reference coordinate system.

[0099]

[0100] Where, x ref y ref , z ref It is the three-dimensional position corresponding to the center of the target in the reference coordinate system.

[0101] Because of the nonlinear state transition and observation matrices, an extended Kalman filter can be used for iterative prediction and updating of the target motion state. This unifies different vehicle-mounted sensing sensors under a single fusion framework for filtering.

[0102] 2) Determine the topology and relative weight matrix of the distributed multi-sensor fusion network, and fuse the local posterior probability densities of the multi-sensors based on the relative weight matrix to obtain the fused probability posterior density.

[0103] Vehicle perception systems involve various types of sensors, including ranging sensors, millimeter-wave radar, lidar, and visual sensors such as monocular and binocular cameras. Each type of sensor may be present in varying quantities to reduce blind spots. Therefore, it is necessary to determine the connection order and weights of local perception information within the distributed multi-sensor fusion network. Specifically, this includes the following steps:

[0104] 2.1) Determine the topology of the distributed multi-sensor fusion network.

[0105] like Figure 2 The diagram illustrates the topology of a distributed multi-sensor fusion network. In designing this topology, each type of sensor is configured with a local central sensor (e.g., a long-range millimeter-wave radar detecting the front of the vehicle) and surrounding sensors (e.g., millimeter-wave angular radar detecting the left and right front of the vehicle). When there are many surrounding sensors, they are fused in pairs according to a predefined order, with the final surrounding sensors transmitting the fusion result to the central sensor. Similarly, for different types of central sensors, pairs of central sensors are fused in a predefined order, with the final central sensor outputting the final fusion result. If there are "blind" sensors—sensors that have no detection information due to sensor malfunction—they do not need to send information to other sensors.

[0106] 2.2) Determine the relative weight matrix C of the distributed multi-sensor fusion network.

[0107] Specifically, it includes the following steps:

[0108] 2.2.1) Using the DeGroot model, the trust matrix A of the distributed multi-sensor fusion network is obtained.

[0109] Where, the trust matrix A = (a ij ),a ij This represents the information weight of sensor i relative to the input of sensor j. Its value is independent of the network structure and is only related to the detection performance (accuracy + stability under the current environment) of sensor j.

[0110] 2.2.2) Based on the topology of the distributed multi-sensor fusion network, the adjacency matrix B of the distributed multi-sensor fusion network is obtained as follows: B = (b ij If sensor i can receive the information weights from sensor j, then b ij =1.

[0111] 2.2.3) Based on the trust matrix A and the adjacency matrix B, determine the relative weight matrix C = (c ij ),in,

[0112] 2.2.4) Based on the relative weight matrix C of the distributed multi-sensor fusion network, the filtering results of the multi-sensor system are allocated.

[0113] Specifically, when assigning information weights based on the results of the relative weight matrix C:

[0114] If c ij=1, then sensor i fully receives the information input from sensor j;

[0115] If c ij =0, then sensor i completely rejects the information input from sensor j;

[0116] If c ij >0 and <1, then according to c ij The actual value is used to determine the information from sensor j.

[0117] Assume x i (k) represents the target state value estimated by sensor i at time k, then the target state value estimated by sensor i at time k+1 is:

[0118]

[0119] 2.3) Based on the relative weight matrix, the local posterior probability densities of multiple sensors are fused to obtain the fused probability posterior density.

[0120] Specifically, it includes the following steps:

[0121] 2.3.1) At time k, the local filter of sensor s obtains a local posterior probability density estimate based on its original measurement information and sends the information to its fusion processing module.

[0122] The local posterior probability density estimate of sensor s is expressed as:

[0123]

[0124] To obtain an analytical solution, the Dobernouri distribution can be approximated as a Gaussian mixture model, from the first to the Jth... s The original probability density is obtained by weighted summation of (l) Gaussian models. It can be represented as:

[0125]

[0126] in, These are the weights of the i-th Gaussian distribution; and is the mean and covariance of the i-th Gaussian distribution.

[0127] 2.3.2) The fusion processing module sends the local probability posterior density estimate to the receiving node p corresponding to the node, and at the same time receives the local probability posterior density estimate of the incoming node q.

[0128] 2.3.3) After obtaining the local posterior probability density of each sensor, under the GCI fusion principle, suboptimal distribution fusion is performed based on the local posterior probability density of each sensor:

[0129]

[0130] Where ω1+ω2=1, ω1 and ω2 represent the weights of the local parts of sensor 1 and sensor 2 in the fusion, which are determined by the relative weight matrix C obtained in step 2); This indicates that at time k, sensor 1 observes... The resulting local posterior probability density estimate for the state set; This indicates that at time k, sensor 2 observes... The resulting local posterior probability density estimate for the state set; This represents the posterior probability density estimate of the state set after fusion based on the observations and local posterior probability density estimates of sensor 1 and sensor 2, using the GCI fusion principle.

[0131] Each node, using the GCI fusion rule, the sensor network topology, and the fusion mapping priority relationship determined by the ranking and allocation method (i.e., the relative weight matrix obtained in step 2), performs pairwise fusion of the local posterior probability density estimated locally with that of other sensors to obtain the updated fused probability posterior density.

[0132]

[0133] 3) Based on the posterior density of the fusion probability, the maximum likelihood estimation method is used to update the target state and quantity, and obtain the distributed multi-sensor fusion tracking results.

[0134] Specifically, it includes the following steps:

[0135] 3.1) Based on the obtained posterior density of fusion probability, branches with weights less than a set threshold are removed through operations such as pruning, truncation, and merging to obtain a simplified posterior density of fusion probability, thereby improving computational efficiency.

[0136] 3.2) The number of targets N is estimated based on the simplified fusion probability posterior density.

[0137] The number of targets N can be determined by weights. To obtain, that is:

[0138]

[0139] 3.3) Select the states corresponding to the first N branches as the estimated multi-target tracking results after fusion.

[0140] Example 2

[0141] Embodiment 1 above provides a distributed multi-sensor fusion tracking method. Correspondingly, this embodiment provides a distributed multi-sensor fusion tracking system. The system provided in this embodiment can implement the distributed multi-sensor fusion tracking method of Embodiment 1. The system can be implemented through software, hardware, or a combination of both. For example, the system may include integrated or separate functional modules or units to execute the corresponding steps in the methods of Embodiment 1. Since the system in this embodiment is basically similar to the method embodiment, the description process in this embodiment is relatively simple. For relevant details, please refer to the description of Embodiment 1. The system embodiment provided in this embodiment is merely illustrative.

[0142] This embodiment provides a distributed multi-sensor fusion tracking system, including:

[0143] The local filtering module is used to acquire the raw measurement information of each sensor and to obtain the local posterior probability density of the multi-sensor system using a pre-established extended label-based multi-Bernoulli filter.

[0144] The fusion module is used to determine the topology and relative weight matrix of the distributed multi-sensor fusion network, and to fuse the local posterior probability densities of the multi-sensor networks based on the relative weight matrix to obtain the fused probability posterior density.

[0145] The tracking module is used to update the target state and quantity based on the posterior density of the fusion probability using the maximum likelihood estimation method, thereby obtaining the distributed multi-sensor fusion tracking results.

[0146] Example 3

[0147] This embodiment provides a processing device corresponding to the distributed multi-sensor fusion tracking method provided in Embodiment 1. The processing device can be a client-side processing device, such as a mobile phone, laptop, tablet computer, desktop computer, etc., to execute the method of Embodiment 1.

[0148] The processing device includes a processor, a memory, a communication interface, and a bus. The processor, memory, and communication interface are connected via the bus to enable communication between them. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the distributed multi-sensor fusion tracking method provided in Embodiment 1.

[0149] In some embodiments, the memory may be high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk storage device.

[0150] In other embodiments, the processor can be a general-purpose processor of various types, such as a central processing unit (CPU) or a digital signal processor (DSP), and is not limited thereto.

[0151] Example 4

[0152] The distributed multi-sensor fusion tracking method of this embodiment 1 can be specifically implemented as a computer program product. The computer program product may include a computer-readable storage medium on which computer-readable program instructions for executing the distributed multi-sensor fusion tracking method of this embodiment 1 are loaded.

[0153] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distributed multi-sensor fusion tracking method, characterized in that, Includes the following steps: The raw measurement information of each sensor is acquired, and a pre-established extended label-based multi-Bernoulli filter is used to obtain the local posterior probability density of the multi-sensor system; wherein, the extended label includes at least category information and target size information. The topology and relative weight matrix of the distributed multi-sensor fusion network are determined, and the local posterior probability densities of the multi-sensor networks are fused based on the relative weight matrix to obtain the fused probability posterior density. Based on the posterior density of the fusion probability, the target state and quantity are updated to obtain the distributed multi-sensor fusion tracking result; The topology of the distributed multi-sensor fusion network is as follows: Each type of sensor is equipped with a local center sensor and surrounding sensors. When there are many surrounding sensors, they are fused in pairs according to a set order, and the last-level surrounding sensors transmit the fusion result to the center sensor. For different types of center sensors, the two center sensors are fused in pairs according to a set order, and the final center sensor outputs the final fusion result. The determination of the relative weight matrix of the distributed multi-sensor fusion network includes: The trust matrix of the distributed multi-sensor fusion network is obtained by adopting the DeGroot model. Based on the topology of the distributed multi-sensor fusion network, the adjacency matrix of the distributed multi-sensor fusion network is obtained; The relative weight matrix of the distributed multi-sensor fusion network is determined based on the trust matrix and the adjacency matrix.

2. The distributed multi-sensor fusion tracking method as described in claim 1, characterized in that, The process of acquiring the raw measurement information from each sensor and using a pre-established extended label-based multi-Bernoulli filter to obtain the multi-sensor local posterior probability density includes: The target of interest in the environment at the current time k is taken as the target set, the measurement information of each sensor is taken as the observation set, and the observation set is filtered based on the extended label multi-Bernoulli filter to obtain the local posterior probability density estimation result of the state set at the current time. Based on the local posterior probability density estimate of the current k-state set and the motion model, for the next time... Prediction is made by estimating the local posterior probability density of the state set; according to The observation set acquired by the time sensor and The local posterior probability density estimation results of the time-state set, for The posterior density of the state set at each time step is updated; Repeat the above steps to obtain the local posterior probability density of the state set of the multi-sensor system at each time step.

3. The distributed multi-sensor fusion tracking method as described in claim 1, characterized in that, The process of determining the topology and relative weight matrix of the distributed multi-sensor fusion network, and fusing the local posterior probability densities of the multi-sensors based on the relative weight matrix to obtain the fused probability posterior density, includes: Determine the topology of the distributed multi-sensor fusion network; Determine the relative weight matrix of the distributed multi-sensor fusion network; The fused probability posterior density is obtained by fusing the local posterior probability densities of multiple sensors based on the relative weight matrix.

4. The distributed multi-sensor fusion tracking method as described in claim 1, characterized in that, The process of fusing the local posterior probability densities of multiple sensors based on the relative weight matrix to obtain the fused probability posterior density includes: exist At any moment, the sensor s The local filter obtains a local posterior probability density estimate based on its original measurement information and sends this information to its fusion processing module; The fusion processing module sends the local probability posterior density estimate to the corresponding receiving node. Simultaneously receive incoming nodes The local probability posterior density estimate; where each node represents a sensor; After obtaining the local posterior probability density of each sensor, under the GCI fusion principle, suboptimal distribution fusion is performed based on the local posterior probability density of each sensor to obtain the fused probability posterior density.

5. The distributed multi-sensor fusion tracking method as described in claim 1, characterized in that, The step of updating the target state and quantity based on the posterior density of the fusion probability to obtain the distributed multi-sensor fusion tracking result includes: The obtained fusion probability posterior density is pruned, truncated, and merged to remove branches with weights less than a set threshold, resulting in a simplified fusion probability posterior density. The number of targets N is estimated based on the simplified fusion probability posterior density. Before choosing N The states corresponding to each branch are used as the estimated multi-target tracking results after fusion.

6. A distributed multi-sensor fusion tracking system, characterized in that, include: The local filtering module is used to acquire the raw measurement information of each sensor and use a pre-established extended label-based multi-Bernoulli filter to obtain the local posterior probability density of the multi-sensor system; wherein, the extended label includes at least category information and target size information. The fusion module is used to determine the topology and relative weight matrix of the distributed multi-sensor fusion network, and to fuse the local posterior probability densities of the multi-sensor networks based on the relative weight matrix to obtain the fused probability posterior density. The tracking module is used to update the target state and quantity based on the posterior density of the fusion probability using the maximum likelihood estimation method, and obtain the distributed multi-sensor fusion tracking results. The topology of the distributed multi-sensor fusion network is as follows: Each type of sensor is equipped with a local center sensor and surrounding sensors. When there are many surrounding sensors, they are fused in pairs according to a set order, and the last-level surrounding sensors transmit the fusion result to the center sensor. For different types of center sensors, the two center sensors are fused in pairs according to a set order, and the final center sensor outputs the final fusion result. The determination of the relative weight matrix of the distributed multi-sensor fusion network includes: The trust matrix of the distributed multi-sensor fusion network is obtained by adopting the DeGroot model. Based on the topology of the distributed multi-sensor fusion network, the adjacency matrix of the distributed multi-sensor fusion network is obtained; The relative weight matrix of the distributed multi-sensor fusion network is determined based on the trust matrix and the adjacency matrix.

7. A processing apparatus, the processing apparatus comprising at least a processor and a memory, the memory storing a computer program, characterized in that, When the processor runs the computer program, it performs steps to implement the distributed multi-sensor fusion tracking method according to any one of claims 1 to 5.

8. A computer storage medium, characterized in that, It stores computer-readable instructions that can be executed by a processor to implement the steps of the distributed multi-sensor fusion tracking method according to any one of claims 1 to 5.

Images