Factor graph optimization based passive detection multi-target tracking method based on gaussian mixture model

Abstract

The application belongs to the technical field of distributed multi-sensor passive detection multi-target tracking. The application provides a factor graph optimization passive detection multi-target tracking method based on a Gaussian mixture model. The embodiment of the disclosure distributes multi-target batch numbers by constructing a distributed passive sensor cooperative coordinate system and combining the same target determination results between two sensors; based on two-dimensional observation of the sensor, a direction finding line is calculated, the same batch number direction finding lines are combined, a least square method is used to obtain a multi-target position estimation point set, and a weighted fusion is used to obtain a multi-target rough positioning point; a Gaussian mixture model is used to model and fuse the measurement distribution characteristics, an expectation maximization algorithm is used to solve the parameters and complete the solvable conversion of the optimization problem; a factor graph optimization model containing multiple factors is constructed, a sliding window optimization is used to realize state estimation, and the Markov distance and the Hungarian algorithm are combined to complete trajectory association and state updating; and the performance of distributed sensor passive detection multi-target tracking is effectively improved.

Description

Technical Field

[0001] This disclosure relates to the field of distributed multi-sensor passive detection and multi-target tracking technology, and in particular to a factor graph optimization passive detection and multi-target tracking method based on a Gaussian mixture model. Background Technology

[0002] Passive sensors are characterized by their high concealment, low power consumption, and strong anti-interference capabilities. They detect targets by receiving electromagnetic signals radiated or reflected by the target itself, without requiring active signal transmission, thus making them difficult to detect and interfere with. In the civilian sector, passive sensors have been widely used in critical scenarios such as low-altitude safety monitoring, urban emergency response, and environmental monitoring, providing crucial target perception technology support for UAV airspace control, intelligent traffic monitoring, and natural disaster early warning. However, due to the physical characteristics of sensors, individual passive sensors generally suffer from inherent defects such as limited detection accuracy, inability to directly obtain target distance information, and susceptibility to obstruction and environmental noise interference, making it difficult to meet the urgent need for high-precision collaborative perception of multiple targets in complex dynamic environments.

[0003] To effectively overcome the inherent limitations of single passive sensors, distributed passive sensor collaborative sensing schemes have emerged, becoming an important direction in the development of target perception technology. This scheme deploys multiple passive sensor nodes at different spatial locations, fully utilizing the geometric advantages of multi-view observation information for collaborative detection. This significantly improves target detection accuracy, greatly expands monitoring coverage, and effectively enhances the overall robustness of the system. Compared to single-sensor schemes, distributed passive sensor systems possess outstanding advantages such as high information redundancy, large spatial coverage, and strong resistance to single-point failures, enabling comprehensive, multi-dimensional, and three-dimensional perception of targets.

[0004] When using distributed passive sensor systems to perform cooperative sensing tasks on multiple non-cooperative targets, the entire process can be decomposed into three interrelated key steps: multi-target cooperative identification and detection, multi-target identity determination, and multi-target localization and tracking. Among these, multi-target identity determination is a crucial prerequisite for achieving accurate multi-target localization and tracking, and current multi-passive target tracking tasks often rely on this determination result. The current mainstream determination method is the geometric-spatiotemporal constraint method, which, while theoretically establishing target correspondences, is susceptible to errors from multiple sources in practical applications, leading to false cross-location points and matching ambiguity, thus hindering the effectiveness of tracking tasks. Looking at the current state of multi-target tracking research, the academic community has conducted extensive research on measurement fusion and algorithm optimization. For example, multi-dimensional allocation two-stage tracking methods have been proposed to control computational load, and a "local tracking-central fusion" architecture has been constructed to address clutter and missed detection problems in asynchronous pure orientation sensor systems. Meanwhile, feature-assisted and algorithm structure optimization, distributed cooperative tracking, and other technologies are also continuously developing, providing pathways to improve tracking robustness. However, existing tracking studies still have significant shortcomings, making it difficult to adapt to scenarios where non-cooperative target features are scarce and observations are incomplete. This further highlights the severe challenges faced by multi-passive target tracking under current technology, and there is an urgent need for targeted technical solutions to address these challenges.

[0005] Based on the latest research, existing passive detection multi-target tracking technologies still have significant technical gaps. First, passive sensors can only acquire two-dimensional angular observations, resulting in incomplete information and potential deviations in tracking initialization and state estimation. Second, when multi-sensor two-dimensional observations are converted to three-dimensional position through triangulation, the fused measurements exhibit a multi-peaked, skewed non-Gaussian distribution, making traditional single Gaussian assumption methods prone to model mismatch. Third, multiple targets, low feature discrimination, and superposition of observation noise easily lead to false associations, resulting in association uncertainty. Traditional recursive filtering is highly sensitive to model errors and noise biases, failing to effectively suppress the propagation of estimation errors caused by false associations, potentially leading to track drift or target loss. These problems ultimately result in insufficient robustness of existing tracking methods in complex scenarios with incomplete observations and fluctuating noise levels, making them unsuitable for real-world dynamic application environments.

[0006] Therefore, it is necessary to improve one or more of the problems existing in the above-mentioned related technical solutions.

[0007] It should be noted that this section is intended to provide background or context for the technical solutions of this disclosure as set forth in the claims. The description herein does not constitute an admission that it is prior art simply because it is included in this section. Summary of the Invention

[0008] The purpose of this disclosure is to provide a factor graph optimization passive detection multi-target tracking method based on Gaussian mixture model, thereby overcoming at least to some extent one or more problems caused by the limitations and defects of related technologies.

[0009] Based on all passive sensors, a main sensor and an auxiliary sensor are defined. The origin of the coordinate system is the position of the main sensor at time zero in the Earth-centered Earth-fixed system, and a distributed passive sensor cooperative coordinate system is constructed.

[0010] In a distributed passive sensor cooperative coordinate system, batch numbers are assigned to each target in space by combining the multi-target identity determination results between any master sensor and auxiliary sensor.

[0011] Based on the batch number of each target, cross-location is performed using the direction finding lines of the same target on different passive sensors to obtain multiple position estimation points. The multiple position estimation points of the same target are then fused to obtain the coarse positioning point of the target at the current time. All two-dimensional observed targets within the main sensor are traversed to obtain the set of coarse positioning points of each target at the current time.

[0012] A Gaussian mixture model is used to model the observation error distribution of the coarse location point set, and the expectation-maximization algorithm is used to solve the parameters of the Gaussian mixture model to update the parameters of the Gaussian mixture model until convergence.

[0013] A multi-target tracking factor graph optimization model is constructed based on a convergent Gaussian mixture model.

[0014] Based on the multi-target tracking factor graph optimization model, a nonlinear least squares optimization problem that minimizes the total error of the error function of all factor nodes is constructed and solved to obtain the optimized estimate set of all target states within the sliding window at the current time.

[0015] The optimized estimation set of all target states within the sliding window at the current moment is correlated with the coarse positioning point set at the next moment, and the factor graph structure and target trajectory are updated based on the trajectory correlation results to complete multi-target tracking.

[0016] Furthermore, based on all passive sensors, the steps of setting a primary sensor and auxiliary sensors, and constructing a distributed passive sensor cooperative coordinate system with the position of the primary sensor at time zero (height 0) in the Earth-centered Earth-fixed system as the origin, include:

[0017] Set one passive sensor as the main sensor and the rest of the passive sensors as auxiliary sensors;

[0018] A distributed passive sensor cooperative coordinate system is constructed with the position point of the main sensor at time zero, where the height is 0, as the origin.

[0019] The position information, attitude information, and target observation information of each auxiliary sensor are uniformly converted into the distributed passive sensor cooperative coordinate system.

[0020] Furthermore, in the distributed passive sensor cooperative coordinate system, the step of assigning batch numbers to various targets in space by combining the multi-target identity determination results between any primary and auxiliary sensors includes:

[0021] The identity of multiple targets in the field of view of different passive sensors is determined, and the one-to-one correspondence between the two-dimensional observation targets of each passive sensor and multiple real targets in space is determined.

[0022] Based on the multi-target identity determination results, the number of the multi-target number in the imaging plane of the main sensor is used as the basis, and the number is used as the unique batch number of the corresponding real target in space.

[0023] For any auxiliary sensor, each target in its imaging plane is assigned the target number of the main sensor associated with it through identity determination, which serves as the batch number of the real spatial target represented by that target.

[0024] Furthermore, based on the batch number of each target, cross-location is performed using the direction finding lines of the same target on different passive sensors to obtain multiple position estimation points. These multiple position estimation points for the same target are then fused to obtain the coarse positioning point of the target at the current moment. The step of traversing all two-dimensional observed targets within the main sensor to obtain the set of coarse positioning points for each target at the current moment includes:

[0025] By combining the pixel coordinates of multiple targets observed in the two-dimensional imaging plane of any passive sensor, the direction finding lines pointing from the optical centers of different passive sensors to all targets are calculated;

[0026] By combining direction finding lines pointing to the same batch of targets in pairs, and using the least squares method to calculate the midpoint of the common perpendicular of any combination of direction finding lines, the estimated position of the same target can be obtained.

[0027] Multiple estimated location points of the same target are fused to obtain the coarse location point of the target;

[0028] Traverse all two-dimensional observation targets within the main sensor to obtain a coarse set of positioning points for multiple targets.

[0029] Furthermore, the process of modeling the observation error distribution of the coarse location point set using a Gaussian mixture model and solving for the parameters of the Gaussian mixture model using the expectation-maximization algorithm to update the Gaussian mixture model parameters until convergence includes:

[0030] Calculate the observation error sequence based on the coarse location point set of multiple targets and the predicted target state values;

[0031] A Gaussian mixture model is used to model the distribution characteristics of the observation error sequence, and the parameters of the Gaussian mixture model are solved by the expectation-maximization algorithm.

[0032] The optimization problem involving nonlinear weighted sums is transformed into a maximum value problem using the expectation-maximization method.

[0033] The problem is transformed into a minimization problem by taking the negative logarithm, in order to update the parameters of the Gaussian mixture model until convergence.

[0034] Furthermore, the steps in constructing a multi-objective tracking factor graph optimization model based on a convergent Gaussian mixture model include:

[0035] Observation factors are constructed using a convergent Gaussian mixture model, and factor nodes are constructed by combining prior factors, motion model factors, and mutually exclusive factors.

[0036] Construct a multi-objective tracking factor graph optimization model based on factor nodes.

[0037] Furthermore, based on the multi-target tracking factor graph optimization model, the step of constructing and solving a nonlinear least squares optimization problem that minimizes the total error of the error function of all factor nodes to obtain the optimized estimate set of all target states within the sliding window at the current time includes:

[0038] Based on the multi-objective tracking factor graph optimization model, we carry out multi-objective factor graph optimization state estimation. We sum the error functions of all factor nodes in the multi-objective tracking factor graph optimization model and construct a nonlinear least squares optimization problem with the goal of minimizing the total error of the factor nodes associated with all target state variables within the sliding window.

[0039] Solving the nonlinear least squares optimization problem yields an optimized estimate set of all target states within the sliding window at the current time.

[0040] Furthermore, the process of associating the optimized estimate set of all target states within the current sliding window with the coarse localization point set at the next time step, and updating the factor graph structure and target trajectory based on the trajectory association results to complete multi-target tracking includes:

[0041] Based on the optimized estimation set of all target states, the trajectory similarity weight between the current target optimized state and the next time positioning observation is calculated using the Mahalanobis distance method;

[0042] Based on trajectory similarity weights, a trajectory association similarity matrix is ​​constructed, and the trajectory association results are obtained by solving the Hungarian algorithm.

[0043] Furthermore, the method also includes:

[0044] Based on the trajectory association results, state updates and trajectory lifecycle management are completed. For successfully matched state variables and positioning observations, new observation factors are constructed and the corresponding state variable nodes are updated.

[0045] For existing state variables that are continuously unmatched, determine their trajectory lifetime. If the lifetime exceeds the threshold, terminate the tracking; otherwise, continue tracking.

[0046] For consecutive unmatched location observations, initialize them as new targets and update the multi-target tracking factor graph optimization model.

[0047] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0048] In the embodiments of this disclosure, the passive detection multi-target tracking method based on Gaussian mixture model and factor graph optimization has the following advantages: First, this method can accurately model the non-Gaussian distribution characteristics of fused measurements to ensure the accuracy of state estimation. By using a Gaussian mixture model to model the non-Gaussian fused measurement distribution characteristics generated by multi-passive sensor fusion positioning, it can more accurately describe the actual observation noise characteristics under complex multi-source error coupling scenarios, thereby significantly improving the accuracy and reliability of multi-target state estimation. At the same time, the expectation-maximization algorithm is used to optimize the parameters of the Gaussian mixture model, which has good convergence and numerical stability, further ensuring the accuracy and stability of state estimation. Second, this method can effectively solve the problem of association matching ambiguity caused by the coupling of cross-positioning pseudo-points and identity determination results of multi-targets, avoid interference of cross-positioning pseudo-points on the tracking algorithm, suppress error propagation, and avoid track drift or target loss. In addition, a trajectory association and lifecycle management mechanism was designed. By combining Mahalanobis distance and Hungarian algorithm, the accuracy of target matching between adjacent time moments is improved. Through dynamic management of trajectory lifecycle, fine control is achieved for maintaining existing target tracking, initializing new targets, and terminating tracking of failed targets. Stable tracking performance can still be maintained even in complex scenarios with sensor position errors and changes in observation noise levels. Attached Figure Description

[0049] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0050] Figure 1 The diagram illustrates the steps of a factor graph optimization passive detection multi-target tracking method based on a Gaussian mixture model in an exemplary embodiment of this disclosure.

[0051] Figure 2 This diagram illustrates a distributed passive sensor collaborative sensing scenario in an exemplary embodiment of this disclosure.

[0052] Figure 3 This diagram illustrates a method for constructing a distributed passive sensor cooperative coordinate system in an exemplary embodiment of this disclosure.

[0053] Figure 4 This diagram illustrates how the results of multi-target identity determination among distributed passive sensors are used to assign batch numbers to multiple targets in an exemplary embodiment of this disclosure.

[0054] Figure 5 This diagram illustrates the calculation of the set of estimated positions of multiple targets at a certain moment in an exemplary embodiment of this disclosure, based on the identity determination result.

[0055] Figure 6 This diagram illustrates a set of coarse localization points for multiple targets obtained through fusion processing in an exemplary embodiment of this disclosure.

[0056] Figure 7 This diagram illustrates the construction of a multi-target fusion measurement Gaussian mixture model in an exemplary embodiment of this disclosure.

[0057] Figure 8 This diagram illustrates the overall structure of the passive detection multi-target tracking framework based on factor graphs in an exemplary embodiment of this disclosure.

[0058] Figure 9 This diagram illustrates a multi-objective factor graph optimization state estimation in an exemplary embodiment of this disclosure.

[0059] Figure 10 The illustration shows a schematic diagram of trajectory correlation solved based on Mahalanobis distance and Hungarian algorithm in an exemplary embodiment of this disclosure. Detailed Implementation

[0060] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0061] Furthermore, the accompanying drawings are merely illustrative diagrams of embodiments of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0062] This example implementation provides a factor graph optimization method for passive detection and multi-target tracking based on a Gaussian mixture model. (Reference) Figure 1 As shown, this factor graph optimization passive detection multi-target tracking method based on Gaussian mixture models can include:

[0063] Step S1: Based on all passive sensors, set the main sensor and auxiliary sensor, and construct a distributed passive sensor cooperative coordinate system with the position point of the main sensor at zero time in the Earth-centered Earth-fixed system at height 0 as the coordinate origin;

[0064] Step S2: In the distributed passive sensor cooperative coordinate system, combine the multi-target identity determination results between any main sensor and auxiliary sensor to assign batch numbers to each target in space;

[0065] Step S3: Based on the batch number of each target, cross-positioning is performed using the direction finding lines of the same target on different passive sensors to obtain multiple position estimation points. The multiple position estimation points of the same target are then fused to obtain the coarse positioning point of the target at the current time. All two-dimensional observed targets within the main sensor are traversed to obtain the set of coarse positioning points of each target at the current time.

[0066] Step S4: Use a Gaussian mixture model to model the observation error distribution of the coarse location point set, and use the expectation-maximization algorithm to solve for the parameters of the Gaussian mixture model, so as to update the parameters of the Gaussian mixture model until convergence.

[0067] Step S5: Construct a multi-target tracking factor graph optimization model based on a convergent Gaussian mixture model;

[0068] Step S6: Based on the multi-target tracking factor graph optimization model, construct and solve the nonlinear least squares optimization problem that minimizes the total error of the error function of all factor nodes to obtain the optimized estimate set of all target states within the sliding window at the current time.

[0069] Step S7: Associate the optimized estimation set of all target states within the sliding window at the current time with the coarse positioning point set at the next time with the trajectory, and update the factor graph structure and target trajectory according to the trajectory association result to complete multi-target tracking.

[0070] The aforementioned passive detection multi-target tracking method based on Gaussian mixture model and factor graph optimization achieves several advantages. Firstly, it accurately models the non-Gaussian distribution characteristics of fused measurements to ensure state estimation accuracy. By employing a Gaussian mixture model to model the non-Gaussian fused measurement distribution characteristics generated by multi-passive sensor fusion positioning, it can more accurately describe the actual observation noise characteristics under complex multi-source error coupling scenarios, thus significantly improving the accuracy and reliability of multi-target state estimation. Simultaneously, the expectation-maximization algorithm is used to optimize the parameters of the Gaussian mixture model, exhibiting good convergence and numerical stability, further ensuring the accuracy and stability of state estimation. Secondly, this method effectively solves the problem of ambiguity in correlation matching caused by the coupling of false points in multi-target cross-positioning and identity determination results, avoiding interference from false points in cross-positioning on the tracking algorithm, suppressing error propagation, and preventing track drift or target loss. Furthermore, a trajectory association and lifecycle management mechanism is designed, combining Mahalanobis distance and the Hungarian algorithm to improve the accuracy of target matching between adjacent time points. Dynamic trajectory lifecycle management enables refined control of existing target tracking maintenance, new target initialization, and termination of tracking for failed targets, maintaining stable tracking performance even in complex scenarios with varying sensor position errors and observation noise levels.

[0071] Below, we will refer to Figures 1 to 10 The steps of the factor graph optimization passive detection multi-target tracking method based on Gaussian mixture model described in this example implementation will be explained in more detail.

[0072] In one embodiment, to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application are clearly and completely described below with reference to the accompanying drawings. The described embodiments are some embodiments of the present application. All other instances obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the protection scope of the present application.

[0073] This application proposes a passive detection multi-target tracking method based on Gaussian mixture model and factor graph optimization. This embodiment details the complete process steps of this application to complete passive detection multi-target tracking based on Gaussian mixture model and factor graph optimization, using a typical practical application scenario where four motorized platforms are equipped with passive sensors to collaboratively detect four targets. The specific process steps are as follows: Figure 1 As shown. In the case of Figure 2In the distributed passive sensor cooperative sensing scenario shown, four moving airborne platforms are each equipped with a passive sensor. Within a certain period, all four passive sensors detect four moving targets within their detection fields of view, forming continuous two-dimensional motion tracks in the imaging planes of each sensor. To highlight the key technical points of this application, the installation position of the airborne passive sensors relative to the airborne platforms is not considered; that is, it is assumed that the centroid coordinates of the airborne platform itself coincide with the spatial coordinates of the sensor's centroid or optical center. Furthermore, the number of passive sensors described in this embodiment is not limited to... Figure 2 The four passive sensors shown in the scene can also be expanded to include multiple passive sensors. According to... Figure 1 The specific process steps shown are combined with Figure 2 To complete the distributed passive sensor collaborative sensing scenario shown in this embodiment, the following steps are required:

[0074] In step S1, a passive sensor in the distributed passive sensor system is selected as the master sensor, and the coordinate origin is set at the position point of the master sensor at time zero in the Earth-centered Earth-fixed system where the height is 0. A distributed passive sensor cooperative coordinate system is constructed.

[0075] like Figure 3 As shown, if passive sensor 1 is selected as the main sensor, then the remaining three passive sensors are auxiliary sensor 1, auxiliary sensor 2, and auxiliary sensor 3. The method for constructing the distributed passive sensor cooperative coordinate system is as follows:

[0076] At any given time, in a distributed infrared sensor cooperative detection system, the sensors are mounted on different airborne platforms. Their position and attitude information is typically measured by an inertial navigation system, and the raw data is usually output based on a geodetic coordinate system. Since the deployment locations of the airborne platforms and the installation attitude and orientation of the sensors are not uniform, a unified cooperative coordinate system needs to be established through spatial registration. This system transforms all sensor position, attitude parameters, and target observation information to this coordinate system, eliminating systematic errors caused by different coordinate references and installation attitudes, and ensuring the consistency of data among multiple sensors. First, the latitude, longitude, and altitude coordinates under the geodetic system are transformed to the geocentric coordinate system. The transformation formula is as follows:

[0077]

[0078] in Indicates a specific moment; subscript Represents the geocentric and geofixed coordinate system; Indicates the first The longitude, latitude, and altitude coordinates of the location of the airborne platform's center of gravity at all times; These are the coefficients of the elliptic expression for the meridional elliptic plane where the airborne platform is located, representing the major axis radius and minor axis radius of the ellipse, respectively. For the first The coordinates of the airborne platform's center of mass in the Earth's solid-state system at all times. To express coefficients through ellipse The first parameter to be solved, This is the second parameter obtained by combining the above parameters.

[0079] A distributed passive sensor cooperative coordinate system is established, following the right-hand rule. Its X-axis points east, Y-axis points north, and Z-axis is perpendicular to the ground and upwards. The moment the main sensor powers on is taken as time zero, and the point where the main sensor's altitude is 0 at that moment is taken as the origin. The purpose is to bring the origin of the cooperative coordinate system close to each sensor to reduce momentary coordinate transformation errors. Let the origin's geocentric coordinates be... Then in Time of the first Auxiliary sensors The position coordinates in the cooperative coordinate system can be obtained using the following formula:

[0080]

[0081] In the formula For auxiliary sensors In the The latitude and longitude coordinates of the moment; For this moment, auxiliary sensor Geocentric coordinates; For this moment, auxiliary sensor Coordinates in the cooperative coordinate system.

[0082] In step S2, in the distributed passive sensor cooperative coordinate system, the multi-target identity determination result between any two passive sensors is used as the multi-target distribution batch number in space.

[0083] The essence of multi-target identity determination is to establish a one-to-one correspondence between each two-dimensional observed target in the field of view of different passive sensors and the same real target in space, thereby eliminating ambiguity in the observed targets between different sensors. For example... Figure 4 As shown in the figure, the black lines clearly indicate the one-to-one correspondence between all targets in the imaging plane of the main sensor and all targets in the imaging planes of the other three auxiliary sensors, determined through multi-target identity determination. As can be seen from the figure, any target represented by any two-dimensional observation in the main sensor has a unique association with one and only one two-dimensional observation target in each auxiliary sensor, and there is no one-to-many or many-to-one situation.

[0084] Based on the aforementioned identity determination results, a unique batch number is assigned to all real targets in space using the "main sensor reference number" principle. This means that the multi-target number within the main sensor's imaging plane is used as the reference, and this number serves as the unique batch number for the corresponding real target in space. For any auxiliary sensor, each target within its imaging plane uses the main sensor target number associated with it through the identity determination, which is then used as the batch number for the real target in space represented by that target. For example, if a two-dimensional observation numbered "Target 1" in the main sensor corresponds to a real "Target 1" in space, then all two-dimensional observations in all auxiliary sensors associated with "Target 1" in the main sensor via the black line will be assigned the batch number "Target 1." This ensures that the same target in space has a unique identifier in all sensor observations, providing a unified basis for subsequent direction finding line combinations and cross-positioning calculations for targets with the same batch number.

[0085] In step S3, the direction finding lines pointing from the optical centers of different sensors to all targets are calculated by combining the pixel coordinates of multiple targets observed in the two-dimensional imaging plane of any sensor. The direction finding lines pointing to the same batch of targets in the multi-target identity determination results are combined in pairs. The midpoint of the common perpendicular of any combination of direction finding lines is calculated by combining the least squares method to obtain the position estimate point of the same target at a certain time. The multiple position estimate points of the same target are fused by the weighted average fusion method to calculate the coarse positioning point set of multiple targets at a certain time.

[0086] At any moment Set the same goal In the main sensor and auxiliary sensors Corresponding observations in the imaging plane and ,sensor and The positions are respectively and Due to noise and calibration errors, both sensors point to the same target. The direction finding lines are mostly skew lines. The midpoint of the common perpendicular is taken as the position estimation point, and its coordinates... It can be represented as:

[0087]

[0088] in This represents the direction vector of the common perpendicular of the two direction finding lines pointing towards the target. For sensors Pointing to target The direction finding line, For sensors Pointing to target The direction finding line.

[0089] Suppose the system includes For each sensor, the direction finding lines of any two sensors pointing to the target are combined in pairs and the solution is obtained. If there are several location estimation points, then the same target can be estimated. The estimated location points are weighted and averaged to obtain the coarse coordinates of the target's location. , means as follows:

[0090]

[0091] The above formula represents the first At that moment, the The coarse location point of the target, also known as the target Fusion measurement. For example... Figure 5 As shown in the figure, there are a total of 4 sensors. Any combination of two sensors can calculate 6 estimated target position points for any target. Using the same method, if there are... There are 10 associated matching results, that is, there are 100 in the space. If the fused measurements of several targets are obtained, then the coarse localization point set of the multiple targets can be represented as:

[0092]

[0093] like Figure 6 As shown, there are 4 targets in space. Six position estimation points can be calculated for any one target. By fusing the position estimation points of the same target, a coarse positioning point of a certain target can be calculated. Furthermore, the set of coarse positioning points of all four targets can be solved.

[0094] In step S4, a Gaussian mixture model is used to model the distribution characteristics of multi-target fusion measurement under associated uncertainty. The Gaussian mixture model parameters are solved by the expectation-maximization algorithm. Then, the optimization problem with nonlinear weighted sum is transformed into a maximum value problem by using the expectation-maximization method. After taking the negative logarithm, it is transformed into a minimum value problem, thus completing the solvable transformation of the fusion measurement modeling and optimization problem.

[0095] The coarse positioning point is obtained by fusing the position estimation points of the same target between multiple pairs of master and auxiliary sensors (i.e., master sensor and auxiliary sensor). The cross-positioning measurement of any pair of master and auxiliary sensors can be assumed to be a single Gaussian distribution. The coarse positioning point is formed by fusing multiple sets of dual-sensor cross-positioning measurements and has a complex nonlinear distribution. The distribution of the fused measurement characteristics is modeled using a Gaussian Mixture Model (GMM).

[0096] Let GMM be... The components are composed of elements, and each component follows a mean of 1. The variance is If the probability distribution is Gaussian, then its probability distribution can be expressed in the following form:

[0097]

[0098] in For the goal The observation error samples represent the fusion measurement. With state estimate The difference between them For the goal Observation error samples, For the parameters of each component, This represents the total number of Gaussian components in the GMM. Indicates the first The weight of each component in the overall GMM. For example... Figure 7 The diagram illustrates the construction of a multi-target fusion measurement Gaussian mixture model, which includes state prediction values ​​and fused measurement values ​​(coarse localization points of multiple targets). The observation error sequence is obtained by calculating the deviation between the current target state prediction value and the coarse localization point (fused measurement value). The structure of the three-dimensional Gaussian mixture model is then presented as a superposition of multiple Gaussian components. Each Gaussian component corresponds to a type of error distribution characteristic in the observation error sequence. Taking target 1 as an example, its observation error sequence is as follows: Based on this sequence, it is possible to construct .

[0099] When solving state estimation based on GMM observation noise, the objective function contains a nonlinear weighted sum, making it unsolvable using the traditional least squares method. Therefore, the Max-Mixture method is used to approximate the solution, transforming the nonlinear weighted sum in the optimization function into a maximum value problem, as shown in the following equation:

[0100]

[0101] in Indicates the first The square root information matrix of the Gaussian components. Under this condition, taking the negative logarithm of the optimization function, based on the monotonicity of the logarithmic function, transforms the maximum value problem into a minimum value problem:

[0102]

[0103] Under the fusion measurement conditions modeled by GMM, the state estimation problem of the target location point can be reformulated as:

[0104]

[0105] in Indicate space target exist The state estimate at time step [time]. In the above process, the parameters of the GMM are solved using the Expectation-Maximization (EM) algorithm. The observed variables in the EM algorithm [are...]. Latent variables and the parameters to be solved as follows:

[0106]

[0107] Assuming initial values ​​for the parameters as follows:

[0108]

[0109] in Given the initial value, the first The weight of each component in the overall GMM, This represents the mean under the initial conditions. This represents the variance under the initial conditions. Let represent the variances of the initial observation errors on the three-dimensional coordinate axes, respectively. First, the latent variable parameters are calculated through the E-step, i.e., the first step is calculated. The observation belongs to the GMM. The probability of Gaussian components As shown in the following formula:

[0110]

[0111] Secondly, the calculation is performed in M ​​steps, maximizing the log-likelihood function based on the latent variable parameters, updating the model parameters until convergence, as shown in the following equation:

[0112]

[0113] In step S5, a multi-target tracking factor graph optimization model is constructed, where variable nodes represent the position and velocity state of the target at each time step, and factor nodes include prior factors, motion model factors, observation factors, and mutually exclusive factors.

[0114] The factor graph consists of variable nodes representing state variables and factor nodes representing constraints and measurements. In the multi-target localization and tracking scenario presented in this paper, the variable nodes are the state variables to be optimized, representing the target's position and velocity state at each time step; the factor nodes contain prior knowledge, motion models, observation models, and mutual exclusion relationships between targets. Let... Time and space target The state variables are Define its three-dimensional observation variables as follows:

[0115]

[0116] In the factor nodes, prior factors are used to ensure the uniqueness of the solution to the optimization problem; state transition factors are used to describe the transition process of the target position between adjacent time steps, which can be understood as the target's motion model; observation factors are used to describe the sensor's observation error of the target; and mutual exclusion factors are used to effectively distinguish two targets that are close to each other. The four types of factor nodes are defined as follows:

[0117]

[0118] in, Indicates and Adjacent times Time of the first The state variables of the target express time The state variables of the target. For example... Figure 8 The diagram shown is an overall schematic of a passive detection multi-target tracking framework based on factor graphs. The diagram contains two targets, namely the target and the target. With the goal Prior factors Weak prior constraints are introduced based on the initial coarse positioning point of the sliding window; state transition factor. The key to achieving tracking smoothness and continuity is to minimize the distance between state variables at adjacent time points; observation factor The difference between the description state and coarse localization; mutual exclusion factor It helps to effectively distinguish between two targets that are close together, when the target and target When the distance between the two targets is less than the safe distance threshold, add this constraint to the state variable nodes corresponding to the two targets.

[0119] In step S6, multi-objective factor graph optimization state estimation is carried out. A nonlinear least squares optimization problem is constructed with the objective of minimizing the total error of the associated factor nodes of all objective state variables within the sliding window. The optimized estimation set of all objective states within the current sliding window is obtained by solving the problem.

[0120] To enhance robustness to fusion errors and false correlations, the observed factors are modeled using a Gaussian mixture model instead of a single Gaussian model, as follows:

[0121]

[0122] in, Let the observation error be in probabilistic form. For the first The variance s of the Gaussian components;

[0123] The aforementioned observation factors provide robust statistical constraints for the initial state nodes. Their mean is derived from the initial observation fusion results, and their covariance is determined by measurement noise.

[0124] for Momentary Goal The state estimation problem, i.e., solving for a state variable. This minimizes the total error of the error functions of all factor nodes connected to it. Therefore, the target localization problem based on factor graph optimization can be formulated as the following nonlinear least squares optimization problem:

[0125]

[0126] By solving this nonlinear least squares problem, we can obtain the optimal estimate set of all target states within the current sliding window. .like Figure 9 The diagram illustrates a multi-target state estimation process using factor graph optimization. It demonstrates the complete workflow for state estimation of multiple targets using passive sensors based on factor graph optimization. Using the factor graph as the core, variable nodes first represent the positional state of multiple targets at each time step. Then, factor nodes, composed of prior factors, motion model factors, observation factors, and mutually exclusive factors, constrain the correlation and motion patterns of the multi-target state variables. Simultaneously, a Gaussian mixture model parameter is solved using the residual between the state prediction and fused measurement values, based on the expectation-maximization algorithm, to accurately represent the non-Gaussian distribution characteristics of the fused measurement. Finally, a nonlinear least-squares optimization problem minimizing the total error of factor nodes within a sliding window is constructed to obtain the multi-target optimized state estimation result set. .

[0127] In step S7, trajectory association processing is performed. The Mahalanobis distance method is used to calculate the trajectory similarity weight between the current target optimization state and the next positioning observation, and a trajectory association similarity matrix is ​​constructed. The trajectory association result is obtained by solving the Hungarian algorithm.

[0128] The optimized state set With the next time-positioning observation set Perform trajectory association matching, where Using the Mahalanobis distance method, we define... Time of the first Each target state and Time of the first Trajectory similarity weights among target localization observations for:

[0129]

[0130] in It is a state The inverse covariance matrix. Based on this, we can construct... time Each target state and time Trajectory correlation similarity matrix between positioning observations .like Figure 10 The image shows a schematic diagram of trajectory correlation solving based on Mahalanobis distance and the Hungarian algorithm. Momentary Goal State estimate and Time of the first Target location observations If there are four targets, a trajectory association similarity matrix can be constructed. The trajectory association results are obtained by solving the following formula:

[0131]

[0132] in, for Momentary Goal State estimate and Time of the first Target location observations The correlation decision variables between them, when Indicates no association, when Indicates a relationship.

[0133] Finally, based on the trajectory association results, state updates and trajectory lifecycle management are completed. For successfully matched state variables and positioning observations, new observation factors are constructed and the corresponding state variable nodes are updated. For existing state variables that are continuously unmatched, their trajectory lifecycle is determined. If the lifecycle exceeds the threshold, tracking is terminated; otherwise, it is retained. For continuous unmatched positioning observations, they are initialized as new targets, and the factor graph structure is updated.

[0134] The aforementioned passive detection multi-target tracking method based on Gaussian mixture model and factor graph optimization achieves several advantages. Firstly, it accurately models the non-Gaussian distribution characteristics of fused measurements to ensure state estimation accuracy. By employing a Gaussian mixture model to model the non-Gaussian fused measurement distribution characteristics generated by multi-passive sensor fusion positioning, it can more accurately describe the actual observation noise characteristics under complex multi-source error coupling scenarios, thus significantly improving the accuracy and reliability of multi-target state estimation. Simultaneously, the expectation-maximization algorithm is used to optimize the parameters of the Gaussian mixture model, exhibiting good convergence and numerical stability, further ensuring the accuracy and stability of state estimation. Secondly, this method effectively solves the problem of ambiguity in correlation matching caused by the coupling of false points in multi-target cross-positioning and identity determination results, avoiding interference from false points in cross-positioning on the tracking algorithm, suppressing error propagation, and preventing track drift or target loss. Furthermore, a trajectory association and lifecycle management mechanism is designed, combining Mahalanobis distance and the Hungarian algorithm to improve the accuracy of target matching between adjacent time points. Dynamic trajectory lifecycle management enables refined control of existing target tracking maintenance, new target initialization, and termination of tracking for failed targets, maintaining stable tracking performance even in complex scenarios with varying sensor position errors and observation noise levels.

[0135] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0136] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A factor graph optimization passive detection multi-target tracking method based on Gaussian mixture model, characterized in that, include: Based on all passive sensors, a main sensor and an auxiliary sensor are defined. The origin of the coordinate system is the position of the main sensor at time zero in the Earth-centered Earth-fixed system, and a distributed passive sensor cooperative coordinate system is constructed. In a distributed passive sensor cooperative coordinate system, batch numbers are assigned to each target in space by combining the multi-target identity determination results between any master sensor and auxiliary sensor. Based on the batch number of each target, cross-location is performed using the direction finding lines of the same target on different passive sensors to obtain multiple position estimation points. The multiple position estimation points of the same target are then fused to obtain the coarse positioning point of the target at the current time. All two-dimensional observed targets within the main sensor are traversed to obtain the set of coarse positioning points of each target at the current time. A Gaussian mixture model is used to model the observation error distribution of the coarse location point set, and the expectation-maximization algorithm is used to solve the parameters of the Gaussian mixture model to update the parameters of the Gaussian mixture model until convergence. A multi-target tracking factor graph optimization model is constructed based on a convergent Gaussian mixture model. Based on the multi-target tracking factor graph optimization model, a nonlinear least squares optimization problem that minimizes the total error of the error function of all factor nodes is constructed and solved to obtain the optimized estimate set of all target states within the sliding window at the current time. The optimized estimation set of all target states within the sliding window at the current moment is correlated with the coarse positioning point set at the next moment, and the factor graph structure and target trajectory are updated based on the trajectory correlation results to complete multi-target tracking.

2. The factor graph optimization passive detection multi-target tracking method based on Gaussian mixture model according to claim 1, characterized in that, Based on all passive sensors, the steps of establishing a distributed passive sensor cooperative coordinate system, with the location of the main sensor at time zero (height 0) in the Earth-centered Earth-fixed system as the origin, include: Set one passive sensor as the main sensor and the rest of the passive sensors as auxiliary sensors; A distributed passive sensor cooperative coordinate system is constructed with the position point of the main sensor at time zero, where the height is 0, as the origin. The position information, attitude information, and target observation information of each auxiliary sensor are uniformly converted into the distributed passive sensor cooperative coordinate system.

3. The factor graph optimization passive detection multi-target tracking method based on Gaussian mixture model according to claim 2, characterized in that, In a distributed passive sensor cooperative coordinate system, the step of assigning batch numbers to various targets in space by combining the multi-target identity determination results between any primary and auxiliary sensors includes: The identity of multiple targets in the field of view of different passive sensors is determined, and the one-to-one correspondence between the two-dimensional observation targets of each passive sensor and multiple real targets in space is determined. Based on the multi-target identity determination results, the number of the multi-target number in the imaging plane of the main sensor is used as the basis, and the number is used as the unique batch number of the corresponding real target in space. For any auxiliary sensor, each target in its imaging plane is assigned the target number of the main sensor associated with it through identity determination, which serves as the batch number of the real spatial target represented by that target.

4. The factor graph optimization passive detection multi-target tracking method based on Gaussian mixture model according to claim 3, characterized in that, Based on the batch number of each target, cross-positioning is performed using the direction finding lines of the same target on different passive sensors to obtain multiple position estimation points. The multiple position estimation points of the same target are then fused to obtain the coarse positioning point of the target at the current moment. The step of traversing all two-dimensional observed targets within the main sensor to obtain the coarse localization set of each target at the current moment includes: By combining the pixel coordinates of multiple targets observed in the two-dimensional imaging plane of any passive sensor, the direction finding lines pointing from the optical centers of different passive sensors to all targets are calculated; By combining direction finding lines pointing to the same batch of targets in pairs, and using the least squares method to calculate the midpoint of the common perpendicular of any combination of direction finding lines, the estimated position of the same target can be obtained. Multiple estimated location points of the same target are fused to obtain the coarse location point of the target; Traverse all two-dimensional observation targets within the main sensor to obtain a coarse set of positioning points for multiple targets.

5. The factor graph optimization passive detection multi-target tracking method based on Gaussian mixture model according to claim 4, characterized in that, The process of modeling the observation error distribution of the coarse location point set using a Gaussian mixture model and solving for the parameters of the Gaussian mixture model using the expectation-maximization algorithm to update the Gaussian mixture model parameters until convergence includes: Calculate the observation error sequence based on the coarse location point set of multiple targets and the predicted target state values; A Gaussian mixture model is used to model the distribution characteristics of the observation error sequence, and the parameters of the Gaussian mixture model are solved by the expectation-maximization algorithm. The optimization problem involving nonlinear weighted sums is transformed into a maximum value problem using the expectation-maximization method. The problem is transformed into a minimization problem by taking the negative logarithm, in order to update the parameters of the Gaussian mixture model until convergence.

6. The factor graph optimization passive detection multi-target tracking method based on Gaussian mixture model according to claim 5, characterized in that, The steps for constructing a multi-objective tracking factor graph optimization model based on a convergent Gaussian mixture model include: Observation factors are constructed using a convergent Gaussian mixture model, and factor nodes are constructed by combining prior factors, motion model factors, and mutually exclusive factors. Construct a multi-objective tracking factor graph optimization model based on factor nodes.

7. The factor graph optimization passive detection multi-target tracking method based on Gaussian mixture model according to claim 6, characterized in that, The steps involved in constructing and solving a nonlinear least squares optimization problem that minimizes the total error of the error functions of all factor nodes, based on a multi-target tracking factor graph optimization model, to obtain the optimized estimate set of all target states within the sliding window at the current time, include: Based on the multi-objective tracking factor graph optimization model, we carry out multi-objective factor graph optimization state estimation. We sum the error functions of all factor nodes in the multi-objective tracking factor graph optimization model and construct a nonlinear least squares optimization problem with the goal of minimizing the total error of the factor nodes associated with all target state variables within the sliding window. Solving the nonlinear least squares optimization problem yields an optimized estimate set of all target states within the sliding window at the current time.

8. The factor graph optimization passive detection multi-target tracking method based on Gaussian mixture model according to claim 7, characterized in that, The steps involved in multi-target tracking include: associating the optimized estimate set of all target states within the current sliding window with the coarse localization point set for the next time step, and updating the factor graph structure and target trajectory based on the trajectory association results. Based on the optimized estimation set of all target states, the trajectory similarity weight between the current target optimized state and the next time positioning observation is calculated using the Mahalanobis distance method; Based on trajectory similarity weights, a trajectory association similarity matrix is ​​constructed, and the trajectory association results are obtained by solving the Hungarian algorithm.

9. The factor graph optimization passive detection multi-target tracking method based on Gaussian mixture model according to claim 8, characterized in that, The method also includes: Based on the trajectory association results, state updates and trajectory lifecycle management are completed. For successfully matched state variables and positioning observations, new observation factors are constructed and the corresponding state variable nodes are updated. For existing state variables that are continuously unmatched, determine their trajectory lifetime. If the lifetime exceeds the threshold, terminate the tracking; otherwise, continue tracking. For consecutive unmatched location observations, initialize them as new targets and update the multi-target tracking factor graph optimization model.

Images