A space target state estimation method based on collaborative adaptive structure IMM

Abstract

The embodiment of the application provides a space target state estimation method based on a cooperative adaptive variable structure IMM, comprising the following steps: calling a pre-trained deep Q network model, selecting an effective model set from a preset basic model set and a candidate model set, and constructing a variable structure interactive multiple model according to the effective model set; determining a state estimation value of the space target by measuring data of the space target and the variable structure interactive multiple model. The application proposes a cooperative adaptive variable structure interactive multiple model algorithm fusing deep reinforcement learning and variational Bayesian inference, realizes intelligent model set adaptation through DQN, solves the non-conjugate problem by using KLA approximation, improves the robustness of the state estimation of the jump Markov system to model mismatch and time-varying noise, and effectively improves the state estimation precision of the space target.

Description

Technical Field

[0001] This invention relates to the field of navigation technology, and more particularly to a method for estimating the state of a space target, and more specifically, to a method for estimating the state of a space target based on a cooperative adaptive variable structure IMM. Background Technology

[0002] Jumping Markov Systems (JMS), as a class of dynamic systems with implicit mode transition characteristics, exhibit both Markovian and continuous dynamic states in their state evolution. They are widely used in complex scenarios with uncertain motion patterns, such as target tracking and autonomous navigation. High-precision JMS state estimation is a core foundation for autonomous intelligent decision-making in fields such as target tracking, space exploration, and strategic decision-making. Multi-model estimation methods, by running a pre-set set of models in parallel and fusing conditional estimates from each model, achieve global state inference and have become a typical paradigm for solving joint state-mode estimation in JMS.

[0003] Multi-model estimation methods have evolved through three generations of theoretical systems: Autonomous Multi-Model (AMM), Interactive Multi-Model (IMM), and Variable Structure Interactive Multi-Model (VSIMM). Among them, AMM and IMM algorithms are based on the assumption of a fixed model set. Their inherent defects expose these methods to the risk of degradation or even divergence in estimation accuracy due to model set mismatch (model redundancy or missing models).

[0004] VSIMM constructs a time-varying estimation space by dynamically adjusting the model set, theoretically overcoming the limitations of pre-defined model sets. However, the Model Set Adaptation (MSA) mechanism has become a key bottleneck restricting its practical engineering application. Existing MSA methods, such as model set switching, likelihood model sets, adaptive meshes, expected model extensions, and improved methods like hybrid meshes and centroid model sets (IMM), can achieve local pattern coverage through parameterized variants, but are limited by structural homogeneity constraints and are only applicable to scenarios with parameter differences under the same structural model. Therefore, constructing an intelligent model set adaptation mechanism with autonomous pattern inference capabilities has become a key path to overcome the theoretical bottleneck of VSIMM.

[0005] Furthermore, time-varying noise is another important factor affecting VSIMM performance. Adaptive filtering methods for time-varying noise can be categorized into covariance matching, correlation, maximum likelihood, and Bayesian methods. Bayesian methods include state augmentation-based methods, multi-model methods, particle methods, and VB inference (VBI) methods, which jointly estimate the system state and measurement noise covariance matrix (MNCM) by selecting appropriate prior conjugate distributions. However, these methods are all limited to IMM. When VB methods are directly applied to VSIMM, the time-varying nature of the model set leads to the model interaction and the resulting mixture distribution not satisfying conjugacy, i.e., the mixture distribution differs from the joint posterior probability density (PDF) form, making it difficult to guarantee the mathematical closure of parameter updates.

[0006] In summary, existing VSIMMs still face challenges: MSA relies on model homogeneity constraints and heuristic rules, lacking in-depth mining of historical decision information and quantitative evaluation of long-term benefits; traditional VBI leads to the non-conjugate nature of VSIMMs. Therefore, it is necessary to develop VSIMMs based on intelligent MSA (or cooperative adaptive variable structure IMMs) under unknown SNCM and MNCM conditions to improve the robustness of state estimation of jumping Markov systems to model mismatch and time-varying noise, thereby effectively improving the state estimation accuracy of space targets. Summary of the Invention

[0007] This invention provides a space target state estimation method based on cooperative adaptive variable structure IMM, which can improve the state estimation accuracy of space targets.

[0008] To achieve the above objectives, embodiments of the present invention provide a spatial target state estimation method based on a collaborative adaptive variable structure IMM, comprising: calling a pre-trained deep Q-network model, selecting an effective model set from a preset basic model set and a candidate model set, and constructing a variable structure interactive multi-model IMM based on the effective model set; and determining the state estimate of the spatial target using measurement data of the spatial target and the variable structure IMM.

[0009] Furthermore, by using measurement data of the space target and the variable structure IMM, the state estimate of the space target is determined, including: performing the following iterative steps on each model in the variable structure IMM: performing model interaction on the model at the current time; performing model filtering on the model interaction results; performing model probability update; and ending the above iterative steps when the last model completes the model probability update.

[0010] The results of model filtering and model probability update are weighted and fused; the state estimate of the space target is determined from the weighted fusion result.

[0011] Furthermore, model interaction is performed on the current time-instance model, specifically including: calculating the model prediction probability based on the model probability and model transition probability matrix of the previous time-instance model; deriving the model mixture probability based on the model prediction probability; using the model mixture probability, interactively mixing the optimal posterior state of the previous time-instance model with the optimal posterior error covariance of the previous time-instance model to obtain the mixture state and mixture error covariance; obtaining the mixture degree of freedom parameters through the optimal posterior degree of freedom parameters of the previous time-instance model and the model mixture probability; obtaining the mixture inverse scaling matrix through the optimal posterior inverse scaling matrix of the previous time-instance model and the model mixture probability, wherein...

[0012] Furthermore, model filtering is performed on the results of model interaction, specifically including: obtaining the predicted degree of freedom parameters based on the mixed degree of freedom parameters; obtaining the predicted inverse scale matrix based on the mixed inverse scale matrix; determining the predicted state of the model at the current time based on the mixed state and the model input of the model at the current time; and determining the predicted error covariance of the model at the current time based on the mixed error covariance and the fused state noise covariance matrix.

[0013] Set the initial value of the iteration number t Then, the following iterative process will begin:

[0014] Calculate the state noise transition matrix using the following formulas. and measurement noise transition matrix :

[0015] ,

[0016] ,

[0017] in, For measurement function exist The Jacobian matrix at point, where State transition function exist Jacobian matrix at the location, The measured value at the current moment. It is in a mixed state. For mixed error covariance, , The number of models in the variable structure IMM;

[0018] The posterior degrees of freedom are obtained from the predicted degrees of freedom parameters. Then, based on the posterior degrees of freedom parameters, the predicted inverse scaling matrix, and the state noise transition matrix... and measurement noise transition matrix The posterior inverse scaling matrix is ​​obtained, where the posterior degrees of freedom parameters include the posterior state noise degrees of freedom parameters. and posterior measurement of noise degrees of freedom parameters The posterior inverse scaling matrix includes the posterior state noise inverse scaling matrix. and posterior measurement noise inverse scaling matrix ;

[0019] The posterior prediction error covariance is obtained using the following formula. :

[0020] ;

[0021] By fusing the measurement noise covariance matrix, the posterior measurement noise covariance is obtained using the following formula. :

[0022] ;

[0023] Based on the posterior prediction error covariance and posterior measurement noise covariance Determine the corrected gain Then, based on the corrected gain The posterior states were obtained respectively. and posterior error covariance ,in, Expressing expectations;

[0024] Update iteration count ;

[0025] when When the maximum number of iterations C is reached, the above iteration process is repeated.

[0026] The posterior state obtained from the last iteration As the optimal posterior state of the model at the current moment, the posterior error covariance obtained from the last iteration is used. The optimal posterior error covariance of the model at the current time is obtained by using the posterior degrees of freedom parameters obtained in the last iteration as the optimal posterior degrees of freedom parameters of the model at the current time, and the optimal posterior inverse scaling matrix obtained in the last iteration as the optimal posterior inverse scaling matrix of the model at the current time.

[0027] Furthermore, model probability updates are performed, specifically including: obtaining the residual covariance based on the posterior prediction error covariance and the posterior measurement noise covariance after each iteration; and then, updating the residual covariance based on the current measurement value. The corrected model likelihood value is obtained by calculating the residual covariance, and then the model probability at the current time step is determined by the corrected model likelihood value. .

[0028] Furthermore, the results of model filtering and model probability updates are weighted and fused, specifically including:

[0029] The optimal posterior state of each model and optimal posterior error covariance respectively with the corresponding model probabilities Perform weighted fusion and calculate the corrected fusion state using the following formula. and corrected fusion error covariance :

[0030] ;

[0031] By using the optimal posterior degrees of freedom parameters and corresponding model probabilities of each model The fusion degree-of-freedom parameters are obtained, including the fusion state noise degree-of-freedom parameters. and fusion measurement of noise degree of freedom parameters ; using the optimal posterior inverse scaling matrix of each model and the corresponding model probability The fused inverse scaling matrix is ​​obtained, which includes the fused state noise inverse scaling matrix. and fusion measurement noise inverse scaling matrix ;

[0032] The fusion state noise covariance matrix is ​​updated using the following formula. and fusion measurement noise covariance matrix :

[0033] ,

[0034] .

[0035] Furthermore, the state estimate of the space target is determined from the weighted fusion result, specifically: using the corrected fused state... As a state estimate of a space target.

[0036] Furthermore, the spatial target state estimation method based on the collaborative adaptive variable structure IMM also includes: calculating the likelihood value of the effective model set and the model selection stability value; determining the reward function based on the likelihood value of the effective model set and the model selection stability value, and updating the pre-trained deep Q network model through the reward function.

[0037] The above technical solution has the following beneficial effects:

[0038] To address the challenges of state estimation in JMS systems, specifically MSA and time-varying noise estimation, this invention proposes a collaborative adaptive variable structure IMM that integrates deep reinforcement learning and variational Bayesian inference. To achieve intelligent MSA with autonomous pattern reasoning capabilities, a weighted average of model set likelihood and model stability is used as the reward function of the DQN. Offline training with a dual-network structure and an experience replay mechanism allows the Q-network to approximate the true cumulative reward function of the model selection problem. Online MSA is then achieved through model decision-making based on DQN, overcoming the dependence of traditional methods on model structure homogeneity and prior knowledge. Simultaneously, under time-varying unknown noise, the inverse Wishart distribution is used as the conjugate prior of the noise covariance. Addressing the non-conjugate nature of model interaction and fusion estimation, a KLA-based VB-VSIMM is proposed to achieve joint posterior estimation of system state, SNCM, and MNCM. This solves the problem of mathematical closure deficiency in VSIMM in traditional variational Bayesian methods, improving the robustness of state estimation for jumping Markov systems to model mismatch and time-varying noise, and effectively enhancing the accuracy of state estimation for spatial targets. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart of a spatial target state estimation method based on cooperative adaptive variable structure IMM according to the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] like Figure 1 As shown, this embodiment of the invention provides a spatial target state estimation method based on cooperative adaptive variable structure IMM, including:

[0043] S101. Call the pre-trained deep Q-network model, select the effective model set from the preset basic model set and candidate model set, and construct the variable structure IMM based on the effective model set;

[0044] S102. The state estimate of the space target is determined by the measurement data of the space target and the variable structure IMM.

[0045] The following detailed explanation of the derivation process of the aforementioned method and formula, as well as its simulation application, is provided through a specific embodiment:

[0046] 1) Variable structure interactive multi-model

[0047] 1.1 Precise Bayesian Filtering

[0048] Consider discrete stochastic nonlinear jumping Markov systems (JMS).

[0049]

[0050] Among them, time, Let be the system state vector. For the system measurement vector, For the system's true model, For pattern space; and These are the state noise and the measurement noise, respectively. The estimation problem for the above system is given by... System status at all times and pattern The optimal estimate under a given optimization criterion. Common estimation criteria include Minimum Mean Square Error (MMSE) and Maximum Likelihood (ML).

[0051] The multi-model approach constructs an approximate model set in the mode space, thereby enabling the system to... Transform into:

[0052]

[0053] in express Real-time system mode The estimation model, express The model set constructed at time step 1 is time-varying in the variable structure multi-model method. The initial time step 2... The prior probabilities of each model are The transformation between models satisfies a first-order homogeneous Markov chain, the th The model is converted to the first... The probability of each model is Obviously, ,in for The modulus, i.e., the number of models. All transition probabilities form the transition probability matrix (TPM). .

[0054] Under the meaning of minimum mean square error The optimal estimate of the system state and mode at time t is:

[0055]

[0056] in Expressing expectations, , These represent the model set sequence and the measurement sequence, respectively.

[0057] ,according to Time model probability With TPM, you can get Predicted probability :

[0058]

[0059] The PDF of the model's state after interaction is as follows:

[0060]

[0061] according to Time model State estimates under and error covariance We can obtain the state estimates and error covariance of each model after interactive mixing:

[0062]

[0063] in for The time-matter model is hour, The time-matter model is The probability,

[0064]

[0065] The state prediction probability density function can be expressed by the Chapman-Kolmogorov equation as follows:

[0066]

[0067] This allows us to obtain the posterior probability density function after conditional filtering of the model:

[0068]

[0069] The state estimate after conditional filtering of the model is The error covariance is .

[0070] Time model The update probability is:

[0071]

[0072] in, As the normalization factor, For the model The likelihood of the multi-model fusion posterior probability density function can be obtained from the model conditional posterior PDF and the update probability:

[0073]

[0074] From the formula , and The IMM fusion state estimate and error covariance can be obtained as follows:

[0075]

[0076] 1.2 Estimation performance analysis under model mismatch:

[0077] like Model sets built at different times With real mode If they match, it is called the optimal model set, denoted as . Number of models The multi-model MMSE estimation under the optimal model set is denoted as:

[0078]

[0079] Without loss of generality, consider any MMSE estimation under time-model mismatch conditions ,Right now

[0080]

[0081] definition and The distance between them is the variance between them, which is the conditional expectation:

[0082]

[0083] remember for and The set of public models, i.e. ,make

[0084]

[0085] Right now The set of redundant and missing models is called the mismatch model set. Let be an example. In the model set at all times and The model update probabilities under the given conditions are as follows:

[0086]

[0087] but , by formula , Time model The predicted probability depends on Time model set and From the middle model to The transition probabilities of the model, therefore

[0088]

[0089] From the formula and ,

[0090]

[0091] Right now , It is a constant. From the above equation, we know that when... hour, ;when hour, .

[0092] time The equivalent MMSE estimate under the model set is ,Right now

[0093]

[0094] Then regarding the estimated value under model mismatch conditions The performance is governed by the following theorem.

[0095] Theorem 1: Under the condition of model mismatch and under the condition of the optimal model set Distance and equivalent estimation under mismatched model set conditions and The distance is directly proportional to the distance, and the proportional relationship is:

[0096]

[0097] Proof: For equivalent estimation ,

[0098]

[0099] The second and third steps can be derived from the formula for calculating the update probability. Therefore, steps three and four can be derived from the formula. We obtain the estimated values ​​under the model mismatch condition. ,

[0100]

[0101] therefore,

[0102]

[0103] This leads to the formula .

[0104] Note: Theorem 1 shows that the performance of variable structure interactive multi-model estimation is determined by the redundancy and missing models in the model set constructed at each time step relative to the optimal model set. Too many or too few models will reduce the estimation accuracy. Therefore, constructing a model set that matches the target true pattern space is the key to improving estimation performance.

[0105] 2) Model set adaptation based on Deep Q Network

[0106] As the analysis above shows, variable structure interactive multi-model estimation is optimal only when the model set at each time step matches the target real pattern, and the key to achieving this goal lies in model set adaptation.

[0107] To design the MSA framework based on deep reinforcement learning, we first denote the set of all possible models for the target as follows: To satisfy the pattern space as much as possible The coverage is divided into two parts: the basic model set. and candidate model set ,in A small set consisting of necessary models, used to ensure that... Basic coverage. The set of models used for state estimation at time step 1 is called the effective model set, and is:

[0108]

[0109] in To be based on a certain criterion from The optimal model set is selected. DQN is used to select the optimal model set. This avoids dependence on real-world model information, and The number of models is not fixed, but rather an optimal number. By designing the DQN state space and reward function, the optimal model set at each time step is selected, improving the adaptability and robustness of the model set.

[0110] 2.1 Deep Q Network

[0111] DQN utilizes deep neural networks to approximate the mapping from an agent's states and actions to a value function—the Q-function—thereby learning the optimal policy that maximizes the expected cumulative reward. DQN models the environment as a Markov Decision Process (MDP), represented as... ,in For the state space of DQN, For the action space, Represents the state transition probability. As a discount factor, This is the reward function. It's important to note the states in DQN. This represents the current state of the agent that needs to make a decision, distinct from the system state in JMS. MDP strategy Indicates the action to be taken in the current state. The probability of this is given, and the goal of DQN is to learn the optimal policy.

[0112]

[0113] in Let be the parameters of the neural network (called a Q-network), and its training loss function be:

[0114]

[0115] in The output of the target network, another neural network with the same architecture in DQN, is defined as...

[0116]

[0117] in These are the parameters of the target network. The parameters remain unchanged in each episode and are set to the Q network parameters after the episode ends to achieve stable updates of the Q network.

[0118] 2.2 Effective Model Set Based on Deep Q Network

[0119] The JMS model selection problem based on DQN is as follows: Given Time-effective model set Basic state estimates of each model and model probability Using DQN from Select And then from Build Time-effective model set. Model selection for each time step is performed using an offline-trained DQN.

[0120] During the offline training process of DQN, each iteration starts from... Select a model from the options and iterate through multiple iterations to obtain... .

[0121] To distinguish between the iteration steps of DQN and the timing of JMS, superscripts can be used. Indicates the former, subscript This refers to the latter. The model selection form is the same at different times, with... Taking the model selection problem at a given time as an example, this explains the state, reward, and training process of DQN.

[0122] The states of a DQN need to contain enough information to accurately fit the value function of the adaptive decision-making model set. Therefore, the state of a DQN is defined as follows:

[0123]

[0124] in express In the model selection problem at time 1 The state of each iteration step for Time of the first The model set for each iteration step, the initial model set. . for Time measurement. For for The estimated value of the system's basic state obtained from the effective model set at time t is calculated using the following formula:

[0125]

[0126] In the model selection problem, define the action for each iteration. That is, from Choose one model Join Formation in ,Right now

[0127]

[0128] reward function Reflects the state Take a certain action The reward obtained. Considering that the actual pattern of the system does not change frequently, and the current pattern is often the same as the previous pattern, the reward function is defined by taking into account the residuals of the model set and the stability of model selection:

[0129]

[0130] in For model set The likelihood is calculated as follows:

[0131]

[0132] in , For the model The likelihood, for The predicted probability. These are the weighting coefficients. Reflecting the stability of model selection and avoiding frequent model switching, it is defined as:

[0133]

[0134] when When this occurs, it indicates that the model has switched, so the aforementioned penalty term is added to the reward function.

[0135] DQN uses an offline training method, and the training data is based on the system model set. This dataset includes multiple target trajectories and corresponding measurement data generated by a sensor measurement model simulation. The model set covers various motion modes with parameter perturbations. To bridge the gap between simulation and reality, unmodeled maneuvers and time-varying sensor noise can be injected during training. Each trajectory and its measurement data are denoted as... , This represents the number of trajectory sampling points.

[0136] The training process of DQN is shown in Algorithm 1 in Table 1.

[0137]

[0138] Table 1. Offline training process of Algorithm 1—DQN

[0139] Note: Experience replay memory The samples in the sample follow the first-in, first-out (FIFO) principle. This indicates that each sampling point can select at most Each model is added to the basic model set. In the middle. Used during training. Greedy strategies select actions to ensure sample diversity. Hours There are enough training samples. Set the desired number of samples. hour, This indicates that the entire Joined At this point, there are no further cumulative rewards. The significance of the target network is that the goal of the Q-network is fixed at each sampling point to achieve stable updates.

[0140] The trained DQN selects models online in the variable structure interactive multi-model algorithm to achieve model set adaptation. The online adaptation process of the model set is shown in Algorithm 2 in Table 2.

[0141]

[0142] Table 2. Algorithm 2—Online Adaptation of Model Sets Based on DQN

[0143] Note: Algorithm 2 uses probability. Updating DQN online can improve its generalization ability.

[0144] 3) Conjugate approximation of VB-VSIMM under time-varying noise

[0145] Here, we propose an approximation method based on weighted KL mean, which approximates the mixture distribution as a product of individual distributions, making them conjugate. This allows us to update the estimated state, the joint posterior PDF of the SNCM and MNCM through the parameters of the distributions. First, we theoretically derive the non-conjugate property of VB-VSIMM, and then present the proposed VB-VSIMM based on weighted KL mean.

[0146] 3.1 Non-conjugation of VB-VSIMM

[0147] In VSIMM, the model The underlying fundamental state PDF follows a Gaussian distribution, i.e.

[0148]

[0149] SNCM and MNCM are the covariance matrices of a Gaussian distribution. The inverse Wishart (IW) distribution is typically chosen to model their conjugate prior probability density.

[0150]

[0151]

[0152] in

[0153]

[0154] For dimension is A symmetric positive definite matrix, and These are the degrees of freedom parameters and the inverse scaling matrix, respectively. and These represent determinant and trace operations, respectively. Its conjugacy guarantees that its posterior PDF is also an IW distribution. The goal is to estimate the posterior PDF given the basic state, SNCM, and MNCM at the previous time step. time The joint posterior PDF. However, for VB-VSIMM, the following theorem holds.

[0155] Theorem 2: For JMS In VB-VSIMM, model interaction and estimation fusion do not satisfy conjugacy.

[0156] Proof: Under a single model, existing variational Bayesian methods have proven that the above distribution is a conjugate prior distribution, guaranteeing that the posterior PDF is identical in form to the prior PDF. Furthermore, based on conditional independence, Time model The joint posterior PDF under the given conditions can be decomposed into a product of posterior PDFs, i.e.

[0157]

[0158] VB-VSIMM first requires model interaction based on the joint posterior PDF under the single model described above, according to Bayes' theorem and the law of total probability.

[0159]

[0160] in As shown in the formula As shown. From the formula The joint distribution after model interaction differs from the joint posterior PDF, making it impossible to obtain the joint PDF through parameter updates, and consequently, also impossible to obtain... The predictive PDF and the posterior PDF at each moment do not satisfy conjugacy. Similarly, the estimation fusion involving a weighted fusion of the joint posterior PDFs of multiple models also does not satisfy conjugacy.

[0161] To address the aforementioned issues, a weighted KLA-based VB-VSIMM is proposed. By using weighted KLA, the joint PDF is approximated as a product of individual distributions, ensuring conjugacy. The specific forms of the joint PDF of model interaction, the joint prediction PDF and posterior PDF of model conditional filtering, and the joint posterior PDF of estimation fusion are derived.

[0162] 3.2 VB-VSIMM based on weighted KLA

[0163] Using the weighted KLA method, the model interaction is improved. The joint PDF and the previous time-instance model The sum of the weighted KL divergences of the joint posterior PDF is minimized, i.e.

[0164]

[0165] in

[0166]

[0167] for and The KL divergence measures the difference between two distributions. Then, regarding the equation... The following theorem holds.

[0168] Theorem 3: The joint PDF after model interaction can be approximated as:

[0169]

[0170] The formulas for calculating the degrees of freedom parameter and inverse scaling matrix of the IW distribution are as follows:

[0171]

[0172] and and The update differs from the moment matching method in VSIMM. , revised to

[0173]

[0174] The time-joint prediction PDF is

[0175]

[0176] Among them, the prediction PDF of SNCM Prediction PDF with MNCM It is unknown; a heuristic approach is used, through the forgetting factor. ( )calculate Time-ahead PDF and The degree of freedom parameters and inverse scaling matrix in the figure:

[0177]

[0178] The prediction PDF of the basic state follows a Gaussian distribution, i.e. ,in

[0179]

[0180] in State transition function exist Jacobian matrix at the location,

[0181]

[0182] Comprehensive - , The time-joint prediction PDF can be represented as

[0183]

[0184] Next, we calculate the joint posterior PDF. The variational Bayesian method transforms it into the computation of a decomposable approximate posterior distribution, i.e.

[0185]

[0186] in It is a model Download Post-Version PDF An approximation. Its solution criterion is to minimize the KL divergence, i.e.

[0187]

[0188] Its optimal solution satisfies:

[0189]

[0190] in for Any element in , To and Relevant constants.

[0191] Since the three approximate posterior distributions are coupled, a fixed-point iterative method is used to estimate them.

[0192] Then the model Next The approximate posterior distributions of the nth iteration are as follows

[0193]

[0194]

[0195]

[0196] The update equations for the parameters are as follows:

[0197]

[0198]

[0199]

[0200] in For measurement function exist Jacobian matrix at the location, , , , See Algorithm 3 for the specific form.

[0201] Fixed-point iteration After this, the approximate posterior distribution is updated to...

[0202]

[0203] The process of updating the model probability is the same as that of VSIMM, as shown in the equation. As shown, the difference is that the model likelihood is corrected as follows:

[0204]

[0205] in,

[0206]

[0207] During the estimation fusion, the joint posterior PDF is:

[0208]

[0209] As can be seen from the above equation, the fused joint posterior PDF also does not satisfy conjugacy. Similarly, using the weighted KLA method, we can obtain...

[0210]

[0211] Note this. and The calculation and the moment matching method in VSIMM Different, corrected to:

[0212]

[0213] The degrees of freedom parameters and inverse scaling matrix of SNCM and MNCM are as follows:

[0214]

[0215] The estimates of SNCM and MNCM are the expectations of the IW distribution, i.e.:

[0216]

[0217]

[0218] The specific implementation process of combining the proposed DQN-based model set adaptation with variational Bayes-based SNCM and MNCM adaptation is shown in Algorithm 3 in Table 3.

[0219]

[0220]

[0221] Table 3. Detailed process of the cooperative adaptive variable structure IMM of the present invention

[0222] Note: Among them Indicates the first The expected operation of the next iteration. Regarding the choice of initial parameters...

[0223]

[0224] This makes , For the first The initial SNCM of the model, This represents the number of models in the initial model set. Similarly, for the initial parameters of MNCM,

[0225]

[0226] Simulation and analysis:

[0227] The effectiveness of the proposed SA-VSIMM is verified through a multi-sensor spatial target tracking problem in a 3D scene. The target alternates between uniform acceleration (CA), cooperative right turn (CRT), cooperative left turn (CLT), and uniform velocity (CV) motions, experiencing ten time-varying motion stages with different model parameters and noise parameters. The sensors measure the relative distance between the target and itself. The state estimation results of VSIMM based on optimal model augmentation (BMA), VSIMM with fixed centroid (CFIMM), multi-model recognition expectation estimation (IEE) based on VBI, VSIMM based on expectation model augmentation and VBI (EVA), and the proposed SA-VSIMM are compared. Among them, BMA and CFIMM use a variable structure model set, IEE uses the VB method to estimate the state and mode, while EVA combines a variable structure model set and a VB-based MNCM noise adaptation method.

[0228] Number of sensors The sampling frequency is 100Hz, and the motion time is... The exercise mode switches every 5 seconds. Target number 1 The SNCM and MNCM of each sampling point are respectively

[0229]

[0230] in The number of sampling points for each motion phase, This represents a stage of the movement. This represents the total number of sampling points. For the first Initial SNCM of the phase, For the initial MNCM, It is an 8th-order identity matrix.

[0231] The initial model sets for all algorithms, as well as the basic model sets in BMA and SA-VSIMM, include CA, CRT, CLT, and CV models, with noise levels set to 0. This means it does not include all ten real-world patterns. During DQN training, the training data is generated as described above, with a reward discount factor... Q network hidden layer node count , Experience replay memory capacity The tuning parameters for VB are: Forgetting factor Number of iterations .

[0232] To evaluate the accuracy of target state estimation, the number of Monte Carlo simulations for each algorithm was compared. The root mean square error (RMSE) and The mean root mean square error (ARMSE) within the range.

[0233]

[0234] Table 4. ARMSE for estimating position and velocity

[0235] Analysis shows that the proposed SA-VSIMM has a lower RMSE than other methods. Especially in transitional sections where motion patterns change, traditional methods experience a rapid increase in localization errors in all directions due to model set and noise mismatch. SA-VSIMM, however, autonomously selects the optimal model set matching the target's motion pattern through a dynamic policy network and adaptively estimates noise using VB, effectively suppressing peak errors and quickly converging to a stable value. Its performance in the stable motion phase of the target is also superior to other algorithms. Table 4 shows that SA-VSIMM's RMSE is also significantly lower than other algorithms.

[0236] To evaluate and compare the computational efficiency of each algorithm, the average single-step computation time of each algorithm was calculated, as shown in Table 5. The hardware environment consisted of an Intel Core i9-12900H processor with a main frequency of 2.50GHz.

[0237]

[0238] Table 5 Algorithm single-step computation time

[0239] As shown in Table 5, IEE is the most efficient due to its fixed model set, while BMA has a higher computational complexity. CFIMM uses a predefined centroid model set, resulting in relatively low computational complexity. The proposed SA-VSIMM uses a pre-trained DQN for intelligent model set adaptation, achieving slightly lower computational efficiency than EVA, but significantly improving estimation accuracy without affecting the algorithm's real-time performance.

[0240] To examine the accuracy of target motion pattern estimation, the estimated probabilities of each algorithm for each motion model were compared. The comparison showed that SA-VSIMM was more accurate in estimating model probabilities than other methods and responded more quickly to motion pattern transitions.

[0241] To evaluate the number of hidden layer nodes in the Q network Optimization parameters and forgetting factor Impact on algorithm performance, Below settings as well as ,exist Below settings as well as ARMSE for position and velocity and By comparison, we can see that: different and The algorithm exhibits high accuracy and relatively consistent estimation performance. Q-networks with too few hidden layer nodes lack the ability to fit the adaptive value function of the model set, while too many hidden layer nodes lead to overfitting; both result in a decline in estimation performance. From the equation... and , The prior strength of the IW distribution's degrees of freedom parameters affects the algorithm's ARMSE; increasing this strength leads to an increase in the algorithm's ARMSE. (From the equation...) , As the noise inertia increases, its adaptability to time-varying noise weakens. This indicates that SNCM and MNCM are constant, therefore the performance degradation is more significant.

[0242] In summary, in state estimation of complex dynamic systems, the proposed SA-VSIMM integrates adaptive statistical inference noise into the intelligent decision-making framework, which can improve the adaptive decision-making capability of the model set and its robustness to inaccurate noise, thereby improving the accuracy of state estimation. Although the current experiments are based on simulation data, theoretical analysis shows that the intelligent decision-making of the model set and the real-time update mechanism of noise statistical characteristics have inherent robustness to model mismatch and inaccurate prior information in real positioning scenarios. Furthermore, the algorithm parameters have clear physical meaning. In the deployment of practical intelligent decision-making systems, It depends on the state and output of DQN. and The optimal value of the algorithm balances the stability and adaptability of the system. This provides theoretical support for the transferability of the algorithm to practical systems. Future work will focus on verifying the performance of this method on measured data, especially exploring its adaptive capabilities under noise coupling from multi-source heterogeneous sensors.

[0243] The disclosed embodiments have been described above to enable any person skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the spirit and scope of this disclosure. Therefore, this disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed herein.

[0244] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A spatial target state estimation method based on cooperative adaptive variable structure IMM, characterized in that, include: A pre-trained deep Q-network model is invoked to select an effective model set from a preset basic model set and candidate model set, and a variable structure interactive multi-model (IMM) is constructed based on the effective model set. The state estimate of the space target is determined by measuring data of the space target and the variable structure IMM.

2. The spatial target state estimation method based on cooperative adaptive variable structure IMM as described in claim 1, characterized in that, The process of determining the state estimate of the space target using measurement data of the space target and the variable structure IMM includes: For each model in the variable structure IMM, perform the following iterative steps: Perform model interaction on the model at the current time; Model filtering is applied to the results of model interactions. Perform model probability updates; Once the last model completes its probability update, the above iteration steps end. The results of model filtering and model probability update are weighted and fused together; The state estimate of the space target is determined from the weighted fusion result.

3. The spatial target state estimation method based on cooperative adaptive variable structure IMM as described in claim 2, characterized in that, The model interaction with the current time-instance model specifically includes: Calculate the model prediction probability based on the model probability and model transition probability matrix of the previous time step; Based on the predicted probabilities of the model, the mixed probabilities of the model are derived; Using the model mixture probability, the optimal posterior state of the model at the previous time step is interactively mixed with the optimal posterior error covariance of the model at the previous time step to obtain the mixed state and the mixed error covariance. The mixture degree of freedom parameters are obtained by using the optimal posterior degree of freedom parameters of the model at the previous time step and the mixture probability of the model. The mixture inverse scaling matrix is ​​obtained by using the optimal posterior inverse scaling matrix of the model at the previous time step and the model mixture probability, where...

4. The spatial target state estimation method based on cooperative adaptive variable structure IMM as described in claim 3, characterized in that, The process of performing model filtering on the results of model interaction specifically includes: Based on the aforementioned mixed degree-of-freedom parameters, the predicted degree-of-freedom parameters are obtained; Based on the hybrid inverse scaling matrix, the prediction inverse scaling matrix is ​​obtained; Based on the mixed state and the model input of the model at the current time, the predicted state of the model at the current time is determined; Based on the hybrid error covariance and the fused state noise covariance matrix, the prediction error covariance of the model at the current time is determined; Set the initial value of the iteration number t Then, the following iterative process will begin: Calculate the state noise transition matrix using the following formulas. and measurement noise transition matrix : ， ， in, For measurement function exist The Jacobian matrix at point, where State transition function exist Jacobian matrix at the location, The measured value at the current moment. It is in a mixed state. For mixed error covariance, , The number of models in the variable structure IMM; Based on the predicted degrees of freedom parameters, the posterior degrees of freedom parameters are obtained. Then, based on the posterior degrees of freedom parameters, the predicted inverse scaling matrix, and the state noise transition matrix... and measurement noise transition matrix The posterior inverse scaling matrix is ​​obtained, wherein the posterior degrees of freedom parameters include the posterior state noise degrees of freedom parameters. and posterior measurement of noise degrees of freedom parameters The posterior inverse scaling matrix includes the posterior state noise inverse scaling matrix. and posterior measurement noise inverse scaling matrix ; The posterior prediction error covariance is obtained using the following formula. : ； By fusing the measurement noise covariance matrix, the posterior measurement noise covariance is obtained using the following formula. : ； Based on the posterior prediction error covariance and the posterior measurement noise covariance Determine the corrected gain Then, based on the corrected gain... The posterior states were obtained respectively. and posterior error covariance ,in, Expressing expectations; Update iteration count ; when When the maximum number of iterations C is reached, the above iteration process is repeated. The posterior state obtained from the last iteration As the optimal posterior state of the model at the current moment, the posterior error covariance obtained from the last iteration is used. The optimal posterior error covariance of the model at the current time is obtained by using the posterior degrees of freedom parameters obtained in the last iteration as the optimal posterior degrees of freedom parameters of the model at the current time, and the optimal posterior inverse scaling matrix obtained in the last iteration as the optimal posterior inverse scaling matrix of the model at the current time.

5. The spatial target state estimation method based on cooperative adaptive variable structure IMM as described in claim 4, characterized in that, The process of updating the model probability specifically includes: The residual covariance is obtained based on the posterior prediction error covariance and the posterior measurement noise covariance after each iteration. Then, based on the measurement value at the current time... The corrected model likelihood value is obtained by calculating the residual covariance, and then the model probability at the current time is determined by the corrected model likelihood value. .

6. The spatial target state estimation method based on cooperative adaptive variable structure IMM as described in claim 5, characterized in that, The weighted fusion of the model filtering result and the model probability update result specifically includes: The optimal posterior state of each model and optimal posterior error covariance respectively with the corresponding model probabilities Perform weighted fusion and calculate the corrected fusion state using the following formula. and corrected fusion error covariance : ； By using the optimal posterior degrees of freedom parameters and corresponding model probabilities of each model The fusion degree-of-freedom parameters are obtained, including the fusion state noise degree-of-freedom parameters. and fusion measurement of noise degree of freedom parameters ; using the optimal posterior inverse scaling matrix of each model and the corresponding model probability The fused inverse scaling matrix is ​​obtained, which includes the fused state noise inverse scaling matrix. and fusion measurement noise inverse scaling matrix ; The fusion state noise covariance matrix is ​​updated using the following formula. and fusion measurement noise covariance matrix : ， 。 7. The spatial target state estimation method based on cooperative adaptive variable structure IMM as described in claim 6, characterized in that, The determination of the state estimate of the space target from the weighted fusion result specifically involves: With the modified fusion state This serves as a state estimate of the space target.

8. The spatial target state estimation method based on cooperative adaptive variable structure IMM as described in claim 6, characterized in that, Also includes: Calculate the likelihood value of the effective model set and the numerical stability of model selection; The reward function is determined based on the likelihood value of the effective model set and the model selection stability value, and the pre-trained deep Q-network model is updated using the reward function.

Images