Estimation method of whittle index based on double-channel neural network and use thereof

Abstract

The application discloses a kind of based on double channel neural network's Whittle index estimation method and its use, comprising: step 1: the partial observable Markov model of each target is established;Step 2: each target is regarded as an arm, and under the premise of subjecting to the partial observable Markov model of each target, the discounted cumulative reward of all targets is used as the optimization objective function to optimize the action decision of each target;Beam scheduling optimization problem is established, Lagrange relaxation is carried out on beam scheduling optimization problem, and a plurality of sub-problems are obtained;Step 4: the Whittle index estimation value of each target under different joint state is obtained by using the trained double channel neural network;The application estimates the Whittle index value of different multidimensional joint state according to the interaction with target tracking environment and the reward obtained, and performs weighted gradient ascent.

Description

Technical Field

[0001] This invention belongs to the field of radar network information fusion, and particularly relates to an estimation method for Whittle index based on a dual-channel neural network and its application. Background Technology

[0002] Multi-target tracking using co-located multiple-input multiple-output (MIMO) radar in multi-beam operation mode plays an important role in both military and civilian fields, and optimized scheduling of multi-beam resources can further enhance the potential of radar systems in improving multi-target tracking performance.

[0003] Beam resource scheduling is typically optimized for short-term tracking performance targets, such as one-step prediction of the Bayesian Cramer–Rao Lower Bound (BCRLB) or one-step prediction of the tracking error covariance matrix. This leads to a decrease and deterioration in long-term multi-target tracking performance.

[0004] For optimizing long-term goals, traditional dynamic programming algorithms and decision trees calculate multi-step decisions within a certain future horizon. However, the size of this decision space is exponential, inevitably leading to the curse of dimensionality. Therefore, the time scale of the future horizon is usually finite, which also reduces the optimization performance of long-term tracking metrics. Existing methods for long-term scheduling performance metrics, such as decision trees and branch and bound methods, model multiple objectives' POMDPs as a single POMDP. This significantly increases the scale of scheduling actions and the dimensionality of multi-objective states, limiting the ability to consider long-term performance. Summary of the Invention

[0005] The purpose of this invention is to provide a Whittle index estimation method based on a dual-channel neural network and its application, in order to solve the problem that due to the increase in the scale of actions and states, the time scale is often within 3 to 5 steps when considering long-term performance, and therefore the potential of long-term scheduling performance cannot be fully realized.

[0006] This invention employs the following technical solution: a method for estimating the Whittle index based on a dual-channel neural network, comprising:

[0007] Step 1: Establish a partially observable Markov model for each target based on its joint state, action space, target motion measurement space, target measurement likelihood, joint state transition function, and direct reward function; where the joint state includes the target's motion state and Bayesian Craméro state.

[0008] Step 2: Treat each objective as an arm, and optimize the action decision for each objective by using the discounted cumulative reward of all objectives as the objective function, under the premise of conforming to the partially observable Markov model of each objective;

[0009] Step 3: Based on the action decisions of each target, optimize the objective function, and the number of beam resources transmitted by the MIMO radar system. The number of targets establishes a beam scheduling optimization problem. Based on the Whittle index strategy, the beam scheduling optimization problem is subjected to Lagrange relaxation, resulting in multiple subproblems.

[0010] Step 4: Extract the features of the Bayesian Craméro state and motion state of each target, and input them into the dual-channel neural network of each target. Then, use the discount cumulative reward in the subproblem to train the dual-channel neural network; and use the trained dual-channel neural network to obtain the Whittle index estimate of each target in different joint states.

[0011] Furthermore, the motion state of the target in step 1 is estimated using an unscented Kalman filter.

[0012] Furthermore, the action space of step 1 is ; 0 indicates no target tracking, 1 indicates target tracking.

[0013] Furthermore, the direct reward function for step 1 is:

[0014]

[0015] In the formula, For the goal The state of motion, For the goal The position of the axis For the goal The speed of the shaft; For the goal The position of the axis For the goal The speed of the shaft, For the goal BCRLB state, This is the trace of the BCRLB matrix.

[0016] Furthermore, in step 4, the input to the first channel of the dual-channel neural network is a one-dimensional vector obtained by concatenating multiple eigenvalues ​​and eigenvectors calculated from the Bayesian Craméro matrix; the input to the second channel of the dual-channel neural network in step 4 is the target and radar system position. relative distance Azimuth sine value and azimuth cosine value .

[0017] Furthermore, the first channel of the dual-channel neural network consists of one input layer, five hidden layers, and one output layer connected in sequence;

[0018] The second channel of a dual-channel neural network consists of one input layer, two hidden layers, and one output layer connected in sequence.

[0019] The output channel of a dual-channel neural network is formed by splicing the output layers of the first and second channels, and then passing through one input layer, two hidden layers, and one output layer in sequence.

[0020] Furthermore, the method for training the dual-channel neural network in step 4 is as follows: compare the Whittle index estimate with the action loss factor, obtain the action at the current moment based on the comparison result, and train the dual-channel neural network using a deep reinforcement learning algorithm.

[0021] Among them, the motion loss factor In the formula, For any joint state of the objective; (·) is a function of the Whittle index estimate obtained through the current two-channel neural network;

[0022] The cross-entropy loss function is:

[0023] In the formula, For action decision-making;

[0024] in, ;

[0025] In the formula, This indicates that the two-channel neural network has certain network parameters. Next pair of joint states Estimated Whittle index; For the Sigmoid function;

[0026] Among them, cumulative discount rewards ,

[0027] In the formula, This is the discount factor.

[0028] A Whittle index estimation method based on a dual-channel neural network is used for radar beam scheduling. The joint state of each objective is input into the corresponding trained dual-channel neural network to obtain... The Whittle index estimates for each objective under different joint states are calculated, and the Whittle index estimates are sorted from largest to smallest. The top estimates are then selected. Track each target; among them... The number of beam resources transmitted by a MIMO radar system.

[0029] The beneficial effects of this invention are:

[0030] This invention utilizes the Whittle index strategy to schedule the multi-beams of MIMO radar. For multi-dimensional joint states and complex state transition processes, it uses a deep reinforcement learning algorithm to learn the Whittle index, thereby enhancing the long-term tracking performance of multiple targets.

[0031] This invention uses the Whittle index strategy as the main structure to decompose the multi-beam scheduling problem of co-located MIMO radar, thereby reducing the problems of dimensionality curse and high computational complexity caused by traditional dynamic programming algorithms when solving this problem;

[0032] This invention establishes a dual-channel neural network to comprehensively learn the features of BCRLB and dynamic states, which is beneficial for estimating the Whittle index value of the multidimensional joint state; it uses a deep reinforcement learning algorithm for training and obtains reward and gradient values ​​through interaction with the target tracking environment to update the network parameters.

[0033] This invention learns the features of two network channels, adaptively adjusts the weights between different channels, captures the correlation and feature importance between BCRLB states and motion states; and performs weighted gradient ascent based on the interaction with the target tracking environment and the rewards obtained, to estimate the Whittle index values ​​of different multidimensional joint states.

[0034] This invention decomposes the original problem, which significantly reduces the size of the actions and states of the subproblems. Therefore, when calculating long-term performance, it can increase the range of future time and enhance long-term scheduling performance. Attached Figure Description

[0035] Figure 1 This is a model diagram of the scheduling problem in this invention;

[0036] Figure 2 This is a diagram of the dual-channel neural network structure of the present invention;

[0037] Figure 3 This is a diagram of the deep reinforcement learning structure of the present invention;

[0038] Figure 4 Analysis of Whittle index characteristics for this invention;

[0039] Figure 5 This invention provides a multi-target real-time trajectory;

[0040] Figure 6 This refers to the performance of WINN;

[0041] Figure 7 This refers to the performance of Myopic;

[0042] Figure 8 This refers to the performance of REINFORCE;

[0043] Figure 9 This refers to the performance of AQL;

[0044] Figure 10 This is the cumulative reward result for the discount in this invention. Detailed Implementation

[0045] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0046] This invention discloses a method for estimating the Whittle index based on a two-channel neural network, including...

[0047] Step 1: Establish a partially observable Markov model for each target based on its joint state, action space, target motion measurement space, target measurement likelihood, joint state transition function, and direct reward function; wherein, the joint state includes the target's motion state and Bayesian Craméro state.

[0048] First, the Bayesian Cramer–Rao Lower Bound (BCRLB) and the motion state are defined as the joint state for each objective. Then, for each objective, a Partially Observable Markov Decision Process (POMDP) ​​model is established.

[0049] Assuming in At any time, in the target tracking scenario there exists One goal, using Indicates the index identifier of the target. (For) The motion state and BCRLB state of the target at each time step are modeled. Let Indicate target The state of motion, its For the goal The position of the axis For the goal The speed of the shaft; For the goal The position of the axis For the goal The velocity of the axis. This motion state can be estimated using an unscented Kalman filter (UKF), let... Indicates the target The estimated state of motion; let Indicate target The BCRLB state represents the target. exist The lower bound of the mean square error at time t. Let and They represent in Time, Goal The joint state of the target and the set of joint states of all targets.

[0050] Therefore, for each target A 6-tuple POMDP model can be built. They represent the joint state and the action space, respectively. The joint state transition function, target motion measurement space, target measurement likelihood distribution, and direct reward function.

[0051] In this model, the target In the next moment joint state Depends on the current moment joint state and actions .make and This indicates the estimated motion state and the BCRLB state in action. The transfer process below. Among them, The transfer process is based on the UKF method, through measurement Estimate the motion state at the next moment This will not be elaborated upon here; The transition process is recursively updated using Bayes' theorem. The following describes the action... and The following is the BCRLB state transition process:

[0052]

[0053]

[0054] in, This indicates the estimation of motion state. Jacobian matrix of the measurement function.

[0055] Both of the above state transition processes include motion state transition matrices in the motion model and measurement model. Process noise covariance matrix Nonlinear measurement function Measurement noise covariance matrix Information. Order Indicate target Relative to the location of the radar system Spatial distance and azimuth measurement information. Let Indicate target At the present moment The direct reward function, where, This represents the trace of the BCRLB matrix. It is worth noting that, in order to improve tracking accuracy, this invention aims to maximize the sum of rewards for multiple targets.

[0056] Step 2: Treat each objective as an arm, and under the premise of conforming to the partially observable Markov model of each objective, optimize the action decision of each objective using the discounted cumulative reward of all objectives as the optimization objective function.

[0057] Treating each target as an arm, the beam scheduling optimization problem is expressed mathematically.

[0058] Each objective is treated as an arm, conforming to the POMDP model established in step 1. A multi-objective discount cumulative reward is set over an infinite time period. To optimize the objective function. Wherein, This represents the discount factor.

[0059] Assuming the MIMO radar system is located in Every moment MIMO radar systems can only transmit Each beam has one resource. Each beam can only track one target. Initialization. The joint state set of the objectives The beam scheduling optimization problem is then established as follows:

[0060]

[0061] In this problem, the MIMO radar system is allocated at each time step. Each beam resource in One goal, Each objective involves state transitions and the acquisition of direct rewards, ultimately maximizing... The reward for each objective aims to improve tracking performance, and the scheduling problem model is as follows: Figure 1 As shown.

[0062] Step 3: Based on the action decisions of each target, the objective function, the number of beam resources transmitted by the MIMO radar system, and the number of targets, establish a beam scheduling optimization problem. Apply Lagrangian relaxation to the beam scheduling optimization problem using the Whittle index strategy to obtain multiple subproblems.

[0063] The original constraint is extended to [the following] in infinite time: Define Lagrange multipliers The original scheduling optimization problem can be transformed into an unconstrained problem after Lagrange relaxation, as shown below:

[0064]

[0065] After omitting the constant term on the rightmost side of the above equation, we can obtain There are several subproblems. Each subproblem can be represented as:

[0066]

[0067] In the Whittle indexing strategy, this Lagrange multiplier It is considered a motion loss factor.

[0068] Step 4: Extract the features of the Bayesian Craméro state and motion state of each target, and input them into the dual-channel neural network of each target. Then, use the discount cumulative reward in the subproblem to train the dual-channel neural network. Use the trained dual-channel neural network to obtain the Whittle index estimate of each target in different joint states. When multiple targets have the same motion model, they can share the same dual-channel neural network.

[0069] First, a two-channel neural network is established, with the input of each channel being the preprocessed state features. The input features of the first channel include multiple feature values ​​and multiple feature vectors; the input features of the second channel include the relative distance, and the sine and cosine values ​​of the azimuth angle.

[0070] Feature extraction: Extract target features separately Joint state BCRLB status and motion state The characteristics of. Among them, Features include: the BCRLB matrix eigenvalues ​​and There are feature vectors, each with a dimension of 4. The size of the dimension representing the motion state; Features include: location relative to radar system relative distance azimuth sine value and azimuth cosine value .

[0071] A dual-channel neural network is constructed, consisting of multiple fully connected layers. The first network channel is oriented towards the BCRLB state, and the input layer is configured to contain... The first network channel consists of 128 neurons in each hidden layer and 32 neurons in each output layer. The second channel, designed for motion states, has an input layer with 3 neurons, connected to 2 hidden layers and 1 output layer. The hidden layers have 64 neurons each, and the output layer has 2 neurons. Finally, the output channel concatenates the output layers from the first two channels, connecting to 1 input layer, 2 hidden layers, and 1 output layer. The number of neurons in each layer is 34, 32, 16, and 1, respectively. This generates an estimated real Whittle index. .in, This represents the weights and biases in the dual-channel neural network. Except for the input layer of the two channels and the output layer of the output channel, all other fully connected layers use the ReLU activation function. It is particularly noteworthy that the number of hidden layers and the corresponding number of neurons in the two channels and the output channel can be flexibly set and may be fine-tuned based on performance requirements according to different application scenarios. The dual-channel neural network structure of this invention is as follows: Figure 2 As shown.

[0072] like Figure 3 As shown, based on the Deep Reinforcement Learning (DRL) algorithm, an action loss factor is preset in the reward function. By utilizing the threshold characteristic of the Whittle indexing strategy, the estimated Whittle index value is compared with the action loss factor, and the action at the current moment is sampled.

[0073] Utilizing the threshold feature in the Whittle indexing strategy, namely: when The optimal action should be 1; conversely, the optimal action should be 0. Therefore, in this invention, before inputting into the neural network, This is the initial joint state of the objective, used for its state transitions; For any joint state of the objective; (·) is a function representing the Whittle index estimate obtained through the current two-channel neural network. Furthermore, it sets the motion loss factor. Among them, neural network parameters These are the parameters of the neural network for the current segment.

[0074] At any moment Calculate the difference This is then input into the Sigmoid function. The action is then calculated. sampling probability Using this sampling probability, the neural network samples the time step. action .

[0075] Based on the Sigmoid function result and action results Calculate time Cross-entropy loss:

[0076] when At that time, its calculation result will be in Within the interval. Otherwise, the calculation result is within... Within the range.

[0077] This invention utilizes mini-batch training to calculate gradients and discounted rewards over a large number of segments. Weighted gradients are then used to perform gradient ascent on the network parameters, completing the training of a two-channel neural network.

[0078] A training dataset is constructed, and a positive semi-definite symmetric BCRLB matrix is ​​obtained by multiplying a random matrix by its transpose. The random matrix has a dimension of 4×4, and each element follows a uniform distribution. Simultaneously, within a spatial distance of 200km and a speed range of -100m / s to 100m / s, the target's motion state is generated. The target is assumed to undergo uniform linear motion, and a motion state transition matrix is ​​provided. Process noise covariance matrix Nonlinear measurement function :

[0079]

[0080] Measurement noise covariance matrix .in, This represents the target tracking time interval. It's worth noting the Jacobian matrix. It can be represented as:

[0081]

[0082] Define a batch containing There are 1 segment, and the duration of each segment is 1. Initialize before the start of each segment. Then, at each time step... Iterative execution, while simultaneously utilizing backpropagation to compute segments. Mid-moment gradient .

[0083] After each segment is trained, the segment is calculated. gradient and and cumulative discount rewards .

[0084] Execution is performed iteratively within a batch, thereby calculating the weighted gradient:

[0085]

[0086] At the end of the batch, the weighted gradient is used. Neural network parameters Perform gradient ascent. Let Let the learning rate be the parameter update process, as shown below:

[0087]

[0088] Multiple batches are set up, and the neural network is trained in each batch. During training, the parameters of all neural networks are... Both updates are performed through backpropagation and gradient ascent. This invention utilizes the Adam optimizer to perform gradient ascent.

[0089] This invention also discloses a Whittle index estimation method based on a dual-channel neural network for radar beam scheduling, which... The joint state of each objective is input into the corresponding trained dual-channel neural network to obtain... The Whittle index estimates for each objective under different joint states are calculated, and the Whittle index estimates are sorted from largest to smallest. The top estimates are then selected. Track each target; among them... The number of beam resources transmitted by a MIMO radar system.

[0090] Example 1

[0091] Training set and test set:

[0092] For the training and test sets, the MIMO radar positions are set. The positive semi-definite symmetric BCRLB matrix is ​​obtained by multiplying the random matrix with its transpose. The random matrix has a dimension of 4×4, where each element follows a uniform distribution. Simultaneously, the distance range along the x-axis is -100km to 100km, and the distance range along the y-axis is 0km to 100km, with the velocities along both the x and y axes ranging from -100m / s to 100m / s. The target's motion state is generated within this range. By combining these two factors, a large number of different combined target states are obtained.

[0093] Simulation scenario:

[0094] Co-located MIMO radar can transmit at every moment Each beam resource is designed for tracking Each of the flying targets moves at a constant velocity in a two-dimensional plane. Figure 5 It displays the realistic tracks of multiple targets. The initial BCRLB state and initial real motion state of each target are shown. As shown in Table 1. Additionally, the initial motion states... With real motion state The variance between them is the same as the variance represented by the initialized BCRLB. The initial estimated error covariance matrix in UKF Same as the BCRLB matrix.

[0095] Table 1 Initial states of multiple objectives

[0096]

[0097] Algorithm parameters:

[0098] For model parameters, set a discount factor. Motion state transition matrix Process noise covariance matrix Nonlinear measurement function Jacobian matrix Measurement noise covariance matrix .

[0099] For the neural network structure, the fully connected layers of the first channel, the second channel, and the output channel, and the number of neurons in each layer are shown in Table 2.

[0100] Table 2 Neural Network Structure

[0101]

[0102] For deep reinforcement learning algorithms, set the batch size. and segment duration Initialize the learning rate. The parameters of the dual-channel neural network are decayed at a rate of 0.9 every 500 segments. Before training, the parameters are randomly initialized. The entire training process consisted of 10,000 segments.

[0103] Performance evaluation:

[0104] Three performance metrics were selected to evaluate the algorithm, including:

[0105] 1) Average selected ratio: Calculated by statistically analyzing the average selected ratio at each time step during multiple Monte Carlo tests. In this context, the frequency at which each target is selected is indicated. This metric reflects the allocation of beam resources.

[0106] 2) Square root of BCRLB trace: For each target's BCRLB state at each time step, the square root of the trace is calculated. This metric reflects the change in the direct BCRLB reward for each target during the tracking process.

[0107] 3) Root Mean Square Error (RMSE) of Target Motion State: The root mean square error between the estimated motion state and the actual motion state, calculated using the following formula:

[0108]

[0109] in, This indicates the number of Monte Carlo simulations. This metric reflects the change in tracking accuracy for each target during the tracking process.

[0110] 4) Discounted cumulative reward: This metric directly reflects the algorithm's optimization performance on the objective function. The performance metric value is... The average of the Monte Carlo tests.

[0111] The algorithm in this embodiment is named WINN, and it is compared with three other algorithms:

[0112] 1) Myopic, or short-sighted strategy: every moment Calculate the one-step predicted BCRLB state under the non-tracking action, and select the previous state accordingly. The target with the largest BCRLB trace.

[0113] 2) REINFORCE: A deep reinforcement learning algorithm based on policy gradient iteration.

[0114] 3) AQL, or Amortized Q-learning, is a model-free deep reinforcement learning algorithm.

[0115] The test results are as follows:

[0116] First, the estimated Whittle index characteristics are analyzed under different BCRLB states and dynamic states. The starting positions of the four targets are set as shown in Table 1. The subsequent target positions vary from index 0 to 19, as follows: Figure 4 As shown in (a). For four objectives, while keeping the BCRLB matrix state unchanged, it is... .from Figure 4 As seen in the Whittle index changes in (b), the Whittle index value increases with the slant range between the target and the MIMO radar station. This is because the greater the target distance, the greater the measurement error in slant range, leading to faster BCRLB deterioration. Therefore, the target will require radar beam tracking more urgently, resulting in a larger Whittle index value. Furthermore, assuming the target position remains constant, as shown in Table 1, the subsequent BCRLB trace gradually decreases, as... Figure 4 As shown in (c), as the BCRLB decreases, the urgency of beam tracking required also decreases, and the Whittle index naturally decreases, as... Figure 4 As shown in (d). In summary, the estimation of the Whittle index value in this invention conforms to practical engineering application experience and can comprehensively consider BCRLB and dynamic state.

[0117] Secondly Figures 6-9 The performance results of the four algorithms are presented in turn. In each figure, (a) to (c) represent three performance metrics: average selection rate, square root of BCRLB trace, and RMSE, respectively. Figure 10 The cumulative reward results for discounts from four algorithms are shown.

[0118] like Figure 7 As shown, Myopic greedily selects the target with the largest BCRLB trace and only considers one future tracking time interval. Because target 2 is farthest from the radar station and target 4 is closest, under the nonlinear measurement equation, target 2's BCRLB state deteriorates more, while target 4's deteriorates the least. The radar system allocates the most beam resources to target 2 and the least to target 4, ultimately making the tracking performance of the four targets converge. Figure 8 and Figure 9 The study shows that the beam resource allocation rate for each target is approximately between 10% and 30%, indicating that REINFORCE and AQL tend to employ a more balanced scheduling strategy. However, this results in a failure to rapidly reduce the BCRLB of distant targets in the early stages of tracking, thus deteriorating the tracking performance for distant targets. WINN combines the advantages of rapidly reducing the BCRLB of distant targets among the four targets in the early stages of tracking with an adaptively balanced scheduling strategy in the later stages of tracking, maintaining the tracking performance of all four targets within a high range. Figure 6 As shown. Additionally, via Figures 6-9 As shown in (c), REINFORCE and AQL ultimately worsened the RMSE of target 2; while Myopic stabilized the RMSE of all four targets at around 30, WINN stabilized the RMSE of targets 1, 3, and 4 at around 20, and the RMSE of target 2 at around 30. Therefore, WINN outperformed the other three algorithms in tracking performance across all targets.

[0119] Discount cumulative reward performance metrics such as Figure 10 As shown in Table 3, the reward values ​​and improvement rates of the four algorithms at fragment 10000 are presented. It can be seen that WINN achieves the highest cumulative discount reward value after fragment 4100 and gradually converges and stabilizes. Compared to the other three algorithms, WINN improves the cumulative discount reward value by at least 1.96%.

[0120] Table 3. Cumulative Discount Rewards at Segment 10000

[0121]

[0122] In summary, WINN outperforms the other three comparison algorithms across all metrics. This superior performance can be attributed to the Whittle index strategy's decomposition of the original scheduling problem, which allows for faster and more stable training and convergence of the neural network through deep reinforcement learning. Simultaneously, the dual-channel neural network can comprehensively learn the features of the joint state, improving the accuracy of Whittle index estimation. These results demonstrate that WINN is superior to Myopic, REINFORCE, and AQL in overall performance regarding MIMO radar beam scheduling and multi-target tracking.

[0123] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for estimating Whittle index based on a two-channel neural network, characterized in that, include: Step 1: Establish a partially observable Markov model for each target based on its joint state, action space, target motion measurement space, target measurement likelihood, joint state transition function, and direct reward function; wherein, the joint state includes the target's motion state and Bayesian Craméro state; Step 2: Treat each objective as an arm, and optimize the action decision for each objective by using the discounted cumulative reward of all objectives as the objective function, under the premise of conforming to the partially observable Markov model of each objective; Step 3: establishing a beam scheduling optimization problem according to the action decision of each target, the optimization target function, and the number of beam resources transmitted by the MIMO radar system , the number of targets, obtaining a plurality of sub-problems by performing Lagrange relaxation on the beam scheduling optimization problem according to a Whittle index strategy; Step 4: Extract the features of the Bayesian Cramer-Roman state and motion state of each target, and input them into the dual-channel neural network of each target. Then, use the discount cumulative reward in the sub-problem to train the dual-channel neural network; and use the trained dual-channel neural network to obtain the Whittle index estimate of each target under different joint states. The input of the first channel of the double-channel neural network in step 4 is a one-dimensional vector obtained by splicing a plurality of characteristic values and a plurality of characteristic vectors calculated by the Bayesian Kramers matrix; the input of the second channel of the double-channel neural network in step 4 is the relative distance , the sine value of the azimuth angle , the cosine value of the azimuth angle , and the cosine value of the azimuth angle of the target and the radar system position . ​ The first channel of the dual-channel neural network consists of one input layer, five hidden layers, and one output layer connected in sequence. The second channel of the dual-channel neural network consists of one input layer, two hidden layers, and one output layer connected in sequence. The output channel of the dual-channel neural network is formed by splicing the output layers of the first and second channels, and then passing through one input layer, two hidden layers, and one output layer in sequence.

2. The method for estimating the Whittle index based on a dual-channel neural network according to claim 1, characterized in that, The motion state of the target in step 1 is estimated using an unscented Kalman filter.

3. The method for estimating the Whittle index based on a dual-channel neural network according to claim 1, and its application, is characterized in that... The motion space of step 1 is ; 0 indicates no target tracking, 1 indicates target tracking.

4. The method for estimating the Whittle index based on a dual-channel neural network according to claim 1, characterized in that, The direct reward function for step 1 is: ； In the formula, For the goal The state of motion, For the goal The position of the axis For the goal The speed of the shaft; For the goal The position of the axis For the goal The speed of the shaft, For the goal BCRLB state, This is the trace of the BCRLB matrix.

5. The method for estimating the Whittle index based on a dual-channel neural network according to claim 1, characterized in that, The method for training the dual-channel neural network in step 4 is as follows: compare the Whittle index estimate with the action loss factor, obtain the action at the current moment based on the comparison result, and train the dual-channel neural network using a deep reinforcement learning algorithm. Among them, the motion loss factor In the formula, For any joint state of the objective; (·) is a function of the Whittle index estimate obtained through the current two-channel neural network; The cross-entropy loss function is: ； In the formula, For action decision-making; in, ； In the formula, This indicates that the two-channel neural network has certain network parameters. Next pair of joint states The estimated Whittle index value; For the Sigmoid function; Among them, cumulative discount rewards , In the formula, This is the discount factor.

6. A Whittle index estimation method based on a dual-channel neural network for radar beam scheduling, characterized in that... Will The joint state of each objective is input into the corresponding trained dual-channel neural network to obtain... The Whittle index estimates for each objective under different joint states are calculated, and the Whittle index estimates are sorted from largest to smallest. The top estimates are then selected. Track each target; among them... The number of beam resources transmitted by a MIMO radar system.

Images