Intelligent track initiation method based on dual-channel depth Q learning model

Abstract

The invention provides an intelligent track initiation method based on a dual-channel depth Q learning model, and the method comprises the steps: setting logic method M and N parameter initial values for track initiation, carrying out the track initiation, and determining a region-of-interest sequence; standardizing the radar video region-of-interest signal sequence; constructing a dual-channel MN parameter optimization neural network model; training a dual-channel MN parameter optimization neural network model by using a deep Q learning method; and obtaining optimized M and N parameters by utilizing the optimized quantity of the obtained M and N parameters after training, and starting the track by utilizing the optimized M and N parameters by using a logic method. The features of the radar video signal and the track initiation information are synchronously extracted by using the deep neural network, the information of the radar signal is fully utilized, and the track initiation performance is effectively improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of track initiation, and particularly relates to an intelligent track initiation method based on a double-channel deep Q learning model. BACKGROUND

[0002] Track initiation is an important process in radar target tracking and is the first step of discovering and tracking targets. Accurate track initiation helps to discover targets in time and reduce the interference of false targets. Existing track initiation methods are divided into sequential processing methods and batch processing methods. The batch processing method has a large amount of calculation and a long initiation cycle, and has poor performance for targets that need to be initiated quickly. The sequential processing method includes intuitive method, logical method and modified logical method, and is suitable for track initiation in a weak clutter background. However, it is easy to initiate too many false tracks in a strong clutter background. These track initiation methods only use the distance, azimuth, speed and other information contained in the plot, and the utilization rate of information is limited, resulting in a decline in track initiation performance in a strong clutter background. SUMMARY

[0003] The application provides an intelligent track initiation method based on a double-channel deep Q learning model.

[0004] The technical scheme for achieving the application is as follows: an intelligent track initiation method based on a double-channel deep Q learning model, comprising the following steps:

[0005] Step 1: setting initial values of M and N parameters of a logical method for track initiation, initiating a track, performing sliding window processing on each temporary track, extracting track points, extracting a signal region around a target from a radar video according to distance, azimuth and time information of the track points, taking the signal region as a sequence of regions of interest, and saving track initiation state parameters of the current temporary track;

[0006] Step 2: standardizing the signal sequence of the region of interest in the radar video;

[0007] Step 3: constructing a double-channel MN parameter optimization neural network model, wherein the double-channel MN parameter optimization neural network model comprises a radar video channel, a track initiation state parameter channel and a parameter optimization module, the radar video channel is used to extract video features according to the input region of interest, the track initiation state parameter channel is used to extract state features according to the track initiation state parameters, and the video features and the state features are fused and input into the parameter optimization module to obtain M and N parameter optimization results;

[0008] Step 4: training the double-channel MN parameter optimization neural network model by using a deep Q learning method;

[0009] Step 5, set initial M, N parameters for each temporary track respectively in the track initiation stage, record and save the association success or failure after each plot association, and input the data corresponding to the last K track points into the trained double-channel MN parameter optimization neural network model, obtain the optimization amount of M, N parameters, and thus obtain the optimized M, N parameters, and use the optimized M, N parameters to perform track initiation using the logical method.

[0010] Compared with the prior art, the present application has the following advantages:

[0011] The present application introduces a double-channel deep Q learning model method to dynamically optimize the track initiation parameters M and N, which can effectively improve the initiation speed of real targets and suppress the initiation of false targets; the present application uses a deep neural network to simultaneously extract the features of radar video signals and track initiation information, fully utilizes the information of radar signals, and effectively improves the track initiation performance.

[0012] The present application will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 It is an intelligent track initiation method flow chart based on a double-channel deep Q learning model.

[0014] Figure 2 It is a schematic diagram of a neural network model with two input channels. DETAILED DESCRIPTION

[0015] An intelligent track initiation method based on a double-channel deep Q learning model, referring to Figure 1 , the method comprises the following steps:

[0016] Step 1: preset the M, N parameters of the logical method initiation, perform track initiation, and perform sliding window processing on each temporary track with a window length of K, extract the last K track points P=(p 1 ,p 2 ,…,p K ), wherein p i =(r i ,a i ), r i represents the distance unit of the i th track point, and a i represents the azimuth unit of the i th track point. Extract K circles of radar video signals S=(S 1 ,S 2 ,…,S K ) corresponding to the last K track points. Determine the center point distance unit and azimuth unit

[0017]

[0018] with the center point distance cell and azimuth cell as the center, K circles of video signals in the radar video signal corresponding to K track points in the vicinity of ±L1 distance cells and ±L2 azimuth cells are extracted as the region of interest signal s=(s 1 ,s 2 ,…,s K ). The track starting state parameters C=(c1, c2, M, N) of the current temporary track are saved, wherein c1 represents the number of successful associations of the temporary track in the starting process, and c2 represents the total number of successful associations of the temporary track. The logic method M and N parameters used for track starting specifically correspond to the number of successful associations and the number of associations of the temporary track when the logic method track starting is used.

[0019] Step 2: The K region of interest signals s=(s 1 ,s 2 ,…,s K ) corresponding to the K track points are respectively normalized, and are normalized to a region range with a mean value of 0 and a variance of 1:

[0020]

[0021] wherein s represents the normalized region of interest signal.

[0022] Step 3: A dual-channel MN parameter optimization neural network model is constructed in combination with the radar video region of interest sequence and the track starting state parameters. The dual-channel MN parameter optimization neural network model includes a radar video channel, a track starting state parameter channel, and a parameter optimization module. The dual-channel MN parameter optimization neural network model outputs the change amount of parameters M and N.

[0023] The radar video channel is a 3D convolution residual network, and the input thereof is the region of interest signal (s 1 ,s 2 ,…,s K ), and video features are extracted. The track starting state parameter channel is a fully connected neural network, and the input thereof is the track starting state parameter C=(c1, c2, M, N), and state features are extracted. The video features and the state features are fused, and are input to the parameter optimization module constructed by the fully connected neural network, and the M and N parameter optimization results are output.

[0024] For example, refer to Figure 2This diagram illustrates a neural network model with two input channels. The radar video channel is constructed using 3D convolutional layers and contains three residual blocks. The track initial state parameter channel is a fully connected neural network containing three fully connected layers, each followed by a BatchNorm layer and a ReLU layer. The outputs of the radar video channel and the track initial state parameter channel are concatenated and fused, and then input into a parameter optimization module containing two fully connected layers. The first fully connected layer is followed by a BatchNorm layer, a ReLU layer, and a Dropout layer. The output of the second fully connected layer has 49 units and can be reconstructed into a 7x7 matrix. The number of rows in the matrix represents the variation of the track initial parameter M from -3 to +3, and the number of columns represents the variation of the track initial parameter N from -3 to +3. The variation values ​​of parameters M and N are... c N c Subtract 4 from the row / column number; for example, the first row and second column correspond to M. c =-3, N c =-2 means subtracting parameter M by 3 and parameter N by 2. The actions corresponding to the row and column with the maximum value in the 7x7 matrix are selected to optimize parameters M and N.

[0025] Step 4: Train a dual-channel neural network model using the deep Q-learning method. The steps are as follows:

[0026] Step 4.1. Deep Q-learning initialization. The state of the deep Q-learning method is represented by the standardized region of interest sequence and the initial state parameters of the track, and the action space is the result of MN parameter optimization.

[0027] Step 4.2. Data Input Stage. After each point association, the data corresponding to the last K track points of each temporary track are input into the dual-channel MN parameter optimization neural network model. The dual-channel MN parameter optimization neural network model outputs the changes M and N. c and N c And according to M c and N c Update parameters M and N:

[0028] M t =M t-1 +M c

[0029] N t =N t-1 +N c

[0030] If M t >N t Let N t =M t Ensure N t Greater than or equal to M t If Mt <1 Let M = 1, to ensure the correctness of the rules.

[0031] Step 4.3. Reward calculation phase: for real target track, if M decreases and N increases, reward the model, if M increases and N decreases, punish the model, set the initial reward function as:

[0032] reward true = -c1*M c + c2*N c

[0033] where c1 and c2 are the importance weights of M c and N c in the reward function. According to the updated M and N parameters, determine whether the temporary track is started or suppressed, if the track is started, give an additional reward V1, let if the track is suppressed, give an additional penalty V2, let If the track is not started or suppressed, let Put the current sample, action, reward and the sample corresponding to the next round of data and the track start and end flags into the experience replay pool; for false target track, if M decreases and N increases, punish the model, if M increases and N decreases, reward the model, set the reward function as:

[0034] reward false = c1*M c - c2*N c

[0035] If the track is started, give an additional penalty V3, let if the track is suppressed, give an additional reward V4, let If the track is not started or suppressed, let Put the current sample, action, reward and the sample corresponding to the next round of data and the track start and end flags into the experience replay pool.

[0036] Step 4.4. Use Q-learning algorithm and its variants, etc. Reinforcement learning models suitable for discrete action space to train neural network model. After a certain amount of data is stored in the experience replay pool, samples are extracted from the experience replay pool to train the model. For each sample in a batch used to train the model, first calculate the Q value of the corresponding action with the current state data of a sample, then calculate the target of the current sample according to the Q value and reward value of the next round of state data, use the MSE loss to evaluate the difference between the Q value of the current round of data and the target, which is used to train the model. Use optimizers such as Adam to optimize model parameters. Train the model according to the above rules until the model converges.

[0037] Step 5: In practical application, initial M, N parameters are set for each temporary track respectively in track initiation stage, record and save the success or failure of each track association, input the data corresponding to the last K track points into the model, modify the M, N parameters according to the M, N values output by the model, and judge whether the temporary track is successful more than or equal to M times in the last N associations under the new M, N parameters. If yes, initiate the track. If the failure times are more than N-M times in the last N associations, delete the track. Otherwise, maintain the current temporary track to continue initiation. c c Step 5: In practical application, initial M, N parameters are set for each temporary track respectively in track initiation stage, record and save the success or failure of each track association, input the data corresponding to the last K track points into the model, modify the M, N parameters according to the M, N values output by the model, and judge whether the temporary track is successful more than or equal to M times in the last N associations under the new M, N parameters. If yes, initiate the track. If the failure times are more than N-M times in the last N associations, delete the track. Otherwise, maintain the current temporary track to continue initiation.​

Claims

1. An intelligent track initiation method based on a two-channel deep Q learning model, characterized in that, Comprise the following steps: Step 1, set the logic method M, N parameter initial value for track initiation, carry out track initiation, carry out sliding window processing on each temporary track, extract track points, extract the signal region around the target from the radar video according to the distance, bearing and time information of the track points as the sequence of regions of interest, and save the track initiation state parameters of the current temporary track; Step 2, standardize the radar video signal sequence of interest region; Step 3, build a double-channel MN parameter optimization neural network model, the double-channel MN parameter optimization neural network model comprises a radar video channel, a track initiation state parameter channel and a parameter optimization module, the radar video channel is used to extract video features according to the input region of interest; The track initiation state parameter channel is used to extract state features according to the track initiation state parameters; the video features and state features are fused and input into the parameter optimization module to obtain the M, N parameter optimization result; Step 4, use deep Q learning method to train the double-channel MN parameter optimization neural network model; Step 5, set initial M, N parameters for each temporary track during track initiation, record and save the successful or failed association after each point track association, and input the data corresponding to the last K track points into the trained double-channel MN parameter optimization neural network model to obtain the optimization amount of M, N parameters, thereby obtaining the optimized M, N parameters, and using the optimized M, N parameters to perform track initiation using the logic method.

2. The intelligent track initiation method based on a two-channel deep Q-learning model according to claim 1, characterized in that, The specific method for extracting the signal region around the target from the radar video as the sequence of regions of interest according to the distance, bearing and time information of the track points is: A center point distance unit determines a center point of a region of interest based on a distance median and an azimuth median of the extracted K number of track points and an azimuth unit With the center point distance unit and the azimuth unit As the center, K circle radar video signals corresponding to K track points in the vicinity of ±L1 distance units of the distance unit and the vicinity of ±L2 azimuth units of the azimuth unit are extracted as the region of interest signal, L1, L2 are natural numbers. 3.The intelligent track initiation method based on the dual-channel deep Q learning model according to claim 1, characterized in that, The track initiation state parameters of the current temporary track include: the number of successful associations of the temporary track during the initiation process, the total number of successful associations of the temporary track, and the initial value of the logic method M, N parameters for track initiation.

4. The intelligent track initiation method based on a two-channel deep Q-learning model according to claim 1, characterized in that, The specific method for standardizing the radar video signal sequence of interest region is: For the signals s = (s) of the K regions of interest corresponding to the K waypoints 1 ,s 2 ,…,s K Standardize them separately to a range with a mean of 0 and a variance of 1: wherein, denotes the normalized region of interest signal.

5. The intelligent track initiation method based on a double-channel deep Q learning model according to claim 1, characterized in that, Step 4.1: represent the state of the deep Q learning method as the standardized sequence of regions of interest and the track initiation state parameters, and the action space as the MN parameter optimization result; Step 4.2: After each plot association, the last K track points of each temporary track correspond to the data input of the double-channel MN parameter optimization neural network model, and the double-channel MN parameter optimization neural network model outputs the change amount M c and N c of M and N, and updates the parameters M and N according to M c and N c : M t = M t-1 + M c N t = N t-1 + N c If M t > N t , then let N t = M t , ensuring that N t is greater than or equal to M t , and if M t < 1, then let M = 1, ensuring the correctness of the rule, M t-1 is the M parameter before updating, N t-1 is the N parameter before updating, M t is the M parameter after updating, and N t is the N parameter after updating; Step 4.

3. For real target track, if M parameter decreases, N parameter increases, reward model, if M parameter increases, N parameter decreases, punish model, set initial reward function reward true ; According to the updated M and N parameters, it is determined whether the temporary track is started or inhibited. If the track is started, an additional reward V1 is given, and let If the track is inhibited, an additional penalty V2 is given, and let If the track is not started or inhibited, let The current sample, action, reward The sample corresponding to the next round of data and the track start / end flag are put into the experience replay pool. For false target tracks, if M decreases and N increases, the model is punished, if M increases and N decreases, the model is rewarded, and an initial reward function is set; If the trajectory is initiated, give an extra penalty V3, let If the trajectory is suppressed, give an extra reward V4, let If the trajectory is not initiated or suppressed, let Put the current sample, action, reward The next round of data corresponding to the sample and whether the trajectory starting and ending flag is put into the experience replay pool; Step 4.

4. Use Q learning algorithm and its variants, etc. Reinforcement learning model suitable for discrete action space to train neural network model; when a certain amount of data is stored in the experience replay pool, samples are extracted from the experience replay pool to train the model; for each sample in a batch used to train the model, first calculate the Q value of the corresponding action with the current state data of a sample, then calculate the target of the current sample according to the Q value and reward value of the next round of state data, use the MSE loss to evaluate the difference between the Q value of the current round of data and the target to train the model; use the Adam optimizer to optimize the model parameters. Train the model according to the above rules until the model converges.

6. The intelligent track initiation method based on a two-channel deep Q-learning model according to claim 5, characterized in that, For real target tracks, the initial reward function is specifically: reward true = -c1*M c + c2*N c where c1 and c2 are control the importance weights of M c and N c in the reward function.

7. The intelligent track initiation method based on a two-channel deep Q-learning model according to claim 5, characterized in that, For the false target track, the initial reward function is specifically: reward false = c1 * M c - c2 * N c where c1 and c2 are control the importance weights of M c and N c in the reward function. 8.The intelligent track initiation method based on the dual-channel deep Q learning model according to claim 1, characterized in that, The radar video channel is a 3D convolution residual network. 9.The intelligent track initiation method based on the dual-channel deep Q learning model according to claim 1, wherein, The track initial state parameter channel is a fully connected neural network.

10. The method of claim 1, wherein the method is based on a dual-channel deep Q-learning model. The parameter optimization module is a fully connected neural network.

Images