Low-speed target adaptive MTI method in reinforcement learning ground feature clutter environment

Abstract

The invention discloses a self-adaptive MTI method for a low-speed target in a reinforcement learning ground clutter environment, and belongs to the field of radar signal processing. According to the method, the low-speed target MTI filter bank is screened by utilizing reinforcement learning, and the filter with the optimal improvement factor for the low-speed target is selected in a self-adaptive manner. The algorithm maintains a left boundary and a right boundary corresponding to filters with the lowest stop band edge frequency and the highest stop band edge frequency, a left segmentation index and a right segmentation index are calculated according to the two boundaries in each cycle, and improvement factors of the filters corresponding to the two indexes serve as current state input DQN for action prediction. The action space comprises two actions, and the first action updates a left boundary into a left segmentation index; and the second action updates the right boundary into a right segmentation index. And repeatedly shrinking the left and right boundaries until the left and right boundaries coincide, and returning the algorithm to the filter corresponding to the current index. According to the method, reinforcement learning is utilized, the filtering frequency is reduced, and the method has the capability of accurately screening the optimal filter.

Description

Technical Field

[0001] The present invention relates to the field of radar signal processing. Background Art

[0002] In modern radar systems, Moving Target Indication (MTI) is a key technology used to detect and track moving targets. Traditional MTI systems rely on fixed parameter filter designs that perform well under specific environments and target characteristics. However, as the complexity of the environment and the diversity of targets increase, the limitations of fixed parameter filters gradually become apparent.

[0003] In low-speed target detection, the statistical characteristics of ground clutter vary in different spatial distributions, and traditional fixed-parameter MTI filters are difficult to effectively suppress them throughout the entire process and at all times. A feasible technical approach is to pre-store MTI filter groups with different parameters, and autonomously select the best MTI filter to achieve clutter suppression according to the changes in the statistical characteristics of clutter. Usually, this type of method relies on expert experience, the filter generalization ability is insufficient, and when all filters need to participate in the operation for evaluation and selection, the algorithm complexity will increase sharply, seriously affecting the real-time performance of the operation.

[0004] In recent years, the application of machine learning technology in radar signal processing has gradually increased. In particular, reinforcement learning technology has shown great potential in adaptive signal processing because it can continuously test and feedback optimization strategies in dynamic environments. The adaptive filter optimization method based on reinforcement learning can automatically learn and adjust the filter parameters to adapt to different environments and target characteristics, significantly improving the detection performance of the MTI system. Summary of the invention

[0005] In order to improve the detection capability of radar system for low-speed targets in complex environments, the present invention proposes a reinforcement learning adaptive MTI method for low-speed targets in ground clutter environment. By introducing a reinforcement learning algorithm based on filter decision interval segmentation, the filter with the optimal improvement factor for low-speed targets can be automatically selected from the filter group to achieve adaptive optimization of the filter, thereby effectively suppressing ground clutter and improving the detection performance of low-speed targets, especially when clutter and targets overlap in the Doppler domain.

[0006] The reinforcement learning algorithm based on filter decision interval segmentation of the present invention maintains two boundaries, namely the left boundary and the right boundary, corresponding to the filters with the lowest and highest stopband edge frequencies. According to the two boundaries, the filter decision interval is segmented, and the left segmentation index and the right segmentation index, as well as the improvement factors of the filters corresponding to the two indexes are further calculated as the current state, and used as the input of DQN for action prediction. The action space contains two actions. Action one is to shrink the left boundary and update the left boundary to the left segmentation index; Action two is to shrink the right boundary and update the right boundary to the right segmentation index. Repeat the contraction of the left and right boundaries until the left and right boundaries coincide, and the algorithm returns the filter corresponding to the current index. During the training process, the real action corresponding to the label is executed, and the DQN gradient is updated; during the test, the action predicted by DQN is executed. The technical solution for implementing the present invention includes:

[0007] Step 1: According to the clutter sequence set {C i |i=0,...,N-1}, FIR clutter suppression filter bank {w j |j=0,...,M-1} and the detection range of the system for low-speed targets in the speed dimension [-v0,v0], calculate the improvement factor sequence set {IF ij |i=0,...,N-1,j=0,...,M-1}, where N represents the number of ground clutter sequences, M represents the number of filters in the clutter suppression filter bank, and IF ij Indicates the use of the jth clutter suppression filter w j For the i-th clutter sequence C i The improvement factor of

[0008] Step 2: For the i-th clutter sequence C i , for a specific i, will improve the factor sequence IF ij The optimal improvement factor IF ij * Index As the clutter sequence C i The filter decision label L i , construct the ground clutter suppression dataset {D i}={(C i ,L i )|i=0,...,N-1}, and the data set {D i} Divide into training sets according to a certain ratio and test set Where P is the number of samples in the training set, and Q is the number of samples in the test set;

[0009] Step 3: Reinforcement learning model training phase, according to the i-th sample in the training set in step 2 Get the filtering decision interval segmentation state space And the filter decision label Li ,use L i Training reinforcement learning models;

[0010] Step 4: Reinforcement learning model testing phase, according to step 2 test set i-th sample Get the filtering decision interval segmentation state space And the filter decision label use Testing reinforcement learning models;

[0011] Step 5: Repeat steps 3 and 4 until a reinforcement learning model is obtained that converges and has the required generalization ability.

[0012] Compared with the prior art, the present invention has the following significant advantages:

[0013] (1) The traditional MTI filter relies on expert experience to optimize the filter parameters based on prior knowledge of clutter. The reinforcement learning algorithm based on the filter decision interval segmentation in the present invention can realize the adaptive optimization of the MTI filter.

[0014] (2) Traditional fixed-parameter MTI filter groups are difficult to effectively suppress ground clutter in complex environments, resulting in limited low-speed target detection performance. The present invention enables the reinforcement learning model to learn filter parameters with greater robustness and generalization capabilities by training in a variety of environments and conditions, ensuring that excellent detection performance can be maintained in various complex environments.

[0015] (3) The present invention adopts a lightweight reinforcement learning network to optimize low-speed targets and has the best improvement factor for low-speed targets. In particular, when the target and clutter are aliased in the Doppler domain, there is no need to traverse the MTI filter group, which can improve the low-speed target detection performance in a clutter environment with low computational complexity.

[0016] The present invention is described in further detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a preferred flow chart of the present invention;

[0018] Figure 2 It is the frequency response diagram of the filter bank using the equiripple method;

[0019] Figure 3 This is the DQN network structure diagram;

[0020] Figure 4 is the boundary and segmentation index relationship graph;

[0021] Figure 5 It is a schematic diagram of the relationship between the filter stopband edge frequency and the improvement factor;

[0022] Figure 6 It is the relationship between the decision interval equal score K and the model accuracy;

[0023] Figure 7 It is a graph showing the relationship between the decision interval equal score and the improvement factor corresponding to the model's wrong prediction result and the average difference / standard deviation of the improvement factor corresponding to the true label;

[0024] Figure 8 This is a graph showing the relationship between the decision interval equal score K and the average number of model steps and filtering times. DETAILED DESCRIPTION

[0025] The present invention proposes a reinforcement learning method for adaptive MTI of low-speed targets in a ground clutter environment. For low-speed targets, a reinforcement learning algorithm based on filter decision interval segmentation is introduced to select a filter with the optimal improvement factor from a high-pass FIR filter group to achieve adaptive optimization of the filter group, overcoming many deficiencies in traditional methods. It can not only effectively suppress ground clutter and other interference signals in complex environments, but also improve the detection accuracy and reliability of low-speed targets, thereby improving the performance of the entire MTI system. At the same time, the present invention uses a lightweight network to achieve its functions, which is suitable for real-time signal processing occasions.

[0026] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0027] like Figure 1 As shown, the preferred process of the present invention comprises the following steps:

[0028] Step 1: According to the clutter sequence set {C i |i=0,...,N-1}, FIR clutter suppression filter bank {w j |j=0,...,M-1} and the detection range of the system for low-speed targets in the speed dimension [-v0,v0], calculate the improvement factor sequence set {IF ij |i=0,...,N-1,j=0,...,M-1}, where N represents the number of ground clutter sequences, M represents the number of filters in the clutter suppression filter bank, and IF ij Indicates the use of the jth clutter suppression filter w j For the i-th clutter sequence C i The improvement factor of

[0029] Step 2: For the i-th clutter sequence C i , for a specific i, will improve the factor sequence IF ij The optimal improvement factor IF ij * Index As the clutter sequence C i The filter decision label Li , construct the ground clutter suppression dataset {D i}={(C i ,L i )|i=0,...,N-1}, and the data set {D i} Divide into training sets according to a certain ratio and test set Where P is the number of samples in the training set, and Q is the number of samples in the test set;

[0030] Step 3: Reinforcement learning model training phase, according to the i-th sample in the training set in step 2 Get the filtering decision interval segmentation state space And the filter decision label L i ,use L i Training reinforcement learning models;

[0031] Step 4: Reinforcement learning model testing phase, according to step 2 test set i-th sample Get the filtering decision interval segmentation state space And the filter decision label use Testing reinforcement learning models;

[0032] Step 5: Repeat steps 3 and 4 until a reinforcement learning model is obtained that converges and has the required generalization ability.

[0033] Preferably, step 2 comprises the following steps:

[0034] Step 2-1: For each clutter sequence C i , select the filter index with the best improvement factor

[0035]

[0036] Step 2-2: Index the filter with the best improvement factor As the clutter sequence C i The filter decision label L i :

[0037]

[0038] Step 2-3: Record each clutter sequence C i and its filtering decision label L i , construct the data set D = {D i}:

[0039] {D i}={(C i ,Li )|i=0,...,N-1} (3)

[0040] Step 2-4: Divide the dataset D into training sets in proportion and test set Where P is the number of samples in the training set, and Q is the number of samples in the test set.

[0041] Preferably, step 3 comprises the following steps:

[0042] Step 3-1: For the training set The training samples in Initialize left and right borders Corresponding to the filter bank {w j}The first and last filters in ;

[0043] Step 3-2: According to the left and right boundaries Calculate the left split index and right split index

[0044]

[0045] Among them, K is the interval equal fraction of each decision.

[0046] Step 3-3: Calculate the filters corresponding to these two segmentation indices and For clutter sequences Improvement factor IF i L and IF i R , the calculation method is specifically described as:

[0047]

[0048] Get training samples The state space

[0049]

[0050] Step 3-4: Use DQN (Deep Q-Network) to transform the state space Mapping to action space Represents a 1×2 vector.

[0051] Step 3-5: Based on the training sample Label L in i Calculate the current state space Corresponding real action

[0052]

[0053] Step 3-6: Execution , the left boundary is updated to the left split index:

[0054]

[0055] , the right boundary is updated to the right split index:

[0056]

[0057] Step 3-7: Calculate cross entropy loss i :

[0058]

[0059] Where N a represents the number of actions in the action space; y i Represents the one-hot encoding of the true action, that is, if the a-th action is the correct action Then y a =1, otherwise equal to 0; Representing the action space Predict the probability of action a.

[0060] Step 3-8: Loss i Perform back propagation, update gradients, and train DQN.

[0061] Step 3-9: When one of the following two situations occurs, the current training sample End of loop:

[0062] 1) That is, the left and right boundaries coincide, and return

[0063] 2) That is, the difference between the left and right boundaries is less than K, and the calculation right Improvement factor Select the index corresponding to the optimal improvement factor

[0064] Return to Index * .

[0065] If the above two conditions are not met, go to step 3-2.

[0066] Step 3-10: Let i←i+1, and re-execute steps 3-1 to 3-9 for the next training set sample Train until the training set is traversed That is, i=P.

[0067] Preferably, step 4 comprises the following steps:

[0068] Step 4-1: For the training set Each training sample in Initialize left and right borders Corresponding to the filter bank {w j}The first and last filters in ;

[0069] Step 4-2: According to the left and right boundaries Calculate the left split index and right split index

[0070]

[0071] Among them, K is the interval equal fraction of each decision.

[0072] Step 4-3: Calculate the filters corresponding to these two segmentation indices and For clutter sequences Calculate the improvement factor IF i L and IF i R The calculation method is specifically described as:

[0073]

[0074] Get the test sample The state space

[0075]

[0076] Step 4-4: Use DQN (Deep Q-Network) to transform the state space Mapping to action space Represents a 1×2 vector. Get the action action according to the action space i :

[0077]

[0078] action i ∈{0,1}. Execute action action i , action i = 0, the left boundary is updated to the left split index:

[0079]

[0080] action i = 1, the right boundary is updated to the right split index:

[0081]

[0082] Step 4-5: When one of the following two situations occurs, the current test sample End of loop:

[0083] 1) That is, the left and right boundaries coincide, and return

[0084] 2) That is, the difference between the left and right boundaries is less than K, and the calculation right Improvement factor Select the index corresponding to the optimal improvement factor

[0085] Return to Index * .

[0086] If the above two conditions are not met, go to step 4-2.

[0087] Step 4-6: Let i←i+1, and re-execute steps 4-1 to 4-5 for the next test set sample D i test Run the test until the test set is exhausted That is, i=Q.

[0088] Preferably, step 5 comprises the following steps:

[0089] Step 5-1: Execute step 3 to get the trained model DQN train , and for the training set D train The accuracy of εtrain ;

[0090] Step 5-2: Execute step 4 and use the model DQN of step 5-1 train , get the test set D test The accuracy of εtest ;

[0091] Step 5-3: Repeat steps 5-1 to 5-3. If the generalization condition is met:

[0092] |δ εtrain -δ εtest |<δ ε (17)

[0093] Then stop training and save the trained model In the formula, δ ε The network model DQN in the training process train Whether the threshold of generalization requirement is met.

[0094] Example:

[0095] 1) Ground clutter suppression dataset parameters

[0096] v0=10m / s, N=44400, M=25, P=37000, Q=7400.

[0097] 2) Radar emitter parameters

[0098] Pulse repetition number N pulse =4096, wavelength λ = 0.0375m, pulse repetition period 125us, Δf D =0.037m / s.

[0099] 3) Filter bank parameters

[0100] f start =0.1m / s,f end =2.5m / s, Δf1=0.3m / s, Δf2=0.1m / s, order=746. Figure 2 For filtering

[0101] Example graph of the frequency response of the three filters in the filter group.

[0102] Figure 3 The DQN model. The model input is a 1×4 state space, which is converted to 1×256 through a fully connected layer, and then through a relu activation function and mapped to a 1×2 action space through a second fully connected layer. The model uses the Adam optimizer, a learning rate of 1e-5, and a cross entropy loss function, δ ε Set to 0.025.

[0103] Figure 4 When K=3, the relationship between the left and right segmentation indexes and the left and right boundaries.

[0104] Figure 5 The relationship curve between the stopband edge frequency and the improvement factor of the filter group for a noise sequence. At this time, the filter with a stopband edge frequency of 1.5 is the filter with the optimal improvement factor.

[0105] Figure 6 This is the relationship curve between the decision interval equalization score K and the model accuracy. When K increases from 2 to 3, the algorithm accuracy improves most significantly, by 11.43%. When K increases from 3 to 4 and from 4 to 5, the accuracy increases by 1.35% and 0.33%, respectively.

[0106] Figure 7is the relationship between the average difference / standard deviation of the improvement factor corresponding to the wrong prediction result and the improvement factor corresponding to the true label. When K is 3, the average difference and standard deviation between the wrong prediction result and the true one are the lowest, which are 0.04dB and 0.08dB.

[0107] Figure 8 It is the relationship curve between the decision interval equal score K and the number of steps and filtering times. It can be seen from the curve that with the increase of K, the interval segmentation becomes finer, and the average number of steps and filtering times will increase, especially when the number of segmentation intervals increases from 3 to 4, the average number of steps increases by 0.61, and the average number of filtering times increases by 1.96.

Claims

1. A reinforcement learning method for adaptive MTI of low-speed targets in a cluttered environment, characterized by: Step 1: According to the clutter sequence set {C i |i=0,...,N-1}, FIR clutter suppression filter bank {w j |j=0,...,M-1} and the detection range of the system for low-speed targets in the speed dimension [-v0,v0], calculate the improvement factor sequence set {IF ij |i=0,...,N-1,j=0,...,M-1}, where C i Represents the complex sequence of the i-th clutter IQ data, and the sequence length is fixed to N pulse ; N represents the number of ground clutter sequences, M represents the number of filters in the clutter suppression filter bank, IF ij Indicates the use of the jth clutter suppression filter w j For the i-th clutter sequence C i The improvement factor of Step 2: For the i-th clutter sequence C i , for a specific i, will improve the factor sequence IF ij The best improvement factor Index As the clutter sequence C i The filter decision label L i , construct the ground clutter suppression dataset {D i }={(C i ,L i )|i=0,...,N-1}, and the data set {D i } Divide into training sets according to a certain ratio and test set Where P is the number of samples in the training set, and Q is the number of samples in the test set; Step 3: Reinforcement learning model training phase, according to the i-th sample in the training set in step 2 Get the filtering decision interval segmentation state space And the filter decision label L i ,use L i Training reinforcement learning models; Step 4: Reinforcement learning model testing phase, according to step 2 test set i-th sample Get the filtering decision interval segmentation state space And the filter decision label use Testing reinforcement learning models; Step 5: Repeat steps 3 and 4 until a reinforcement learning model is obtained that converges and has the required generalization ability.

2. According to claim 1, a reinforcement learning method for adaptive MTI of low-speed targets in a cluttered environment, characterized in that: The FIR clutter suppression filter bank in step 1 includes: {w j |j=0,...,M-1} (2) The filter bank is designed using the equiripple method, and the stopband edge frequencies are evenly distributed in [v start ,v end ] range, where v start is the minimum value of the half-spectrum width of the clutter supported by the system, v end is the maximum value of the half-spectrum width of the clutter supported by the system, and the transition band width is fixed to Δv1. Δv1 is specifically described as: Δv1=α1Δv D (3) Where Δv D is the Doppler velocity resolution, α1 is an adjustable parameter. The smaller α1 is, the narrower the transition band is, the stronger the resolution of clutter and target is, but the order of the filter will increase. The size of the filter order should ensure that (order+N c -1) close to but not exceeding a power of 2, where N c is the length of the noise sequence; The spacing between filters is fixed to Δv2: Δv2=α2Δv D (4) α2 is an adjustable parameter, and M represents the number of filters in the clutter suppression filter bank, including: In the formula, This is a round-up operation.

3. The method of adaptive MTI for low-speed targets in a reinforcement learning environment with ground clutter according to claim 1, characterized in that: The improvement factor sequence set in step 1 {IF ij The calculation methods of} include: Find the autocorrelation matrix of the low-speed target. The detection range of the low-speed target in the speed dimension is [-v0, v0]. The autocorrelation function of the target signal is: In the formula, τ mn =t m -t n =(mn) / fr is the correlation time, λ is the wavelength, fr is the pulse repetition frequency, and the autocorrelation matrix R of the signal of N pulses is obtained. s for: According to the low-speed target autocorrelation matrix R s Find the FIR clutter suppression filter w j For the i-th clutter sequence C i Improvement factor IF ij :

4. The method of adaptive MTI for low-speed targets in a reinforcement learning environment with ground clutter according to claim 2, characterized in that: The step 2 comprises: Step 2-1: For each clutter sequence C i , select the filter index with the best improvement factor Step 2-2: Index the filter with the best improvement factor As the clutter sequence C i The filter decision label L i : Step 2-3: Record each clutter sequence C i and its filtering decision label L i , construct the data set D = {D i }: {D i }={(C i ,L i )|i=0,...,N-1} (11) Step 2-4: Divide the dataset D into training sets in proportion and test set Where P is the number of samples in the training set, and Q is the number of samples in the test set.

5. The method of adaptive MTI for low-speed targets in a reinforcement learning environment with ground clutter according to claim 3, characterized in that: The step 3 comprises: Step 3-1: For the training set The training samples in Initialize left and right borders Corresponding to the filter bank {w j }The first and last filters in ; Step 3-2: According to the left and right boundaries Calculate the left split index and right split index Among them, K is the interval equal fraction of each decision; Step 3-3: Calculate the filters corresponding to these two segmentation indices and For clutter sequences Improvement factor and Get training samples The state space Step 3-4: Use DQN (Deep Q-Network) to transform the state space Mapping to action space Represents a 1×2 vector; Step 3-5: Based on the training sample Label L in i Calculate the current state space Corresponding real action Step 3-6: Execution , the left boundary is updated to the left split index: , the right boundary is updated to the right split index: Step 3-7: Calculate cross entropy loss i : Where N a represents the number of actions in the action space; y i Represents the one-hot encoding of the true action, that is, if the a-th action is the correct action Then y a =1, otherwise equal to 0; Representing the action space Predict the probability of action a; Step 3-8: Loss i Perform back propagation, update gradients, and train DQN; Step 3-9: When one of the following two situations occurs, the current training sample End of loop: 1) That is, the left and right boundaries coincide, and return 2) That is, the difference between the left and right boundaries is less than K, and the calculation right Improvement factor Select the index corresponding to the optimal improvement factor Return to Index * ; If the above two conditions are not met, go to step 3-2; Step 3-10: Let i←i+1, and re-execute steps 3-1 to 3-9 for the next training set sample Train until the training set is traversed That is, i=P.

6. The method of adaptive MTI for low-speed targets in a reinforcement learning environment with ground clutter according to claim 4, characterized in that: The step 4 comprises: Step 4-1: For the training set Each training sample in Initialize left and right borders Corresponding to the filter bank {w j }The first and last filters in ; Step 4-2: According to the left and right boundaries Calculate the left split index and right split index Among them, K is the interval equal fraction of each decision; Step 4-3: Calculate the filters corresponding to these two segmentation indices and For clutter sequences Calculate the improvement factor and Get the test sample The state space Step 4-4: Use DQN (Deep Q-Network) to transform the state space Mapping to action space Represents a 1×2 vector; obtains the action action according to the action space i : action i ∈{0,1}; execute action action i , action i = 0, the left boundary is updated to the left split index: action i = 1, the right boundary is updated to the right split index: Step 4-5: When one of the following two situations occurs, the current test sample End of loop: 1) That is, the left and right boundaries coincide, and return 2) That is, the difference between the left and right boundaries is less than K, and the calculation right Improvement factor Select the index corresponding to the optimal improvement factor Return to Index * ; If the above two conditions are not met, go to step 4-2; Step 4-6: Let i←i+1, and re-execute steps 4-1 to 4-5 for the next test set sample Run the test until the test set is exhausted That is, i=Q.

7. The method of adaptive MTI for low-speed targets in a reinforcement learning environment with ground clutter according to claim 5, characterized in that: The step 5 comprises: Step 5-1: Execute step 3 to get the trained model DQN train , and for the training set D train The accuracy of εtrain ; Step 5-2: Execute step 4 and use the model DQN of step 5-1 train , get the test set D test The accuracy of εtest ; Step 5-3: Repeat steps 5-1 to 5-3. If the generalization condition is met: d εtrain -d εtest |<d ε (25) Then stop training and save the trained model In the formula, δ ε The network model DQN in the training process train Whether the threshold of generalization requirement is met.

Images