Interference Decision Method and System Combining Behavioral Cloning and Proximal Policy Optimization Algorithm

Abstract

The present invention discloses an interference decision-making method and system combining behavior cloning and proximal policy optimization algorithm. The method of the present invention includes the following steps: S1. Randomly initialize the network parameters of the policy network and the network parameters of the value network; S2. Based on the behavior cloning method, train the policy network; S3. The jammer interacts with the radar according to the current policy, samples trajectory data, and stores it in the dataset; S4. The jammer uses the proximal policy optimization algorithm to optimize the policy network using the trajectory data in the dataset; S5. Repeat steps S3 and S4 until the optimal policy is obtained. The present invention improves the speed of policy training while greatly enhancing the security of the jammer.

Description

Technical Field

[0001] The present invention belongs to the field of radar electronic countermeasures, and specifically relates to an interference decision-making method and system combining behavior cloning and proximal policy optimization algorithm. The technical solution of the present invention can be applied to an electronic countermeasure system. Background Art

[0002] The rapid development of modern science and technology has promoted the shift of the focus of military confrontation from physical warfare to information warfare. Electronic warfare is the most important part of information warfare. The core of electronic warfare is centered around radar, competing for the use and control of the electromagnetic spectrum. With the support of modern technology, modern radars have increasingly rich functions and stronger anti-jamming capabilities, making it more and more difficult to interfere with enemy radars. How to select the most suitable interference pattern to produce the most effective interference on enemy radars has become a key research point in the field of electronic countermeasures.

[0003] The existing research on radar interference decision-making methods focuses on the application of reinforcement learning algorithms. Reinforcement learning algorithms can dynamically adjust strategies according to the actual environment in an interactive manner. In the initial stage of strategy adjustment, reinforcement learning algorithms explore the environment through random strategies. However, in the field of radar interference decision-making, random strategies will decide ineffective interference patterns. Ineffective interference behaviors not only cannot interfere with enemy radars, but will instead expose the position of our jammer in advance, making it impossible to be directly applied to radar interference decision-making. Therefore, how to improve the security of radar interference decision-making algorithms has extremely high application value. Summary of the Invention

[0004] Aiming at the problem of how to safely and effectively interfere with enemy multifunctional radars in a complex electromagnetic environment, the present invention proposes an interference decision-making method and system combining behavior cloning and proximal policy optimization algorithm.

[0005] To achieve the above-mentioned invention purpose, the present invention adopts the following technical solutions:

[0006] The interference decision-making method combining behavior cloning and proximal policy optimization algorithm is carried out according to the following steps:

[0007] S1. Randomly initialize the network parameters θ of the policy network π θ and the network parameters φ of the value network V φ ;

[0008] S2. Train the policy network π θ based on the behavior cloning method;

[0009] S3. The jammer interacts with the radar according to the current policy to sample trajectory data τ and store it in the data set D k ;

[0010] S4. The jammer uses the proximal policy optimization algorithm to optimize the policy network using the trajectory data in dataset D k ;

[0011] S5. Repeat steps S3 and S4 alternately until the optimal policy is obtained

[0012] As an optimal solution, in step S2, based on the behavior cloning method, the policy network π θ is trained, and the process includes the following steps:

[0013] S21. Generate a prior dataset D p ={(s i , j i )|i = 1, …, N}, where (s i , j i ) represents the i-th radar state-jamming pattern pair;

[0014] S22. Use dataset D p to train the policy network according to the objective function, and the objective function is:

[0015]

[0016] where N is the size of the expert knowledge base and i indicates that s i and j i are at the same time step, and ||x||2 represents the 2-norm of x.

[0017] As an optimal solution, in step S3, the jammer interacts with the radar according to the current policy and samples the trajectory data τ, and the process includes the following steps:

[0018] S31. Initialization: The jammer flies through the radar signal coverage area along the established route.

[0019] S32. The radar continuously emits pulse signals to scan the adjacent airspace and tracks and guides strikes on the detected targets according to the radar working logic.

[0020] S33. The jammer receives the radar pulse signal within a processing time after time t and extracts the pulse description word (PDW) of each pulse.

[0021] The pulse description word (PDW) contains 6 radar pulse parameters: time of arrival (TOA), direction of arrival (DOA), pulse width (PW), bandwidth (BW), carrier frequency (CF), and in-pulse modulation characteristic (F). The PDW of the i-th pulse can be expressed as

[0022] PDW i ={TOAi , DOA i , PW i , BW i , CF i , F i}

[0023] S34. The jammer identifies the radar operating mode through a set of pulse description words and encodes the radar status s t

[0024]

[0025] Among them, is the vector concatenation symbol, s t represents the radar status at time t; f n , n ∈ [1, N] represents the radar function corresponding to the radar status s t ; p m , m ∈ [1, M] represents the radar phrase included in the radar function f n ; {w i , w j , w k , w l}, i, j, k, l ∈ [1, T] represents the radar word corresponding to the radar phrase p m .

[0026] S35. The jammer obtains the jamming pattern t and strategy and sends the corresponding jamming signal to the radar. Common jamming patterns include: noise frequency modulation jamming, noise amplitude modulation jamming, noise phase modulation jamming, amplitude modulation frequency modulation jamming, range deception jamming, velocity deception jamming, range-velocity combined deception jamming, dense false target jamming, and smart jamming.

[0027]

[0028] S36. After being jammed, the radar changes its working state according to the processing result of the radar receiver. The jammer obtains the encoding s of the new radar state by re-implementing step S34 t+1 , and according to the change of the radar state s t → s t+1 , obtains the reward r according to the reward function R t .

[0029] The reward function is defined as follows according to the threat level of the radar working state:

[0030]

[0031] ​S37, repeat steps S32-S36 and record the interaction trajectory sequence (s t ,j t ,r t ) until the jammer flies out of the radar signal coverage area or is destroyed by radar guidance. The trajectory τ is expressed as:

[0032] τ={(s1,j1,r1),(s2,j2,r2),…,(s T ,j T ,r T )}

[0033] Among them, T is the termination time, that is, at time T the jammer flies out of the radar signal coverage area (penetration successful) or is destroyed by radar guidance (penetration failed).

[0034] S38, will be based on the strategy The sampled trajectory data is stored in set D k .

[0035] As a preferred solution, in step S4, the jammer uses the data set D k The strategy network is optimized based on the trajectory data in the dataset. The process includes the following steps:

[0036] S41, extract the trajectory sequence set of the jammer and radar interaction according to the strategy The trajectory τ obtained by interaction,τ∈D k Calculate the reward G for each step in the trajectory t

[0037]

[0038] Where γ is the discount factor

[0039] S42. Calculate advantage function A based on current value function V

[0040] A t (s t ,j t )=r t +V(s t+1 )-V(s t )

[0041] S43. Objective function based on proximal strategy optimization algorithm:

[0042] L=min(r t (θ)A t ,clip(r t (θ),1-ε,1+ε)A t )

[0043] Update the policy network parameters θ using the stochastic gradient descent algorithm

[0044]

[0045] Here, argmax is a function that finds the parameters of a function. y = argmax f(t) means: y is the parameter t in the f(t) function that will produce the maximum output. is the importance sampling ratio, π θ is the output of the current policy network, is the output of the policy network used when sampling trajectory τ. T is the length of trajectory τ. |D k |For use strategy The total number of trajectories sampled. t (θ),1-ε,1+ε) is the trimming function, which is used to reduce the strategy ratio r t (θ) is constrained to be in the range [1-ε,1+ε], if r t (θ) is less than 1-ε, then output 1-ε; if r t If (θ) is greater than 1+ε, then the output is 1+ε. ε is a hyperparameter, usually set to 0.2.

[0046] S44. Use the stochastic gradient descent algorithm to fit the value network parameter φ through the mean square error

[0047]

[0048] The present invention further provides an interference decision system that combines behavioral cloning and a proximal strategy optimization algorithm, and applies the decision method described in any of the above solutions. The decision system includes the following modules:

[0049] An acquisition module, used to acquire time domain pulse signals of a multi-function radar with multiple working modes;

[0050] Parameter identification module, used to extract PDW parameters of radar time domain pulse signals;

[0051] A radar state encoding module, used for encoding the radar state according to the PDW of a group of radar pulse signals;

[0052] An interference generation module is used to generate a time domain interference signal according to the interference pattern output by the strategy network;

[0053] A data storage module for storing data generated by an expert prior knowledge base and trajectory sequence data of the interaction between the jammer and the radar;

[0054] The network optimization module is used to optimize the policy network and value network according to the corresponding objective function.

[0055] Compared with the existing technologies, the advantages of the present invention are as follows:

[0056] The proximal policy optimization algorithm is used to optimize the policy network. In the initial stage of network iterative optimization, the network performance is extremely unstable, and it is easy to make wrong interference patterns in decision-making, resulting in the jammer being exposed to the enemy radar. Therefore, the present invention combines the behavior cloning method with the proximal policy optimization, and integrates the existing expert prior knowledge into the policy network, so that the policy network can avoid obvious wrong decisions, while improving the safety of the jammer, accelerating the convergence of the policy network. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a flowchart of the interference decision-making method combining behavior cloning and proximal policy optimization algorithm in an embodiment of the present invention;

[0058] Figure 2 is an interaction flowchart between a radar and a jammer in an embodiment of the present invention;

[0059] Figure 3 is a schematic diagram of a multi-functional radar hierarchical model in an embodiment of the present invention;

[0060] Figure 4 is a schematic diagram of a CLIP function in an embodiment of the present invention;

[0061] Figure 5 is a block diagram of an interference decision-making system combining behavior cloning and proximal policy optimization algorithm of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] In order to more clearly illustrate the embodiments of the present invention, the specific implementation manners of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to these drawings, and other implementation manners can be obtained.

[0063] The interference decision-making method of the present invention combining behavior cloning and proximal policy optimization algorithm includes first randomly initializing a policy network and a value network; then using the behavior cloning method to generate a prior data set according to the existing prior knowledge, and then training the policy network through a mean square error loss function using a stochastic gradient descent optimization algorithm to integrate the prior knowledge into the policy network; finally using the proximal policy optimization algorithm to iteratively optimize the policy network through the actual interaction data between the jammer and the enemy radar, and finally obtaining the optimal policy.

[0064] As Figure 1 shown, the interference decision-making method of the embodiment of the present invention combining behavior cloning and proximal policy optimization algorithm specifically includes the following steps:

[0065] S1. Randomly initialize the policy network πθ network parameters θ and value network V φ network parameters φ

[0066] Among them, both the policy network and the value network are deep neural networks. Both the policy network and the value network have 3 hidden layers, and the hyperbolic tangent function is used as the activation function. The input of the policy network is the radar state encoding, and the output is the interference pattern number j ∈ N; the input of the value network is the radar state encoding, and the output is the evaluation value of the output of the policy network under this type of state encoding. The radar state encoding is an abstract representation of the radar state, and the specific form is referred to in step S34.

[0067] S2. Train the policy network π based on the behavior cloning method θ .

[0068] Due to the danger in the field of radar electronic countermeasures, the jammer cannot use conventional reinforcement learning algorithms to optimize the strategy. To make full use of the existing expert prior knowledge and avoid the dangerous actions generated by the proximal policy optimization algorithm's random exploration in the early stage to the greatest extent, the present invention uses the behavior cloning method to pre-train the policy network.

[0069] S21. Generate a prior dataset D according to expert prior knowledge p ={(s i , j i ) | i = 1, …, N} (s i , j i ) represents the i-th radar state - interference pattern pair.

[0070] The radar state encoding - interference pattern pair (s i , j i ) in the prior dataset is generated according to the existing expert prior knowledge, which can be fuzzy and plays a guiding role in the training of the policy. And under the condition of satisfying the MDP assumption (that is, the optimal interference only depends on the current radar state), the order of the radar state encoding - interference pair can be interrupted during training.

[0071] S22. Train the policy network using the stochastic gradient descent algorithm (SGD) according to the dataset D p according to the objective function, and the objective function is:

[0072]

[0073] where N is the size of the expert knowledge base and i indicates that s i and j i are at the same time step, and ||x||2 represents the 2-norm of x.

[0074] The termination condition for training is that the objective function is less than the threshold η, i.e., L < η. The threshold η is an empirical value and can be adjusted according to the actual training results.

[0075] S3. The jammer interacts with the radar according to the current strategy to sample the trajectory data τ and store it in the dataset D k ;

[0076] Specifically, as Figure 2 shown, step S3 of this embodiment includes the following steps:

[0077] S31. Initialization: The jammer flies through the radar signal coverage area along a pre-determined route.

[0078] S32. The radar continuously emits pulse signals for scanning, and tracks and guides the strikes on the detected targets according to the radar working logic.

[0079] The radar is an electronic reconnaissance device used to detect enemy targets in the airspace, and has three basic functions: search, tracking, and guidance. Search is to search for targets in a specified area and provide information for tracking. When the radar first discovers a target, to confirm whether the target is a false alarm or a real target, it will irradiate the area where the target is located again. If the target is rediscovered within the range gate centered on the position where the target was first discovered, it is confirmed as a real target, and then it enters the tracking mode. The tracking mode is to continuously track a target for a period of time. When the radar continuously tracks a target for a period of time, it can enter the guidance mode, providing information such as the flight trajectory and speed of the target for the weapon system, and predicting the target position at the next moment. After entering the guidance mode, the target can basically not escape the strike, and at this time, it can be determined that the target penetration mission fails.

[0080] The phased array radar is the most representative multi-functional radar among them. The phased array radar can use the phased array antenna to perform multiple tasks simultaneously and derive composite working modes based on the three basic functions. The typical composite working modes of a phased array radar mainly include four types: Track While Search (TWS), Track And Search (TAS), Single-Target Tracking (STT), and Multi-Target Tracking (MTT).

[0081] S33. The jammer receives the radar pulse signals within a processing time period after time t and extracts the Pulse Description Word (PDW) of each pulse by the pre-parameter recognition module.

[0082] The Pulse Description Word (PDW) contains six radar pulse parameters: Time of Arrival (TOA), Direction of Arrival (DOA), Pulse Width (PW), Bandwidth (BW), Carrier Frequency (CF), and Intra-Pulse Modulation Feature (F). The PDW of the i-th pulse can be expressed as:

[0083] PDW i ={TOA i ,DOA i ,PW i ,BW i ,CF i ,F i}

[0084] S34. The jammer identifies the radar operating mode through a set of pulse description words and encodes the radar status s t .

[0085] The PDW of the pulse signal belongs to the lower-level information, and high-level information related to the radar status needs to be further extracted from it. According to the multi-functional radar hierarchical model (as Figure 3 shown), the jammer identifies the radar operating mode through a set of pulse description words and encodes the radar status.

[0086] The radar function (i.e., the radar operating mode) is represented by the symbol f n ; each radar function contains multiple radar phrases p m , and the radar phrase consists of multiple radar words p m =(w i w j w k w l ), and the radar word is a description of the radar pulse. The radar status encoding is:

[0087]

[0088] where ο is the vector concatenation symbol, and s t represents the radar status at time t; f n ,n∈[1,N] represents the radar function corresponding to the radar status s t ; p m ,m∈[1,M] represents the radar phrase contained in the radar function f n ; {w i ,w j ,w k ,w l},i,j,k,l∈[1,T] represents the radar word corresponding to the radar phrase p m .

[0089] The corresponding radar word number sequence can be obtained based on the PDW of a group of pulse signals. According to the combination order of radar words, the radar phrase number sequence can be obtained; based on multiple groups of radar phrase number sequences, the radar function number can be obtained; finally, the radar state code is obtained according to the combination of the radar function number and the radar word number sequence.

[0090] S35. The jammer obtains the jamming pattern i and strategy according to the radar state code s and sends the corresponding interference time-domain signal to the radar.

[0091] There are 9 common jamming patterns: noise frequency modulation jamming, noise amplitude modulation jamming, noise phase modulation jamming, amplitude modulation frequency modulation jamming, range deception jamming, velocity deception jamming, range-velocity combined deception jamming, dense false target jamming, and smart jamming.

[0092] S36. After being jammed, the radar will change its working state according to the processing result of the radar receiver. The jammer obtains the code s of the new state of the radar by repeating step S4 t+1 , and according to the change of the radar state s t →s t+1 , the reward r is obtained according to the reward function R t .

[0093] The reward function is defined based on the threat level of the radar working state. For the convenience of explanation, the working modes of a certain multi-functional radar and their threat level assessments are given here

[0094]

[0095] The reward function is defined as follows:

[0096]

[0097] S37. Repeat steps S32 - S36 and record the interaction trajectory sequence (s t , j t , r t ) of the jammer and the radar at each moment until the jammer flies out of the radar signal coverage area or the jammer is destroyed by radar guidance. The expression of the trajectory τ is:

[0098] τ = {(s1, j1, r1), (s2, j2, r2), …, (s T , j T , r T )}

[0099] where T is the termination time, that is, at time T, the jammer flies out of the radar signal coverage area or the jammer is destroyed by radar guidance.

[0100] S38. According to the strategy The sampled trajectory data is stored in the set D k

[0101] S4. The jammer uses the proximal policy optimization algorithm to optimize the policy network using the dataset D k in the trajectory data for policy network optimization.

[0102] S41. Take out all the trajectories τ obtained according to the policy from the dataset D , τ ∈ D k , and calculate the return G for each step in each trajectory t :

[0103]

[0104] where γ is the discount factor, which determines the present value of future rewards: the reward at future time k is only γ times its current value k-1 times, usually 0 < γ ≤ 1, and r t is the reward at time t in the trajectory τ.

[0105] S42. Calculate the advantage function A based on the current value function V φ A(s

[0106] , j t ) = r t + V t (s φ ) - V t+1 (s φ ) t )

[0107] The advantage function describes how much better it is for the jammer to select the jamming pattern j when the radar is in state s t than to randomly select a jamming pattern. t

[0108] S43. Based on the objective function of the proximal policy optimization algorithm

[0109] L t (θ) = min(r t (θ)A t , clip(r t (θ), 1 - ε, 1 + ε)A t )

[0110] Use the stochastic gradient descent algorithm to update the policy network parameters θ

[0111]

[0112] ​Among them, argmax is a function that finds the argument (set) of a function. y = argmax f(t) means that y is the argument t in the function f(t) that produces the maximum output. min(A,B) takes the smaller value between A and B. is the importance sampling ratio, π θ is the output of the current policy network, is the output of the policy network used when sampling the trajectory τ. T is the length of the trajectory τ. |D k | is the total number of trajectories sampled using the policy clip(r t (θ), 1 - ε, 1 + ε) is a clipping function that constrains the importance sampling ratio r t (θ) within the range [1 - ε, 1 + ε]. If r t (θ) is less than 1 - ε, then 1 - ε is output; if r t (θ) is greater than 1 + ε, then 1 + ε is output, as Figure 4 shown. ε is a hyperparameter, usually taken as 0.2. S44. Use the stochastic gradient descent algorithm to fit the value network parameters φ through the mean squared error

[0113]

[0114] S5. Repeat steps S3 and S4 alternately until the optimal policy is obtained

[0115] as Figure 5 shown, the present invention also discloses an interference decision-making system that combines behavior cloning and proximal policy optimization algorithms and is applied to the interference decision-making method of the above embodiment. The interference decision-making system includes the following modules:

[0116] Acquisition module, used to acquire the time-domain pulse signals of a multi-functional radar with multiple working modes;

[0117] Parameter identification module, used to extract the PDW parameters of the radar time-domain pulse signals;

[0118] Radar state encoding module, used to encode the radar state according to the PDW of a group of radar pulse signals;

[0119] Interference generation module, used to generate time-domain interference signals according to the interference patterns output by the policy network;

[0120] Data storage module, used to store the data generated by the expert prior knowledge base and the trajectory sequence data of the interaction between the jammer and the radar;

[0121] Network optimization module, used to optimize the policy network and the value network according to the corresponding objective function.

[0122] The above description only elaborates in detail on the preferred embodiments and principles of the present invention. For those of ordinary skill in the art, according to the idea provided by the present invention, there will be changes in the specific implementation manners, and these changes should also be regarded as the protection scope of the present invention.

Claims

1. An interference decision-making method combining behavior cloning and proximal policy optimization algorithm, characterized in that It includes the following steps: S1. Randomly initialize the network parameters θ of the policy network π θ and the network parameters φ of the value network V φ ; S2. Train the policy network π based on the behavior cloning method θ ; S3. The jammer interacts with the radar according to the current strategy to sample the trajectory data τ and store it in the dataset D k ; S4. The jammer uses the proximal policy optimization algorithm to optimize the policy network using the trajectory data in dataset D k ; Step S4 specifically includes the following steps: S41. Take out the trajectory τ obtained according to the policy from the set of trajectory sequences of the interaction between the jammer and the radar, where τ ∈ D obtained through interaction k Calculate the return G for each step in the trajectory t ; where γ is the discount factor; S42. Calculate the advantage function A based on the current value function V; A t (s t ,j t ) = r t +V(s t+1 ) - V(s t ) S43. Based on the objective function, L = min(r t (θ)A t , clip(r t (θ), 1 - ε, 1 + ε)A t ) where ε is a hyperparameter, taking 0.2; Update the policy network parameter θ using the stochastic gradient descent algorithm; Among them, argmax is a function that finds the argument of a function; y = argmax f(t) means that y is the argument t that produces the maximum output in the function f(t). is the policy ratio, π θ is the output of the current policy network, is the output of the policy network used when sampling the trajectory τ; T is the length of the trajectory τ; |D k | is the total number of trajectories obtained by sampling using the policy ​ clip(r t (θ), 1 - ε, 1 + ε) is a clipping function that constrains the policy ratio r t (θ) within the range [1 - ε, 1 + ε]. If r t (θ) is less than 1 - ε, then 1 - ε is output; if r t (θ) is greater than 1 + ε, then 1 + ε is output; S44. Use the stochastic gradient descent algorithm to fit the value network parameter φ through the mean square error; S5. Repeat steps S3 and S4 to obtain the optimal strategy 2. The interference decision-making method combining behavior cloning and proximal policy optimization algorithm according to claim 1, characterized in that, Step S2 specifically includes the following steps: S21. Generate a prior dataset D according to existing prior knowledge p ={(s i , j i )|i = 1, …, N}, where (s i , j i ) represents the i-th radar state-jamming pattern pair; S22. Adopt the data set D p Train the policy network according to the objective function, where the objective function is: where N is the size of the expert knowledge base, i represents s i and j i are at the same time step, and ||x||2 represents the 2-norm of x.

3. The interference decision-making method combining behavior cloning and proximal policy optimization algorithm according to claim 2, characterized in that Step S3 specifically includes the following steps: S31. Initialization: The jammer flies over the radar signal coverage area according to the established route; S32. The radar continuously emits pulse signals to scan the adjacent airspace, and tracks and guides strikes on the detected targets according to the radar working logic; S33. The jammer receives the radar pulse signal within a processing time period after time t and extracts the pulse description word (PDW) of each pulse; The pulse description word PDW contains six radar pulse parameters as follows: time of arrival (TOA) of the pulse, direction of arrival (DOA) of the pulse, pulse width (PW), bandwidth (BW), carrier frequency (CF), and in-pulse modulation characteristic (F); The PDW of the i-th pulse is expressed as: PDW i = {TOA i , DOA i , PW i , BW i , CF i , F i} S34. The jammer identifies the radar operating mode through a set of pulse description words and encodes the radar status s t : Among them, is the vector splicing symbol, s t represents the radar state at time t; f n , n ∈ [1, N] represents the radar function corresponding to the radar state s t ; p m , m ∈ [1, M] represents the radar phrase included in the radar function f n ; {w i , w j , w k , w l}, i, j, k, l ∈ [1, T] represents the radar word corresponding to the radar phrase p m ; S35. The jammer encodes s according to the radar status t and the strategy to obtain the jamming pattern and sends the corresponding jamming signal to the radar; S36. After the radar is interfered, it changes its working state according to the processing result of the radar receiver. The jammer obtains the code s of the new state of the radar by re-implementing step S34 t+1 , and through the change s of the radar state t →s t+1 , the reward r is obtained according to the reward function R t ; The reward function is defined as follows according to the threat level of the radar working state: S37. Repeat steps S32 - S36 and record the interaction trajectory sequence (s t ,j t ,r t ) of the jammer and the radar each time until the jammer flies out of the radar signal coverage area or the jammer is destroyed by radar guidance; the expression of the trajectory τ is: τ = {(s1, j1, r1), (s2, j2, r2), …, (s T , j T , r T )} where T is the termination time, that is, at time T, the jammer flies out of the radar signal coverage area or is destroyed by radar guidance; S38. Store the trajectory data obtained by sampling according to the policy into the set D k .

4. A jamming decision-making system combining behavioral cloning and proximal policy optimization algorithm, applied to the jamming decision-making method according to any one of claims 1-3, characterized in that, The described interference decision-making system includes the following modules: An acquisition module, used to acquire the time-domain pulse signals of a multi-functional radar with multiple working modes; A parameter identification module, used to extract the PDW parameters of the radar time-domain pulse signals; A radar state encoding module, used to perform radar state encoding according to the PDW of a group of radar pulse signals; An interference generation module, used to generate time-domain interference signals according to the interference patterns output by the policy network; A data storage module, used to store the data generated by the expert prior knowledge base and the trajectory sequence data of the interaction between the jammer and the radar; A network optimization module, used to optimize the policy network and the value network according to the corresponding objective function.

Images