Electronic countermeasure double-layer modeling and decision optimization method, system, equipment and medium

Abstract

The invention discloses an electronic countermeasure double-layer modeling and decision optimization method, system and device and a medium, and aims at an electronic countermeasure scene to carry out Markov decision process modeling on two electronic countermeasure parties from different time scales. According to the method, the problem that in the electronic countermeasure process, signal level parameters do not have Markov performance under the time scale of function level parameter change frequency is solved, meanwhile, decision optimization of an anti-interference mode and an interference pattern can be conducted according to a model, and the countermeasure performance is improved.

Description

Technical Field

[0001] The present invention relates to the field of electronic countermeasure technology, and in particular to an electronic countermeasure double-layer modeling and decision optimization method, system, equipment and medium. Background Art

[0002] The Markov decision process is a time-related mathematical framework consisting of four elements: state, action, state transition function, and reward function. In the Markov decision process, there is usually an agent that performs an action. The Markov decision process is a process of continuous interaction between the agent and the environment. At a certain moment, the agent selects an action based on the current state and submits it to the environment. The environment obtains the reward value and the next state feedback to the agent based on the state transition function and the reward function. The function that the agent selects an action based on the current state is called a strategy. Generally speaking, the purpose of the agent is to maximize the cumulative reward, so the agent needs to update the strategy based on the feedback from the environment during each interaction, and finally obtain an optimal strategy.

[0003] At present, there are already quite a few electronic confrontation scenarios based on Markov decision process modeling, but most of them remain at the functional level, have not been refined to the signal level, and often only involve unilateral confrontation. Therefore, the confrontation performance needs to be improved. Summary of the invention

[0004] The purpose of the present invention is to provide an electronic countermeasure double-layer modeling and decision optimization method, system, equipment and medium, which can perform electronic countermeasure double-layer from the functional level and signal level, and optimize the decision, so as to improve the countermeasure performance.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] An electronic countermeasure double-layer modeling and decision optimization method, comprising:

[0007] The two parties of the electronic countermeasure are modeled as corresponding agents, the radar party is modeled as a radar agent, and the jammer party is modeled as a jammer agent;

[0008] Establish a corresponding time scale model according to different time intervals of confrontation, model the electronic confrontation process of radar agent and jammer agent based on Markov decision process and game theory, and optimize the radar agent and jammer agent. The optimization process includes: the radar agent and the jammer agent each make action decisions and execute them according to the current state based on the Markov decision process, and then obtain the corresponding reward value and new state based on game theory. The radar agent and the jammer agent update the strategy network and value network based on their respective reward values ​​and new states; wherein the strategy network and value network are both used in the action decision process, the strategy network is used to output the probability distribution of the action, and the value network is used to output the value of the action; after the optimization is completed, the equilibrium strategy of the corresponding time scale model is obtained;

[0009] The equilibrium strategy of the corresponding time scale model is used to guide the one-to-one radar confrontation decision-making process.

[0010] An electronic countermeasure double-layer modeling and decision optimization system, comprising:

[0011] An agent modeling unit is used to model the two electronic countermeasure parties as corresponding agents, the radar party is modeled as a radar agent, and the jammer party is modeled as a jammer agent;

[0012] The electronic countermeasure double-layer modeling and decision optimization unit is used to establish a corresponding time scale model according to the time interval of the confrontation, model the electronic countermeasure process of the radar agent and the jammer agent based on the Markov decision process and game theory, and optimize the radar agent and the jammer agent. The optimization process includes: the radar agent and the jammer agent each make action decisions and execute them according to the current state based on the Markov decision process, and then obtain the corresponding reward value and new state based on game theory. The radar agent and the jammer agent update the strategy network and the value network based on the reward value and the new state obtained by each agent. The strategy network and the value network are both used in the action decision process, the strategy network is used to output the probability distribution of the action, and the value network is used to output the value of the action. After the optimization is completed, the equilibrium strategy of the corresponding time scale model is obtained;

[0013] The decision-making unit is used to guide the one-to-one radar confrontation decision-making process using the equilibrium strategy of the corresponding time scale model.

[0014] A processing device, comprising: one or more processors; a memory for storing one or more programs;

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.

[0016] A readable storage medium stores a computer program, which implements the above method when the computer program is executed by a processor.

[0017] It can be seen from the technical solution provided by the present invention that, for electronic confrontation scenarios, Markov decision process models are performed on both sides of the electronic confrontation from different time scales to solve the problem that signal-level parameters in the electronic confrontation process do not have Markov properties at the time scale of the function-level parameter change frequency. At the same time, the decision optimization of anti-interference methods and interference styles can be performed according to the model to improve the confrontation performance (i.e., improve the anti-interference capability of the radar and improve the interference capability of the jammer). BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0019] Figure 1 A flowchart of an electronic countermeasure double-layer modeling and decision optimization method provided by an embodiment of the present invention;

[0020] Figure 2 A search task flow chart provided for an embodiment of the present invention;

[0021] Figure 3 A confirmation task flow chart provided for an embodiment of the present invention;

[0022] Figure 4 A flow chart of a tracking task provided by an embodiment of the present invention;

[0023] Figure 5 A flow chart of a lost tracking task provided by an embodiment of the present invention;

[0024] Figure 6 A schematic diagram of the switching logic of a radar agent provided by an embodiment of the present invention;

[0025] Figure 7 A schematic diagram of a large time scale model solving and optimization process provided by an embodiment of the present invention;

[0026] Figure 8 A schematic diagram of a small time scale model solving and optimization process provided by an embodiment of the present invention;

[0027] Fig. 9 A schematic diagram of an electronic countermeasure dual-layer modeling and decision optimization system provided by an embodiment of the present invention;

[0028] Fig.10A schematic diagram of a processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0030] First, the terms that may be used in this article are explained as follows:

[0031] The terms "include", "comprises", "contains", "has" or other descriptions with similar semantics should be interpreted as non-exclusive inclusion. For example, including certain technical feature elements (such as raw materials, components, ingredients, carriers, dosage forms, materials, dimensions, parts, components, mechanisms, devices, steps, procedures, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products or products, etc.) should be interpreted as including not only certain technical feature elements explicitly listed, but also other technical feature elements known in the art that are not explicitly listed.

[0032] The term "consisting of..." means excluding any technical feature elements not explicitly listed. If this term is used in a claim, it will make the claim closed, so that it does not contain technical feature elements other than the technical feature elements explicitly listed, except for the conventional impurities related to them. If this term only appears in a clause of a claim, it only limits the elements explicitly listed in the clause, and the elements recorded in other clauses are not excluded from the overall claim.

[0033] Unless otherwise specified or limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this article can be understood according to specific circumstances.

[0034] The following is a detailed description of an electronic countermeasure double-layer modeling and decision optimization method, system, device and medium provided by the present invention. The contents not described in detail in the embodiments of the present invention belong to the prior art known to professional and technical personnel in the field. If no specific conditions are specified in the embodiments of the present invention, the conventional conditions in the field or the conditions recommended by the manufacturer are followed. If the manufacturer of the instruments used in the embodiments of the present invention is not specified, they are all conventional products that can be purchased commercially.

[0035] Embodiment 1

[0036] The embodiment of the present invention provides a two-layer modeling and decision optimization method for electronic countermeasures, such as Figure 1 As shown, it mainly includes the following steps:

[0037] Step 1: Model both parties of the electronic confrontation as corresponding intelligent agents.

[0038] In the embodiment of the present invention, the two electronic countermeasure parties include: a radar and a jammer. The radar party is modeled as a radar agent, and the jammer party is modeled as a jammer agent.

[0039] Step 2: Establish corresponding time scale models according to different time intervals of confrontation, model the electronic confrontation process of radar agent and jammer agent based on Markov decision process and game theory, and optimize the radar agent and jammer agent.

[0040] The optimization process includes: the radar agent and the jammer agent each make action decisions and execute them according to the current state based on the Markov decision process, and then obtain the corresponding reward value and new state based on game theory. The radar agent and the jammer agent update the strategy network and the value network based on their respective reward values ​​and new states; wherein the strategy network and the value network are both used in the action decision process, the strategy network is used to output the probability distribution of the action, and the value network is used to output the value of the action; after the optimization is completed, the equilibrium strategy of the corresponding time scale model is obtained.

[0041] Step 3: Use the equilibrium strategy of the corresponding time scale model to guide the one-to-one radar confrontation decision-making process, that is, use the equilibrium strategy to decide the actions of the radar and jammer.

[0042] In order to more clearly demonstrate the technical solution and technical effects provided by the present invention, the method provided by the embodiment of the present invention is described in detail with reference to specific embodiments below.

[0043] This paper focuses on the scenario of one-on-one electronic confrontation in air combat. Based on the Markov decision process, the confrontation process model of two time scales with different perspectives is studied and optimized. Specifically, this paper defines two time intervals. One time interval is the time for the radar to execute a scheduling cycle, and the corresponding time scale is called the large time scale model; the other time interval is the radar coherent processing interval t CPI , the corresponding time scale is called the small time scale model. The following is a detailed introduction to the two time scale models.

[0044] 1. Large time scale model.

[0045] The parameters of the radar agent are expressed as:

[0046]

[0047] Among them, s R (n), o R (n), a R (n) corresponds to the state, observation, and action of the radar agent in the nth time slot, S R , O R , A R Correspondingly represents the state space, observation space, and action space, r R is the revenue function of the radar agent, W R (n) is the working mode, RL R (n) is the track list; a J (n) is the action of the jammer agent in the nth time slot.

[0048] Those skilled in the art can understand that a time slot is the length of time occupied by a round. Since the time intervals of a large time scale model and a small time scale model are different, the lengths of their time slots are different.

[0049] The observation is the echo signal received by the radar agent, and the echo signal refers to the echo signal received after executing the beam request. Figure 2 to Figure 5 The figure shows the simulation process of executing beam request. The radar executes a beam request among search, confirmation, tracking and loss of tracking according to the current working mode and track list. After executing the beam request, the actions of both radar and jammer jointly generate echo signals, which are received by radar. After that, the echo signals are processed in the simulation program to obtain the target motion information. x is the coordinate in the spherical coordinate system with the radar as the origin and the front as the polar axis. is the radial velocity of the target.

[0050] The working mode is the scheduling logic of the radar agent, which affects the execution order of the four beam requests: search, confirmation, tracking, and loss of tracking. At the same time, it will switch according to the observed value at the end of the time slot. The switching logic is as follows Figure 6 As shown, Figure 6 Medium R i (m) is the radius of the jammer in the spherical coordinate system with the radar as the center, β i (m) is the elevation angle of the jammer in the spherical coordinate system with the radar as the center, and R1, R2, ..., R4 and β0 are setting parameters.

[0051] The action is the anti-interference method adopted by the radar agent, including: frequency agility, RF shielding, and frequency modulation slope agility.

[0052] The track list is divided into a temporary track list and a stable track list, which respectively represent the target motion information history of the confirmed target and the tracked target. After each beam request is executed, the track list for the next time slot is maintained.

[0053] The profit function is expressed as:

[0054]

[0055] Among them, -η1, η2, η3 are the reward values ​​in different situations.

[0056] The parameters of the jammer agent are expressed as:

[0057]

[0058] Among them, s J (n), o J (n), a J (n) corresponds to the state, observation, and action of the jammer agent in the nth time slot, S J , O J , A J Correspondingly represents the state space, observation space, and action space, r J is the reward function of the jammer agent; l J (n),H J (n) is the jammer's own motion information and observation history; a R (n) is the action of the radar agent in the nth time slot;

[0059] The observation is the pulse descriptor of the radar signal intercepted by the jammer agent, that is:

[0060]

[0061] Where T is the transposed symbol, each item represents a pulse description word, and the subscript is the sub-pulse number reached in the nth time slot, K J is the number of sub-pulses intercepted in the nth time slot.

[0062] The jammer's own motion information is consistent with the target information structure obtained by the radar, that is, Considering the target moving model with constant speed, so l J The transfer of (n) can be expressed as:

[0063] l J (n) = Fl J (n-1)

[0064] Among them, the matrix T0 is the time slot length of the large time scale model.

[0065] The transfer of observation history can be obtained by the following formula

[0066] H J (n+1)= H J (n)∪o J (n)

[0067] The action is the jamming style taken by the jammer agent, and the jammer agent generates jamming signals based on the observations and actions.

[0068] The reward function is set to the opposite of the radar perspective model reward. This is to facilitate the solution of the model and set to a zero-sum game. Those skilled in the art can understand that game theory is a reasonable means to study confrontation problems. In the game, it is often assumed that all participants can find the optimal strategy. In particular, the game in which the sum of the gains and losses of each party is always zero is called a zero-sum game. It can be mathematically proved that for all zero-sum game problems, an equilibrium solution with mutually optimal strategies can be obtained by solving linear programming problems.

[0069] The optimization process of solving large time scale models is as follows Figure 7 As shown, input the radar simulation platform parameter structure MPARParam (describing some constants of the radar, such as power, working airspace range, antenna gain, etc.), the number of training rounds e num , exploration rate ε; then, initialize π R (Radar Agent Strategy),π J (Jammer Agent Strategy),Q R (Radar Agent Action Value),Q J (the jammer agent action value), V R (Radar agent state value), V J (Disruptor Agent State Value),s R ,s J ; In each time slot of each iteration (Maxtime is an external hyperparameter that limits the maximum time a game can last), the radar agent and the jammer agent each make an action decision based on the current state with a probability of 1-ε based on the Markov decision process, or randomly select an action according to the uniform distribution with a probability of ε; perform a confrontation simulation for a scheduling cycle to obtain the tracking status of the radar after the confrontation, and the state s′ of the radar agent after the transfer R and the jammer agent state s′ J , thereby obtaining the reward values ​​of the two agents and updating the Q table (a table of long-term rewards corresponding to states and actions, including the aforementioned Q R ,Q J ), strategy (i.e. π R ,π J) and V table (the table of expected long-term returns corresponding to the state corresponds to the aforementioned V R ,V J ), in the relevant formula, α and γ are two hyperparameters, a′ R and a′ J It is a temporary variable in the process of finding the minimum or sum (corresponding to the actions of the two agents). Execute Isterminal(s R ,s j ), that is, to determine whether the current state is the terminal state, that is, whether the game between the radar and the jammer has produced a winner; repeat the iteration until the set number of training rounds e is reached num , output the Nash equilibrium strategy.

[0070] Those skilled in the art can understand that the Nash equilibrium strategy is a pair of strategies in which the strategies of the two opposing parties are mutually optimal solutions, that is, both parties use the optimal interference strategy and anti-interference strategy respectively, that is, when one party adopts the Nash equilibrium strategy, the other party does not adopt the Nash equilibrium strategy, and only suboptimal results can be obtained. Therefore, compared with the unilateral confrontation of the existing scheme, the double-layer electronic confrontation of the present invention can achieve an improvement in confrontation performance.

[0071] 2. Small time scale model.

[0072] The parameters of the radar agent are expressed as:

[0073] {s r (n)={F(n),f c (n),PRF,Z(n)}∈S r ,a r (n)∈A r ,o r (n)∈O r ,r r (s r (n),a r (n),a j (n)),W R ,a R}

[0074] Among them, s r (n), o r (n), a r (n) corresponds to the state, observation, and action of the radar agent in the nth time slot, S r , O r , A r Correspondingly represents the state space, observation space, and action space, r r is the revenue function of the radar agent, W R with a RCorrespondingly, they represent fixed working modes and anti-interference actions. Both are parameters in the large time scale model and can be directly obtained synchronously from the large time scale model. Both will directly affect the radar status of the small time scale model, as shown in Table 1 below; a j (n) is the action of the jammer agent in the nth time slot.

[0075] Similar to the large time scale model, the observation is the echo signal received by the radar agent, and the echo signal is processed to obtain the target motion information.

[0076] Status r (n)={F(n),f c (n),PRF,Z(n)}, where F(n) = {0,1} is the radio frequency shielding flag. When its value is 0, the signal is processed according to the normal process. When its value is 1, the first sub-pulse observed in the nth time slot is used as the shielding pulse, and no signal processing is performed. c (n)={f0,f1,...,f K″} is the carrier frequency, each item is a different carrier frequency point, the subscript is the carrier frequency point number, K″ is the total number of carrier frequency points, and Z(n)={-1,1} is the frequency modulation slope agility mark.

[0077] The action is a binary vector of 1*Np. When the i-th bit is set to 1, the corresponding anti-interference action is taken to make s r (n) The corresponding parameter changes and issues a r (n) The corresponding sub-pulse signal, s r The parameter variation range of (n) is shown in Table 1.

[0078] Table 1: s r (n) Parameter range

[0079]

[0080] The settings of other working modes and anti-interference actions are omitted in Table 1. The working mode settings can be flexibly adjusted according to the actual scenario, so they are not omitted.

[0081] The sub-pulse signal waveform is expressed as follows:

[0082]

[0083] Among them, A is the pulse amplitude, rect is the rectangular window function, and f d is the Doppler frequency, j is the imaginary unit, t is the time variable, B is the bandwidth, t p is the pulse width, and π is the symbol of pi.

[0084] After sending a signal, the radar will receive an echo. The target echo is in the form of:

[0085]

[0086] Where τ(n) is the delay time of the signal return, v(n) is the Doppler frequency, and φ j is an interference signal.

[0087] Revenue function r r Defined as the signal-to-interference ratio of the received echo.

[0088] The parameters of the jammer agent are expressed as:

[0089] {s j (n) = {H j (n)}∈S j ,a j (n)∈A j ,o j (n)∈O j ,r j (s j (n),a j (n),a r (n)),a J}

[0090] Among them, s j (n), o j (n), a j (n) corresponds to the state, observation, and action of the jammer agent in the nth time slot, S j , O j , A j is the corresponding representation of state space, observation space, and action space, r j is the reward function of the jammer agent, H j (n) is the observation history, a J is the interference signal pattern transmitted by the large time scale model. This parameter is synchronized from the large time scale model and used as a parameter for simulating the signal processing process in the small time scale model. r (n) is the action of the radar agent in the nth time slot.

[0091] The observation is the pulse description word of the radar signal intercepted by the jammer agent. In the case of transmission and reception isolation (i.e., the jammer cannot transmit the jamming signal when intercepting the signal), assuming that the duration of each interception by the jammer agent is equal to the repetition interval of the radar pulse, the observation o is defined as j (n) = [o j (n,1),...o j (n,m),...,o j (n,Np)], where oj (n,m) is the observed value of the jammer agent at the mth pulse repetition interval in the nth time slot, defined as:

[0092]

[0093] in, is the pulse description word obtained in the nth time slot, that is, the kth pulse description word obtained in the current large time slot. j A pulse description word, a j (n,m)=1 means the jammer takes reconnaissance action at the mth pulse repetition interval of the nth time slot, a j (n,m)=0, then according to a j (n) and o j (n,m) transmits interference signal.

[0094] The state is defined as H j (n) = {o j (1),…,o j (n-1)}, that is, the set of historical observations; the transfer equation can be expressed as

[0095] The actions of the jammer agent are defined as: Where Np is the total number of pulses transmitted by the radar agent in one time slot.

[0096] Similar to the large time scale model, the jammer agent payoff function is defined as the inverse of the radar payoff function.

[0097] The optimization process of solving small time scale models is as follows Figure 8 As shown, input radar simulation platform parameter structure MPARParam (describing some constants of radar, such as power, working airspace range, antenna gain, etc.), anti-interference method a R , interference pattern a J , the number of training rounds e num , exploration rate ε; then, initialize π r (Radar Agent Strategy),π j (Jammer Agent Strategy),Q r (Radar Agent Action Value),Q j (the jammer agent action value), V r (Radar agent state value), V j (Disruptor Agent State Value),s r ,s j; In each iteration, the radar agent and the jammer agent each make an action decision based on the current state with a probability of 1-ε based on the Markov decision process, or randomly select an action according to the uniform distribution with a probability of ε; simulate the signal-level radar countermeasure process to obtain the signal-to-interference ratio after signal processing and the radar agent state s′ after transfer r and the jammer agent state s′ j , thereby obtaining the reward values ​​of the two agents and updating the Q table (a table of long-term rewards corresponding to states and actions, including the aforementioned Q r ,Q j ), strategy (i.e. π r ,π j ) and V table (the table of expected long-term returns corresponding to the state corresponds to the aforementioned V r ,V j ); Repeat the iteration until the set number of training rounds e is reached num , output the Nash equilibrium strategy. Considering that both the small time scale model and the aforementioned large time scale model are optimized by multi-agent reinforcement learning algorithm, the meanings of the parameters involved are basically the same. The main difference is that the two belong to different time scale models. Therefore, the subscripts are distinguished by uppercase and lowercase.

[0098] The above solution provided by the embodiment of the present invention mainly has the following advantages:

[0099] The present invention aims at specific scenarios, starting from the functional level and signal level in the radar confrontation process, and establishing models for both parties based on Markov decision process and game theory and solving optimization. At a large time scale, the definition of state, action, and benefit in the model is given based on the basic elements in the radar confrontation process, and according to the process of radar signal processing, the real process corresponding to the model state transfer is explained, and the large-time scale Markov modeling of the radar perspective and the jammer perspective is completed. The decision optimization of the anti-interference mode and the jamming style can be performed according to the model. At a small time scale, based on the fact that the antenna is isolated from the transmitter and receiver, the game of intermittent observation of the jammer and the agile change of radar waveform parameters is focused on, and the small-time scale Markov modeling of the radar perspective and the jammer perspective is completed, solving the problem that the changes of some parameters in the small-time scale model of radar confrontation do not conform to the Markov property. In terms of model solution optimization, a multi-agent reinforcement learning algorithm is used to perform model solution optimization on a radar confrontation simulation platform.

[0100] For ease of understanding, the following example is provided using the one-on-one radar confrontation process in air combat as an example; the main process of this example is as follows:

[0101] 1. Adjust the size of the model state space and action space according to the number of radar working modes, the number of anti-interference actions, and the number of jammer interference patterns.

[0102] 2. According to the feedback from the radar simulation program, run the small-time-scale model solving algorithm for each anti-interference action-interference pattern pair to obtain the equilibrium strategy of the small-time-scale model.

[0103] 3. Since the benefit of the large-time-scale model depends on the simulation of the signal processing process, and the simulation results of the signal processing process are affected by the equilibrium strategy of the small-time-scale model, the equilibrium strategy of the small-time-scale model is brought into the solution optimization process of the large-time-scale model to obtain the equilibrium strategy of the large-time-scale models of both parties.

[0104] 4. According to the equilibrium strategy of the large-time scale model and the equilibrium strategy of the small-time scale model, guide the decision-making process of one-on-one radar confrontation in air combat. Specifically: first use the large-time scale model to determine the interference style or anti-interference action adopted in the current state, and then use the small-time scale model to determine the intermittent observation behavior in the interference style or anti-interference action.

[0105] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiments can be implemented by software, or by means of software plus necessary general hardware platforms. Based on such understanding, the technical solutions of the above embodiments can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0106] Embodiment 2

[0107] The present invention also provides an electronic countermeasure dual-layer modeling and decision optimization system, which is mainly used to implement the method provided in the above embodiment, such as Fig. 9 As shown, the system mainly includes:

[0108] An agent modeling unit is used to model the two electronic countermeasure parties as corresponding agents, the radar party is modeled as a radar agent, and the jammer party is modeled as a jammer agent;

[0109] The electronic countermeasure double-layer modeling and decision optimization unit is used to establish a corresponding time scale model according to the time interval of the confrontation, model the electronic countermeasure process of the radar agent and the jammer agent based on the Markov decision process and game theory, and optimize the radar agent and the jammer agent. The optimization process includes: the radar agent and the jammer agent each make action decisions and execute them according to the current state based on the Markov decision process, and then obtain the corresponding reward value and new state based on game theory. The radar agent and the jammer agent update the strategy network and the value network based on the reward value and the new state obtained by each agent. The strategy network and the value network are both used in the action decision process, the strategy network is used to output the probability distribution of the action, and the value network is used to output the value of the action. After the optimization is completed, the equilibrium strategy of the corresponding time scale model is obtained;

[0110] The decision-making unit is used to guide the one-to-one radar confrontation decision-making process using the equilibrium strategy of the corresponding time scale model.

[0111] Considering that the main technical details involved in the system have been introduced in detail in the previous embodiments, they will not be repeated here.

[0112] Technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above.

[0113] Embodiment 3

[0114] The present invention also provides a processing device, such as Fig.10 As shown, it mainly includes: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the methods provided in the aforementioned embodiments.

[0115] Furthermore, the processing device also includes at least one input device and at least one output device; in the processing device, the processor, memory, input device, and output device are connected via a bus.

[0116] In the embodiment of the present invention, the specific types of the memory, input device and output device are not limited; for example:

[0117] The input device may be a touch screen, an image acquisition device, a physical button or a mouse, etc.;

[0118] The output device may be a display terminal;

[0119] The memory may be a random access memory (RAM) or a non-volatile memory, such as a disk memory.

[0120] Embodiment 4

[0121] The present invention also provides a readable storage medium storing a computer program, which implements the method provided in the above embodiment when the computer program is executed by a processor.

[0122] In the embodiment of the present invention, the readable storage medium is a computer-readable storage medium and can be set in the aforementioned processing device, for example, as a memory in the processing device. In addition, the readable storage medium can also be a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk, etc., which can store program codes.

[0123] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims. The information disclosed in the background technology section of this article is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as an admission or in any form that the information constitutes prior art known to those skilled in the art.

Claims

1. A two-layer modeling and decision optimization method for electronic countermeasures, characterized in that: include: The two parties of the electronic countermeasure are modeled as corresponding agents, the radar party is modeled as a radar agent, and the jammer party is modeled as a jammer agent; Establish a corresponding time scale model according to different time intervals of confrontation, model the electronic confrontation process of radar agent and jammer agent based on Markov decision process and game theory, and optimize the radar agent and jammer agent. The optimization process includes: the radar agent and the jammer agent each make action decisions and execute them according to the current state based on the Markov decision process, and then obtain the corresponding reward value and new state based on game theory. The radar agent and the jammer agent update the strategy network and value network based on their respective reward values ​​and new states; wherein the strategy network and value network are both used in the action decision process, the strategy network is used to output the probability distribution of the action, and the value network is used to output the value of the action; after the optimization is completed, obtain the equilibrium strategy of the corresponding time scale model; The equilibrium strategy of the corresponding time scale model is used to guide the one-to-one radar confrontation decision-making process.

2. The electronic countermeasure double-layer modeling and decision optimization method according to claim 1 is characterized in that: The establishing of a corresponding time scale model according to different time intervals of the confrontation includes: Two time intervals are defined. One time interval is the time for the radar to execute a scheduling cycle, and the corresponding time scale is called the large time scale model. The other time interval is the coherent processing interval of the radar, and the corresponding time scale is called the small time scale model. For the large time scale model, during the optimization process, the reward value of the radar agent is determined in combination with the radar's tracking situation; for the small time scale model, during the optimization process, the reward value of the radar agent is determined in combination with the signal-to-interference-noise ratio of the received echo signal; in both types of time scale models, the reward value of the jammer agent is the opposite value of the reward value of the radar agent in the corresponding time scale model.

3. The electronic countermeasure double-layer modeling and decision optimization method according to claim 1 or 2, characterized in that: The radar agent and the jammer agent each make action decisions based on the Markov decision process according to the current state, including: According to the set exploration rate ε, the radar agent and the jammer agent each make an action decision based on the current state with a probability of 1-ε based on the Markov decision process, or randomly select an action according to the uniform distribution with a probability of ε.

4. The electronic countermeasure double-layer modeling and decision optimization method according to claim 2 is characterized in that: In the large time scale model, the parameters of the radar agent are expressed as: Among them, s R (n), o R (n), a R (n) corresponds to the state, observation, and action of the radar agent in the nth time slot, S R , O R , A R Correspondingly represents the state space, observation space, and action space, r R is the revenue function of the radar agent, W R (n) is the working mode, RL R (n) is the track list; a J (n) is the action of the jammer agent in the nth time slot; observation is the target motion information obtained by the radar agent by processing the received echo signal, and the echo signal refers to the echo signal received after executing the beam request; the working mode is the scheduling logic of the radar agent, which affects the execution order of the beam request; the action is the anti-interference method adopted by the radar agent; the track list is divided into a temporary track list and a stable track list, which respectively represent the target motion information history of the confirmed target and the tracked target; The profit function is expressed as: Among them, -η1, η2, η3 are the reward values ​​in different situations.

5. The electronic countermeasure double-layer modeling and decision optimization method according to claim 2 or 4, characterized in that: In the large time scale model, the parameters of the jammer agent are expressed as: Among them, s J (n), o J (n), a J (n) corresponds to the state, observation, and action of the jammer agent in the nth time slot, S J , O J , A J Correspondingly represents the state space, observation space, and action space, r J is the reward function of the jammer agent; l J (n),H J (n) is the jammer's own motion information and observation history; a R (n) is the action of the radar agent in the nth time slot; the observation is the pulse descriptor of the radar signal intercepted by the jammer agent; the action is the jamming style adopted by the jammer agent.

6. The electronic countermeasure double-layer modeling and decision optimization method according to claim 2 is characterized in that: In the small time scale model, the parameters of the radar agent are expressed as: {s r (n) = {F(n), f c (n), PRF, Z(n)} ∈ S r , a r (n) ∈ A r , o r (n) ∈ O r , r r (s r (n), a r (n), a j (n)), W R , a R} Wherein, s r (n), o r (n), a r (n) corresponds to the state, observation, and action of the radar agent in the nth time slot, S r , O r , A r Correspondingly represents the state space, observation space, and action space, r r is the profit function of the radar agent; W R with a R The corresponding representations are fixed working modes and anti-interference actions, both of which are parameters in the large time scale model and are obtained synchronously from the large time scale model; a j (n) is the action of the jammer agent in the nth time slot; observation is the target motion information obtained by the radar agent processing the received echo signal; state s r (n)={F(n),f c (n),PRF,Z(n)}, where F(n) = {0,1} is the radio frequency shielding flag. When its value is 1, the first sub-pulse observed in the nth time slot is used as the shielding pulse, and no signal processing is performed. c (n)={f0,f1,...,f K″ } is the carrier frequency, each item is a different carrier frequency point, the subscript is the carrier frequency point number, K″ is the total number of carrier frequency points, Z(n)={-1,1} is the frequency modulation slope agility mark; the benefit function r r Defined as the signal-to-interference ratio of the received echo; The action is a binary vector of 1*Np. When the nth bit is 1, the corresponding anti-interference action is taken to make s r (n) The corresponding parameter changes and issues a r (n) The corresponding sub-pulse signal, the sub-pulse signal waveform is expressed as follows: Among them, A is the pulse amplitude, rect is the rectangular window function, and f d is the Doppler frequency, j is the imaginary unit, t is the time variable, B is the bandwidth, t p is the pulse width, and π is the symbol of pi.

7. The electronic countermeasure double-layer modeling and decision optimization method according to claim 2 or 6, characterized in that: In the small time scale model, the parameters of the jammer agent are expressed as: {s j (n)={H j (n)}∈S j ,a j (n)∈A j ,o j (n)∈O j ,r j (s j (n),a j (n),a r (n)),a J } Among them, s j (n), o j (n), a j (n) corresponds to the state, observation, and action of the jammer agent in the nth time slot, S j , O j , A j is the corresponding representation of state space, observation space, and action space, r j is the reward function of the jammer agent, H j (n) is the observation history, a J The interference signal pattern delivered to the large time scale model; a r (n) is the action of the radar agent in the nth time slot; the state is defined as H j (n) = {o j (1),…,o j (n-1)}, that is, the set of historical observations; The observation is the pulse description word of the radar signal intercepted by the jammer agent. In the case of transmission and reception isolation, assuming that the duration of each interception by the jammer agent is equal to the repetition interval of the radar pulse, the observation o is defined j (n) = [o j (n,1),...o j (n,m),...,o j (n,Np)], where o j (n,m) is the observed value of the jammer agent at the mth pulse repetition interval in the nth time slot, defined as: in, is the pulse description word obtained in the nth time slot, that is, the kth pulse description word obtained in the current large time slot. j A pulse description word, a j (n,m)=1 means the jammer takes reconnaissance action at the mth pulse repetition interval of the nth time slot, a j (n,m)=0, then according to a j (n) and o j (n,m) transmits interference signal; The action of the jammer agent is defined as: a j (n) = [a j (n,1),...,a j (n,m),...,a j (n,Np)], where Np is the total number of pulses transmitted by the radar agent in one time slot.

8. An electronic countermeasure two-layer modeling and decision optimization system, characterized in that: include: An agent modeling unit is used to model the two electronic countermeasure parties as corresponding agents, the radar party is modeled as a radar agent, and the jammer party is modeled as a jammer agent; The electronic countermeasure double-layer modeling and decision optimization unit is used to establish a corresponding time scale model according to the time interval of the confrontation, model the electronic countermeasure process of the radar agent and the jammer agent based on the Markov decision process and game theory, and optimize the radar agent and the jammer agent. The optimization process includes: the radar agent and the jammer agent each make action decisions and execute them according to the current state based on the Markov decision process, and then obtain the corresponding reward value and new state based on game theory. The radar agent and the jammer agent update the strategy network and the value network based on the reward value and the new state obtained by each agent. The strategy network and the value network are both used in the action decision process, the strategy network is used to output the probability distribution of the action, and the value network is used to output the value of the action. After the optimization is completed, the equilibrium strategy of the corresponding time scale model is obtained; The decision-making unit is used to guide the one-to-one radar confrontation decision-making process using the equilibrium strategy of the corresponding time scale model.

9. A processing device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images