Radar intelligent anti-interference method combining attention mechanism and A2C algorithm

Abstract

The invention specifically relates to a radar intelligent anti-interference method combining an attention mechanism and an A2C algorithm, and the method comprises the steps: constructing a Markov decision process MDP model corresponding to a radar, a target and a jammer, and defining a state space, an action space and a multi-target reward function corresponding to the MDP model; performing feature extraction on the state data by using a preprocessing layer based on a self-attention mechanism to obtain corresponding state features; wherein the state data comprises interference signal frequency information; determining a frequency agility strategy corresponding to the state characteristic by using an actor network; controlling the radar to execute a frequency agility strategy so as to acquire updated state data; the updated state data and the frequency agility strategy are input into a commentator network, and the time difference deviation of the action value function in the current state is obtained; and performing gradient updating on the actor network by using the time difference deviation, and updating network parameters of the actor network by using the updated gradient so as to generate a frequency agility strategy by using the actor network with the updated network parameters.

Description

Technical Field

[0001] The present invention relates to the field of radar technology, and in particular to a radar intelligent anti-interference method combining an attention mechanism and an A2C algorithm. Background Art

[0002] Among related technologies, the development of radar anti-jamming technology stems from its application in the military. In modern electronic warfare, radars perform crucial tasks such as detection and tracking, and ensuring their proper functioning is crucial to their users. Consequently, electronic countermeasures (ECM) methods have become widely used. Jammers disrupt enemy radars' effective use of the electromagnetic spectrum, thereby achieving interference. When jammed, radars are unable to effectively receive information-carrying signals and may even be misled. To minimize the impact of jamming on radars, developing more efficient and effective radar anti-jamming technologies is crucial in ECM.

[0003] In electronic countermeasures, jamming techniques can be categorized as active or passive, depending on whether the jammer can actively emit electromagnetic waves. Active jamming, in which a jammer emits jamming signals to disrupt radar operation, has drawn considerable attention. Currently, active jamming techniques have evolved into sophisticated and diverse methods, including aiming and sweeping frequency jamming. In practice, radars face a complex environment characterized by diverse interference, placing higher demands on the adaptability of radar anti-jamming technology. Furthermore, jammers are becoming increasingly intelligent. In some radar anti-jamming challenges, jammers can utilize advanced techniques to perform detailed analysis of radar anti-jamming strategies and refine their own jamming signal transmission strategies. Therefore, addressing complex and intelligent jamming methods is a key topic in radar anti-jamming research.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0005] The present invention provides a radar intelligent anti-interference method, a computer program product, an electronic device, and a storage medium that combine an attention mechanism and an A2C algorithm. The method can adaptively adjust the radar anti-interference strategy and overcome the defects of the existing technology to a certain extent.

[0006] Other features and advantages of the present invention will become apparent from the following detailed description, or may be learned in part by practice of the present invention.

[0007] According to a first aspect of the present invention, a radar intelligent anti-interference method combining an attention mechanism and an A2C algorithm is provided, the method comprising:

[0008] Construct a Markov decision process (MDP) model corresponding to the radar, target, and jammer, and define the state space, action space, and multi-objective reward function corresponding to the MDP model;

[0009] A pre-processing layer based on a self-attention mechanism is used to extract features from the state data to obtain corresponding state features; wherein the state data includes the frequency information of the interference signal;

[0010] Using the actor network to determine the frequency agility strategy corresponding to the state characteristics; and controlling the radar to execute the frequency agility strategy to collect updated state data;

[0011] Input the updated state data and frequency agility strategy into the critic network to obtain the time difference deviation of the action value function in the current state;

[0012] The actor network is gradient updated using the temporal difference bias, and the network parameters of the actor network are updated using the updated gradient, so as to generate a frequency agility strategy using the actor network after the network parameters are updated.

[0013] In some exemplary embodiments, the state space includes frequency information of the radar transmission signal and frequency information of the interference signal, which are configured as observation information; the action space includes the frequency of the radar; and the multi-objective reward function includes: the detection probability of the radar and the signal-to-interference-noise ratio of the radar.

[0014] In some exemplary embodiments, constructing a Markov decision process (MDP) model corresponding to the radar, the target, and the jammer includes:

[0015] Define the action space, including: at time t, the radar agent observes the current environment and obtains the corresponding state s t , and for state s t Determine the corresponding frequency agility strategy; the frequency agility strategy includes: carrier frequency a t ; The carrier frequency is selected from the given M frequency points;

[0016] Define the state space, including: the radar agent observes the current environment to obtain the corresponding state s t , including the frequency information of radar transmission signals and the frequency information of interference signals transmitted by jammers;

[0017] Define a multi-objective reward function, including: the state s at time t t Transfer to the next state s t+1 , the radar agent receives a reward r t , which is used to describe the radar agent's selection of frequency agility strategy for radar anti-interference effect.

[0018] In some exemplary embodiments, the frequency information of the interference signal includes a binary group based on a center frequency and a bandwidth.

[0019] In some exemplary embodiments, the multi-objective reward function includes: a radar detection probability indicator and a radar signal-to-interference-and-noise ratio indicator.

[0020] In some exemplary embodiments, a preprocessing layer based on a self-attention mechanism is used to extract features from state data to obtain corresponding state features, including:

[0021] Map state data into query matrix Q, key matrix K and value matrix V;

[0022] Configure the attention weight matrix based on the query matrix Q, key matrix K and value matrix V;

[0023]

[0024] Among them, d k is the matrix dimension;

[0025] The state feature vector after weighting the state data is calculated using the attention weight matrix; the state feature vector includes: according to the current frequency agility strategy π θ In state s t Select radar frequency hopping action a t , radar agent performs frequency hopping action a t And transmit the signal, the interference signal emitted by the jammer in the environment, calculate the detection probability and signal to interference noise ratio according to the frequency information of the radar and the jammer in the environment, and get the reward r t and the state s at the next moment t+1 .

[0026] In some exemplary embodiments, the updated state data and the frequency agility strategy are input into the critic network to obtain the time difference deviation of the action-value function in the current state, including:

[0027] δ t =r t +γV φ (s t+1 )-V φ (s t )

[0028] Among them, V φ (s t ) is the state value function;

[0029] Construct the MSE loss function and minimize it to update the network parameters.

[0030] In some exemplary embodiments, the method further includes: outputting an optimal action for the radar based on the current state of the radar and feedback from the commentator network, including:

[0031] Calculate the advantage function A φ (s t ,a t ) and uses it to compute the gradients of the actor network, including:

[0032]

[0033] Use the gradients to update the actor network parameters, including:

[0034]

[0035] Among them, s t is the state at time t, a t is the radar frequency modulation action at time t.

[0036] According to a second aspect of the present invention, a computer program product is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned radar intelligent anti-interference method combining the attention mechanism and the A2C algorithm is implemented.

[0037] According to a third aspect of the present invention, there is provided an electronic device, comprising:

[0038] processor; and

[0039] a memory for storing executable instructions of the processor;

[0040] The processor is configured to implement the above-mentioned radar intelligent anti-interference method combining the attention mechanism and the A2C algorithm by executing the executable instructions.

[0041] According to a fourth aspect of the present invention, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the radar intelligent anti-interference method combining the attention mechanism and the A2C algorithm is implemented.

[0042] The radar intelligent anti-interference method provided by the embodiment of the present invention, which combines the attention mechanism and the A2C algorithm, can accurately extract key signals by setting a preprocessing layer based on the self-attention mechanism, thereby effectively improving the decision-making accuracy of the frequency agility strategy. The reinforcement learning method based on the actor-critic architecture enables strategy optimization and value estimation to be carried out in parallel, converges faster, and avoids the computational difficulties of traditional methods in the high-dimensional state space. At the same time, the system continuously updates the strategy through a real-time feedback mechanism, so that the radar can adapt to the dynamically changing interference environment and improve its anti-interference ability. Overall, this solution has more advantages than the existing technology in terms of intelligent decision-making, learning efficiency and environmental adaptability of frequency agility, and can effectively improve the performance of the radar system in complex electromagnetic environments.

[0043] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings are incorporated into and constitute a part of this specification, illustrate embodiments consistent with the present invention, and together with the description, serve to explain the principles of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and it is clear that those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0045] Figure 1 A schematic diagram schematically illustrates an exemplary embodiment of the present invention, a radar intelligent anti-interference method combining an attention mechanism and an A2C algorithm;

[0046] Figure 2 A schematic diagram schematically illustrates the principle of a radar intelligent anti-interference method combining an attention mechanism and an A2C algorithm according to an exemplary embodiment of the present invention;

[0047] Figure 3 A schematic diagram schematically illustrates a process flow of a radar intelligent anti-interference method combining an attention mechanism and an A2C algorithm according to an exemplary embodiment of the present invention;

[0048] Figure 4 A schematic diagram schematically illustrates the performance of different algorithms when a reward is a signal-to-interference-and-noise ratio according to an exemplary embodiment of the present invention;

[0049] Figure 5 A schematic diagram schematically illustrates the performance of different algorithms when the reward is detection probability according to an exemplary embodiment of the present invention;

[0050] Figure 6 A schematic diagram schematically illustrates a convergence comparison when SINR is a reward according to an exemplary embodiment of the present invention;

[0051] Figure 7A schematic diagram schematically illustrates a convergence comparison when the detection probability is a reward according to an exemplary embodiment of the present invention;

[0052] Figure 8 The figure schematically shows the composition of an electronic device in an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0053] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0054] In addition, the accompanying drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0055] Traditional radar anti-jamming methods rely on fixed frequency hopping rules or simple reinforcement learning algorithms, making them incapable of coping with the dynamic game of intelligent jammers. Policy gradient algorithms suffer from high variance, while value function-based methods (such as DQN) are inefficient in continuous action spaces. Furthermore, radar signal feature extraction lacks the ability to focus on key jammers, resulting in insufficient decision robustness.

[0056] In view of the shortcomings and deficiencies of the existing technology, this example embodiment provides a radar intelligent anti-interference method that combines the attention mechanism and the A2C algorithm. By dynamically modeling the game process between the radar and the jammer, enhancing the key signal recognition capability, and optimizing the strategy update mechanism, high-precision, low-latency anti-interference decision-making is achieved. Figure 1 As shown in FIG, the radar intelligent anti-interference method combining the attention mechanism and the A2C algorithm can specifically include the following steps:

[0057] Step S11, constructing a Markov decision process (MDP) model corresponding to the radar, target, and jammer, and defining the state space, action space, and multi-objective reward function corresponding to the MDP model;

[0058] Step S12: extracting features from the state data using a preprocessing layer based on a self-attention mechanism to obtain corresponding state features; wherein the state data includes interference signal frequency information;

[0059] Step S13, using the actor network to determine a frequency agility strategy corresponding to the state feature; and controlling the radar to execute the frequency agility strategy to collect updated state data;

[0060] Step S14: input the updated state data and frequency agility strategy into the critic network to obtain the time difference deviation of the action-value function in the current state;

[0061] Step S15: performing gradient update on the actor network using the time difference deviation, and updating the network parameters of the actor network using the updated gradient, so as to generate a frequency agility strategy using the actor network after the network parameters are updated.

[0062] Below, the various steps of the radar intelligent anti-interference method combining the attention mechanism and the A2C algorithm in this example implementation will be described in more detail with reference to the accompanying drawings and embodiments.

[0063] In step S11, a Markov decision process (MDP) model corresponding to the radar, target, and jammer is constructed, and the state space, action space, and multi-objective reward function corresponding to the MDP model are defined.

[0064] Exemplarily, the state space includes frequency information of the radar transmission signal and frequency information of the interference signal, which are configured as observation information; the action space includes the frequency of the radar; and the multi-objective reward function includes: the detection probability of the radar and the signal-to-interference-noise ratio of the radar.

[0065] Exemplarily, the step of constructing a Markov decision process (MDP) model corresponding to the radar, the target, and the jammer includes:

[0066] Define the action space, including: at time t, the radar agent observes the current environment and obtains the corresponding state s t , and for state s t Determine the corresponding frequency agility strategy; the frequency agility strategy includes: carrier frequency a t ; The carrier frequency is selected from the given M frequency points;

[0067] Define the state space, including: the radar agent observes the current environment to obtain the corresponding state s t , including the frequency information of radar transmission signals and the frequency information of interference signals transmitted by jammers;

[0068] Define a multi-objective reward function, including: the state s at time t t Transfer to the next state st+1 , the radar agent receives a reward r t , which is used to describe the radar agent's selection of frequency agility strategy for radar anti-interference effect.

[0069] Exemplarily, the frequency information of the interference signal includes a binary group based on a center frequency and a bandwidth.

[0070] Specifically, we can first define the Markov decision process model of radar anti-interference. Specifically, at time t, the radar obtains the state s at time t by observing the environment. t The radar agent selects an action, that is, it selects the current state s according to the current strategy π t The corresponding carrier frequency a t and transmits radar RF signals. At the same time, the jammer can be regarded as part of the environment, which can transmit jamming signals Interference signal is a two-tuple consisting of the center frequency and bandwidth. Then the state will change from s t Transfer to the next moment s t+1 , the radar agent will receive a reward r t , which describes the pros and cons of the radar agent's choice of frequency hopping action on the radar's anti-interference effect.

[0071] Combined with the construction of the radar anti-interference model, the key elements of the MDP can be defined in the radar anti-interference reinforcement learning algorithm as follows:

[0072] The model's action space describes the radar's behavior, specifically, the radar's frequency agility according to the strategy. For each radar pulse, its carrier frequency can be selected from any of M given frequency points. M represents the size of the action space in reinforcement learning. Since the frequency of the frequency-agile radar's transmitted waveform changes linearly with time, the agent's action can be set to select an appropriate frequency from among these M frequency points. This can be expressed as:

[0073]

[0074] in, is the starting frequency of the LFM waveform (linear frequency modulation waveform) emitted by the radar, is the end frequency of the LFM waveform in Hertz.

[0075] For the model's state space, consider using interference detectors and other methods to perceive interference signals during radar operation. During this process, the radar's perception of interference signals is configured as the radar's observations. By perceiving and analyzing interference signals, the radar agent can more accurately understand the current electromagnetic environment, providing important reference information for formulating and adjusting anti-interference strategies. The current state of the radar's anti-interference environment can be defined as:

[0076]

[0077] Where i∈{0,1,…,t}, and They are the frequency information of the signals transmitted by radar and jammer respectively.

[0078] In order to increase the information obtained by the radar agent and learn historical experience, the agent can use historical information H t Instead of the current state information to learn and make decisions. In RL (Reinforcement Learning) theory, historical information H t It can be seen as a t and observe t A set of sequences is represented as:

[0079] H t =a0,o1,…,a t-1 ,o t

[0080] Among them, t is the radar’s observation result of the interference signal, that is, the frequency information of the interference signal. In practical applications, if the radar makes decisions only based on the data of interaction with the environment, the history contains all the information about the environment. t Frequency information of radar transmission signal.

[0081] Obviously, as time t increases, H t The size of will also increase significantly, which brings difficulties to the input of the model. Therefore, the radar cannot directly use it as the input state. To solve this problem, the kth-order history method can be used to obtain an approximate result of historical information. This method uses a set of sequences consisting of k observations and actions in the past to approximate the historical information H t Therefore, the state of the frequency agile radar agent at time t can be expressed as:

[0082] s t =a t ,o t ,a t-1 ,…,ot-k

[0083] in, Indicates the carrier information of the radar and jammer at time t, that is, the starting frequency and ending frequency in a cycle, representing the transmitted signal information. t is the action taken by the radar at time t, that is, the frequency information of the radar's transmitted signal. The number of radar's selected frequency bands is m. Since the linear frequency modulation signal has a starting frequency and an ending frequency, the size of the action space is 1×m×m. The radar's observation space dimension is 3×k, where the first dimension is 3, indicating k is the length of the truncated historical information.

[0084] The model's rewards can be used to evaluate the agent's strategy. The goal of radar anti-jamming is to ensure the radar's normal operation and complete detection and other tasks. Therefore, rewards should also be formulated based on principles that reflect the radar's anti-jamming capabilities. This chapter develops evaluation criteria for radar anti-jamming strategies, primarily based on two metrics: radar detection probability and radar signal-to-interference-noise ratio.

[0085] The detection probability of a radar refers to the probability that the system can correctly detect a target in the presence of noise interference and a target object. A high detection probability means that the radar can still effectively detect the target in various interference environments, reflecting the reliability and sensitivity of the radar system. Generally speaking, when the false alarm probability value set by the frequency agile radar is very small and the single pulse detection method is used, the detection probability P of the frequency agile radar is d It can be calculated by the following formula, which is expressed as:

[0086]

[0087] Among them, S N is the signal-to-noise ratio of a single pulse, y0=-ln(P f ), P f is the false alarm probability.

[0088] When the false alarm probability P of the frequency agile radar is f =10 -6 When the number of pulse accumulations n received by the frequency agile radar is large, the detection probability of the frequency agile radar on the target can be calculated using the following formula:

[0089]

[0090] Where n is the accumulated value of radar pulses in one scan, s N is the signal-to-noise ratio of a single pulse of the radar.

[0091]

[0092] In the calculation formula of detection probability, D M =n γ , γ is the efficiency of accumulation, and its value is related to the size of n.

[0093] In most cases, the value of γ is between 0.7 and 0.9, and only when it is very large will its value be close to 0.5. f is the upper α quantile of the false alarm probability.

[0094] The signal-to-interference-and-noise ratio (SIR) of a radar is the ratio of the radar signal to the sum of interference and noise. Its practical meaning is the ratio between the strength of the signal received by the radar that is useful for its work and the strength of the interference signal (noise plus interference) received by the radar. The formula is:

[0095]

[0096] The signal-to-interference-and-noise ratio is an important indicator to measure the extent of interference to the radar caused by the jammer. The larger the signal-to-interference-and-noise ratio, the less interference the radar suffers and the better the anti-interference effect.

[0097] These two metrics enable us to comprehensively evaluate the performance of radar anti-interference strategies. During learning, the agent continuously optimizes its frequency hopping strategy to maximize detection probability and signal-to-interference-noise ratio, thereby improving the radar system's anti-interference capabilities in complex electromagnetic environments.

[0098] In step S12, a preprocessing layer based on a self-attention mechanism is used to extract features of the state data to obtain corresponding state features; wherein the state data includes interference signal frequency information.

[0099] Exemplarily, the state data is mapped into a query matrix Q, a key matrix K, and a value matrix V;

[0100] Configure the attention weight matrix based on the query matrix Q, key matrix K and value matrix V;

[0101]

[0102] Among them, d k is the matrix dimension;

[0103] The state feature vector after weighting the state data is calculated using the attention weight matrix; the state feature vector includes: according to the current frequency agility strategy π θ In state s t Select radar frequency hopping action a t , radar agent performs frequency hopping action a tAnd transmit the signal, the interference signal emitted by the jammer in the environment, calculate the detection probability and signal to interference noise ratio according to the frequency information of the radar and the jammer in the environment, and get the reward r t and the state s at the next moment t+1 .

[0104] Specifically, the calculation formula of the self-attention mechanism includes:

[0105]

[0106] Where Q = XW Q ; K = XW K ; V = XW V . X is the input feature, W Q 、W K and W V is a trainable parameter matrix.

[0107] Since the input feature form is s t =a t ,o t-1 ,a t-1 ,…,o t-k ], k=4. So we can get:

[0108] Q=s t W Q

[0109] K=s t W K

[0110] V=s t W V

[0111] The preprocessing layer based on the self-attention mechanism outputs the weighted state feature vector, including the current frequency hopping strategy π θ In state s t Select radar frequency hopping action a t , radar agent performs frequency hopping action a t And transmit the signal, the jammer in the environment transmits the jamming signal, and the detection probability and signal to interference noise ratio are calculated according to the frequency information of the radar and the jammer in the environment, and the reward r is obtained. t and the state s at the next moment t+1 .

[0112] Specifically, by introducing a preprocessing layer based on the self-attention mechanism before the first layer of the actor network in the Attention-A2C algorithm, the state (including radar and environmental information) is processed. The self-attention layer enables the model to better focus on important parts of the state space. This not only improves the performance and generalization of the model, but also helps the model capture long-range dependencies in the state space. Specifically, the self-attention layer can better understand and process sequential data by processing state changes in historical radar information.

[0113] In step S13, the actor network is used to determine the frequency agility strategy corresponding to the state characteristics; and the radar is controlled to execute the frequency agility strategy to collect updated state data.

[0114] For example, in the actor network, the optimal action of the radar, namely the frequency agility strategy, can be output according to the current state of the radar and the feedback from the critic network.

[0115] Specifically, the radar agent obtains the jammer frequency information through observation and combines it with its own actions to form an action space, which can be expressed as:

[0116]

[0117] Construct radar frequency hopping action and observations of jammers The optimal frequency agility strategy is selected from a predefined set of candidate frequencies and frequency hopping is performed.

[0118] In step S14, the updated state data and frequency agility strategy are input into the critic network to obtain the time difference deviation TD of the action-value function in the current state.

[0119] For example, for the critic network, a value function can be constructed based on the current state and action of the radar to estimate the long-term cumulative reward of the radar.

[0120] Specifically, in the critic network, the calculation of the Q value can be divided into two parts: the state value function V(s) and the advantage value A(s,a). The formula is expressed as:

[0121] Q(s,a)=V(s)+A(s,a)

[0122] The A2C algorithm uses the advantage function A π (s t ,a t ), which is used to replace the value function in the critic network. The advantage of this function is that it can be used to evaluate the value of the selected agent's action and compare it with the average value of all the actions to be selected. The result obtained through this comparison is an indicator of the quality of the selected action value.

[0123] The advantage function can be defined as:

[0124]

[0125] Among them, Q π (s t ,a t ) is the action value function, V π (s t ) is the state value function. Using advantage functions instead of value functions can avoid penalizing certain actions because the agent currently adopts a bad strategy. Similarly, it also avoids not rewarding actions when the agent currently follows a good strategy. π (s t ) is unknown, and a parameterized network is needed to estimate it, which can be recorded as V φ (s), the policy gradient formula in A2C becomes:

[0126]

[0127] Here A φ (s t ,a t ) can use the state value function V φ (s t ) is estimated and calculated. Similar to the AC algorithm, V in A2C φ (s t ) also uses the temporal difference method to update it. The TD target here is:

[0128] y=r t +γV φ (s t+1 )

[0129] The form of TD deviation is expressed as:

[0130] δ t =r t +γV φ (s t+1 )-V φ (s t )

[0131] The parameters of the value network can be updated by constructing an MSE loss function and minimizing it.

[0132] A φ (s t ,a t) is greater than 0, indicating that the selected action is better than the average action. In this case, its value is positive, which allows the network parameters θ to follow the positive gradient. Conversely, when its value is less than 0, it means that the agent's action is inferior to the average action, which should be avoided. Its negative value allows the network parameters θ to follow the negative gradient.

[0133] In step S15, the actor network is gradient updated using the time difference deviation, and the network parameters of the actor network are updated using the updated gradient, so as to generate a frequency agility strategy using the actor network after the network parameters are updated.

[0134] For example, the actor network can output the optimal action of the radar based on the current state of the radar and the feedback of the critic network. Specifically, the advantage function A can be calculated φ (s t ,a t ) and use it to compute the gradient of the actor network:

[0135]

[0136] Then use the gradient to update the actor network parameters: expressed as:

[0137]

[0138] Specifically, the radar performs frequency agility actions based on the output of the actor network, while simultaneously observing environmental feedback to update the radar's state and rewards. This method is repeated over and over again, achieving intelligent anti-interference for the radar.

[0139] For example, to prove the effectiveness of the present invention, the following simulation experiment is used for further explanation.

[0140] The dataset uses the radarsimpy library to model and simulate the radar. We modeled the radar and jammer separately in detail, allowing us to adjust parameters such as power and frequency for each to interact under different conditions. In the simulation, the radar's center frequency was set to 6 GHz, varying in steps of Δf = 50 MHz. The radar can perform frequency agility across 4 to 12 subbands, selecting the appropriate frequency band to output the radar signal. The radar transmits N = 4 pulses in a pulse train, with each pulse lasting 8 × 10 -6 s. The radar power is set to 20dBm, and the jammer power is 30dBm. The radar receiver noise figure is 2dB, and the total baseband gain is 60dB.

[0141] The technical solution of the present invention is compared with various existing technical standards in the radar anti-interference model, including the dominant actor-critic algorithm (A2C), deep Q-network (DQN), dual deep Q-network (DDQN), policy gradient learning (SARSA) and random strategy (Random).

[0142] Prediction index formulation. The performance of all models is evaluated by the commonly used prediction accuracy probability indicators in the radar anti-interference strategy of the present invention, including detection probability and signal-to-interference-noise ratio. The performance of these algorithms is intuitively reflected through different action spaces. In addition, the convergence between the present invention and A2C is also compared. The size of the action space in the continuous action space represents the range of optional frequency points of the radar agent, and the interference method is frequency scanning interference. The detection probability refers to the probability that the radar correctly detects the target, that is, the possibility that the radar successfully identifies the target when the target exists. It is usually affected by factors such as signal strength, noise level, and interference intensity. The higher the detection probability, the stronger the radar detection capability. The signal-to-interference-noise ratio represents the ratio of the strength of the received signal to the interference and noise.

[0143] refer to Figure 4-Figure 7 FIG. 1 shows the preliminary verification and comparison results of the present invention and the existing advanced technology in this research field.

[0144] refer to Figure 4 The figure shows the performance of different algorithms in a jamming environment, using signal-to-interference-noise ratio (SIR) and detection probability as rewards. The horizontal axis represents the size of the radar agent's action space, while the vertical axis represents the SIR and detection probability, respectively. As the action space increases, the radar's anti-jamming effectiveness improves. When the SIR is the reward, the performance of the Attention-A2C algorithm is comparable to that of several reinforcement learning algorithms, such as DQN and DDQN, when the action space is small. However, when the action space increases, the Attention-A2C and A2C algorithms significantly outperform the other algorithms, demonstrating their superior performance in larger action spaces.

[0145] When the reward is detection probability, Attention-A2C and A2C outperform the other algorithms even in smaller action spaces, and maintain their advantage as the action space increases. The results for DQN and DDQN are similar, slightly exceeding those of SARSA. This suggests that while DQN and DDQN are better at handling interference, Attention-A2C and A2C still have a significant advantage in improving detection probability.

[0146] The Attention-A2C algorithm performs well in aiming interference environments with varying action space sizes and exhibits strong adaptability. Its introduced attention mechanism not only improves the algorithm's performance but also enhances its performance in large action spaces.

[0147] In addition to the algorithm's reward, its convergence is also an important indicator of its effectiveness. Convergence primarily measures the speed and stability with which the algorithm approaches the optimal solution during the iteration process. Better convergence indicates a more practical algorithm, as it indicates that the algorithm can find the optimal solution more quickly and stably.

[0148] refer to Figure 5-Figure 7 The figure shows a comparative analysis of the convergence optimization effect of the attention mechanism on the proposed method and the A2C algorithm. The performance of the A2C and Attention-A2C algorithms over 300 training rounds is compared. The figure shows the change in signal-to-noise ratio and detection probability over training rounds.

[0149] In terms of convergence speed, when using detection probability as the reward, the Attention-A2C algorithm completed convergence after 60 training episodes, significantly faster than the A2C algorithm, which required 100 training episodes to complete convergence. The Attention-A2C algorithm also converged faster than the A2C algorithm in both signal-to-noise ratio and detection probability.

[0150] In terms of convergence stability, both algorithms demonstrated good stability, but in experiments using signal-to-noise ratio as a reward, the Attention-A2C algorithm performed better. Its introduced attention mechanism enables the radar agent to more accurately focus on key states and signals, thereby improving learning efficiency and stability.

[0151] In general, the addition of the attention mechanism not only significantly improves the convergence speed of the radar anti-interference algorithm, but also enhances the convergence stability.

[0152] The method provided by the embodiment of the present invention is referred to Figure 3As shown, compared to existing technologies, this solution boasts stronger adaptability and optimization efficiency. Traditional methods often rely on fixed rules or heuristic algorithms, making it difficult to flexibly adapt to complex interference environments. However, this solution utilizes reinforcement learning to construct an MDP model, enabling the radar to dynamically perceive the environment and adaptively adjust its frequency agility strategy. Furthermore, existing methods often use simple weighted averaging when processing historical radar observations, resulting in low information utilization. This solution, however, introduces a self-attention mechanism to accurately extract key signals and improve decision-making accuracy. The reinforcement learning method, based on an actor-critic architecture, enables policy optimization and value estimation to proceed in parallel, resulting in faster convergence and avoiding the computational difficulties inherent in the high-dimensional state space of traditional methods. Furthermore, the system continuously updates its strategy through a real-time feedback mechanism, enabling the radar to adapt to dynamically changing interference environments and enhance its anti-interference capabilities. Overall, this solution offers advantages over existing technologies in terms of intelligent frequency agility decision-making, learning efficiency, and environmental adaptability, effectively improving the performance of radar systems in complex electromagnetic environments.

[0153] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention and are not intended to be limiting. It is readily understood that the processes illustrated in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0154] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0155] Figure 8 A schematic diagram of an electronic device suitable for implementing an embodiment of the present invention is shown.

[0156] It should be noted that Figure 8 The electronic device 1000 shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0157] like Figure 8As shown, electronic device 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to the program stored in read-only memory (ROM) 1002 or the program loaded from storage portion 1008 into random access memory (RAM) 1003. Various programs and data required for system operation are also stored in RAM 1003. CPU 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.

[0158] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output section 1007 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 1008 including a hard disk and the like; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. Removable media 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1010 as needed, so that computer programs read therefrom can be installed into the storage section 1008 as needed.

[0159] In particular, according to an embodiment of the present invention, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product that includes a computer program carried on a storage medium, the computer program containing program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1009 and / or installed from a removable medium 1011. When the computer program is executed by the central processing unit (CPU) 1001, the various functions defined in the system of the present application are performed.

[0160] It should be noted that the storage medium shown in the embodiments of the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any storage medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on the storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0161] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0162] The units involved in the embodiments of the present invention may be implemented in software or hardware, and the units described may also be provided in a processor. In some cases, the names of these units do not limit the units themselves.

[0163] It should be noted that, as another aspect, the present application also provides a storage medium, which can be included in an electronic device; or it can exist independently without being installed in the electronic device. The above storage medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the following embodiments. For example, the electronic device can implement the following Figure 1 The individual steps of the method are shown.

[0164] In one embodiment, the present application provides a computer program product, including a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0165] Furthermore, the above-described figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention and are not intended to be limiting. It is readily understood that the processes illustrated in the above-described figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0166] Other embodiments of the present invention will readily occur to those skilled in the art after considering the specification and practicing the invention herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the claims.

[0167] It should be understood that the present invention is not limited to the exact construction described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof, which is limited only by the appended claims.

Claims

1. A radar intelligent anti-interference method combining attention mechanism and A2C algorithm, characterized in that: The method comprises: Construct a Markov decision process (MDP) model corresponding to the radar, target, and jammer, and define the state space, action space, and multi-objective reward function corresponding to the MDP model; A pre-processing layer based on a self-attention mechanism is used to extract features from the state data to obtain corresponding state features; wherein the state data includes the frequency information of the interference signal; Using the actor network to determine the frequency agility strategy corresponding to the state characteristics; and controlling the radar to execute the frequency agility strategy to collect updated state data; Input the updated state data and frequency agility strategy into the critic network to obtain the time difference deviation of the action value function in the current state; The actor network is gradient updated using the temporal difference bias, and the network parameters of the actor network are updated using the updated gradient, so as to generate a frequency agility strategy using the actor network after the network parameters are updated.

2. The method according to claim 1, characterized in that The state space includes frequency information of radar transmission signals and frequency information of interference signals, which are configured as observation information; The action space includes the frequency of the radar; The multi-objective reward function includes: radar detection probability and radar signal-to-interference-noise ratio.

3. The method according to claim 2, characterized in that The construction of the Markov decision process MDP model corresponding to the radar, target and jammer includes: Define the action space, including: at time t, the radar agent observes the current environment and obtains the corresponding state s t , and for state s t Determine the corresponding frequency agility strategy; the frequency agility strategy includes: carrier frequency a t ; The carrier frequency is selected from the given M frequency points; Define the state space, including: the radar agent observes the current environment to obtain the corresponding state s t , including the frequency information of radar transmission signals and the frequency information of interference signals transmitted by jammers; Define a multi-objective reward function, including: the state s at time t t Transfer to the next state s t+1 , the radar agent receives a reward r t , which is used to describe the radar agent's selection of frequency agility strategy for radar anti-interference effect.

4. The method according to claim 3, characterized in that The frequency information of the interference signal includes a binary group based on the center frequency and the bandwidth.

5. The method according to claim 3, characterized in that Multi-objective reward function, including: radar detection probability index and radar signal-to-interference-noise ratio index.

6. The method according to claim 2, characterized in that The preprocessing layer based on the self-attention mechanism is used to extract features from the state data and obtain the corresponding state features, including: Map state data into query matrix Q, key matrix K and value matrix V; Configure the attention weight matrix based on the query matrix Q, key matrix K and value matrix V; Among them, d k is the matrix dimension; The state feature vector after weighting the state data is calculated using the attention weight matrix; the state feature vector includes: according to the current frequency agility strategy π θ In state s t Select radar frequency hopping action a t , radar agent performs frequency hopping action a t And transmit the signal, the interference signal emitted by the jammer in the environment, calculate the detection probability and signal to interference noise ratio according to the frequency information of the radar and the jammer in the environment, and get the reward r t and the state s at the next moment t+1 .

7. The method according to claim 1, characterized in that The updated state data and frequency agility strategy are input into the critic network to obtain the time difference deviation of the action value function in the current state, including: δ t =r t +γV φ (s t+1 )-V φ (s t ) Among them, V φ (s t ) is the state value function; Construct the MSE loss function and minimize it to update the network parameters.

8. The method according to claim 1, characterized in that The method further includes: outputting an optimal action of the radar based on the current state of the radar and feedback from the critic network, including: Calculate the advantage function A φ (s t ,a t ) and uses it to compute the gradients of the actor network, including: Use the gradients to update the actor network parameters, including: Among them, s t is the state at time t, a t is the radar frequency modulation action at time t.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the radar intelligent anti-interference method combining the attention mechanism and the A2C algorithm according to any one of claims 1 to 8 is implemented.

10. An electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor is configured to execute the radar intelligent anti-interference method combining the attention mechanism and the A2C algorithm according to any one of claims 1 to 8 by executing the executable instructions.

Images