Distributed jamming array intelligent optimization method and system based on deep deterministic policy gradient

Abstract

The application discloses a kind of distributed interference arraying intelligent optimization method and system based on deep deterministic policy gradient, mainly solve the problem that the existing technology cooperation effect is poor, the amount of calculation is large, optimization precision is limited by discretization, it is difficult to utilize space-time adaptive processing feedback, its implementation scheme is: the optimization scene including target radar, space-time adaptive processing module and multiple jammer is constructed;Arraying parameter optimization is modeled as Markov decision process;Actor network, critic network and its target network output and evaluate continuous action vector, and it is mapped as interference azimuth, power weight, retransmission pulse width and false target number;According to the suppression notch depth of each interference azimuth in space-time power spectrum, construct reward function and train network, obtain optimal arraying strategy.The application can realize the continuous space joint optimization of multiple jammer arraying parameters, improve the real-time performance and precision of arraying decision and adaptive ability, enhance the interference efficiency of space-time adaptive processing radar, and can be used for distributed electronic countermeasure.

Description

Technical Field

[0001] This invention belongs to the field of signal processing technology, and specifically relates to a method and system for intelligent optimization of distributed jamming arrays, which can be used in distributed electronic warfare. Background Technology

[0002] With the development of phased array radar and space-time adaptive processing technology, traditional monostation jamming methods, due to their limited spatial degrees of freedom, struggle to achieve stable and effective suppression in complex adversarial environments. Space-time adaptive processing technology can jointly utilize the spatial information of the radar array and the temporal information of coherent pulses to adaptively suppress clutter and interference in the two-dimensional space-time domain, thereby significantly improving the anti-jamming capability of the radar system. Especially in multi-channel array radars, STAP processing can adaptively form a weight vector based on the covariance matrix of the received data, creating a suppression notch near the incident direction of the interference, significantly weakening the interference signal. Therefore, a single jamming source or a simple fixed array configuration is insufficient to fully overcome the space-time suppression capability achieved by STAP processing.

[0003] Distributed jamming, through the coordinated operation of multiple jammers, can jointly jam target radars from different azimuths, distances, or spatiotemporal dimensions, offering greater spatial freedom and collaborative countermeasure potential compared to monostation jamming. By rationally designing the array azimuth, power allocation, relay parameters, and jamming patterns of multiple jammers, the spatiotemporal distribution characteristics of the jamming signal at the radar receiver can be altered, thereby affecting the spatiotemporal power spectrum and suppression notch distribution after STAP processing. However, in actual combat scenarios, there are strong coupling relationships among the jammer array parameters. Parameters such as array angle, power weight, relay pulse width, and the number of false targets jointly influence the residual jamming effect after STAP processing, resulting in the distributed jamming array optimization problem exhibiting high-dimensional, nonlinear, and continuous action space characteristics. If a random array method is adopted, the azimuth and parameters of each jammer lack coordinated optimization, making it difficult to fully leverage the advantages of distributed jamming, and the jamming effect exhibits significant randomness and instability. Therefore, optimization of the distributed jamming array is necessary.

[0004] Existing distributed jamming array optimization methods typically employ exhaustive search, local search, or heuristic intelligent optimization methods. For example, exhaustive search methods usually discretize the jammer array angles at certain intervals, traverse different combinations within a given set of angles, and select the optimal array scheme with the goal of minimizing the STAP output signal-to-interference-plus-noise ratio or raising the jamming suppression notch. This type of method is intuitive and can achieve certain optimization results when the number of jammers is small and the array precision requirements are not high. However, as the number of jammers increases and the angle division precision improves, the search space expands dramatically, and the computational complexity increases significantly, making it difficult to meet the rapid response requirements in real-world adversarial scenarios. Furthermore, the optimization accuracy of exhaustive search is limited by the discretization interval; the truly optimal array scheme may fall outside the discrete grid, resulting in accuracy loss.

[0005] Patent document CN201710678602.6 discloses a "radar cooperative jamming method based on distributed platforms." This method addresses scenarios where distributed platforms are scanned or tracked by radar. It first calculates the jamming power, radar echo power, and effective jamming sector for each platform entering the radar receiver. Then, it determines the minimum jamming sector required to cover all platforms based on their actual distribution. When a platform is tracked by radar, it further divides the cooperative and non-cooperative zones of other platforms relative to the tracked platform. Based on the enemy radar's location, it determines the platforms that can participate in cooperative jamming and whether to use suppression or deception jamming. While this method can achieve multi-platform cooperative jamming decision-making to some extent, it primarily relies on platform geometry, radar beam range, and jamming suppression ratio for rule-based judgment, focusing on cooperative platform selection and jamming method selection. It does not jointly optimize continuous parameters such as multi-jammer array azimuth, power weight, relay pulse width, and number of false targets, nor does it consider the space-time adaptive processing of the space-time suppression notch formed by the radar on the jamming signal. Therefore, it is difficult to adaptively adjust the distributed jamming array scheme based on STAP processing feedback, and it is impossible to solve the problem of intelligent array optimization of STAP radar by distributed jammers in complex dynamic combat environments.

[0006] Patent document CN202210151246.3 discloses a "cluster cooperative jamming planning method based on an optimal jamming planning model." This method constructs a cluster cooperative system composed of multiple jammers and mission aircraft. The optimization objective is the ratio of the mission aircraft's flight distance and the jammer's power to the target radar to the distance between the jammer and the target radar. A multi-target particle swarm optimization algorithm is used to solve for the jammer's spatial position, jamming power allocation, and mission aircraft's flight path, thereby improving the cooperative jamming effect and jamming resource utilization efficiency of the radar network. While this method can solve the multi-jammer cooperative planning problem to some extent, it primarily focuses on radar network detection range compression and safe flight path generation. Its optimization indicators emphasize mission aircraft flight paths and jamming power resource allocation. It does not establish a jamming effectiveness evaluation model for radars with space-time adaptive processing capabilities, nor does it use the notch depth of the space-time power spectrum after STAP processing as array optimization feedback. Therefore, this method struggles to perform closed-loop optimization of distributed jamming array parameters based on STAP processing feedback, failing to meet the needs of rapid and adaptive array deployment in complex adversarial environments. Meanwhile, because this method uses a multi-objective particle swarm optimization algorithm for offline iterative optimization, it is easily affected by the initial particle distribution and local optima. Furthermore, it is difficult to achieve fast and adaptive deployment decisions when the array parameters change continuously, the radar status changes dynamically, or the jamming pattern switches.

[0007] Patent document CN202510538927.9 discloses a "Method for Reinforcement Learning-Based Anti-Distributed Cooperative Track Deception Jamming in Networked Radars." From the perspective of radar anti-jamming, it inserts a verification time slot during the tracking process of networked radars, schedules multi-domain agile pulse trains through reinforcement learning, and identifies distributed cooperative track deception jamming by combining verification radar selection and binary hypothesis testing. The core of this method is to improve the radar system's ability to identify and resist cooperative deception jamming. It belongs to the radar-side protection and identification method, rather than designing the array parameters of the distributed jammer from the jammer's perspective, nor does it establish an array optimization mechanism for the jamming notch rise problem after STAP processing. Furthermore, the reinforcement learning in this method is mainly used for anti-jamming scheduling of radar verification time slots, verification radars, and pulse resources. Its optimization object is the radar-side detection and identification strategy, rather than continuous parameters such as the jammer's azimuth array, power allocation, relay pulse width, and number of false targets. Therefore, this method cannot solve the problems of strong coupling between the array parameters of multiple jammers, difficulty in finding the optimal space for continuous action, and the need to adjust the jamming effect in real time according to the feedback from STAP processing. It is difficult to directly use it to improve the active jamming effectiveness of distributed jamming systems against space-time adaptive processing radars.

[0008] In summary, while existing distributed cooperative jamming technologies can achieve certain levels of cooperative platform selection, jamming mode determination, spatial location planning, and power resource allocation, they still mainly rely on geometric rules, traditional optimization, or radar-side anti-jamming decisions, lacking jamming effectiveness evaluation and closed-loop optimization capabilities oriented towards the STAP processing mechanism. Especially in complex adversarial scenarios involving continuously changing jammer array parameters, dynamic adjustment of radar space-time adaptive weight vectors, and jamming pattern switching, existing methods struggle to fully characterize the coupling relationship between multi-jammer array parameters and the STAP suppression notch, and also fail to achieve fast, accurate, and adaptive array decision-making. Summary of the Invention

[0009] The purpose of this invention is to address the shortcomings of the existing technologies by proposing a distributed jamming array intelligent optimization method and system based on deep deterministic strategy gradient. This method aims to improve the real-time performance, accuracy, and jamming effectiveness of distributed jamming array optimization against space-time adaptive processing radars, while avoiding the poor collaborative effect of random array deployment, the large computational load of exhaustive search, and the tendency of heuristic optimization methods to get trapped in local optima.

[0010] The technical approach to achieving the objective of this invention is as follows: By modeling the optimization problem of multi-jammer collaborative array parameters as a Markov decision process, collaborative optimization of multi-jammer parameters in random arrays is achieved; by employing a deep deterministic strategy gradient agent to directly output continuous action vectors and map them to continuous array parameters, the computational load is reduced and the optimization accuracy is improved; through the Actor-Critic network structure, experience replay, and target network update mechanism, global optimization and adaptation to dynamic adversarial environments are achieved; based on radar echo simulation and STAP processing results, the spatiotemporal power spectrum notch depth is calculated to guide the agent to learn distributed jamming array strategies that weaken the STAP suppression effect, thereby achieving feedback processing of radar echoes and STAP.

[0011] Based on the above ideas, the technical solution of the present invention includes the following:

[0012] 1. A distributed interference array intelligent optimization method based on deep deterministic policy gradient, characterized in that it includes:

[0013] (1) Construct a distributed jamming array optimization scenario including target radar, space-time adaptive processing module and multiple jammers;

[0014] (2) The problem of optimizing the parameters of the multi-interference machine cooperative array is modeled as a Markov decision process, defining the system state, action space and constructing the reward function;

[0015] (3) Select the existing actor network, commentator network, target actor network and target commentator network as the multi-interference machine cooperative array parameter optimization network;

[0016] (4) Obtain the parameters such as interference azimuth angle, power weight, forwarding pulse width, number of false targets and STAP feedback information, and input them into the actor network to output a continuous action vector. Then decode the continuous action vector into a new interference azimuth angle, power weight, forwarding pulse width and number of false targets, and generate a distributed interference signal based on the decoding result.

[0017] (5) Input the parameters obtained in (4) and the output continuous action vector into the commentator network, output the value of the interference effect, and update the actor network parameters;

[0018] (6) Perform space-time adaptive processing on the distributed interference signal to obtain the space-time power spectrum;

[0019] (7) Extract the suppression notch depth at each interference location based on the spatiotemporal power spectrum and obtain the reward;

[0020] (8) Input the interactive experience consisting of system state, action, reward and new system state into the target actor network and target commentator network to calculate the target interference effect value, and update the commentator network parameters to approximate the target interference effect value;

[0021] (9) Repeat steps (4) to (8) until the set termination condition is met to obtain the optimal actor network;

[0022] (10) Input the current interference azimuth, power weight, forwarding pulse width and number of false targets into the optimal actor network, and output the interference azimuth, power weight, forwarding pulse width and number of false targets of the optimized distributed jammer array to complete the intelligent optimization of the distributed jammer array.

[0023] Furthermore, in step (8), the system state, actions, rewards, and new system state are combined to form interactive experience, which is then input into the target actor network and the target commentator network to calculate the target interference effect value. The commentator network parameters are then updated to approximate the target interference effect value. This process includes:

[0024] 8a) The system state, actions, rewards, and new system state obtained during the interaction at time t constitute the interaction experience: ;

[0025] 8b) Random selection Group interaction experience:

[0026] 8c) New system states in the interactive experience Input into the target actor network to obtain the target action at the next moment. ;

[0027] 8d) Change the new system state and target action Inputting the data into the target commentator network yields the target state-action value. ;

[0028] 8e) Based on the reward and target state-action value Calculate the value of target interference effect ;

[0029] 8f) Change the current system state and current action Input the commentator network to obtain the current interference effect value. ;

[0030] 8g) Construct the commentator network loss function based on the current interference effect value and the target interference effect value. ;

[0031] 8h) Update the commentator network parameters according to the negative gradient direction of the loss function;

[0032] 8i) Perform soft updates to the target actor network and the target commentator network.

[0033] 2. A distributed interference array intelligent optimization system based on deep deterministic policy gradient, characterized in that it includes:

[0034] The distributed jamming array scenario construction module is used to construct an optimized distributed jamming array scenario that includes a target radar, a space-time adaptive processing module, and multiple jammers.

[0035] The Markov decision process modeling module is used to model the multi-interference machine cooperative deployment parameter optimization problem as a Markov decision process and define the system state, action space and reward function.

[0036] The deep deterministic policy gradient agent decision-making module is used to construct actor networks, commentator networks, target actor networks, and target commentator networks, and output continuous action vectors based on the current system state.

[0037] The action decoding and parameter mapping module is used to map continuous action vectors to the jamming azimuth angle, power weight, forwarding pulse width and number of false targets of multiple jammers;

[0038] The radar echo and space-time adaptive processing module is used to generate radar received data based on interference parameters, calculate the space-time adaptive processing weight vector, and output the space-time power spectrum.

[0039] The reward calculation module is used to extract the suppression notch depth at each interference azimuth based on the spatiotemporal power spectrum and calculate the reward function value.

[0040] The experience replay and network training module is used to store experience samples generated by the interaction between the agent and the environment, and to train and update the actor network and commentator network.

[0041] The array strategy output module is used to output the optimal distributed interference array parameters after training is completed.

[0042] Compared with the prior art, the present invention has the following advantages:

[0043] First, this invention models the problem of multi-jamming machine cooperative array parameter optimization as a Markov decision process, and uses the jamming azimuth angle, power weight, forwarding pulse width, number of false targets and STAP feedback information as the system state. This overcomes the shortcomings of existing random array methods, such as the lack of coordination among jammer parameters and the difficulty in adjusting them based on radar processing feedback. This enables the distributed jamming array optimization process to have environmental awareness and closed-loop optimization capabilities.

[0044] Secondly, this invention uses an actor network to output continuous action vectors and decodes these continuous action vectors into new interference azimuth angles, power weights, forwarding pulse widths, and the number of false targets. This overcomes the shortcomings of traditional exhaustive search methods, which require discrete traversal of array parameters, have a large computational load, and whose optimization accuracy is limited by discrete intervals. This allows the array parameters of multiple jammers to be jointly optimized in a continuous space, improving the accuracy and efficiency of array optimization.

[0045] Third, this invention evaluates the system state and continuous action vectors through a commentator network, outputs the interference effect value corresponding to the current deployment action, and uses this value to guide the update of the actor network parameters. This overcomes the problem that heuristic optimization methods are easily affected by initial conditions and prone to getting trapped in local optima, enabling the actor network to gradually learn better deployment strategies during the training process, thereby improving the stability and global optimization capability of distributed interference deployment optimization.

[0046] Fourth, this invention performs space-time adaptive processing on distributed interference signals, extracts the suppression notch depth at each interference azimuth in the space-time power spectrum, and uses the suppression notch depth as the reward basis. This overcomes the problem that existing interference planning methods do not directly optimize for the STAP suppression mechanism, so that the learning objective of the agent directly corresponds to the suppression effect of weakening the STAP on the interference signal, thereby improving the interference effectiveness of the distributed interference system against space-time adaptive processing radar.

[0047] Fifth, this invention utilizes the target actor network and the target commentator network to calculate the value of the target interference effect, and continuously updates the parameters of the actor network and the commentator network through the training process. This overcomes the problem that existing deployment methods mainly rely on offline planning and are difficult to adapt to complex dynamic adversarial environments. After training, the actor network can quickly output the optimized deployment scheme based on the current interference parameters and STAP feedback, thereby improving the real-time performance and adaptability of distributed interference deployment decisions. Attached Figure Description

[0048] Figure 1 This is a schematic diagram illustrating the implementation of the intelligent optimization method for distributed interference array based on deep deterministic strategy gradient of the present invention.

[0049] Figure 2 This is a diagram of the distributed interference array scenario constructed in this invention;

[0050] Figure 3 This is a schematic diagram of the actor network structure used in this invention;

[0051] Figure 4 This is a schematic diagram of the commentator network structure used in this invention;

[0052] Figure 5 This is a block diagram of the distributed interference array intelligent optimization system based on deep deterministic strategy gradient of the present invention;

[0053] Figure 6 These are images showing the noise convolution interference effects after arraying using the present invention and the existing exhaustive method, respectively.

[0054] Figure 7 These are images showing the effects of dense false target interference after arraying using the present invention and existing exhaustive methods, respectively.

[0055] Figure 8 These are diagrams showing the intermittent sampling frequency shifting and forwarding interference effects after array deployment using the present invention and existing exhaustive methods, respectively. Detailed Implementation

[0056] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention and not all embodiments. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative effort should all fall within the protection scope of the present invention.

[0057] Example 1: Intelligent optimization method for distributed interference array based on deep deterministic policy gradient.

[0058] Reference Figure 1The implementation steps of this example include the following:

[0059] Step 1: Establish a distributed interference array optimization scenario.

[0060] Reference Figure 2 The distributed jamming array optimization scenario established in this step includes a target radar, a space-time adaptive processing module, and multiple jammers. The target radar is an airborne phased array radar with STAP processing capabilities, and multiple jammers coordinate to jam the target radar from different azimuths. After receiving the distributed jamming signals, the target radar obtains the space-time power spectrum through space-time adaptive processing and forms a suppression notch at the jamming azimuth.

[0061] Unlike random deployment methods, this step does not randomly assign the deployment positions of multiple jammers. Instead, it uses the target radar, STAP processing results, and multiple jammers to form a closed-loop optimization environment. This allows the agent to adaptively adjust the jamming deployment based on STAP feedback, thereby avoiding the problems of poor coordination and unstable jamming results caused by random deployment.

[0062] Step 2: Establish a Markov decision process model for optimizing the parameters of the multi-interference machine cooperative array.

[0063] To enable the distributed jamming array optimization process to make continuous decisions based on the current array state and STAP processing feedback, this step models the multi-jammer cooperative array parameter optimization problem as a Markov decision process. The Markov decision process describes the interaction between the agent and the spatiotemporal adaptive processing environment through system state, action space, and reward function. The system state characterizes the current jamming azimuth, power weight, forwarding pulse width, number of false targets, and STAP feedback information; the action space represents the continuous array parameters such as jamming azimuth, power weight, forwarding pulse width, and number of false targets that the agent network can output and adjust; and the reward function evaluates the jamming effect of the current array action based on the suppression notch depth after STAP processing. Through this modeling, the multi-jammer cooperative array parameter optimization problem can be transformed from a traditional static parameter search problem into a decision-making problem where the agent learns the optimal array strategy step-by-step based on environmental feedback in a continuous action space. Its implementation includes:

[0064] 2.1) Define system state:

[0065] The system state is composed of parameters such as the jamming azimuth angle, power weight, relay pulse width, number of false targets, and STAP feedback information of each jammer at time t. :

[0066] ,

[0067] Where J represents the number of jammers. Let be the jamming azimuth angle of the j-th jammer at time t. Let be the power weight of the j-th jammer at time t. Let be the relay pulse width of the j-th jammer at time t. Let be the number of false targets of the j-th jammer at time t. Let t be the depth of the suppression notch at the corresponding jamming azimuth of the j-th jammer after STAP processing at time t.

[0068] 2.2) Define the action space:

[0069] In Markov decision process modeling, the action space describes the range within which an agent can adjust the parameters of a distributed jamming array. This embodiment employs a deep deterministic policy gradient agent for continuous action decision-making. The agent uses an actor-commentator network structure, where the actor network outputs continuous action vectors based on the current system state, and the commentator network evaluates the state-action pairs composed of the system state and the continuous action vectors, outputting the corresponding jamming effect value. That is, the actor network is responsible for generating array adjustment actions, and the commentator network is responsible for evaluating the impact of these actions on the distributed jamming effect and providing a basis for subsequent actor network parameter updates. The jamming azimuth angle, power weight, relay pulse width, and number of false targets of multiple jammers are used as optimization parameters corresponding to the action space. The continuous action vector output by the actor network is represented as... :

[0070] ,

[0071] Among them, dimensions 1 to J are used for interference azimuth coding, dimensions J+1 to 2J are used for power weight coding, dimensions 2J+1 to 3J are used for forwarding pulse width coding, dimensions 3J+1 to 4J are used for false target number coding, and the value range of each element in the action vector is [-1,1].

[0072] 2.3) Define the reward function:

[0073] Using the suppression notch depth at each interference orientation after spatiotemporal adaptive processing as the evaluation criterion for the reward function, the learning objective of the agent directly corresponds to the reduction of the STAP's suppression effect on the interference signal, which is expressed as:

[0074] ,

[0075] Where R is the reward value and J is the number of jammers. The depth of the suppression notch at the corresponding jamming azimuth of the j-th jammer after STAP processing.

[0076] By defining the system state, action space, and reward function, the interaction between the agent and the STAP processing environment is established. This transforms the problem of optimizing the parameters of a multi-interference machine cooperative deployment into a continuous action space optimization problem based on a Markov decision process. This modeling approach overcomes the limitation of traditional exhaustive search, which can only traverse a discrete set of angles, and enables joint optimization of multiple types of interference parameters within a continuous action space.

[0077] Step 3: Construct a deep deterministic policy gradient agent network based on an actor-commentator structure.

[0078] Reference Figure 3 and Figure 4 The implementation of this step includes:

[0079] 3.1) Select an actor network that includes a state input layer, a first fully connected layer, a first nonlinear activation layer, a second fully connected layer, a second nonlinear activation layer, and a tanh activation layer, and set its input to the current system state. The output is a continuous action vector. Its expression is:

[0080] ,

[0081] in, Indicates actor network, The parameters of the actor network are represented; the output layer dimension of the actor network is 4J, which corresponds to the jamming azimuth coding, power weight coding, forwarding pulse width coding and false target number coding of J jammers respectively. The tanh activation layer is used to restrict each element in the continuous action vector to the range of [-1,1].

[0082] 3.2) Select a commenter network that includes a state input branch, an action input branch, a convolutional neural network, a feature connection layer, and a fully connected neural network, and set its input to the current system state. and the continuous motion vectors output by the actor network The output is the value of the interference effect of the current state and the action. Its expression is:

[0083] ,

[0084] in, This refers to the commentator network. The commentator network parameters are represented; the state input branch is used to extract system state features, the action input branch is used to extract continuous action features, the feature splicing layer is used to fuse system state features and continuous action features, and the value output layer is used to output the interference effect value corresponding to the current array action.

[0085] 3.3) Select a target actor network that includes a state input layer, a first fully connected layer, a first nonlinear activation layer, a second fully connected layer, a second nonlinear activation layer, and a tanh activation layer, and set its input to the system state. The output is a continuous action vector. Its expression is:

[0086] ,

[0087] in, Indicates the target actor network, The target actor network parameters are represented; the target actor network is used to generate target actions based on the new system state during training, providing action input for calculating the value of target interference effects;

[0088] 3.4) Select a target commentator network that includes a state input branch, an action input branch, a convolutional neural network, a feature connection layer, and a fully connected neural network. Set its network structure to be the same as the commentator network, and use the new system state as the input. and the target actor network output target continuous motion vector The output is the target state - action value. Its expression is:

[0089] ,

[0090] in, This indicates a network of target commentators. The target commentator network parameters are represented; the target commentator network is used to calculate the value of the target interference effect as a reference for updating the commentator network parameters.

[0091] 3.5) Connect the actor network and the commentator network to form the main network, and connect the target actor network and the target commentator network to form the target network; then connect the main network and the target network in parallel to form a deep deterministic policy gradient agent network, wherein:

[0092] The main network is used for generating, evaluating, and updating parameters of the current deployment actions. The actor network outputs a continuous action vector based on the current system state. This continuous action vector, after action decoding, is mapped to the jamming azimuth, power weight, relay pulse width, and number of false targets for multiple jammers. This continuous action vector, along with the current system state, is input into the commentator network, which outputs the current jamming effect value.

[0093] The target network is used to calculate the target interference effect value based on the new system state and target action, providing a stable target value for updating the commentator network in the main network. The target actor network outputs the target action based on the new system state, and the target action and the new system state are input into the target commentator network. The target commentator network outputs the target state-action value, which is used to guide the parameter update of the commentator network in the main network.

[0094] In the main network, the actor network and commentator network achieve online learning through parameter updates, while the target actor network and target commentator network in the target network follow the parameter changes of the main network through soft updates. This forms a parallel and collaborative structure where the main network makes real-time decision-making evaluations and the target network provides stable auxiliary training. Compared with existing heuristic optimization methods such as particle swarm optimization, this step evaluates the continuous actions output by the actor network through the commentator network and uses this evaluation to guide the actor network's updates. This reduces the degree to which the algorithm is affected by initial particles or local search paths, mitigating the problem of heuristic optimization methods easily getting trapped in local optima.

[0095] Step 4: Use the actor network to output continuous motion vectors and perform motion decoding.

[0096] 4.1) Set the current system state Input the actor network to obtain continuous motion vectors Mapping dimensions 1 to J of the action vector to the jamming azimuth angle yields the original jamming azimuth angle of the j-th jammer:

[0097] ,

[0098] in To interfere with the upper limit of the azimuth angle, This is the lower limit of the interference azimuth angle;

[0099] 4.2) Preset minimum azimuth interval Apply a minimum spacing constraint to the azimuth angle of the interference:

[0100] ,

[0101] 4.3) Map the (J+1)th to 2Jth dimensions of the action vector to power weights to obtain intermediate power weights. :

[0102] ,

[0103] in, This is the power weight encoding value corresponding to the action vector;

[0104] 4.4) Normalize the intermediate power weights to obtain the power weights of the j-th jammer. :

[0105] ,

[0106] in, This is used to avoid extremely small positive numbers with a denominator of zero;

[0107] 4.5) Map the 2J+1 to 3J dimensions of the action vector to the forwarding pulse width. :

[0108] ,

[0109] in, This is the lower limit of the forwarding pulse width. This is the upper limit of the forwarding pulse width.

[0110] 4.6) Map the 3J+1 to 4J dimensions of the action vector to the number of false targets. :

[0111] ,

[0112] in, This is the lower limit for the number of false targets. The upper limit for the number of false targets. This indicates the rounding operation.

[0113] This step directly generates multiple array parameters through continuous action vectors, avoiding the problem of exhaustive search requiring discrete traversal of azimuth, power, and interference parameters, thus reducing computational complexity and minimizing the loss of optimization accuracy caused by discretization intervals.

[0114] Step 5: Generate a distributed interference signal based on the decoding results.

[0115] To transform the continuous motion vectors output by the actor network into distributed jamming signals perceptible to the target radar receiver, this step generates delayed forwarding components corresponding to each jammer based on the parameters decoded in step 4, and performs spatial superposition of the jamming signals from multiple jammers, providing input for subsequent STAP processing and reward calculation. Its implementation includes:

[0116] 5.1) Based on the jamming azimuth angle of the j-th jammer obtained from decoding... Power weight Forwarding pulse width and the number of false targets ,generate There are m-th delayed forwarding components, and the m-th delayed forwarding component is represented as... :

[0117] ,

[0118] in, For intercepted or referenced radar signals, The delay of the m-th false target generated by the j-th jammer. For amplitude coefficient, For phase modulation amount, Forwarding pulse width A defined pulse width window function;

[0119] 5.2) The j-th jammer The delayed forwarding components are superimposed and combined with power weights. Obtain the jamming signal from the j-th jammer. :

[0120] ;

[0121] 5.3) Based on the jamming azimuth angle of the j-th jammer Construct the spatial steering vector corresponding to the target radar receiving array. :

[0122] ,

[0123] in, In the formula, For the spacing between array elements, The wavelength is the radar wavelength, and T represents the matrix transpose.

[0124] 5.4) The jamming signals from multiple jammers are spatially superimposed to obtain a distributed jamming signal that acts on the target radar receiver. :

[0125] .

[0126] This step integrates array orientation, power allocation, relay pulse width, and the number of false targets into the distributed jamming signal generation process, which can characterize the coupling relationship between multiple jamming machine parameters and overcome the problem that existing methods only optimize array orientation or power resources and cannot jointly describe the jamming effect of multiple parameters.

[0127] Step 6: Perform STAP processing on the distributed interference signal to obtain the space-time power spectrum.

[0128] To evaluate the actual interference effect of the distributed interference signal generated in step 5 after the target radar's space-time adaptive processing, the distributed interference signal, along with the target echo, clutter, and noise, are used as input data for the target radar's receiving array for STAP processing. This process obtains the response intensity at different azimuths and Doppler frequencies, providing a basis for subsequent extraction of the suppression notch depth at each interference azimuth and construction of the reward function.

[0129] 6.1) Based on normalized spatial frequency Constructing the spatial steering vector of the target radar receiving array :

[0130] ,

[0131] in, , The element spacing of the target radar receiving array. The wavelength of the target radar's operating signal is T, which represents the transpose.

[0132] 6.2) Based on normalized Doppler frequency Constructing a time-guided vector :

[0133] ;

[0134] 6.3) Construct a spacetime guidance vector based on the spatial guidance vector and the time guidance vector. :

[0135] ,

[0136] in, Represents the Kronecker product;

[0137] 6.4) Take a common sample from both sides of the target distance cell. The reference distance cell, and the first reference distance cell, and the second The received data from each reference range cell on N array elements and M coherent pulses is expanded into spacetime snapshot data. ;

[0138] 6.5) Based on the time-space snapshot data of the reference distance unit Estimating the covariance matrix :

[0139] ,

[0140] Where H represents the conjugate transpose;

[0141] 6.6) Calculate the space-time adaptive processing weight vector based on the minimum variance criterion of linear constraints. :

[0142] ,

[0143] in, The target spacetime guidance vector;

[0144] 6.7) Based on the space-time adaptive processing weight vector and spacetime steering vector Calculation of space-time two-dimensional power spectrum :

[0145] .

[0146] This step directly incorporates the target radar's STAP processing results. Compared to interference planning methods that do not consider STAP feedback, this allows the subsequent reward function to truly reflect the residual interference effect of the distributed interference signal after it has undergone space-time adaptive suppression by the target radar.

[0147] Step 7: Extract the suppression notch depth based on the spatiotemporal power spectrum and obtain the reward.

[0148] In order to transform the spatiotemporal adaptive processing results into a feedback signal that the agent can learn, this step calculates the suppression notch depth at the corresponding interference azimuth of each jammer based on the spatiotemporal two-dimensional power spectrum obtained in step 6, and uses it as the evaluation basis of the reward function, so that the agent can judge the interference effect of the current deployment action based on the reward value, and provide feedback for the subsequent training and update of the actor network and commentator network.

[0149] 7.1) Based on the jamming azimuth angle of the j-th jammer Extract the spatiotemporal power spectrum response at the corresponding interference azimuth. :

[0150] ,

[0151] in, To normalize the Doppler frequency for the target;

[0152] 7.2) Calculate the depth of the suppression notch at the jamming azimuth corresponding to the j-th jammer. :

[0153] ,

[0154] in, The maximum response value of the azimuth tangent of the power spectrum when empty;

[0155] 7.3) Calculate the reward value based on the depth of the suppression notch at all interference locations. :

[0156] ,

[0157] Where J represents the number of jammers.

[0158] This step uses the depth of the suppression notch after STAP processing as the reward criterion, allowing the agent's optimization objective to move beyond geometric layout or power allocation and directly learn the STAP suppression effect. By maximizing the reward value R, the agent's output actions can be optimized towards increasing the spacetime power spectral response at the interference location and reducing the STAP's effect on distributed interference suppression.

[0159] Step 8: Calculate the target interference effect value based on the target network and update the commentator network parameters.

[0160] To leverage historical deployment experience gained during the interaction between the agent and the spatiotemporally adaptive processing environment, thereby improving the accuracy of the commentator network's evaluation of the interference effects of different deployment actions and enhancing the stability of the training process, this step comprises the interaction experience consisting of system state, actions, rewards, and the new system state. The target interference effect value is calculated based on the target actor network and the target commentator network, and then the commentator network parameters are updated using the error between the current interference effect value and the target interference effect value. Simultaneously, a soft update mechanism for the target network is implemented to allow it to slowly follow changes in the main network, avoiding drastic fluctuations in the target value during training. This mechanism includes:

[0161] 8.1) The system state, actions, rewards, and new system state obtained during the interaction at time t constitute the interaction experience: ,in, Let t be the system state. Let be the action vector at time t. The reward at time t, The new system state obtained after performing the action at time t;

[0162] 8.2) Select Group interaction experience as training samples: ,in, Let i be the current system state of the i-th training sample. For the current action of the i-th training sample, The reward for the i-th training sample. Let i be the next system state for the i-th training sample;

[0163] 8.3) Set the next system state of the i-th training sample. Input into the target actor network to obtain the target motion. :

[0164] ,

[0165] in, For the target actor network, For target actor network parameters;

[0166] 8.4) Set the next system state for the i-th training sample. and target action Input the target commentator network to obtain the target state-action value. :

[0167] ,

[0168] in, For the target commentator network, For target commentator network parameters;

[0169] 8.5) Based on rewards and target state-action value Calculate the value of target interference effect :

[0170] ,

[0171] in, Discount factor;

[0172] 8.6) Set the current system state of the i-th training sample group. and current action Input into the commentator network to obtain the current interference effect value. :

[0173] ,

[0174] in, For commentator network, For commentator network parameters;

[0175] 8.7) Construct the commentator network loss function based on the current interference effect value and the target interference effect value. :

[0176] ,

[0177] 8.8) Update the commentator network parameters according to the negative gradient direction of the loss function:

[0178] ,

[0179] in, For commentators' online learning rate, This represents the gradient of the loss function with respect to the commentator network parameters;

[0180] 8.9) Perform soft updates to the target actor network and the target commentator network:

[0181] ,

[0182] ,

[0183] in, For the target network soft update coefficient, For actor network parameters.

[0184] This step improves the stability of the training process through the target network and soft update mechanism. Compared with traditional offline optimization methods such as particle swarm optimization, it can learn array experience in continuous interactive samples and avoids starting the search from scratch every time the environment changes.

[0185] Step 9: Update actor network parameters.

[0186] Based on the state-action value function output by the commentator network, its gradient with respect to the continuous action vector is calculated. Combined with the gradient of the actor network's output action with respect to the actor network parameters, the actor network parameters are updated according to the deterministic policy gradient. :

[0187] ,

[0188] in, This represents the gradient of the commentator's network output state-action value with respect to continuous action vectors. This represents the gradient of the actor network output action with respect to the actor network parameters.

[0189] This step optimizes the continuous motion vectors output by the actor network in a direction that increases the value of the interference effect, thereby gradually learning a distributed interference strategy that is superior to random array, exhaustive array, and heuristic array.

[0190] Step 10: Repeat the training and output the optimal distributed interference array strategy.

[0191] Set training termination conditions: including reaching a preset number of training rounds, convergence of reward value, or convergence of commentator network loss function;

[0192] Repeat steps 4 through 9 until one of the set training termination conditions is met to obtain the optimal actor network.

[0193] After training, the current jamming azimuth, power weights, relay pulse width, and number of false targets are input into the optimal actor network, and the optimized distributed jammer array parameters are output:

[0194] ,

[0195] in, , , and These represent the optimized jamming azimuth angle, power weight, forwarding pulse width, and number of false targets for the j-th jammer.

[0196] Through the above steps, this embodiment addresses the problem of poor collaborative performance of random array deployment by introducing system state and STAP feedback information to achieve closed-loop optimization; it addresses the problems of large computational load and limited discretization accuracy of exhaustive search by using continuous action space to output array parameters; it addresses the problem of heuristic algorithms easily getting trapped in local optima by using interactive training of actor networks and commentator networks to learn better array strategies; and it addresses the problem of existing methods lacking STAP processing feedback by using the suppression of notch depth as the reward criterion, thereby completing the intelligent optimization of distributed jamming array deployment based on depth deterministic policy gradient, thus improving the jamming effectiveness of the distributed jamming system against space-time adaptive processing radar.

[0197] It should be noted that the step numbers in this specification and claims are only for the purpose of clearly describing the embodiments of the present invention and facilitating understanding, and their order is not limited.

[0198] Example 2: Intelligent optimization system for distributed interference array based on deep deterministic strategy gradient.

[0199] Reference Figure 5 This example includes: a distributed interference deployment scenario construction module 1, a Markov decision process modeling module 2, a deep deterministic policy gradient agent decision-making module 3, an action decoding and parameter mapping module 4, a radar echo and spatiotemporal adaptive processing module 5, a reward calculation module 6, an experience playback and network training module 7, and a deployment policy output module 8. The experience playback and network training module 7 includes: an interaction experience construction submodule 71, a sample sampling submodule 72, a target action generation submodule 73, a target value calculation submodule 74, a commentator network update submodule 75, an actor network update submodule 76, and a target network soft update submodule 77.

[0200] The working principle of the entire system is as follows:

[0201] The distributed jamming array scenario construction module 1 is used to construct a distributed jamming array optimization scenario. Its inputs are target radar parameters, the number of multiple jammers, and their initial array conditions; its processing involves setting the adversarial relationship between the target radar, the space-time adaptive processing module, and the multiple jammers, and initializing the jamming azimuth angle, power weight, relay pulse width, and number of false targets for each jammer; its output is the distributed jamming array optimization scenario parameters, which are transmitted to the Markov decision process modeling module 2.

[0202] The Markov decision process modeling module 2 is used to establish a Markov decision process model for optimizing the parameters of the multi-jamming array cooperative deployment based on the distributed jamming array optimization scenario parameters and the space-time adaptive processing feedback information. Its inputs are the distributed jamming array optimization scenario parameters output by the distributed jamming array scenario construction module 1, and the space-time adaptive processing feedback information from the radar echo and the space-time adaptive processing module 5. Its processing involves defining the system state, action space, and reward function. The system state includes the jamming azimuth angle, power weight, relay pulse width, number of false targets, and space-time adaptive processing feedback information. The action space corresponds to the jamming azimuth angle, power weight, relay pulse width, and number of false targets for multiple jammers. The reward function is used to evaluate the impact of the current deployment scheme on the space-time adaptive processing suppression notch. Its output is the system state, which is then input to the deep deterministic policy gradient agent decision module 3.

[0203] The deep deterministic policy gradient agent decision module 3 is used to generate continuous action vectors based on the current system state using the actor network, and to evaluate the state-action pairs composed of the system state and the continuous action vectors using the commentator network. At the same time, it provides target actions and target state-action values ​​for network training through the target actor network and the target commentator network, and inputs the output continuous action vectors to the action decoding and parameter mapping module 4.

[0204] The action decoding and parameter mapping module 4 is used to decode the continuous action vector input by the deep deterministic policy gradient agent decision-making module 3 into the interference azimuth angle, power weight, forwarding pulse width and number of false targets of multiple jammers according to different dimensions. It applies minimum interval constraints to the interference azimuth angle, normalizes the power weight, and inputs the array parameters output to the radar echo and space-time adaptive processing module 5.

[0205] The radar echo and space-time adaptive processing module 5 is used to generate radar received data according to the array parameters and perform space-time adaptive processing on the radar received data. That is, it constructs a spatial steering vector according to the interference azimuth angle, generates interference signals corresponding to multiple jammers according to the power weight, relay pulse width and number of false targets, and combines the distributed interference signals with the target echo, clutter and noise to form radar received data. Then, it estimates the covariance matrix according to the reference range cell, calculates the space-time adaptive processing weight vector, and obtains the space-time two-dimensional power spectrum. Its output includes the space-time two-dimensional power spectrum and space-time adaptive processing feedback information. The space-time two-dimensional power spectrum is input to the reward calculation module 6, and the space-time adaptive processing feedback information is fed back to the Markov decision process modeling module 2 to update the system state in the next round.

[0206] The reward calculation module 6 is used to extract the azimuth tangent of the spatiotemporal power spectrum at the target normalized Doppler frequency based on the spatiotemporal two-dimensional power spectrum input from the radar echo and the spatiotemporal adaptive processing module 5, as well as the interference azimuth angle corresponding to each jammer, calculate the spatiotemporal power spectrum response and suppression notch depth at each interference azimuth, calculate the reward function value based on the suppression notch depth at all interference azimuths, and input the reward function value to the experience playback and network training module 7.

[0207] The experience replay and network training module 7 is used to train the deep deterministic policy gradient agent network based on the system state, continuous action vectors, reward function values, and the new system state. Wherein:

[0208] The interaction experience construction submodule 71 is used to assemble the system state, action, reward and new system state obtained by the agent in a single interaction with the space-time adaptive processing environment into an interaction experience according to their correspondence, and input it into the sample sampling submodule 72.

[0209] The sample sampling submodule 72 is used to randomly select several groups from multiple interaction experiences as the network training sample set, so as to reduce the correlation between samples and improve the stability of network training. The training sample set is then input into the target action generation submodule 73, the target value calculation submodule 74, the commentator network update submodule 75, and the actor network update submodule 76, respectively.

[0210] The target action generation submodule 73 is used to input the new system state from the training samples into the target actor network, generate the target action, and input it into the target value calculation submodule 74.

[0211] The target value calculation submodule 74 is used to input the new system state and target action into the target commentator network, obtain the target state-action value, calculate the target interference effect value, and output it to the commentator network update submodule 75.

[0212] The commentator network update submodule 75 is used to input the current system state and current action into the commentator network to obtain the current interference effect value, and then construct the commentator network loss function based on the difference between the current interference effect value and the target interference effect value. The commentator network parameters are updated according to the descent direction of the loss function, and the updated commentator network parameters are input into the target network soft update submodule 77.

[0213] The actor network update submodule 76 is used to update the actor network parameters according to the gradient of the state-action value to the action output by the commentator network, so that the action output by the actor network is optimized in the direction of improving the value of the interference effect, and the updated actor network parameters are input into the target network soft update submodule 77 and the array strategy output module 8.

[0214] The target network soft update submodule 77 is used to make the target actor network parameters and target commentator network parameters slowly follow the changes of the actor network parameters and commentator network parameters according to the updated actor network parameters, commentator network parameters, target actor network parameters and target commentator network parameters, according to a preset soft update coefficient, and then send the updated target actor network parameters and target commentator network parameters back to the target action generation submodule 73 and the target value calculation submodule 74.

[0215] The array strategy output module 8 is used to output the optimal continuous action vector based on the actor network parameters after training, and output the optimized interference azimuth, power weight, forwarding pulse width and number of false targets after action decoding.

[0216] Through the collaborative connection between the above modules, this embodiment can unify the distributed jamming array scenario, Markov decision process modeling, continuous action decision-making, jamming parameter mapping, radar space-time adaptive processing feedback, reward calculation, and network training update into the same closed-loop system. This enables the system to continuously adjust the distributed jamming array parameters based on the STAP processing feedback, achieving joint intelligent optimization of the jamming azimuth angle, power weight, forwarding pulse width, and number of false targets for multiple jammers.

[0217] The effectiveness of this invention can be further illustrated by the following simulation results:

[0218] I. Simulation Conditions

[0219] The number of jammers is set to 4, and each jammer is configured with a specific jamming azimuth angle, power weight, repeater pulse width, and decoy.

[0220] Four types of optimization parameters for the number of labels;

[0221] The actor network output continuous motion vector is set to 16 dimensions, the target radar receiver array has 8 elements, the number of coherent pulses is set to 32, and the radar pulse width is set to 50. .

[0222] The batch size for training a deep deterministic policy gradient agent is set to 64, and the experience pool capacity is [missing information]. The discount factor is 0.99, and the target network smoothing coefficient is... The training rounds were 100, and the initial variance of the exploration noise was 0.2.

[0223] II. Simulation Content

[0224] Simulation 1: Under the above conditions, the noise convolution interference effect after arraying is simulated using both the present invention and existing exhaustive methods. The results are as follows: Figure 6 ,in:

[0225] 6(a) shows the space-time two-dimensional power spectrum after the array is arranged using the existing exhaustive method;

[0226] 6(b) is a cross-sectional view of the space-time power spectrum after the array is arranged using the existing exhaustive method;

[0227] 6(c) is the space-time two-dimensional power spectrum after arraying according to the method of the present invention;

[0228] 6(d) is a cross-sectional view of the space-time power spectrum after arraying according to the method of the present invention.

[0229] from Figure 6 (a) It can be seen that after using the existing exhaustive array method, the distributed noise convolution interference energy is still adaptively suppressed by the STAP processing on the two-dimensional space-time plane, and obvious notches are formed at multiple interference locations.

[0230] from Figure 6 (b) As can be seen, the jammer array angles obtained by the existing exhaustive method are [20°, 23°, 26°, 30°], and the corresponding suppression notch depths at the jamming azimuth are -25.440dB, -26.403dB, -34.670dB, and -25.640dB, respectively. Compared with the suppression notch depths of -44.773dB, -31.204dB, -34.960dB, and -27.668dB without array optimization, the existing exhaustive method reduces the average suppression effect of STAP on distributed noise convolutional interference by about 6.6dB, indicating that the exhaustive method can improve the distributed jamming array effect to a certain extent.

[0231] from Figure 6 (c) It can be seen that after arraying using the method of the present invention, the suppression notch at the interference orientation in the spatiotemporal two-dimensional power spectrum becomes shallower overall, indicating that STAP's ability to suppress distributed noise convolution interference is weakened.

[0232] from Figure 6 (d) It can be seen that the jammer array angles obtained by the method of the present invention are [19.7°, 22°, 26.7°, 28°], and the corresponding suppression notch depths at the jamming azimuth are -25.391dB, -23.486dB, -17.810dB and -15.876dB, respectively.

[0233] Compared with the existing exhaustive array method, the present invention can further reduce the average suppression effect of STAP on interference by about 7.4dB; compared with no array optimization, the present invention reduces the average suppression effect of STAP on distributed noise convolution interference by about 14dB.

[0234] Simulation results show that, under noisy convolutional interference scenarios, this invention can learn a non-uniform distributed interference array scheme, significantly raising the STAP suppression notch at the interference azimuth, thereby reducing the target radar's spatiotemporal adaptive suppression capability against distributed interference. Compared to existing exhaustive methods, this invention has stronger array optimization capabilities and improved interference effectiveness.

[0235] Simulation 2: Under the above conditions, the interference effect of the dense false targets after array deployment was simulated using both the present invention and the existing exhaustive method. The results are as follows: Figure 7 ,in:

[0236] 7(a) shows the space-time two-dimensional power spectrum after the array is arranged using the existing exhaustive method;

[0237] 7(b) is a cross-sectional view of the space-time power spectrum after the array is arranged using the existing exhaustive method;

[0238] 7(c) is the space-time two-dimensional power spectrum after arraying according to the method of the present invention;

[0239] 7(d) is a cross-sectional view of the space-time power spectrum after arraying according to the method of the present invention.

[0240] from Figure 7 (a) It can be seen that after using the existing exhaustive array method, the dense false target interference still has a relatively obvious STAP suppression region in the two-dimensional power spectrum of space and time.

[0241] from Figure 7 (b) As can be seen, the jammer array angles obtained by the existing exhaustive method are [20°, 22°, 24°, 26°], and the corresponding suppression notch depths at the jamming azimuth are -40.065dB, -32.582dB, -29.105dB, and -30.039dB, respectively. Compared with the suppression notch depths of -45.955dB, -36.667dB, -34.232dB, and -36.196dB without array optimization, the existing exhaustive method reduces the average suppression effect of STAP on distributed dense false target jamming by about 5.3dB.

[0242] from Figure 7 (c) It can be seen that after the array is arranged using the method of the present invention, the residual response of the dense false target interference in the space-time two-dimensional power spectrum is enhanced, and the notch depth at multiple interference azimuths is significantly reduced.

[0243] from Figure 7(d) As can be seen, the jammer array angles obtained by the method of the present invention are [20.1°, 31.9°, 37.1°, 45.3°], and the corresponding suppression notch depths at the jamming azimuth are -20.743dB, -29.653dB, -19.156dB, and -19.759dB, respectively. Compared with the existing exhaustive array method, the present invention further reduces the average suppression effect of STAP on distributed dense false target jamming by about 10.6dB; compared with no array optimization, the present invention reduces the average suppression effect of STAP on distributed dense false target jamming by about 15.9dB.

[0244] Simulation results show that, in dense decoy interference scenarios, this invention can obtain array parameters that are more favorable to the distribution of interference energy through continuous action space search, significantly reducing the suppression notches at multiple interference locations and enhancing the residual response of dense decoy interference after STAP processing. Compared with existing exhaustive methods, this invention has a more significant optimization effect on dense decoy interference scenarios.

[0245] Simulation 3: Under the above conditions, the intermittent sampling frequency shifting and forwarding interference effect after array deployment is simulated using the present invention and the existing exhaustive method, respectively. The results are as follows: Figure 8 ,in:

[0246] 8(a) shows the space-time two-dimensional power spectrum after the array is arranged using the existing exhaustive method;

[0247] 8(b) is a cross-sectional view of the space-time power spectrum after the array is arranged using the existing exhaustive method;

[0248] 8(c) is the space-time two-dimensional power spectrum after arraying according to the method of the present invention;

[0249] 8(d) is a cross-sectional view of the space-time power spectrum after arraying according to the method of the present invention.

[0250] from Figure 8 (a) It can be seen that after using the existing exhaustive array method, the intermittent sampling frequency shifting and forwarding interference is still processed by STAP in the space-time two-dimensional power spectrum to form multiple suppression notches.

[0251] from Figure 8 (b) As can be seen, the jammer array angles obtained by the existing exhaustive method are [20°, 21°, 22°, 23°], and the corresponding suppression notch depths at the jamming azimuth are -31.265dB, -29.060dB, -28.779dB, and -26.552dB, respectively. Compared with the suppression notch depths of -35.405dB, -31.962dB, -37.238dB, and -33.024dB without array optimization, the existing exhaustive method reduces the average suppression effect of STAP on distributed intermittent sampling frequency shifting and forwarding interference by approximately 5.5dB.

[0252] from Figure 8 (c) It can be seen that after arraying using the method of the present invention, the residual response near the interference azimuth in the two-dimensional power spectrum of space-time is enhanced, indicating that the present invention can reduce the suppression capability of STAP for intermittent sampling frequency shifting and forwarding interference.

[0253] from Figure 8 (d) As can be seen, the jammer array angles obtained by the method of the present invention are [22°, 26.7°, 35.1°, 42°], and the corresponding suppression notch depths at the jamming azimuth are -16.120dB, -26.098dB, -23.533dB, and -21.341dB, respectively. Compared with the existing exhaustive array method, the present invention further reduces the average suppression effect of STAP on distributed intermittent sampling frequency shifting and forwarding interference by about 7.1dB; compared with no array optimization, the present invention reduces the average suppression effect of STAP on distributed intermittent sampling frequency shifting and forwarding interference by about 12.6dB.

[0254] The simulation results show that, in the intermittent sampling frequency shift forwarding interference scenario, the present invention can obtain a better distributed interference array strategy based on the feedback learning of STAP processing, thereby raising the spatiotemporal power spectrum response at the interference location and weakening the suppression effect of STAP on this type of forwarding interference signal.

[0255] The above descriptions are merely two specific examples of the present invention and do not constitute any limitation on the present invention. Obviously, those skilled in the art, after understanding the content and principles of the present invention, may make various modifications, substitutions, or equivalent changes to the specific implementation, parameter settings, network structure, interference patterns, and module connection methods of the present invention without departing from the principles and structure of the present invention. For example, modifications or substitutions can be made in the following five aspects:

[0256] ① In the specific implementation process, the parameters such as the number of jammers, the number of radar array elements, the number of coherent pulses, the radar pulse width, the jamming azimuth range, the minimum azimuth interval, and the number of reference range units in this invention can all be adjusted according to the actual application scenario, radar system, and jamming resource conditions.

[0257] ② In Markov decision process modeling, the system state includes not only the jamming azimuth, power weight, relay pulse width, number of false targets, and STAP feedback information, but also information such as target azimuth, target Doppler frequency, radar operating mode, jamming pattern, signal-to-interference-plus-noise ratio, or historical reward value; the action space includes not only the above four types of array parameters, but can also be extended to other controllable jamming parameters such as jamming frequency, modulation parameters, relay delay, pulse repetition period, jamming duty cycle, or waveform coding parameters.

[0258] ③ Regarding action decoding, reward function, and network structure, the range of interference azimuth mapping, power normalization method, range of forwarding pulse width values, rounding method for the number of false targets, and minimum interval constraint processing method can all be replaced with equivalent values; the reward function can be the average notch depth, weighted notch depth, maximum notch lift, interference azimuth residual power response, or a combination thereof; the number of layers, number of neurons, activation function, optimizer, learning rate, exploration noise, and soft update method of the actor network, commentator network, and target network can also be adjusted according to training stability and computational resources.

[0259] ④ In terms of radar echo and space-time adaptive processing and jamming signal generation, covariance matrix estimation, space-time power spectrum calculation, reference range cell selection, space-time steering vector construction and STAP processing methods can all be replaced by equivalent substitutions; jamming signals can be noise convolution jamming, dense false target jamming, intermittent sampling frequency shifting and forwarding jamming, or other suppression, deception or composite jamming signals.

[0260] ⑤ In terms of system implementation, each module of the present invention can be implemented in software, hardware or a combination of software and hardware, and can be deployed in general-purpose computers, embedded processors, digital signal processors, graphics processors, field-programmable gate arrays or application-specific integrated circuits. The modules can also be implemented in serial, parallel, distributed or cloud-edge collaborative manner.

[0261] However, these modifications and alterations based on the ideas of this invention are still within the scope of protection of the claims of this invention.

Claims

1. A distributed interference array intelligent optimization method based on deep deterministic policy gradient, characterized in that, include: (1) Construct a distributed jamming array optimization scenario including target radar, space-time adaptive processing module and multiple jammers; (2) The problem of optimizing the parameters of the multi-interference machine cooperative array is modeled as a Markov decision process, that is, the system state, action space and reward function are defined; (3) Select the existing actor network, commentator network, target actor network and target commentator network as the multi-interference machine cooperative array parameter optimization network; (4) Obtain the parameters such as interference azimuth angle, power weight, forwarding pulse width, number of false targets and STAP feedback information, and input them into the actor network to output a continuous action vector. Then decode the continuous action vector into a new interference azimuth angle, power weight, forwarding pulse width and number of false targets, and generate a distributed interference signal based on the decoding result. (5) Input the parameters obtained in (4) and the output continuous action vector into the commentator network, output the value of the interference effect, and update the actor network parameters; (6) Perform space-time adaptive processing on the distributed interference signal to obtain the space-time power spectrum; (7) Extract the suppression notch depth at each interference azimuth based on the spatiotemporal power spectrum and obtain the reward; (8) Input the interactive experience consisting of system state, action, reward and new system state into the target actor network and target commentator network to calculate the target interference effect value, and update the commentator network parameters to approximate the target interference effect value; (9) Repeat steps (4) to (8) until the set termination condition is met to obtain the optimal actor network; (10) Input the current interference azimuth, power weight, forwarding pulse width and number of false targets into the optimal actor network, and output the interference azimuth, power weight, forwarding pulse width and number of false targets of the optimized distributed jammer array to complete the intelligent optimization of the distributed jammer array.

2. The method according to claim 1, characterized in that, In (2), the problem of optimizing the parameters of the multi-interference machine cooperative array is modeled as a Markov decision process, the implementation of which includes: 2a) Define the interference azimuth angle, power weight, forwarding pulse width, number of false targets, and STAP feedback information as the system state; 2b) Define the jamming azimuth, power weight, relay pulse width, and number of false targets of multiple jammers as the optimization parameters corresponding to the action space; 2c) Define the suppression notch depth at each interference azimuth after spatiotemporal adaptive processing as the evaluation criterion for the reward function; 2d) By defining the system state, action space and reward function as described above, the interaction between the optimization network and the space-time adaptive processing environment is established, and the optimization of the multi-interference machine cooperative array parameters is transformed into continuous action space optimization based on Markov decision process.

3. The method according to claim 1, characterized in that, The system state and action space are defined in (2), and the reward function is constructed. Its implementation includes: 2e) Define system state for: ， Where J represents the number of jammers. Let be the jamming azimuth angle of the j-th jammer at time t. Let be the power weight of the j-th jammer at time t. Let be the relay pulse width of the j-th jammer at time t. Let be the number of false targets of the j-th jammer at time t. The depth of the suppression notch at the j-th interference orientation after time-space adaptive processing at time t; 2f) Define the action space for: ， Where J is the number of jammers, the first to J dimensions are used for jamming azimuth coding, the J+1 to 2J dimensions are used for power weight coding, the 2J+1 to 3J dimensions are used for forwarding pulse width coding, the 3J+1 to 4J dimensions are used for false target number coding, and the value range of each element in the action vector is [-1,1]. 2g) Construct the reward function: 2g1) Using the suppression notch depth at each interference azimuth after space-time adaptive processing as the evaluation criterion, calculate the target Doppler frequency. Azimuth section of space-time power spectrum extracted : ； 2g2) According to the orientation tangent Calculate the depth of the suppression notch at the j-th interference location. : ； 2g3) Calculate the average value of the suppression notch depth at each interference azimuth to obtain the reward function. : ， Where J represents the number of jammers. Let be the jamming azimuth angle of the j-th jammer. Let J be the space-time power spectrum response at the j-th interference azimuth. Let be the depth of the suppression notch at the j-th interference location.

4. The method according to claim 1, characterized in that, In step (4), the continuous motion vector output by the actor network is decoded into new interference azimuth angle, power weight, forwarding pulse width, and number of false targets. This is achieved by: 4a) Convert the action vector The first to J dimensions are mapped to the jamming azimuth angles, thus obtaining the original jamming azimuth angles of the j-th jammer. : ， in, To prevent interference with the lower limit of the azimuth angle, To interfere with the upper limit of the azimuth angle, This is the azimuth encoding value corresponding to the action vector; 4b) Regarding the azimuth angle of interference Apply a minimum spacing constraint to ensure that the azimuth angles of any two jammers are within a certain range. and satisfy: ， in, The minimum azimuth interval is preset; 4c) Transfer the action vector The (J+1)th to (2J)th dimensions are mapped to power weights, thus obtaining the intermediate power weights of the j-th jammer. : ， in, This is the power weight encoding value corresponding to the action vector; 4d) Weighting of intermediate power After normalization, the power weight of the j-th jammer is obtained. : ， in, This is used to avoid extremely small positive numbers with a denominator of zero; 4e) The action vector The 2J+1 to 3J dimensions are mapped to the forwarding pulse width, and the forwarding pulse width of the j-th jammer is calculated. : ， in, This is the lower limit of the forwarding pulse width. This is the maximum forwarding pulse width. This is the forwarding pulse width encoding value corresponding to the action vector; 4f) The action vector The 3J+1 to 4J dimensions are mapped to the number of false targets, and the number of false targets of the j-th jammer is calculated. : ， in, This is the lower limit for the number of false targets. The maximum number of false targets. This indicates the rounding operation. This is the encoding value for the number of fake targets corresponding to the action vector.

5. The method according to claim 1, characterized in that, The generation of distributed interference signals based on the decoding results in (4) includes the following implementation: 4g) The jamming azimuth angle of the j-th jammer is obtained by decoding the continuous motion vectors output by the actor network. Power weight Forwarding pulse width and the number of false targets ,in J represents the number of jammers; 4h) Based on the number of false targets of the j-th jammer generate One delayed forwarding component : ， in, For the m-th delayed forwarding component, For intercepted or referenced radar signals, The delay of the m-th false target generated by the j-th jammer. For amplitude coefficient, For phase modulation amount, Forwarding pulse width A defined pulse width window function; 4i) The j-th jammer The delayed forwarding components are superimposed and combined with power weights. Obtain the jamming signal from the j-th jammer. : ； 4j) Based on the jamming azimuth angle of the j-th jammer Construct the spatial guidance vector of the j-th jammer : ， in It is the normalized spatial frequency corresponding to the j-th jammer. : 4k) Based on the jamming signal of the j-th jammer and spatial guidance vector Distributed interference signals were obtained: 。 6. The method according to claim 1, characterized in that, In step (5), the parameters obtained in step (4) and the output continuous action vector are input into the commentator network, the value of the interference effect is output, and the actor network parameters are updated. This process includes: 5a) The interference azimuth angle, power weight, forwarding pulse width, number of false targets, and STAP feedback information obtained in (4) are used to form the current system state: ， in, Let J be the system state at time t, and J be the number of jammers. Let be the jamming azimuth angle of the j-th jammer at time t. Let be the power weight of the j-th jammer at time t. Let be the relay pulse width of the j-th jammer at time t. Let be the number of false targets of the j-th jammer at time t. The depth of the suppression notch at the j-th interference orientation after time-space adaptive processing at time t; 5b) The continuous motion vectors output by the actor network are represented as: ， in, Indicates actor network, Indicates actor network parameters; 5c) Change the current system state and continuous action vectors By inputting a commentator network, the interference effect value corresponding to the current deployment action is obtained. : ， Where Q represents the commentator network, This indicates the commentator's network parameters. Used to evaluate the interference effect of the distributed interference array scheme corresponding to the current continuous action vector; 5d) Aiming to improve the interference effect value of commentator network output, based on system state and the continuous motion vectors output by the actor network Construct the optimization objective function for the actor network: ， in, The number of samples in a batch. This represents the system state in the i-th sample. 5e) Based on the state-action value output by the commentator network, calculate its gradient relative to the continuous action vector, and combine it with the gradient of the actor network's output action relative to the actor network parameters to obtain the actor network parameter update amount. : ， in, This represents the gradient of the commentator's network output state-action value with respect to continuous action vectors. This represents the gradient of the actor network output action with respect to the actor network parameters.

7. The method according to claim 1, characterized in that, The process of performing space-time adaptive processing on the distributed interference signal in (6) to obtain the space-time power spectrum includes the following: 6a) Based on normalized spatial frequency Constructing the spatial steering vector of the target radar receiving array : ， Where N is the number of radar receiver array elements, and d is the element spacing. The radar wavelength; 6b) Based on normalized Doppler frequency Constructing the time steering vector of the target radar receiving array : ， Where M is the number of coherent pulses; 6c) Based on the spatial guidance vector and time-guided vector Construct the space-time steering vector of the target radar receiving array : ， in, Represents the Kronecker product; 6d) Select common points on both sides of the radar receiving array of the target to be processed. The reference distance cell, and the first reference distance cell, and the second The received data from each reference range cell on N array elements and M coherent pulses is expanded into spacetime snapshot data. ; 6e) Based on the number of reference distance units and the Spacetime snapshot data of each reference unit Estimating the covariance matrix : ， Where H represents the conjugate transpose; 6f) Calculate the space-time adaptive processing weight vector based on the minimum variance criterion of linear constraints. : ， in, The target spacetime guidance vector; 6g) Based on the space-time adaptive processing weight vector and spacetime steering vector Calculation of space-time two-dimensional power spectrum : 。 8. The method according to claim 1, characterized in that, The process of extracting the suppression notch depth at each interference azimuth based on the spatiotemporal power spectrum in (7) and obtaining the reward includes the following: 7a) Based on the jamming azimuth angle of the j-th jammer Extract the spatiotemporal power spectrum response at the corresponding interference azimuth. : ， in, To normalize the Doppler frequency for the target; 7b) Based on the interference azimuth angle Calculate the depth of the suppression notch: ， in, Let be the depth of the suppression notch at the interference azimuth corresponding to the j-th jammer. The maximum value of the power spectral response in spacetime; 7c) Calculate the reward value based on the depth of the suppression notch at all interference locations. : ， Where J represents the number of jammers.

9. The method according to claim 1, characterized in that, In step (8), the system state, actions, rewards, and new system state are combined to form interactive experience, which is then input into the target actor network and the target commentator network to calculate the target interference effect value. The commentator network parameters are updated to approximate the target interference effect value. Its implementation includes: 8a) The system state, actions, rewards, and new system state obtained during the interaction at time t constitute the interaction experience: ， in, Let t be the system state. Let be the action vector at time t. The reward at time t, The new system state obtained after performing the action at time t; 8b) Random selection Group interaction experience: ， 8c) New system states in the interactive experience Input into the target actor network to obtain the target action at the next moment. : ， in, Indicates the target actor network, Indicates the network parameters of the target actor; 8d) Change the new system state and target action Inputting the data into the target commentator network yields the target state-action value. : ， in, This indicates a network of target commentators. Indicates the network parameters of the target commentator; 8e) Based on the reward and target state-action value Calculate the value of target interference effect : ， in, The value of the target interference effect corresponding to the i-th sample group is... Discount factor; 8f) Change the current system state and current action Input the commentator network to obtain the current interference effect value. : ， Where Q represents the commentator network, Indicates the commentator's network parameters; 8g) Construct the commentator network loss function based on the current interference effect value and the target interference effect value: ； 8h) Update the commentator network parameters according to the negative gradient direction of the loss function: ， in, For the commentator network loss function, For commentator network parameters, For commentators' online learning rate, This represents the gradient of the loss function with respect to the commentator network parameters; 8i) Perform soft updates to the target actor network and the target commentator network: ， ， in, For actor network parameters, For commentator network parameters, denoted as the target network soft update coefficient.

10. A distributed interference array intelligent optimization system based on deep deterministic policy gradient, characterized in that, include: The distributed jamming array scenario construction module is used to construct an optimized distributed jamming array scenario that includes a target radar, a space-time adaptive processing module, and multiple jammers. The Markov decision process modeling module is used to model the multi-interference machine cooperative deployment parameter optimization problem as a Markov decision process and define the system state, action space and reward function. The deep deterministic policy gradient agent decision-making module is used to construct actor networks, commentator networks, target actor networks, and target commentator networks, and output continuous action vectors based on the current system state. The action decoding and parameter mapping module is used to map continuous action vectors to the jamming azimuth angle, power weight, forwarding pulse width and number of false targets of multiple jammers; The radar echo and space-time adaptive processing module is used to generate radar received data based on interference parameters, calculate the space-time adaptive processing weight vector, and output the space-time power spectrum. The reward calculation module is used to extract the suppression notch depth at each interference azimuth based on the spatiotemporal power spectrum and calculate the reward function value. The experience replay and network training module is used to store experience samples generated by the interaction between the agent and the environment, and to train and update the actor network and commentator network. The array strategy output module is used to output the optimal distributed interference array parameters after training is completed.

11. The system according to claim 10, characterized in that, The experience replay and network training module includes: The interactive experience construction submodule is used to combine the system state, actions, rewards and new system state obtained during an interaction into an interactive experience. The sample sampling submodule is used to randomly sample multiple sets of training samples from the interaction experience; The target action generation submodule is used to input the new system state into the target actor network to obtain the target action; The target value calculation submodule is used to input the new system state and target action into the target commentator network, obtain the target state-action value, and calculate the target interference effect value; The commentator network update submodule is used to construct a loss function based on the current interference effect value and the target interference effect value, and to update the commentator network parameters; The actor network update submodule is used to update the actor network parameters based on the gradient of the state-action value to the action output by the commentator network, so that the action output by the actor network is optimized in the direction of improving the value of the interference effect. The Target Network Soft Update submodule is used to perform soft updates on the Target Actor Network and the Target Commentator Network.

Images