Space-based radar time domain waveform adaptive method based on reinforcement learning

Abstract

The invention provides a space-based radar time domain waveform adaptive method based on reinforcement learning, and the method comprises the following steps: firstly, constructing a radar signal echo model and a radar clutter echo model according to an inter-pulse frequency modulation pulse Doppler radar; calculating a signal clutter echo distance Doppler matrix according to the radar signal echo model and the radar clutter echo model; establishing a reinforcement learning environment and introducing a reward function to obtain a large amount of reinforcement learning data pairs; taking a large amount of reinforcement learning data as input, and combining with a reinforcement learning algorithm to train the intelligent agent; and finally, outputting an optimized inter-pulse frequency modulation pulse Doppler waveform by applying the trained intelligent agent, and realizing clutter position regulation and control. According to the method, the pulse Doppler waveform of inter-pulse frequency modulation is provided, so that the radar has the capability of actively regulating and controlling the clutter position.

Description

Technical Field

[0001] The present invention relates to the field of electronic information technology, in particular to the field of radar signal processing technology, and in particular to a method for improving radar signal processing performance by using an artificial intelligence method, specifically referring to a time domain waveform adaptation method for airborne radar based on reinforcement learning. Background Art

[0002] Reinforcement learning is to train an intelligent agent to make autonomous decisions based on the current environment. The intelligent agent continuously interacts with the environment, outputs actions based on the state of the environment, and obtains rewards and penalties from environmental feedback to build an experience library. After continuous iterative learning of the experience library, it eventually achieves actions that can output the maximum reward in all environments and avoids actions with penalty items. Unlike supervised learning in deep learning, reinforcement learning does not require a large amount of labeled data to be provided in advance.

[0003] Airborne radar refers to radar mounted on an aerial platform, such as airborne radar, missile-borne radar, etc. Airborne radar has important applications in the fields of civilian environmental monitoring, traffic management, and military air defense and anti-missile. In particular, with the development of my country's low-altitude economy, the increasing number of aerial vehicles, led by drones, has brought urgent needs for efficient low-altitude traffic control. Given the wide field of view and rapid action capabilities of airborne platforms, airborne radars have a natural advantage in such tasks.

[0004] Airborne radars generally operate in pulse Doppler mode, that is, by sending multiple identical pulses, the phase change value between pulses is obtained and converted into target Doppler information. However, airborne radars are easily affected by ground clutter when performing downward observation of low-altitude targets. Ground clutter mainly includes mainlobe clutter, sidelobe clutter and altitude line clutter. Mainlobe clutter is mainly caused by the mainlobe irradiating the ground, sidelobe clutter is caused by the ground clutter signal entering from the antenna sidelobe, and altitude line clutter is caused by the strong ground clutter mirror reflection caused by the large ground rubbing angle at the altitude line. Once the target enters the clutter area, it will be covered by the clutter, resulting in the loss of target tracking. When the pulse Doppler radar operates at a high repetition rate, the clutter folding caused by range ambiguity will further expand the clutter area, bringing challenges to the radar's target detection and tracking. The methods for processing ground clutter on traditional motion observation platforms include offset phase center antenna (DPCA) and space-time adaptive processing technology (STAP). The offset phase center antenna technology can also be regarded as a method of joint processing in space and time domains. However, the challenge of implementing an offset phase center antenna is that the antenna array has errors such as amplitude, phase, and mutual coupling, which makes it difficult to ensure that two or more sub-apertures with phase center offset have the same amplitude and phase response. Space-time adaptive processing technology sets waveform weights in both the time domain and the spatial domain, so that the echo forms a concave response at the clutter, thereby achieving clutter suppression. However, the performance of space-time adaptive processing technology is based on the premise of obtaining the clutter covariance matrix, but the number of pulses that can be used to estimate the covariance matrix under actual application conditions is limited. In addition, both the offset phase center antenna and space-time adaptive processing technology are post-processing or passive processing technologies, that is, processing performed when the radar waveform is fixed. How to adaptively design the waveform to achieve active control of clutter so that it avoids the target is a solution with application prospects. Summary of the invention

[0005] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and propose a time-domain waveform adaptation method for airborne radar based on reinforcement learning to achieve active control of clutter.

[0006] In order to solve the above technical problems, the technical solution of the present invention is:

[0007] A time-domain waveform adaptation method for airborne radar based on reinforcement learning comprises the following steps:

[0008] Step 1, constructing a radar signal echo model and a radar clutter echo model according to an inter-pulse frequency modulation pulse Doppler radar respectively;

[0009] Step 2: Calculate the signal clutter echo range Doppler matrix according to the radar signal echo model and the radar clutter echo model;

[0010] Step 3: Establish a reinforcement learning environment and introduce a reward function to obtain a large number of reinforcement learning data pairs;

[0011] Step 4: Take a large number of reinforcement learning data pairs as input and combine them with reinforcement learning algorithms to train the intelligent agent;

[0012] Step 5: Apply the trained intelligent agent to output the optimized inter-pulse frequency modulation pulse Doppler waveform to achieve the control of the clutter position.

[0013] Preferably, the radar signal echo model is constructed by:

[0014] Assuming that the starting frequency of the pulse frequency modulation is f0, the nth transmitted pulse can be expressed as s n (t) = exp(j2π(f0+nΔf)t), n = 1, 2, ..., N, where j is the complex imaginary unit, t is the fast time, and Δf is the pulse-to-pulse frequency step;

[0015] Assuming the pulse repetition interval is T, then p = Δf / T is defined as the interpulse modulation frequency;

[0016] The nth pulse echo signal of the target at the distance R0 from the radar can be written as Among them, σ is the echo amplitude value of the target radar scattering cross-section after attenuation by the radar equation, c is the speed of light, and f d is the Doppler frequency of the echo signal;

[0017] In the inter-pulse frequency modulation system, Where v is the velocity of the target relative to the radar.

[0018] Preferably, after transmitting and receiving N pulses through the radar signal echo model, the row vector of the radar signal echo is x=[x1, x2, ..., x N ].

[0019] Preferably, the radar clutter echo model is constructed by:

[0020] Set the observation distance range [R min ,R max ], where R min =h is the radar altitude, R max is the farthest sensitive distance;

[0021] In the distance range [R min ,R max ], and the L distance grids after division are set as R1, R2, ..., R L ;

[0022] Traverse these L grids: for the lth distance grid R l , and its corresponding ground rings are spaced by azimuth Divide the azimuth grid into equal intervals within [0,2π];

[0023] Assume that the lth range ring is divided into M azimuth grids through azimuth grid division, set as Each Doppler grid corresponds to a clutter unit. There are M clutter units in total. The area of ​​the clutter unit is

[0024] The radar cross-sectional area σ of the clutter unit is obtained from the clutter unit area l =σ0ΔA, where σ0 is the electromagnetic wave backscattering coefficient of the ground surface; for the mth clutter unit, its velocity v relative to the radar can be obtained from its geometric relationship with the radar: m , combined with the pulse modulation frequency p, and then the Doppler frequency is obtained

[0025] So the echo of the nth pulse, the lth range unit, and the mth clutter unit is Among them, σ l ′ is the value obtained by the radar equation l The amplitude value of the clutter after attenuation, φ0 is the random phase added to simulate the randomness of the clutter;

[0026] The echoes of all M clutter units corresponding to the lth range unit are superimposed to obtain By l,n , l=1,2,...,L,n=1,2,...,N, we can get the clutter echo matrix Y with L rows and N columns, where the element in the lth row and the nth column of the matrix Y is y l,n .

[0027] Preferably, the R max Selected value: needs to be greater than the target distance, and the maximum value is when the echo power of the ground clutter at location is less than the specific value of the echo power of the ground clutter at location.

[0028] As a preference, in the distance range [R min ,R max ] is divided into grids at intervals of the radar's range resolution ΔR, where ΔR = cτ / 2, and τ is the width of the radar's transmitted pulse.

[0029] Preferably, the azimuth interval The selection criteria are: make sure that the Doppler variation in the clutter unit corresponding to each azimuth interval does not exceed the radar's Doppler resolution Δf d , where Δf d =1 / (TN).

[0030] Preferably, in step 2, the signal clutter echo range Doppler matrix is ​​calculated by: adding the row vector x of the radar signal echo to the row corresponding to the clutter echo matrix Y according to the distance of the target to obtain the signal clutter matrix Z; and then calculating the signal clutter matrix Z according to the maximum unambiguous distance unit number of the radar. The signal-clutter matrix Z is folded and superimposed row by row to obtain the L′ row and N column signal-clutter matrix Z′; finally, each row of Z′ is fast Fourier transformed to obtain the signal-clutter echo-range Doppler matrix Z′ f .

[0031] As a preferred method, the method for establishing the reinforcement learning environment is: setting the state s of the environment to be the signal clutter range Doppler matrix Z′ f , the action value a of the agent is the interpulse modulation frequency p; the agent generates action a according to the state s of the current environment; the nth transmission pulse of the next frame radar waveform changes to s according to the interpulse modulation frequency p n (t) = exp(j2π(f0+npT)t), the environment calculates the new state s' according to the signal clutter echo range Doppler matrix, that is, the signal clutter range Doppler matrix Z' of the radar frame f，new , and give a reward value r, repeat the above steps to obtain a large number of data pairs {s,a,s',r}.

[0032] Preferably, the reward value r is calculated as follows:

[0033] The reward function is designed as follows: r = ε + c1SCR - c2abs(p), where abs(·) represents the absolute value, SCR is the signal-to-noise ratio of the target, and when SCR is greater than the signal-to-noise ratio threshold of the radar echo signal set by the user, ε is set to 1, otherwise ε is set to 0, and c1 and c2 are weight coefficients.

[0034] Preferably, the state s of the environment includes the position information of the radar and the angle of the target relative to the radar.

[0035] Preferably, the reinforcement learning method is a nearest neighbor strategy optimization algorithm or a soft actor-critic algorithm.

[0036] The present invention has the following characteristics and beneficial effects:

[0037] By adopting the above technical solution and proposing a pulse Doppler waveform with inter-pulse frequency modulation, the radar has the ability to actively control the position of clutter; combined with the reinforcement learning processing framework and algorithm, the radar becomes an intelligent entity, which has the ability to adaptively optimize the radar time domain waveform according to the radar echo, thereby avoiding clutter and improving the target signal-to-clutter ratio. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0039] Figure 1 It is a flowchart of the overall process of the method of the present invention.

[0040] Figure 2 It is the trajectory diagram of radar and target.

[0041] Figure 3 It is a simulation comparison of the adaptive waveform signal-to-noise ratio improvement effect of the method of the present invention. DETAILED DESCRIPTION

[0042] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0043] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and the like are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", and the like may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0044] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood by specific circumstances.

[0045] The present invention provides a time domain waveform adaptive method for airborne radar based on reinforcement learning, such as Figure 1 As shown, the following steps are included:

[0046] Step 1: Modeling of pulse Doppler radar signal echo: Assuming that the starting frequency of the pulse frequency modulation is f0, the nth transmitted pulse can be expressed as s n (t) = exp(j2π(f0+nΔf)t), n = 1, 2, ..., N. Where j is the complex imaginary unit, t is the fast time, and Δf is the pulse frequency step. Assuming the pulse repetition interval is T, p = Δf / T is defined as the pulse frequency. In this embodiment, the radar pulse repetition interval T = 6.4us. The nth pulse echo signal of the target at the distance R0 from the radar can be written as Among them, σ is the echo amplitude value of the target radar scattering cross-section after attenuation by the radar equation, c is the speed of light, and f d is the Doppler frequency of the echo signal. In the pulse frequency modulation system, Where v is the speed of the target relative to the radar. After transmitting and receiving N pulses, the signal row vector is x = [x1, x2, ..., x N ] In this embodiment, the number of radar pulses N is set to 128.

[0047] Step 2: Modeling of pulse Doppler radar clutter echo: Set the observation distance range [R min ,R max ], where R min =h is the radar altitude. In this embodiment, the radar altitude is 3.5km, and the target altitude is 1km.

[0048] R max is the farthest sensitive distance, the R max The minimum value needs to be greater than the target distance, and the maximum value is max The echo power of the ground clutter is less than R min The specific value of the ground clutter echo power. It can be understood that the target distance refers to the distance between the radar and the target. In addition, in this embodiment, the specific value is 30dB, 60dB, or 90dB. It can be understood that the larger the value is, the higher the R max The larger the value, the higher the clutter modeling accuracy, but the amount of calculation also increases. max The value is set to 10km.

[0049] In the distance range [R min ,R max ] is divided into grids with the radar range resolution ΔR as the interval. ΔR=cτ / 2, τ is the width of the radar transmission pulse. The L range grids after division are set as R1, R2, ..., R L . Traverse these L grids: for the lth distance grid R lIn this embodiment, the number of radar pulses N is set to 128, so the total number of range units is set to L=1024.

[0050] The corresponding ground rings are spaced by azimuth The azimuth grid is divided into equal intervals in [0,2π].

[0051] The selection criterion is to ensure that the Doppler variation in the clutter unit corresponding to each azimuth interval does not exceed the radar's Doppler resolution Δf d , where Δf d =1 / (TN). Assume that the lth range ring is divided into M azimuth grids through azimuth grid division, set as Each Doppler grid corresponds to a clutter unit, and there are M clutter units in total. The area of ​​the clutter unit is The radar cross-sectional area σ of the clutter unit is obtained from the clutter unit area l =σ0ΔA, where σ0 is the electromagnetic wave backscattering coefficient of the ground surface. Different ground environments have different values, which can be obtained by referring to relevant empirical formulas. For the mth clutter unit, its velocity v relative to the radar can be obtained from its geometric relationship with the radar. m , combined with the pulse modulation frequency p, and then the Doppler frequency is obtained So the echo of the nth pulse, the lth range unit, and the mth clutter unit is Among them, σ l ′ is the value obtained by the radar equation l The amplitude value of the clutter after attenuation, φ0 is the random phase added to simulate the randomness of the clutter. The echoes of all M clutter units corresponding to the lth range unit are superimposed to obtain By l,n , l=1,2,...,L,n=1,2,...,N, we can get the clutter echo matrix Y with L rows and N columns, where the element in the lth row and the nth column of the matrix Y is y l,n .

[0052] According to the settings of step 1 and step 2, the movement trajectory of the target and the radar is as follows Figure 2 A total of 96 frames of data are simulated.

[0053] Step 3: Calculate the signal-clutter-echo range Doppler matrix: According to the target distance, add the signal row vector x to the corresponding row of the clutter matrix Y to obtain the signal-clutter matrix Z. Then, according to the maximum unambiguous range unit number of the radar, In this embodiment, the number of unambiguous range units is L′=128. The signal clutter matrix Z is folded and superimposed row by row to obtain an L′ row and N column signal clutter matrix Z′. Finally, each row of Z′ is fast Fourier transformed to obtain a signal clutter range Doppler matrix Z′f .

[0054] Step 4: Establish reinforcement learning environment and design reward function: Set the state s of the environment to be the signal clutter range Doppler matrix Z′ f , the action value a of the agent is the pulse modulation frequency p. The agent is based on the state s of the current environment, that is, the matrix Z′ f , generating action a, i.e., the interpulse modulation frequency p applied to the next radar frame; the nth transmission pulse of the next radar waveform changes to s according to the interpulse modulation frequency p n (t) = exp(j2π(f0+npT)t), the environment calculates the new state s' according to the signal clutter echo range Doppler matrix, that is, the signal clutter range Doppler matrix Z' of the radar frame f，new , and gives a reward value r. The agent generates new actions based on the new state s', and repeats this process.

[0055] Furthermore, in this embodiment, the state s of the environment also adds the position information of the radar and the angle of the target relative to the radar.

[0056] The reward function is designed as r = ε + c1SCR - c2abs(p). Where abs(·) represents the absolute value, and SCR is the signal-to-noise ratio of the target. When SCR is greater than a certain value, the value is the radar echo signal signal-to-noise ratio threshold value desired by the user. In this embodiment, it is set to 10dB, 15dB, or 20dB.

[0057] Furthermore, ε=1 is set, otherwise ε=0. c1 and c2 are weight coefficients. In this embodiment, when training the agent, the reward function coefficients c1=0.001 and c2=0.1.

[0058] It should be noted that the data pair {s, a, s', r} is single-step reinforcement learning data.

[0059] Step 5: Waveform design based on reinforcement learning agent training: Continuously simulate the reinforcement learning environment. Through multiple interactions between the agent and the environment, a large number of reinforcement learning data pairs in the form of {s, a, s', r} can be generated. A large number of reinforcement learning data pairs are used as input and combined with existing reinforcement learning algorithms to train the agent.

[0060] In this embodiment, the reinforcement learning algorithm may select the Proximal Policy Optimization (PPO) algorithm or the Soft Actor-Critic (SAC) algorithm.

[0061] It is understandable that the process can be repeated, with the environment interacting to generate data pairs, the agent training, and then the environment interacting to generate data pairs again, and so on, until a preset number of cycles is reached.

[0062] Step 6, waveform design agent application: After the agent training is completed, the new signal clutter distance Doppler matrix is ​​input to the agent, and the agent will adaptively give the optimal inter-pulse modulation frequency p. By designing a new inter-pulse frequency modulation pulse Doppler waveform with this inter-pulse modulation frequency, the SCR of the target in the next frame can be improved, that is, the clutter position is regulated to avoid the target.

[0063] The technical solution provided by the above embodiment is compared with the traditional fixed waveform. The comparison results are as follows: Figure 3 As shown in the figure, in the initial 50 frames or so, the signal-to-noise ratio of the adaptive waveform is similar to that of the fixed waveform. However, after 50 frames, the target begins to be obscured by clutter, causing the signal-to-noise ratio of the fixed waveform to drop rapidly until the target passes through the clutter, and the signal-to-noise ratio recovers by more than ten dB. However, the signal-to-noise ratio of the adaptive waveform proposed in the present invention is completely unaffected by clutter, and the signal-to-noise ratio shows an increasing trend, which is consistent with the increasing distance between the radar and the target.

[0064] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments including components are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.

Claims

1. A time domain waveform adaptation method for airborne radar based on reinforcement learning, characterized in that: The steps include: Step 1, constructing a radar signal echo model and a radar clutter echo model according to an inter-pulse frequency modulation pulse Doppler radar respectively; Step 2: Calculate the signal clutter echo range Doppler matrix according to the radar signal echo model and the radar clutter echo model; Step 3: Establish a reinforcement learning environment and introduce a reward function to obtain a large number of reinforcement learning data pairs; Step 4: Take a large number of reinforcement learning data pairs as input and combine them with reinforcement learning algorithms to train the intelligent agent; Step 5: Apply the trained intelligent agent to output the optimized inter-pulse frequency modulation pulse Doppler waveform to achieve the control of the clutter position.

2. The method for time-domain waveform adaptation of airborne radar based on reinforcement learning according to claim 1, characterized in that: The radar signal echo model is constructed as follows: Assuming that the starting frequency of the pulse frequency modulation is f0, the nth transmitted pulse can be expressed as s n (t) = exp(j2π(f0+nΔf)t), n = 1, 2, ..., N, where j is the complex imaginary unit, t is the fast time, and Δf is the pulse-to-pulse frequency step; Assuming the pulse repetition interval is T, then p = Δf / T is defined as the interpulse modulation frequency; The nth pulse echo signal of the target at the distance R0 from the radar can be written as Among them, σ is the echo amplitude value of the target radar scattering cross-section after attenuation by the radar equation, c is the speed of light, and f d is the Doppler frequency of the echo signal; In the inter-pulse frequency modulation system, Where v is the velocity of the target relative to the radar.

3. The method for time-domain waveform adaptation of airborne radar based on reinforcement learning according to claim 2, characterized in that: After transmitting and receiving N pulses through the radar signal echo model, the row vector of the radar signal echo is obtained as x=[x1, x2, ..., x N ].

4. The method for time-domain waveform adaptation of airborne radar based on reinforcement learning according to claim 3 is characterized in that: The construction method of the radar clutter echo model: Set the observation distance range [R min ,R max ], where R min =h is the radar altitude, R max is the farthest sensitive distance; In the distance range [R min ,R max ], and the L distance grids after division are set as R1, R2, ..., R L ; Traverse these L grids: for the lth distance grid R l , and its corresponding ground rings are spaced by azimuth Divide the azimuth grid into equal intervals within [0,2π]; Assume that the lth range ring is divided into M azimuth grids through azimuth grid division, set as Each Doppler grid corresponds to a clutter unit. There are M clutter units in total. The area of ​​the clutter unit is The radar cross-sectional area σ of the clutter unit is obtained from the clutter unit area l =σ0ΔA, where σ0 is the electromagnetic wave backscattering coefficient of the ground surface; for the mth clutter unit, its velocity v relative to the radar can be obtained from its geometric relationship with the radar: m , combined with the pulse modulation frequency p, and then the Doppler frequency is obtained So the echo of the nth pulse, the lth range unit, and the mth clutter unit is Among them, σ l ′ is the value obtained by the radar equation l The amplitude value of the clutter after attenuation, φ0 is the random phase added to simulate the randomness of the clutter; The echoes of all M clutter units corresponding to the lth range unit are superimposed to obtain By l,n , l=1,2,...,L,n=1,2,...,N, we can get the clutter echo matrix Y with L rows and N columns, where the element in the lth row and the nth column of the matrix Y is y l,n .

5. The method for time-domain waveform adaptation of airborne radar based on reinforcement learning according to claim 4 is characterized in that: The R max Selected value: needs to be greater than the target distance, and the maximum value is when the echo power of the ground clutter at location is less than the specific value of the echo power of the ground clutter at location.

6. The method for time-domain waveform adaptation of airborne radar based on reinforcement learning according to claim 4, characterized in that: In the distance range [R min ,R max ] is divided into grids at intervals of the radar's range resolution ΔR, where ΔR = cτ / 2, and τ is the width of the radar's transmitted pulse.

7. The method for time-domain waveform adaptation of airborne radar based on reinforcement learning according to claim 4, characterized in that: The azimuth interval The selection criteria are: make sure that the Doppler variation in the clutter unit corresponding to each azimuth interval does not exceed the radar's Doppler resolution Δf d , where Δf d =1 / (TN).

8. The method for time-domain waveform adaptation of airborne radar based on reinforcement learning according to claim 4, characterized in that: In step 2, the signal clutter echo range Doppler matrix is ​​calculated by: adding the row vector x of the radar signal echo to the row corresponding to the clutter echo matrix Y according to the distance of the target to obtain the signal clutter matrix Z; and then calculating the signal clutter matrix Z according to the maximum unambiguous distance unit number of the radar. The signal-clutter matrix Z is folded and superimposed row by row to obtain the L′ row and N column signal-clutter matrix Z′; finally, each row of Z′ is fast Fourier transformed to obtain the signal-clutter echo-range Doppler matrix Z′ f .

9. The method for time-domain waveform adaptation of airborne radar based on reinforcement learning according to claim 8, characterized in that: The method for establishing the reinforcement learning environment is as follows: setting the state s of the environment to be the signal clutter range Doppler matrix Z′ f , the action value a of the agent is the interpulse modulation frequency p; the agent generates action a according to the state s of the current environment; the nth transmission pulse of the next frame radar waveform changes to s according to the interpulse modulation frequency p n (t) = exp(j2π(f0+npT)t), the environment calculates the new state s' according to the signal clutter echo range Doppler matrix, that is, the signal clutter range Doppler matrix Z' of the radar frame f，new , and give a reward value r, repeat the above steps to obtain a large number of data pairs {s,a,s',r}.

10. The method for time-domain waveform adaptation of airborne radar based on reinforcement learning according to claim 9, characterized in that: The calculation method of the reward value r is: The reward function is designed as follows: r = ε + c1SCR - c2abs(p), where abs(·) represents the absolute value, SCR is the signal-to-noise ratio of the target, and when SCR is greater than the signal-to-noise ratio threshold of the radar echo signal set by the user, ε is set to 1, otherwise ε is set to 0, and c1 and c2 are weight coefficients.

11. The method for time-domain waveform adaptation of airborne radar based on reinforcement learning according to claim 9, characterized in that: The state s of the environment includes the position information of the radar and the angle of the target relative to the radar.

12. The method for time-domain waveform adaptation of airborne radar based on reinforcement learning according to claim 1, characterized in that: The reinforcement learning method is a nearest neighbor strategy optimization algorithm or a soft actor-critic algorithm.

Images