Radar adaptive spectrum resource allocation method based on target clustering

Abstract

The invention provides a radar adaptive spectrum resource allocation method based on target clustering, which comprises the following steps that: at the beginning of each time slot, each radar in a radar system clusters detected targets according to detected information; and the radar selects a corresponding beam direction by referring to the classification result and self observation, and maintains the beam direction in a plurality of next time slots. Then, each radar selects a frequency modulation slope and a sub-band transmitting signal, calculates a return according to a received signal, and updates a strategy; through information interaction with the environment, the radar system learns an optimal strategy of adaptive spectrum resource configuration so as to solve a radar spectrum resource complex decision problem when facing dynamic multiple targets in a complex electromagnetic environment.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a radar adaptive spectrum resource configuration method based on target clustering, which is particularly suitable for multi-target detection mission scenarios in complex electromagnetic environments. Background Art

[0002] With the widespread application of radars and the rapid growth of the number of radars, the spectrum in which radars are located is constantly being squeezed, and their anti-interference capabilities are constantly being tested. Radar systems often face complex and uncertain electromagnetic spectrum environments when performing tasks, which requires their radars to identify effective information from a large amount of collected information, so as to effectively decide the next action. In our research, each radar must allocate sub-bands, frequency modulation slopes, and beam directions in the crowded spectrum to avoid mutual interference and successfully detect targets. This is a sequential decision problem. Usually, the Markov decision process is used to model the above decision problems. However, if the problem is modeled as a Markov decision process, it will make it difficult for the optimization problem to find the global optimal solution and converge to a suboptimal solution, and the number of action spaces will become very complicated. It may even cause the algorithm to fail to converge. In addition, due to interference between radars, it is impossible to effectively judge whether the target is real within a time slot. Therefore, multiple time slots are required to observe and judge, which also makes the problem a complex decision problem with different time scales. Summary of the invention

[0003] Aiming at the problem of spectrum resource allocation of radar system when facing dynamic multiple targets in a crowded spectrum environment, the present invention proposes a radar adaptive spectrum resource configuration method based on target clustering. Through information interaction with the environment, the radar system learns the optimal strategy of adaptive spectrum resource configuration to solve the complex decision-making problem of radar spectrum resources when facing dynamic multiple targets in a complex electromagnetic environment.

[0004] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0005] A radar adaptive spectrum resource configuration method based on target clustering, the radar adaptive spectrum resource configuration method comprising the following steps:

[0006] Each radar clusters the detected targets according to the detected information, and selects the corresponding beam direction using the beam direction strategy network based on the current beam direction, the number of detected targets, the measured target distance and the target clustering result information, and maintains the beam direction in the next few time slots; then, each radar selects the frequency modulation slope and sub-band transmission signal of each time slot based on the selected beam direction, spectrum occupancy and relative noise level estimation value using the spectrum configuration strategy network;

[0007] The return is calculated according to the received signal, and the outputs of the beam direction value network and the spectrum configuration value network are combined to update the beam direction strategy network parameters and the beam direction value network parameters and the spectrum configuration strategy network parameters and the spectrum configuration value network parameters.

[0008] Furthermore, the radar adaptive spectrum resource configuration method based on target clustering comprises the following steps:

[0009] Step 1: Obtain the radar position at the current time slot t, the information detected by each radar at the previous time slot t-1, and the selected beam direction to form a feature matrix;

[0010] Step 2: The communication relationship between the radar and the target is used to form an adjacency matrix, and the feature matrix and the adjacency matrix are input into the convolutional neural network to infer the target clustering result;

[0011] Step 3: Obtain the radar beam direction, the number of detected targets, the measured target distance and the target clustering result information at the current time slot t, and use it as the state of the current radar beam direction decision;

[0012] Step 4: Use the beam direction strategy network to select the beam direction action, and select a direction from the preset beam direction set as the signal beam transmission direction for the next few time slots according to the output action selection probability;

[0013] Step 5: Obtain the beam direction, spectrum occupancy and relative noise level estimation value of the radar at the current time slot t, and use it as the status of the current radar spectrum configuration decision;

[0014] Step 6: Use the spectrum configuration strategy network to select the frequency modulation slope and sub-band action, and select the frequency modulation slope and sub-band from the preset action collection as the signal transmission parameters for the next time slot t+1 according to the output action selection probability;

[0015] Step 7: When the radar transmits a signal in the next time slot t+1 and receives the signal, the first time difference error is calculated according to the output of the spectrum configuration value network and the reward value of the spectrum configuration decision, and the spectrum configuration strategy network and the spectrum configuration value network are trained according to the first time difference error;

[0016] Step 8: After NΔt time slots, the second time difference error is calculated according to the output of the beam direction value network and the reward value of the beam direction decision, and the beam direction strategy network and the beam direction value network are trained according to the second time difference error;

[0017] Step 9: At each N Δt Repeat steps 1 to 8 for each time slot until the stop condition is reached, at which point the action selection strategy converges.

[0018] Step 2 further includes:

[0019] Constructing the feature matrix Where P t represents the radar position coordinate set in the radar system, w t represents the set of directions of each radar beam, represents a collection of range measurements for each radar target;

[0020] Construct an adjacency matrix A, which consists of the communication relationship between radar i and target k. If the target is detected, the corresponding position A(k, i) is set to 1, otherwise it is set to 0;

[0021] Graph convolution is performed in a convolutional neural network as follows:

[0022]

[0023] in I N is the identity matrix, yes The degree matrix, H l is the feature matrix of each layer; σ(·) is the softmax function; W l Represents the weight matrix of the lth layer of the convolutional neural network; the target clustering result Re is obtained at the output layer i Afterwards, the cross entropy loss function is calculated:

[0024]

[0025] Where T i is the label matrix using one-hot encoding; after pre-training, W l Target clustering is directly applied to beam direction decision, and the clustering result is used as one of the observations to help search for targets.

[0026] Step 2 further includes:

[0027] The entire spectrum is divided into Y non-overlapping sub-bands. The radar selects one sub-band in each time slot. The number of sub-bands is greater than the number of radars. The MIMO radars used by each radar i are M t transmit antennas and M r The MIMO radar with 100 receiving antennas is arranged in a uniform linear array, and the interval between adjacent transmitting / receiving antennas is d=λ / 2, where λ is the wavelength.

[0028] The mth t The signal transmitted by the transmitting antenna is:

[0029]

[0030] Where B represents the signal bandwidth, represents the linear frequency modulation interval of the ith radar, represents the carrier frequency used by the i-th radar;

[0031] Beamforming technology is used to concentrate the main lobe energy of the MIMO radar transmission signal. The codebook used for:

[0032]

[0033] Where P represents the total number of codewords in the DFT codebook, and p = 0, 1, ..., P-1; M t Indicates the total number of transmitting antennas of each radar;

[0034] The mth MIMO radar on the i-th radar after beamforming t The signal transmitted by the transmitting antenna is:

[0035]

[0036] Determine whether the target is within the main lobe range of the radar transmission signal. If the target is within the main lobe range, it is considered that there is a target within the beam range of the radar transmission signal, and the transmission signal is mirror-reflected on the target and forms an echo signal; otherwise, it is considered that the radar has not detected the target;

[0037] When the first airborne MIMO radar detects the target, its Mth r The signal received by the receiving antenna is:

[0038]

[0039] Where x represents the xth radar that uses the same carrier frequency as the ith radar, n i (t) represents Gaussian white noise, γ represents the antenna reception coefficient, is the mth MIMO radar on the i-th node t The signal from the mth transmitting antenna is reflected to the target and r The time delay of receiving antennas, γ xi is the receiving coefficient of the th radar receiving the transmitted signal of the xth radar (x≠i), is the mth radar t The signal of the mth transmitting antenna is reflected by the mirror on the target and is the mth radar. r The time delay of receiving by each receiving antenna;

[0040] Part is the target echo of the i-th radar receiving its own transmission signal, satisfying γ i=Gρ / (R i ) 4 as well as Where m = m t m r , c is the speed of light, R i is the distance between the target and the th radar, ρ is the target reflection coefficient, G represents the radar antenna gain, θ i is the azimuth of the target relative to the th radar, v i is the relative speed between the target and the i-th radar; Part of it is the interference signal of other radars using the same sub-frequency band to the first radar, satisfying γ xi =Gρ / (R xi ) 4 as well as Where R xi is the relative distance between the xth radar and the ith radar, v xi is the speed of the x-th radar relative to the ith radar, θ xi is the azimuth of the x-th radar relative to the ith radar;

[0041] Perform one-dimensional constant false alarm detection on the difference frequency signal along the frequency axis, and generate a detection threshold by calculating the background noise and interference noise adaptively. Compare the detection unit amplitude with the detection threshold to get the result of whether the target is detected; detect each detection unit that detects the target. If the detection unit is adjacent to another detection unit that detects the target, it is considered to be the same target, otherwise it is another target. Finally, the number of targets Y is obtained. i，t .

[0042] Furthermore, in the beam direction Markov decision process, given the state space The state observed by agent i in time slot t is defined as in represents the relative distance between agent i and the target, Y i，t represents the number of targets detected by CFAR; the beam direction action determined by agent i in time slot t is defined as The decision is immediately executed in time slot t, and the radar transmits a signal with a new beam direction w i，t+1 , the beam direction remains unchanged until is executed in time slot t+1; the immediate reward is defined as:

[0043]

[0044] Two fully connected neural networks are used to approximate the beam direction policy network π(s; θ) and the beam direction value network V(s; ω). The input of both networks is the state observation of the agent. The output of the beam direction value network is the state The corresponding value The output of the beam direction strategy network is the state The probability corresponding to each action; define the policy network parameters and value network parameters is an intelligent agent, and its parameters need to be iteratively optimized; in time slot t, the beam direction decision selects the action The strategy is expressed as: Indicates that in state s i，t Next, according to the policy network parameters Select an action with a certain probability among the output options The output layer of the beam direction strategy network uses a softmax activation function, which satisfies:

[0045]

[0046] Based on the accumulated rewards And the output of the value network, calculate the first time difference error:

[0047]

[0048] The beam direction value network updates the parameters of the beam direction value network with the goal of minimizing the mean square error of the time difference, and the loss function is defined as:

[0049]

[0050] Where β is the loss coefficient; then the update formula of the beam direction value network parameters is:

[0051]

[0052] Where α is the weight factor. Under the guidance of the first time difference error, the beam direction strategy network follows the loss function The gradient direction of the network parameters is updated, and its loss function is defined as:

[0053]

[0054] The update formula of the beam direction strategy network parameters is:

[0055]

[0056] Furthermore, in the spectrum configuration Markov decision process, the state space and action space are respectively and The state observed by agent j in time slot t is defined as in The action performed by the upper layer for beam direction decision, fj，t It indicates the spectrum occupancy status at the current time slot t observed by the radar through the signal receiving device. represents the relative noise level estimate; the spectrum decision action determined by agent j is in and α j，t They represent the sub-band and frequency modulation slope selected by agent j in time slot t respectively; each agent j takes action according to its selection and To transmit a signal, get a received signal;

[0057] Perform fast Fourier transform on the received signal to obtain its spectrum R at the current time slot t j，t (f), using the sampling interval Δf to calculate R j，t (f) Sampling is performed, and the sampling interval Δf is determined by the sampling rate and the number of FFT points to obtain the sequence Among them, M f is the number of FFT points;

[0058] Pair Sequence Sort in ascending order to get a new sequence

[0059]

[0060] By discarding the largest N t samples, eliminating the target echo and obtaining an estimate of the noise level:

[0061]

[0062] Where N t is the number of discards, In the formula, l max is the length of the largest target, n ma x is the maximum number of targets; the relative noise level estimate is given by the noise level estimate Relative noise level with no interference The division of is obtained, which is expressed as:

[0063]

[0064] Using the spectrum of the received signal to get an estimate of the relative noise level, the instantaneous return is expressed as:

[0065]

[0066] Where η0 is the set threshold;

[0067] Define policy network parameters and value network parameters In time slot t, agent j observes that the upper-level decision is executing action Then choose actions according to the strategy Specifically:

[0068]

[0069] The spectrum configuration decision action is made in each small time slot Δt, and the reward is obtained in the current time slot after the signal transmission action is completed.

[0070] Calculate the second time difference error:

[0071]

[0072] The spectrum configuration value network updates the value network parameters with the goal of minimizing the mean square error of the time difference The loss function in each small slot is:

[0073]

[0074] Spectrum Configuration Value Network Parameters Update according to the loss function, According to the loss function of the value network, the loss function of the strategy network is obtained After that, the policy network parameters Update according to the formula:

[0075]

[0076] Compared with the prior art, the present invention has the following beneficial effects:

[0077] The present invention proposes a radar adaptive spectrum resource configuration method based on target clustering. At the beginning of each time slot, each radar in the radar system clusters the detected targets according to the detected information; the radar refers to the classification result and its own observation, selects the corresponding beam direction, and maintains the beam direction in the next few time slots. Then, each radar selects the frequency modulation slope and sub-band transmission signal, calculates the return according to the received signal, and updates the strategy. In order to obtain the optimal strategy for adaptive spectrum resource configuration of the radar system, it is modeled as a semi-Markov decision process. A radar adaptive spectrum resource configuration and target clustering algorithm based on hierarchical reinforcement learning is provided, which uses the radar adaptive spectrum resource configuration based on the actor-judge algorithm and the target clustering based on the convolutional neural network under the hierarchical decision framework. The present invention successfully solves the complex decision-making problem of radar spectrum resources when facing dynamic multiple targets in a complex electromagnetic environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1This is a flow chart of the radar adaptive spectrum resource configuration method based on target clustering of the present invention.

[0079] Figure 2 It is a scene schematic diagram of the present invention.

[0080] Figure 3 The present invention is a specific operation flow chart of the radar adaptive spectrum resource configuration method based on target clustering. DETAILED DESCRIPTION

[0081] The embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0082] In the scenario studied, Figure 2 As shown, there are a total of I radars and K moving targets, and the goal of the radars is to reduce mutual interference and accurately detect target information. Among them, each radar is a millimeter wave multiple-input multiple-output (MIMO) radar.

[0083] Assume that the entire spectrum is equally divided into Y non-overlapping sub-bands. At the same time, the radar selects a sub-band in each time slot. To simulate the actual crowded frequency band scenario, Y is slightly greater than I, that is, the number of sub-bands is slightly greater than the number of radars.

[0084] Each radar i uses M MIMO radars. t transmit antennas and M r The MIMO radar with 100 receiving antennas adopts a uniform linear array arrangement, and the interval between adjacent transmitting / receiving antennas is d=λ / 2, where λ is the wavelength.

[0085] Then the mth MIMO radar t The signal transmitted by the transmitting antenna is:

[0086]

[0087] Where B represents the signal bandwidth, represents the linear frequency modulation interval of the ith radar, represents the linear frequency modulation interval of the ith radar, Indicates the carrier frequency used by the i-th radar.

[0088] In order to enable the radar to detect targets more accurately, this paper uses beamforming technology to concentrate the main lobe energy of the MIMO radar transmission signal. Therefore, according to the Discrete Fourier Transform (DFT) codebook, for:

[0089]

[0090] Where P represents the total number of codewords in the DFT codebook, and p = 0, 1, ..., P-1. t Indicates the total number of transmitting antennas for each radar.

[0091] At this time, the mth MIMO radar on the i-th radar after beamforming t The signal transmitted by the transmitting antenna is:

[0092]

[0093] Determine whether the target is within the main lobe range of the radar's transmitted signal. If the target is within the main lobe range, it is considered that there is a target within the beam range of the radar's transmitted signal, and then the transmitted signal is mirror-reflected on the target and forms an echo signal; otherwise, it is considered that the radar has not detected the target. At the same time, since the distance between the radar and the target is much greater than the distance between the radars, when the radar's beam hits the target, it is approximately mirror-reflected on the target, thereby interfering with other radars in the same sub-band.

[0094] Therefore, when the i-th airborne MIMO radar detects the target, the signal received by its M-th receiving antenna is:

[0095]

[0096] Where x represents the xth radar that uses the same carrier frequency as the ith radar, n i (t) represents Gaussian white noise, and γ represents the antenna reception coefficient, such as antenna gain, path loss and target reflection coefficient. is the mth MIMO radar on the ,th node t The signal from the mth transmitting antenna is reflected to the target and r The time delay of receiving antennas. xi is the receiving coefficient of the th radar receiving the transmitted signal of the xth radar (x≠i), is the mth radar t The signal of the transmitting antenna is reflected by the mirror on the target and is the mth radar of the ith radar. r The time delay of receiving at each receiving antenna.

[0097] In the above formula, Part is the target echo of the i-th radar receiving its own transmission signal, satisfying γ i =Gρ / (R i ) 4 as well as Where m = m t m r , c is the speed of light, R iis the distance between the target and the th radar, ρ is the target reflection coefficient, G represents the radar antenna gain, θ i is the azimuth of the target relative to the i-th radar, v i is the relative speed between the target and the first radar. In the time delay, we can see that the first half of the delay is caused by the relative distance and relative speed between the target, while the second half is caused by the wave path difference caused by the distance between the antennas.

[0098] and Part of it is the interference signal of other radars using the same sub-band to the i-th radar, satisfying γ xi =Gρ / (R xi ) 4 as well as Where R xi is the relative distance between the xth radar and the ith radar, v xi is the speed of the x-th radar relative to the ith radar, θ xi is the azimuth of the xth radar relative to the th radar.

[0099] The energy of the target echo signal in the difference frequency signal is concentrated on several specific harmonics, while the swept frequency interference appears as a superposition of a series of linear frequency modulation signals. Therefore, one-dimensional constant false alarm detection can be performed on the difference frequency signal along the frequency axis, and the detection threshold is adaptively generated by calculating the background noise and interference noise, thereby reducing interference.

[0100] In mean-based CFAR, the detection threshold is estimated using the sample mean in the reference window on both sides of the detection unit. If the detection unit amplitude exceeds the threshold, the target is detected, otherwise, the target is not detected. The detection threshold calculation first requires calculating the estimated mean Z of the reference window:

[0101]

[0102] where s i is the sample value of the signal to be measured in the ,th reference window, N CFAR is the number of sample points in the reference window. Next, we need to calculate the false alarm probability P FA Calculate the threshold factor B:

[0103]

[0104] The detection threshold T = BZ is obtained based on the estimated mean Z of the reference window and the threshold factor B. The detection unit amplitude is then compared with the detection threshold T to obtain the result of whether the target is detected. At the same time, in order to effectively detect the number of targets, each detection unit that detects a target is further tested. If the detection unit is adjacent to another detection unit that detects a target, it is considered to be the same target, otherwise it is another target. Finally, the number of targets Y is obtained. i,t .

[0105] The adaptive spectrum resource configuration problem of the radar system is transformed into a top-down two-layer hierarchical decision. The two decision tasks are beam direction decision and spectrum configuration decision. The beam direction decision is the upper-level decision of the semi-Markov decision process across multiple time steps, and the spectrum decision is the lower-level decision of the semi-Markov decision process that performs basic actions. The specific modeling method of the semi-Markov decision process is given below. In the beam direction decision, given the state space The state observed by agent i in time slot t is defined as in represents the relative distance between agent i and the target, Y i，t represents the number of targets detected by CFAR; the beam direction action determined by agent i in time slot t is defined as The decision is immediately executed in time slot t, and the radar transmits a signal with a new beam direction w i，t+1 , the beam direction remains unchanged until is executed in time slot t+1; the immediate reward is defined as:

[0106]

[0107] In the spectrum decision Markov decision process, the state space and action space are and The state observed by agent j in time slot t is defined as in The action performed by the upper layer for beam direction decision, f j，t It indicates the spectrum occupancy status at the current time slot t observed by the radar through the signal receiving device. represents the relative noise level estimate; the spectrum decision action determined by agent j is in and α j，t They represent the sub-band and frequency modulation slope selected by agent j in time slot t respectively; each agent j takes action according to its selection and The received signal is transmitted and the received signal is obtained. The received signal is subjected to a fast Fourier transform to obtain its spectrum R at the current time slot t. j，t(f), using the sampling interval Δf to calculate R j，t (f) Sampling is performed, and the sampling interval Δf is determined by the sampling rate and the number of FFT points to obtain the sequence Among them, M f is the number of FFT points. Then sort the sequence in ascending order to get a new sequence

[0108]

[0109] Since the power of the target echo is mainly concentrated in the spectrum R j，t (f). Therefore, by discarding the largest N t samples, the target echo can be eliminated and an estimate of the noise level can be obtained:

[0110]

[0111] Where N t is the number of discards. N t The value of can be chosen approximately as:

[0112]

[0113] Among them l max is the length of the largest target, n max is the maximum number of targets. The discard value pairs other than the target echo signal are The estimate of has little effect, because the estimate of the noise level is the average of all values, and M f >>N t Therefore, N t The value of can be set larger than the above formula to ensure that all peaks corresponding to the target echo are discarded. Then the estimated value of the relative noise level can be obtained by the estimated value of the noise level Relative noise level with no interference The division of can be expressed as:

[0114]

[0115] Therefore, the relative noise level can be estimated by using the spectrum of the received signal, so as to judge the degree of interference to the radar. The instantaneous return can be expressed as:

[0116]

[0117] Where η0 is the set threshold. Applying the hierarchical structure to this problem not only greatly reduces the complexity of each sub-problem, but also improves the overall learning efficiency through lower-level strategies.

[0118] Therefore, the optimization problem for the above semi-Markov decision process can be expressed as:

[0119] and Where N Δt It is a multiple of the time slot length between the beam direction decision and the spectrum configuration decision.

[0120] In order to solve the problem of radar adaptive resource configuration based on target clustering, a target clustering method based on convolutional neural network and a radar adaptive spectrum resource configuration method based on actor-critic algorithm were proposed and embedded in a hierarchical decision framework to form a radar adaptive spectrum resource configuration method based on target clustering.

[0121] Radar adaptive spectrum resource configuration method based on target clustering:

[0122] Target clustering, as one of the observations of the upper-level radar beam direction decision, is of great significance to radar target detection. Therefore, it is necessary to pre-train the graph convolutional neural network first, and then embed the trained graph convolutional neural network into hierarchical reinforcement learning to perform target clustering for beam direction decision. To achieve target clustering using a graph convolutional neural network, it is first necessary to construct an adjacency matrix A and a feature matrix x, where the adjacency matrix A consists of the communication relationship between radar i and target k. If the target is detected, the corresponding position A(k, i) is set to 1, otherwise it is set to 0. The feature matrix X consists of key information that the radar itself can detect, including the radar's position, beam direction, and the distance of the detected target. After obtaining the above matrix, graph convolution is performed in the convolutional neural network, as follows:

[0123]

[0124] in I N is the identity matrix. yes The degree matrix of H. l is the feature matrix of each layer, and is the feature matrix for the input layer. σ(·) is the softmax function. W l Represents the weight matrix of the lth layer of the convolutional neural network; the target clustering result Re is obtained at the output layer i After that, we need to calculate the cross entropy loss function:

[0125]

[0126] Where T i is the label matrix using one-hot encoding. After pre-training, W l Target clustering will be directly applied to beam direction decision, and the clustering result will be used as one of the observations to help search for targets.

[0127] Hierarchical reinforcement learning solves complex problems by optimizing the system structure, that is, the behavior strategies of agents at each layer. Therefore, we propose a hierarchical actor-critic algorithm based on hierarchical reinforcement learning combined with target clustering to solve the above problems.

[0128] The actor-critic method usually has good convergence. It only uses a small step size to update the policy in the direction of the policy gradient, which means that changes in the value function will only lead to small changes in the policy, thereby ensuring that there is little or no oscillation in the policy. Two fully connected neural networks are used to approximate the policy function π(s; θ) and the state value function V(s; ω). First, for the beam direction decision, the input of both networks is the state observation of the agent. The output of the beam direction value network is the state The corresponding value The output of the beam direction strategy network is the state The probability corresponding to each action; define the policy network parameters and value network parameters is the parameter that needs to be iteratively optimized for agent i; in time slot t, the beam direction decision selects the action The strategy is expressed as:

[0129]

[0130] Indicates that in state s i，t Next, according to the policy network parameters Select an action with a certain probability among the output options The output layer of the beam direction strategy network uses a softmax activation function, which satisfies:

[0131]

[0132] The program continues in N Δt At the end of a small time slot, only one time slot has passed for a large time slot. According to the accumulated rewards obtained And the output of the value network, calculate the first time difference error:

[0133]

[0134] The value network updates the value network parameters with the goal of minimizing the mean square error of the time difference, and defines the loss function as:

[0135]

[0136] Where β is the loss coefficient. Then the update formula of the value network parameters is:

[0137]

[0138] Under the guidance of the time difference error, the policy function network follows the loss function J(θ i ) updates the network parameters in the gradient direction. The loss function is defined as:

[0139]

[0140] The update formula of the policy network parameters is:

[0141]

[0142] Similarly, in the spectrum configuration decision plan, define the strategic network parameters and value network parameters In time slot t, agent j observes that the upper-level decision is executing action Then choose actions according to the strategy Specifically:

[0143]

[0144] Different from the beam direction decision in multiple time slots, the spectrum configuration decision action needs to be made within each small time slot Δt, and the reward is obtained in the current time slot after the signal transmission action is completed. Then calculate the second time difference error:

[0145]

[0146] The value network also updates the value network parameters with the goal of minimizing the mean square error of the time difference. The loss function in each small slot is:

[0147]

[0148] Spectrum Configuration Value Network Parameters Update according to the loss function, According to the loss function of the value network, the loss function of the strategy network is obtained After that, the policy network parameters Update according to the formula,

[0149]

[0150] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0151] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0154] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0155] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A radar adaptive spectrum resource configuration method based on target clustering, characterized in that: The radar adaptive spectrum resource configuration method comprises the following steps: Each radar clusters the detected targets according to the detected information, and selects the corresponding beam direction using the beam direction strategy network based on the current beam direction, the number of detected targets, the measured target distance and the target clustering result information, and maintains the beam direction in the next few time slots; then, each radar selects the frequency modulation slope and sub-band transmission signal of each time slot based on the selected beam direction, spectrum occupancy and relative noise level estimation value using the spectrum configuration strategy network; The return is calculated according to the received signal, and the outputs of the beam direction value network and the spectrum configuration value network are combined to update the beam direction strategy network parameters and the beam direction value network parameters and the spectrum configuration strategy network parameters and the spectrum configuration value network parameters.

2. The radar adaptive spectrum resource configuration method based on target clustering according to claim 1 is characterized in that: The radar adaptive spectrum resource configuration method based on target clustering comprises the following steps: Step 1: Obtain the radar position at the current time slot t, the information detected by each radar at the previous time slot t-1, and the selected beam direction to form a feature matrix; Step 2: The communication relationship between the radar and the target is used to form an adjacency matrix, and the feature matrix and the adjacency matrix are input into the convolutional neural network to infer the target clustering result; Step 3: Obtain the radar beam direction, the number of detected targets, the measured target distance and the target clustering result information at the current time slot t, and use it as the state of the current radar beam direction decision; Step 4: Use the beam direction strategy network to select the beam direction action, and select a direction from the preset beam direction set as the signal beam transmission direction for the next few time slots according to the output action selection probability; Step 5: Obtain the beam direction, spectrum occupancy and relative noise level estimation value of the radar at the current time slot t, and use it as the status of the current radar spectrum configuration decision; Step 6: Use the spectrum configuration strategy network to select the frequency modulation slope and sub-band action, and select the frequency modulation slope and sub-band from the preset action collection as the signal transmission parameters for the next time slot t+1 according to the output action selection probability; Step 7: When the radar transmits a signal in the next time slot t+1 and receives the signal, the first time difference error is calculated according to the output of the spectrum configuration value network and the reward value of the corresponding spectrum configuration decision, and the spectrum configuration strategy network and the spectrum configuration value network are trained according to the first time difference error; Step 8: After NΔt time slots, the second time difference error is calculated according to the output of the beam direction value network and the reward value of the corresponding beam direction decision, and the beam direction strategy network and the beam direction value network are trained according to the second time difference error; Step 9: At each N Δt Repeat steps 1 to 8 for each time slot until the stop condition is reached, at which point the action selection strategy converges.

3. The radar adaptive spectrum resource configuration method based on target clustering according to claim 2 is characterized in that: Step 2 further includes: Constructing the feature matrix Where P t represents the radar position coordinate set in the radar system, w t represents the set of directions of each radar beam, represents a collection of range measurements for each radar target; Construct an adjacency matrix A, which consists of the communication relationship between radar i and target k. If the target is detected, the corresponding position A(k, i) is set to 1, otherwise it is set to 0; Graph convolution is performed in a convolutional neural network as follows: in I N is the identity matrix, yes The degree matrix, H l is the feature matrix of each layer; σ(·) is the softmax function; W l Represents the weight matrix of the lth layer of the convolutional neural network; the target clustering result Re is obtained at the output layer i Afterwards, the cross entropy loss function is calculated: Where T i is the label matrix using one-hot encoding; W after pre-training l Target clustering is directly applied to beam direction decision, and the clustering result is used as one of the observations to help search for targets.

4. The radar adaptive spectrum resource configuration method based on target clustering according to claim 2 is characterized in that: Step 2 further includes: The entire spectrum is divided into Y non-overlapping sub-bands. The radar selects one sub-band in each time slot. The number of sub-bands is greater than the number of radars. The MIMO radars used by each radar i are M t transmit antennas and M r The MIMO radar with 100 receiving antennas is arranged in a uniform linear array, and the interval between adjacent transmitting / receiving antennas is d=λ / 2, where λ is the wavelength. The mth t The signal transmitted by the transmitting antenna is: Where B represents the signal bandwidth, represents the linear frequency modulation interval of the ith radar, represents the carrier frequency used by the i-th radar; Beamforming technology is used to concentrate the main lobe energy of the MIMO radar transmission signal. The codebook used for: Where P represents the total number of codewords in the DFT codebook, and p = 0, 1, ..., P-1; M t Indicates the total number of transmitting antennas of each radar; The mth MIMO radar on the i-th radar after beamforming t The signal transmitted by the transmitting antenna is: Determine whether the target is within the main lobe range of the radar transmission signal. If the target is within the main lobe range, it is considered that there is a target within the beam range of the radar transmission signal, and the transmission signal is mirror-reflected on the target and forms an echo signal; otherwise, it is considered that the radar has not detected the target; When the th airborne MIMO radar detects the target, the signal received by its Mth receiving antenna is: Where x represents the xth radar that uses the same carrier frequency as the ith radar, n i (t) represents Gaussian white noise, γ represents the antenna reception coefficient, is the mth MIMO radar on the i-th node t The signal from the mth transmitting antenna is reflected to the target and r The time delay of receiving antennas, γ xi is the reception coefficient of the ith radar receiving the transmission signal of the xth radar (x≠i), is the mth radar t The signal of the transmitting antenna is reflected by the mirror on the target and is the mth radar of the ith radar. r The time delay of receiving by each receiving antenna; Part is the target echo of the i-th radar receiving its own transmission signal, satisfying γ i =Gρ / (R i ) 4 as well as Where m = m t m r , c is the speed of light, R i is the distance between the target and the i-th radar, ρ is the target reflection coefficient, G represents the radar antenna gain, θ i is the azimuth of the target relative to the i-th radar, v i is the relative speed between the target and the i-th radar; Part of it is the interference signal of other radars using the same sub-band to the i-th radar, satisfying γ xi =Gρ / (R xi ) 4 as well as Where R xi is the relative distance between the xth radar and the ith radar, v xi is the speed of the x-th radar relative to the ith radar, θ xi is the azimuth of the x-th radar relative to the ith radar; Perform one-dimensional constant false alarm detection on the difference frequency signal along the frequency axis, and generate a detection threshold by calculating the background noise and interference noise adaptively. Compare the detection unit amplitude with the detection threshold to get the result of whether the target is detected; detect each detection unit that detects the target. If the detection unit is adjacent to another detection unit that detects the target, it is considered to be the same target, otherwise it is another target. Finally, the number of targets Y is obtained. i，t .

5. The radar adaptive spectrum resource configuration method based on target clustering according to claim 2, characterized in that: In the beam direction Markov decision process, given the state space The state observed by agent i in time slot t is defined as in represents the relative distance observed by agent i to the target, Yi,t represents the number of targets detected by CFAR; the beam direction action determined by agent i in time slot t is defined as The decision is immediately executed in time slot t, and the radar transmits a signal with a new beam direction w i，t+1 , the beam direction remains unchanged until is executed in time slot t+1; the immediate reward is defined as: Two fully connected neural networks are used to approximate the beam direction policy network π(s; θ) and the beam direction value network V(s; ω). The input of both networks is the state observation of the agent. The output of the beam direction value network is the state The corresponding value The output of the beam direction strategy network is the state The probability corresponding to each action; define the policy network parameters and value network parameters is an intelligent agent, and its parameters need to be iteratively optimized; in time slot t, the beam direction decision selects the action The strategy is expressed as: Indicates that in state si, t, according to the policy network parameters Select an action with a certain probability among the output options The output layer of the beam direction strategy network uses a softmax activation function, which satisfies: Based on the accumulated rewards And the output of the value network, calculate the first time difference error: The beam direction value network updates the parameters of the beam direction value network with the goal of minimizing the mean square error of the time difference, and the loss function is defined as: Where β is the loss coefficient; then the update formula of the beam direction value network parameters is: Where α is the weight factor. Under the guidance of the first time difference error, the beam direction strategy network follows the loss function The gradient direction of the network parameters is updated, and its loss function is defined as: The update formula of the beam direction strategy network parameters is:

6. The radar adaptive spectrum resource configuration method based on target clustering according to claim 2 is characterized in that: In the spectrum configuration Markov decision process, the state space and action space are and The state observed by agent j in time slot t is defined as in The action performed by the upper layer for beam direction decision, f j，t It indicates the spectrum occupancy status at the current time slot t observed by the radar through the signal receiving device. represents the relative noise level estimate; the spectrum decision action determined by agent j is in and α j，t They represent the sub-band and frequency modulation slope selected by agent j in time slot t respectively; each agent j takes action according to its selection and To transmit a signal, get a received signal; Perform fast Fourier transform on the received signal to obtain its spectrum R at the current time slot t j，t (f), using the sampling interval Δf to calculate R j，t (f) Sampling is performed, and the sampling interval Δf is determined by the sampling rate and the number of FFT points to obtain the sequence Among them, M f is the number of FFT points; Pair Sequence Sort in ascending order to get a new sequence By discarding the largest N t samples, eliminating the target echo and obtaining an estimate of the noise level: Where N t is the number of discards, In the formula, l max is the length of the largest target, n max For the most Large number of targets; The relative noise level estimate is given by the noise level estimate Relative noise level with no interference The division of is obtained, which is expressed as: Using the spectrum of the received signal to get an estimate of the relative noise level, the instantaneous return is expressed as: Where η0 is the set threshold; Define policy network parameters and value network parameters In time slot t, the agent observes that the upper-level decision is executing actions Then choose actions according to the strategy Specifically: The spectrum configuration decision action is made in each small time slot Δt, and the reward is obtained in the current time slot after the signal transmission action is completed. Calculate the second time difference error: The spectrum configuration value network updates the value network parameters with the goal of minimizing the mean square error of the time difference The loss function in each small slot is: Spectrum Configuration Value Network Parameters Update according to the loss function, According to the loss function of the value network, the loss function of the strategy network is obtained After that, the policy network parameters Update according to the formula:

Images