A radar adaptive spectrum resource configuration method based on target clustering

Abstract

The application provides a radar adaptive spectrum resource configuration method based on target clustering, which comprises the following steps: at the beginning of each time slot, each radar in a radar system clusters the detected targets according to the detected information; the radar selects the corresponding beam direction according to the reference classification result and the observation of the radar, and maintains the beam direction in the next several time slots. Then, each radar selects a frequency modulation slope and a sub-band to transmit a signal, and calculates a return according to a received signal and updates a strategy. Through the interaction with the information of the environment, the radar system learns the optimal strategy of the adaptive spectrum resource configuration, so as to solve the radar spectrum resource complex decision problem when facing dynamic multi-targets in a complex electromagnetic environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wireless communication, in particular to a radar adaptive spectrum resource configuration method based on target clustering, which is especially suitable for multi-target detection task scenarios in complex electromagnetic environments. BACKGROUND

[0002] With the wide application of radars and the rapid growth of the number of radars, the spectrum in which the radars are located is continuously occupied, and the anti-interference ability of the radars is continuously tested. When performing a task, a radar system often faces a complex and uncertain electromagnetic spectrum environment, which requires the radar to distinguish effective information from a large amount of collected information, so as to effectively make a decision on the next action. In our research, each radar must allocate a sub-band, a frequency modulation slope and a beam direction in a crowded spectrum to avoid mutual interference and successfully detect targets, which is a sequential decision problem. Generally, a Markov decision process is used to model the above decision problem. However, if the problem is modeled as a Markov decision process, it will be difficult to find a global optimal solution to the optimization problem and converge to a suboptimal solution, and the number of action spaces will also become very complex. Even the algorithm cannot converge. In addition, due to the interference between radars, it is not possible to effectively determine whether a target is real in a time slot. Therefore, multiple time slots are needed to observe and determine, which also leads to a complex decision problem with different time scales. SUMMARY

[0003] In view of the radar system spectrum resource allocation problem in a crowded spectrum environment when facing dynamic multiple targets, the present application proposes a radar adaptive spectrum resource configuration method based on target clustering, which learns the optimal strategy of adaptive spectrum resource configuration through information interaction with the environment, so as to solve the complex decision problem of radar spectrum resources when facing dynamic multiple targets in a complex electromagnetic environment.

[0004] To achieve the above technical purposes, the technical scheme adopted by the present application is as follows:

[0005] A radar adaptive spectrum resource configuration method based on target clustering, the radar adaptive spectrum resource configuration method comprising the following steps:

[0006] Each radar clusters the detected targets according to the detected information, combines the current beam direction, the number of detected targets, the measured target distance and the target clustering result information, selects a corresponding beam direction using a beam direction strategy network, and maintains the beam direction in the next several time slots; then, each radar combines the selected beam direction, the spectrum occupation situation and the relative noise level estimate value, and selects a frequency modulation slope and a sub-band transmission signal for each time slot using a spectrum configuration strategy network.

[0007] According to the received signal, the return is calculated, and the output of the beam direction value network and the spectrum configuration value network is combined respectively to update the beam direction strategy network parameters and the beam direction value network parameters and the spectrum configuration strategy network parameters and the spectrum configuration value network parameters.

[0008] Further, the target clustering-based radar adaptive spectrum resource configuration method comprises the following steps:

[0009] Step 1: Obtain the radar positions of the current time slot t, the information detected by each radar in the last time slot t-1, and the selected beam direction, to form a feature matrix;

[0010] Step 2: Form an adjacency matrix from the communication relationship between the radar and the target, and input the feature matrix and the adjacency matrix into a convolutional neural network to infer the target clustering result;

[0011] Step 3: Obtain the beam direction of the current time slot t radar, the number of detected targets, the measured target distance, and the target clustering result information, which are used as the state of the current radar beam direction decision;

[0012] Step 4: Perform beam direction action selection using the beam direction strategy network, and select a direction from a preset beam direction set as the signal beam emission direction for the next few time slots according to the output action selection probability;

[0013] Step 5: Obtain the beam direction of the current time slot t radar, the spectrum occupation situation, and the relative noise level estimate value, which are used as the state of the current radar spectrum configuration decision;

[0014] Step 6: Perform frequency slope and sub-band action selection using the spectrum configuration strategy network, and select a frequency slope and sub-band from a preset action set as the signal emission parameters for the next time slot t+1 according to the output action selection probability;

[0015] Step 7: When the radar transmits a signal in the next time slot t+1, and after receiving the signal, the first time difference error is calculated according to the output of the spectrum configuration value network and the return value of the spectrum configuration decision, and the spectrum configuration strategy network and the spectrum configuration value network are trained according to the first time difference error;

[0016] Step 8: After N Δt time slots, the second time difference error is calculated according to the output of the beam direction value network and the return value of the beam direction decision, and the beam direction strategy network and the beam direction value network are trained according to the second time difference error;

[0017] Step 9: Repeat the above steps 1 to 8 every N Δt time slots until the stopping condition is reached, at which time the action selection strategy converges.

[0018] Step 2 further includes:

[0019] Constructing the feature matrix Where P t w represents the set of radar position coordinates in a radar system. t This represents the set of directions for each radar beam. This represents the set of range measurements for each radar target.

[0020] Construct an adjacency matrix A, which consists of the communication relationships between radar i and target k. If the target is detected, the corresponding position A(k,i) is set to 1, otherwise it is set to 0.

[0021] Graph convolution is performed in a convolutional neural network as follows:

[0022]

[0023] in I N It is the identity matrix. yes The degree matrix, H l The feature matrix for each layer; σ(·) is the softmax function; W l This represents the weight matrix of the l-th layer of the convolutional neural network; the target clustering result Re is obtained in the output layer. i Next, calculate the cross-entropy loss function:

[0024]

[0025] Where T i The label matrix uses one-hot encoding; W after pre-training. l Target clustering, which is directly applied to beam direction decision-making, uses the clustering results as one of the observations to help search for targets.

[0026] Step 2 further includes:

[0027] The entire spectrum is divided into Y non-overlapping sub-bands. Each radar selects one sub-band for each time slot, and the number of sub-bands is greater than the number of radars. Each radar i uses an M MIMO radar. t One transmitting antenna and M r The MIMO radar with one receiving antenna adopts a uniform linear array arrangement, and the spacing between adjacent transmitting / receiving antennas is d = λ / 2, where λ is the wavelength;

[0028] The m-th MIMO radar of the i-th MIMO radar t The transmitting antennas transmit signals as follows:

[0029]

[0030] Where B represents the signal bandwidth. This represents the linear frequency modulation interval of the i-th radar. This represents the carrier frequency used by the i-th radar;

[0031] Beamforming technology is used to concentrate the main lobe energy of the MIMO radar transmitted signal, and a codebook is employed. for:

[0032]

[0033] Where P represents the total number of codewords in the DFT codebook, and p = 0, 1, ..., P-1; M t This indicates the total number of each radar transmitting antenna;

[0034] The m-th MIMO radar on the i-th radar after beamforming t The transmitting antennas transmit signals as follows:

[0035]

[0036] The system determines whether the target is within the main lobe of the radar signal. If the target is within the main lobe, it is considered that the target exists within the radar signal beam, and the transmitted signal is specularly reflected on the target to form an echo signal; otherwise, it is considered that the radar has not detected the target.

[0037] When the i-th airborne MIMO radar detects the target, its m-th... r The signal received by each receiving antenna is:

[0038]

[0039] Where x represents the x-th radar that uses the same carrier frequency as the i-th radar, and n i (t) represents Gaussian white noise, and γ represents the antenna receiving coefficient. For the m-th MIMO radar on the i-th node t The signal from the m-th transmitting antenna is reflected to the target and is then received by the m-th antenna. r The time delay of reception by each receiving antenna, γ xi Let be the reception coefficient of the i-th radar receiving the transmitted signal from the x-th radar (x≠i). For the x-th radar, the m-th... t The signal from each transmitting antenna is mirror-reflected on the target and is used as the m-th signal of the i-th radar. r The time delay of reception by each receiving antenna;

[0040] Part of it is the target echo received by the i-th radar from its own transmitted signal, satisfying γ iGp / (R i ) 4 and where m = m t m r , c is the speed of light, R i is the distance between the target and the i-th radar, p is the reflection coefficient of the target, G represents the antenna gain of the radar, q i is the azimuth angle of the target relative to the i-th radar, v i is the relative speed of the target relative to the i-th radar; Part is the interference signal of other radars using the same sub-band to the i-th radar, satisfying y xi = Gp / (R xi ) 4 and where R xi is the relative distance between the x-th radar and the i-th radar, v xi is the speed of the x-th radar relative to the i-th radar, q xi is the azimuth angle of the x-th radar relative to the i-th radar;

[0041] A one-dimensional constant false alarm rate (CFAR) detection is performed on the difference frequency signal along the frequency axis. A detection threshold is adaptively generated by calculating the background noise and the interference noise. The detection threshold is compared with the amplitude of the detection unit to obtain the result of whether the target is detected. Each detection unit that detects the target is detected. If the detection unit is adjacent to another detection unit that detects the target, it is considered to be the same target, otherwise it is another target. Finally, the number of targets Y i,t is obtained.

[0042] Further, in the beam direction Markov decision process, given the state space The state observed by the agent i in the time slot t is defined as where represents the relative distance of the target observed by the agent i, Y i,t represents the number of targets detected by the CFAR; the beam direction action decided by the agent i in the time slot t is defined as The decision obtained is immediately executed in the time slot t, and the radar transmits a signal using a new beam direction w i,t+1 The beam direction remains unchanged until is executed in the time slot t+1; the immediate reward is defined as:

[0043]

[0044] Two fully connected neural networks are respectively used to approximate the beam direction policy network p(s; q) and the beam direction value network V(s; w). The input of the two networks is the state observation of the agent i The output of the beam direction value network is the state The corresponding value The output of the beam direction policy network is the state The probability of each action; define the policy network parameters And the value network parameters The parameters that need to be iteratively optimized for the agent i; in time slot t, the beam direction decision selects the action The policy is represented as: Indicates that in the state s i,t According to the policy network parameters Select the action with a certain probability among the output options The output layer of the beam direction policy network uses the softmax activation function, which satisfies:

[0045]

[0046] According to the obtained cumulative reward And the output of the value network, the first time difference error is calculated:

[0047]

[0048] The beam direction value network updates the beam direction value network parameters with the goal of minimizing the time difference mean square error, and the loss function is defined as:

[0049]

[0050] Where β is the loss coefficient; then the update formula of the beam direction value network parameters is:

[0051]

[0052] Where α is the weight factor. The beam direction policy network updates the network parameters in the gradient direction of the loss function Under the guidance of the first time difference error, and its loss function is defined as:

[0053]

[0054] The update formula of the beam direction policy network parameters is:

[0055]

[0056] Further, in the spectrum configuration Markov decision process, the state space and action space are And The state observed by the agent j in time slot t is defined as Where is the action performed by the upper beam direction decision, fj,t represents the spectrum occupancy state observed by the radar through the signal receiving device at the current time slot t, represents the relative noise level estimate; the spectrum decision action decided by the agent j is wherein and α j,t respectively represent the sub-band and the frequency modulation slope selected by the agent j at the time slot t; each agent j transmits a signal according to the selected action and to obtain a received signal;

[0057] a fast Fourier transform is performed on the received signal to obtain its spectrum R j,t (f) at the current time slot t, R j,t (f) is sampled using a sampling interval Δf, the sampling interval Δf being determined by a sampling rate and the number of FFT points, to obtain a sequence wherein M f is the number of FFT points;

[0058] the sequence is sorted in ascending order to obtain a new sequence

[0059]

[0060] by discarding the largest N t samples, the target echo is eliminated to obtain an estimate of the noise level:

[0061]

[0062] wherein N t is the number of discarded samples, in the formula, l max is the length of the largest target, and n max is the number of the largest targets; the estimate of the relative noise level is obtained by dividing the estimate of the noise level by the relative noise level without interference, and is represented as:

[0063] The estimate of the relative noise level is obtained using the spectrum of the received signal, and the instantaneous return is represented as:

[0064]

[0065]

[0066] wherein η0 is a set threshold value;

[0067] the policy network parameters and the value network parameters ​​In time slot t, the agent j observes the upper decision to execute action Then select action according to policy Specifically:

[0068]

[0069] The spectrum configuration decision action makes a decision in each hour slot Δt, and obtains the reward in the current time slot at the end of the execution of the transmit signal action

[0070] Calculate the second time difference error:

[0071]

[0072] The spectrum configuration value network updates the value network parameters with the minimum time difference mean square error as the goal The loss function in each hour slot is:

[0073]

[0074] Spectrum configuration value network parameters According to the loss function, After obtaining the policy network loss function according to the loss function of the value network Then, the policy network parameters Update according to the formula:

[0075]

[0076] Compared with the prior art, the present application has the following beneficial effects:

[0077] The present application provides a radar adaptive spectrum resource configuration method based on target clustering. At the beginning of each time slot, each radar in the radar system clusters the detected targets according to the detected information. The radar selects the corresponding beam direction according to the reference classification result and its own observation, and maintains the beam direction in the next several time slots. Then, each radar selects the frequency modulation slope and the sub-band transmit signal, and calculates the reward according to the received signal and updates the policy. In order to obtain the optimal policy of the radar system adaptive spectrum resource configuration, it is modeled as a semi-Markov decision process. A radar adaptive spectrum resource configuration and target clustering algorithm based on hierarchical reinforcement learning is provided, which uses the radar adaptive spectrum resource configuration based on the actor-critic algorithm under the hierarchical decision framework and the target clustering based on the convolutional neural network. The present application successfully solves the complex decision problem of radar spectrum resource in the face of dynamic multi-target in a complex electromagnetic environment. BRIEF DESCRIPTION OF DRAWINGS

[0078] Figure 1Flow chart of target clustering based radar adaptive spectrum resource configuration method of the present application.

[0079] Figure 2 Scenario schematic diagram of the present application.

[0080] Figure 3 Specific operation flow chart of target clustering based radar adaptive spectrum resource configuration method of the present application. DETAILED DESCRIPTION

[0081] The embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0082] In the studied scenario, as shown in Figure 2 , there are I radars and K moving targets in total, and the goal of the radars is to reduce mutual interference and accurately detect target information. Among them, each radar is a millimeter wave multiple-input multiple-output (MIMO) radar.

[0083] It is assumed that the entire frequency spectrum is equally divided into Y non-overlapping sub-bands. At the same time, each radar selects a sub-band per time slot. To simulate the actual frequency band crowded scenario, Y is slightly larger than I, i.e., the number of sub-bands is slightly larger than the number of radars.

[0084] Each radar i uses a MIMO radar with M t transmit antennas and M r receive antennas, all arranged in a uniform linear array, with adjacent transmit / receive antennas spaced d = λ / 2, λ being the wavelength.

[0085] The signal transmitted by the m t th transmit antenna of the i th MIMO radar is:

[0086]

[0087] where B represents the signal bandwidth, represents the linear frequency modulation interval of the i th radar, represents the linear frequency modulation interval of the i th radar, represents the carrier frequency used by the i th radar.

[0088] In order for the radar to be able to more accurately detect targets, this paper uses beamforming technology to concentrate the main lobe energy of the MIMO radar transmit signal. Therefore, according to the Discrete Fourier Transform (DFT) codebook has :

[0089]

[0090] Where P represents the total number of codewords in the DFT codebook, and p = 0, 1, ..., P-1. M t This indicates the total number of each radar transmitting antenna.

[0091] At this point, the m-th MIMO radar on the i-th radar after beamforming... t The transmitting antennas transmit signals as follows:

[0092]

[0093] The system determines whether the target is within the main lobe of the radar's transmitted signal. If the target is within the main lobe, it is considered to exist within the radar's beam, and the transmitted signal is then specularly reflected off the target to form an echo signal. Otherwise, it is considered that the radar has not detected the target. Simultaneously, because the distance between the radar and the target is much greater than the distance between radars, when the radar beam hits the target, it is approximately specularly reflected off the target, thus interfering with other radars operating in the same sub-band.

[0094] Therefore, when the i-th airborne MIMO radar detects the target, its m-th... r The signal received by each receiving antenna is:

[0095]

[0096] Where x represents the x-th radar that uses the same carrier frequency as the i-th radar, and n i (t) represents Gaussian white noise, and γ represents the antenna receiving coefficient, which includes antenna gain, path loss and target reflection coefficient. For the m-th MIMO radar on the i-th node t The signal from the m-th transmitting antenna is reflected to the target and is then received by the m-th antenna. r The time delay of reception by each receiving antenna. And γ xi Let be the reception coefficient of the i-th radar receiving the transmitted signal from the x-th radar (x≠i). For the x-th radar, the m-th... t The signal from each transmitting antenna is mirror-reflected on the target and is used as the m-th signal of the i-th radar. r The time delay of reception by each receiving antenna.

[0097] In the formula above, Part of it is the target echo received by the i-th radar from its own transmitted signal, satisfying γ i =Gρ / (R) i ) 4 as well as Where m = m t m r c is the speed of light, R iwhere R is the distance between the target and the ith radar, p is the reflection coefficient of the target, G represents the antenna gain of the radar, and θ i is the azimuth angle of the target relative to the ith radar, and v i is the relative velocity of the target relative to the ith radar. The first half of the time delay is caused by the relative distance and the relative velocity between the target and the radar, and the second half is caused by the wave path difference caused by the distance between the antennas.

[0098] And is the interference signal of the ith radar from other radars using the same sub-band, and satisfies γ xi = Gp / (R xi ) 4 And where R xi is the relative distance between the xth radar and the ith radar, v xi is the velocity of the xth radar relative to the ith radar, θ xi is the azimuth angle of the xth radar relative to the ith radar.

[0099] The energy of the target echo signal in the difference frequency signal is concentrated on several specific harmonics, and the sweep jamming appears as the superposition of a series of linear frequency modulation signals, so a one-dimensional constant false alarm detection can be performed on the difference frequency signal along the frequency axis, and a detection threshold is adaptively generated by calculating the background noise and the jamming noise, thereby reducing the jamming.

[0100] In the mean CFAR, the detection threshold is estimated using the sample mean in the reference window on both sides of the detection unit. If the amplitude of the detection unit exceeds the threshold, it is determined that the target is detected, otherwise, it is determined that the target is not detected. The detection threshold calculation first needs to calculate the estimated mean Z of the reference window:

[0101]

[0102] where s i is the sample value of the signal to be measured in the ith reference window, N CFAR is the number of sample points in the reference window. Then, the threshold factor B needs to be calculated according to the false alarm probability P FA

[0103]

[0104] ​The detection threshold T = BZ is obtained according to the estimated mean value Z of the reference window and the threshold factor B, and then the detection unit amplitude is compared with the detection threshold T to obtain the result of whether the target is detected. Meanwhile, in order to effectively detect the number of targets, each detection unit of the detected target is further detected, if the detection unit is adjacent to another detection unit of the detected target, it is considered as the same target, otherwise it is another target, and finally the number of targets Y is obtained i,t .

[0105] The adaptive spectrum resource configuration problem of the radar system is converted into a top-down two-layer hierarchical decision. Two decision tasks are beam direction decision and spectrum configuration decision, wherein the beam direction decision is an upper layer decision of a semi-Markov decision process across multiple time steps, and the spectrum decision is a lower layer decision of a semi-Markov decision process for performing basic actions. The specific modeling method of the semi-Markov decision process is given below. In the beam direction decision, the state space is given as The state observed by the agent i in the time slot t is defined as wherein represents the relative distance of the target observed by the agent i, Y i,t represents the number of targets detected by the CFAR; the beam direction action decided by the agent i in the time slot t is defined as The decision is immediately executed in the time slot t, and the radar transmits a signal in a new beam direction w i,t+1 The beam direction remains unchanged until is executed in the time slot t+1; the immediate reward is defined as:

[0106]

[0107] In the spectrum decision Markov decision process, the state space and the action space are and The state observed by the agent j in the time slot t is defined as wherein is the action performed by the upper layer beam direction decision, f j,t represents the current time slot t spectrum occupation state observed by the radar through the signal receiving device, represents the relative noise level estimation value; the spectrum decision action decided by the agent j is wherein and α j,t respectively represent the sub-band and the frequency modulation slope selected by the agent j in the time slot t; each agent j transmits a signal according to the selected action and to obtain a received signal. The received signal is subjected to a fast Fourier transform to obtain its spectrum R j,t(f) R is sampled with sampling interval Δf j,t (f) R is sampled with sampling interval Δf where M f is the number of FFT points. Then the sequence is sorted in ascending order to get a new sequence

[0108]

[0109] Since the power of target echoes is mainly concentrated in the peaks of the spectrum R j,t (f), the target echoes can be eliminated by discarding the largest N t samples, and the estimate of noise level is obtained:

[0110]

[0111] where N t is the number of discarded samples. The value of N t may be approximately selected as:

[0112]

[0113] where l max is the length of the largest target, and n max is the number of the largest targets. The discarded values other than the target echo signals have little effect on the estimate of , because the estimate of noise level is the average of all values, and M f >> N t . Therefore, the value of N t may be set to be greater than the above formula to ensure that all peaks corresponding to target echoes are discarded. Then the estimate of relative noise level can be obtained by dividing the estimate of noise level by the relative noise level without interference , which can be expressed as:

[0114]

[0115] Therefore, the estimate of relative noise level can be obtained by using the spectrum of the received signal, so as to judge the degree of interference of the radar. Then the immediate reward can be expressed as:

[0116]

[0117] where η0 is a set threshold. Applying the hierarchical structure to this problem not only greatly reduces the complexity of each sub-problem, but also improves the overall learning efficiency through the strategy of lower levels.

[0118] Therefore, the optimization problem of the semi-Markov decision process described above can be expressed as:

[0119] and where N Δt is the multiple of the time slot length between the beam direction decision and the spectrum configuration decision.

[0120] In order to solve the target clustering-based radar adaptive resource allocation problem, a target clustering method based on a convolutional neural network and a radar adaptive spectrum resource allocation method based on an actor-critic algorithm are proposed, and are embedded in a hierarchical decision framework to form a target clustering-based radar adaptive spectrum resource allocation method.

[0121] The target clustering-based radar adaptive spectrum resource allocation method comprises the following steps:

[0122] Target clustering, as one of the observations of the upper layer radar beam direction decision, is of great significance to the radar detected target, and therefore it is necessary to first pre-train the graph convolutional neural network, and then embed the trained graph convolutional neural network into the hierarchical reinforcement learning to cluster the targets for the beam direction decision. In order to realize target clustering using the graph convolutional neural network, it is necessary to first construct an adjacency matrix A and a feature matrix X, wherein the adjacency matrix A is composed of the communication relationship between the radar i and the target k, and if the target is detected, the corresponding position A(k, i) is set to 1, otherwise it is set to 0. The feature matrix X is composed of the key information that can be detected by the radar itself, including the position of the radar, the beam direction and the detected target distance. After obtaining the above matrices, graph convolution is performed in the convolutional neural network, which is specifically as follows:

[0123]

[0124] wherein I N is the identity matrix. is the degree matrix of H l is the feature matrix of each layer, and for the input layer, it is the feature matrix. σ(·) is a softmax function. W l represents the weight matrix of the lth layer of the convolutional neural network; and the target clustering result Re i is obtained at the output layer.

[0125]

[0126] wherein T i is a label matrix using one-hot encoding. After the pre-training of W l , it will be directly applied to the target clustering of the beam direction decision, and the clustering result will be used as one of the observations to help searching for the target.

[0127] Hierarchical reinforcement learning optimizes the system structure, i.e. the behavior policy of each layer agent, to solve complex problems. Therefore, we propose a hierarchical actor-critic algorithm based on hierarchical reinforcement learning combined with target clustering to solve the above problems.

[0128] Actor-critic methods generally have good convergence, which only uses a small step to update the policy in the direction of the policy gradient, which means that the change of the value function will only cause a small change in the policy, thus ensuring that there is little or no oscillatory behavior in the policy, and two fully connected neural networks are used to approximate the policy function π(s; θ) and the state value function V(s; ω), respectively. First, for beam direction decision, the input of the two networks is the state observation of agent i The output of the beam direction value network is the state The corresponding value The output of the beam direction policy network is the state The corresponding probability of each action; define the policy network parameters and the value network parameters as the parameters that agent i needs to iteratively optimize; in time slot t, the beam direction decision selects action The policy is represented as:

[0129]

[0130] represents the probability of selecting action i,t according to the policy network parameters from the output of each scheme with a certain probability The output layer of the beam direction policy network uses the softmax activation function, which satisfies:

[0131]

[0132] The scheme ends after N Δt hours of time slots, which is only one time slot for large time slots, and the first time difference error is calculated according to the cumulative reward and the output of the value network:

[0133]

[0134] The value network updates the value network parameters with the goal of minimizing the time difference mean square error, and the loss function is defined as:

[0135]

[0136] where β is the loss coefficient. Then the update formula of the value network parameters is:

[0137]

[0138] The policy function network updates the network parameters along the gradient direction of the loss function J(0 i ) under the guidance of the time difference error. The loss function is defined as:

[0139]

[0140] The update formula of the policy network parameters is:

[0141]

[0142] Similarly, in the spectrum configuration decision scheme, the policy network parameters and the value network parameters are defined. After the upper layer decision executes the action , the agent j observes the state and selects the action

[0143] according to the policy. Specifically:

[0144] Unlike the multi-time slot beam direction decision, the spectrum configuration decision action needs to be decided within each time slot Δt, and the reward is obtained at the end of the execution of the transmit signal action in the current time slot Then calculate the second time difference error:

[0145]

[0146] The value network also updates the value network parameters with the goal of minimizing the time difference mean square error.

[0147]

[0148] The spectrum configuration value network parameters are updated according to the loss function, and the policy network loss function is obtained according to the loss function of the value network , the policy network parameters are updated according to the formula,

[0149]

[0150] Those skilled in the art will appreciate that embodiments of the application can be readily used as software, hardware, or a combination of software and hardware. In one embodiment, the application can be implemented in software and / or firmware. In addition, the software implementation can be implemented by one or more computer programs.

[0151] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart

[0152] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart

[0153] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart

[0154] While the preferred embodiments of the application have been described, additional variations and modifications can be employed. Therefore, the terms and expressions

[0155] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A target clustering based radar adaptive spectrum resource configuration method, characterized in that, The radar adaptive spectrum resource configuration method comprises the following steps: Step 1: obtaining the radar position of the current time slot t, the information detected by each radar of the previous time slot t-1 and the selected beam direction, forming a feature matrix; Step 2: forming an adjacency matrix composed of the communication relationship between the radar and the target, and inputting the feature matrix and the adjacency matrix into a convolutional neural network to obtain a target clustering result; Step 3: obtaining the beam direction of the current time slot t, the number of detected targets, the measured target distance and the target clustering result information, which are used as the state of the current radar beam direction decision; Step 4: selecting a certain direction from the preset beam direction set as the signal beam emission direction of the next few time slots according to the output action selection probability by using the beam direction strategy network; Step 5: obtaining the beam direction of the current time slot t, the spectrum occupation situation and the relative noise level estimation value, which are used as the state of the current radar spectrum configuration decision; Step 6: selecting the frequency modulation slope and sub-band from the preset action set as the signal emission parameters of the next time slot t+1 according to the output action selection probability by using the spectrum configuration strategy network; Step 7: when the radar transmits a signal in the next time slot t+1 and receives the signal, calculating a first time difference error according to the output of the spectrum configuration value network and the corresponding spectrum configuration decision reward value, and training the spectrum configuration strategy network and the spectrum configuration value network according to the first time difference error; Step 8: After N Δt After each time slot, the second time difference error is calculated based on the output of the beam direction value network and the corresponding reward value of the beam direction decision, and the beam direction policy network and beam direction value network are trained based on the second time difference error. Step 9: Repeat the above steps 1 to 8 for every N Δt slots until a stopping condition is reached, at which time the action selection policy converges.

2. The target cluster based radar adaptive spectrum resource configuration method of claim 1, wherein, Step 2 further comprises: Constructing the feature matrix where P t represents a set of radar position coordinates in the radar system, w t represents a set of radar beam directions for each radar, represents a set of radar target distance measurements for each radar. An adjacency matrix A is constructed, which is composed of the communication relationship between the radar i and the target k. If the target is detected, the corresponding position A(k, i) is set to 1, otherwise it is set to 0; Graph convolution is performed in the convolutional neural network, which is specifically as follows: wherein I N is the identity matrix, is the degree matrix of H l is the feature matrix of each layer; σ(·) is the softmax function; W l denotes the weight matrix of the l-th layer of the convolutional neural network; and Re i After that, the cross-entropy loss function is calculated: where T i is the label matrix with one-hot encoding; W l is directly applied to the target clustering for beam direction decision, and the clustering result is taken as one of the observations to help search the target. 3.The target cluster based radar adaptive spectrum resource configuration method of claim 1, wherein, Step 2 further comprises: The whole spectrum is equally divided into Y non-overlapping sub-bands, and each time slot selects one sub-band, and the number of sub-bands is greater than the number of radars; each radar i uses MIMO radar with M t transmit antennas and M r receive antennas, and all the MIMO radars are arranged in a uniform linear array, and the interval d of adjacent transmit / receive antennas is λ / 2, wherein λ is the wavelength. The signal transmitted by the mth transmit antenna of the ith MIMO radar is: t ​ where B denotes the signal bandwidth, denotes the linear frequency spacing of the i-th radar, denotes the carrier frequency used by the i-th radar; Adopting beam forming technology makes MIMO radar transmit signal main lobe energy concentrate, the codebook adopted is: : where P denotes the total number of code words of the DFT codebook, and p = 0, 1,..., P - 1; M t denotes the total number of each radar transmit antenna; The signal transmitted by the mth transmit antenna of the MIMO radar on the ith radar after beamforming is: t ​ It is judged whether the target is within the main lobe range of the radar transmitted signal. If the target is within the main lobe range, it is considered that there is a target within the beam range of the radar transmitted signal, the signal is specularly reflected on the target and forms a return signal; otherwise, it is considered that the radar does not detect the target; When the ith airborne MIMO radar detects a target, the signal received by the mth receiving antenna of the ith airborne MIMO radar is: r ​ where x represents other radar with the same carrier frequency as the ith radar, n i (t) represents Gaussian white noise, γ represents the antenna receiving coefficient, is the time delay of the signal reflected by the mth t transmitting antenna of the MIMO radar on the ith node to the target and received by the mth r receiving antenna, γ xi is the receiving coefficient of the ith radar receiving the transmitting signal of the xth (x≠i) radar, is the time delay of the signal specularly reflected by the mth t transmitting antenna of the xth radar on the target and received by the mth r receiving antenna of the ith radar. For the target echo of the i-th radar receiving its own transmitted signal, γ i = Gp / (R i ) 4 and where m = m t m r , c is the speed of light, R i is the distance between the target and the i-th radar, p is the target reflection coefficient, G represents the antenna gain of the radar, θ i is the azimuth angle of the target relative to the i-th radar, v i is the relative speed of the target relative to the i-th radar; is the interference signal of the i-th radar from other radars using the same sub-band, γ xi = Gp / (R xi ) 4 and where R xi is the relative distance between the x-th radar and the i-th radar, v xi is the speed of the x-th radar relative to the i-th radar, θ xi is the azimuth angle of the x-th radar relative to the i-th radar; A one-dimensional constant false alarm rate (CFAR) detection is performed on the difference frequency signal along the frequency axis, a detection threshold is adaptively generated by calculating the background noise and the interference noise, the amplitude of the detection unit is compared with the detection threshold, and a result of whether a target is detected is obtained; each detection unit in which a target is detected is detected, if the detection unit is adjacent to another detection unit in which a target is detected, the two detection units are considered to be the same target, otherwise, the two detection units are considered to be different targets, and finally a number Y of targets is obtained i,t .

4. The target cluster based radar adaptive spectrum resource configuration method of claim 1, wherein, In the beam direction semi-Markov decision process, given state space The state observed by agent i in time slot t is defined as where represents the relative distance of the target observed by agent i, Y i,t represents the number of targets detected by CFAR; The action of beam direction decided by agent i in time slot t is defined as The decision is immediately executed in time slot t, the radar transmits signals with a new beam direction w i,t+1 The beam direction remains unchanged until It is executed in time slot t+1; the immediate reward is defined as: Two fully connected neural networks are used to approximate the beam-direction policy network π(s; θ) and the beam-direction value network V(s; ω), respectively. The input to both networks is the state observation of agent i. The output of the beam-direction value network is the state. Corresponding value The output of the beam direction policy network is a state. Define the probability of each action; define the policy network parameters. and value network parameters The parameters that agent i needs to iteratively optimize; the beam direction decision-making action in time slot t. The strategy is represented as: Indicates that in state s i,t Based on the policy network parameters Select an action with a certain probability from the output options. The output layer of the beam direction policy network uses the softmax activation function, which satisfies: According to the obtained cumulative reward and the output of the value network, a first time difference error is calculated: The beam direction value network updates the beam direction value network parameters with the minimization of the time difference mean square error as the target, and the loss function is defined as: Where β is the loss coefficient; then the update formula of the beam direction value network parameters is: where a is a weight factor; the beam direction policy network updates the network parameters along the gradient direction of the loss function under the guidance of the first time difference error, and the loss function thereof is defined as The update formula of the beam direction strategy network parameters is:

5. The target cluster based radar adaptive spectrum resource configuration method of claim 1, wherein, In the spectrum configuration semi-Markov decision process, the state space and action space are respectively and The state observed by the agent j in the time slot t is defined as Where is the action performed by the upper beam direction decision, f j,t represents the spectrum occupation state observed by the radar through the signal receiving device in the current time slot t, represents the relative noise level estimation value; the spectrum decision action decided by the agent j is Where and α j,t respectively represent the sub-band and the frequency modulation slope selected by the agent j in the time slot t; each agent j transmits a signal according to the selected action and to obtain a received signal; R is the received signal in the time domain j,t (f), R is sampled with a sampling interval Δf, which is determined by the sampling rate and the number of FFT points, resulting in a sequence j,t (f) is sampled with a sampling interval Δf, which is determined by the sampling rate and the number of FFT points, resulting in a sequence where M f is the number of FFT points; The sequence is sorted in ascending order to obtain a new sequence The sequence is sorted in ascending order to obtain a new sequence By discarding the largest N t samples, cancel the target echo, and obtain an estimate of the noise level: where N t is the number of dropped, where l max is the length dimension of the largest object, and n max is the number of largest objects. The estimate of the relative noise level is obtained from the estimate of the noise level and the estimate of the relative noise level without interference is obtained by division, and is expressed as: The relative noise level estimation value is obtained by using the spectrum of the received signal, and the immediate reward is represented as: Where η0 is a set threshold value; Define policy network parameters And value network parameters In time slot t, agent j observes the upper layer decision to perform action Then select action according to policy Specifically: The spectrum configuration decision action decides in each hour slot At, and obtains the reward in the current time slot at the end of performing the transmit signal action The second time difference error is calculated: The spectrum configuration value network updates the value network parameters with a time difference mean square error minimization as a target The loss function within each hour slot is: Spectrum configuration value network parameters updating according to the loss function, The policy network loss function is obtained according to the loss function of the value network After that, the policy network parameters are updated according to the formula:

Images