A method for avoiding intelligent interference in networked drone communications

Abstract

The present invention discloses a method for intelligent interference avoidance in networked unmanned aerial vehicle (UAV) communications, which specifically includes the following steps: initializing base station sector information, CUAV information and external interference source information; establishing a transmission signal model and an uplink and downlink interference signal model based on the base station sector information, the CUAV information and the external interference source information, and constructing a weighted sum minimization model of the interference intensity suffered by the downlink communication links of all CUAVs and the interference intensity caused by the uplink communication links based on the transmission signal model and the uplink and downlink interference signal model; discretizing the optimization problem corresponding to the weighted sum minimization model of the interference intensity, and mapping the discretized optimization problem into a partially observable Markov decision process; the present invention proposes a path planning algorithm that combines an upper confidence interval exploration strategy and multi-agent reinforcement learning, providing an efficient, intelligent and reliable solution for communication interference avoidance of multiple CUAVs in urban inspections.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) path planning, and in particular to a method for avoiding intelligent interference in networked UAV communications. Background Art

[0002] Entering the 21st century, the accelerated convergence of artificial intelligence, 5G communications, and the Internet of Things (IoT) has driven the development of drones (UAVs) from short-range communications to wider-area, intelligent connectivity. CUAVs have gradually become a cutting-edge technology hotspot. However, when performing urban inspection missions, the high-altitude line-of-sight (LoS) propagation characteristics of CUAVs also pose more severe and complex interference challenges. Specifically, in the downlink, compared to ground users, CUAVs experience severe inter-cell interference from more non-correlated co-channel base stations, significantly increasing the interference component in the downlink received signal and significantly reducing the achievable rate of the CUAV. In the uplink, the CUAV's LoS link signal may also be received by multiple neighboring base stations, causing strong interference to the communications of a large number of co-channel ground users, resulting in a significant drop in the ground uplink transmission rate. Furthermore, to protect sensitive areas in cities (such as military bases and nuclear power plants), active ground-based jammers are often deployed to block communications from potential threats in those areas. The introduction of these jammers not only exacerbates existing interference issues, but their non-selective signal suppression can also cause communications to be interrupted for legitimate inspection drones approaching the edge of sensitive protected areas.

[0003] Although some studies have explored CUAV path planning methods to enhance the mission execution efficiency of CUAV air-ground communication systems and reduce the probability of communication interruption and system energy consumption, the research on communication interference avoidance strategies for multiple CUAVs still faces many challenges, including the following:

[0004] 1) Existing methods for managing communication interference with multiple CUAVs primarily rely on signal-level collaboration and processing between base stations, such as inter-cell interference coordination, beamforming, and power control. However, these methods are typically applicable to local areas. When faced with larger and more complex air-to-air-to-ground interference scenarios involving multiple CUAVs, they are unable to fundamentally address the communication interference challenges they face.

[0005] 2) As the number of CUAVs increases, the dimensionality of the state space and action space increases significantly. Existing trajectory planning algorithms for multiple CUAVs, such as beam search algorithms, ant colony algorithms, and genetic algorithms, typically rely on global environmental information to calculate the optimal solution. However, their applicability to complex and dynamic communication interference scenarios is significantly limited. This is because the environmental state in a dynamic environment changes over time, causing the initial calculation results to become invalid. Furthermore, the recalculation process of these algorithms in dynamic environments often requires a large amount of computing resources, making it difficult to meet real-time requirements, achieve dynamic trade-offs between conflicting indicators such as flight time and communication reliability, and effectively address the coupled problem of communication interference avoidance and collision risk avoidance.

[0006] 3) Intelligent interference avoidance for multiple CUAV communications in the presence of external malicious interference sources has yet to be reported. On the one hand, the introduction of external malicious interference sources not only exacerbates existing interference issues, but their non-selective signal suppression methods can also cause legitimate patrol CUAVs to experience unexpected communication interruptions when approaching the edge of sensitive protection zones. On the other hand, minimizing the weighted sum of interference experienced by multiple CUAVs in downlink communications and interference to uplink communications in complex interference environments is a dynamic, non-convex problem under multiple constraints, making it difficult to solve directly.

[0007] In summary, in an urban environment with external interference sources, how to ensure the communication connectivity of multiple CUAVs and intelligently avoid interference to efficiently complete inspection tasks has become a key issue that needs to be urgently addressed in the path planning of multiple CUAVs. Summary of the Invention

[0008] The purpose of the present invention is to improve and innovate the shortcomings and problems existing in the background technology and provide a method for avoiding intelligent interference in networked drone communications.

[0009] According to a first aspect of the present invention, a method for avoiding intelligent interference in networked drone communications is provided, which specifically includes the following steps:

[0010] Step S1: method starts;

[0011] Step S2: Initialize base station sector information, CUAV information, and external interference source information. The base station sector information includes the number of base station sectors and the coordinates of the base stations where each sector is located. The CUAV information includes the number of CUAVs and the flight starting point and mission destination of each CUAV. The external interference source information includes the location coordinates of the external interference source.

[0012] Step S3: Establish a transmission signal model and an uplink and downlink interference signal model based on the base station sector information, CUAV information, and external interference source information. Based on the transmission signal model and the uplink and downlink interference signal model, construct a weighted sum minimization model of the interference intensity suffered by the downlink communication link of all CUAVs and the interference intensity caused by the uplink communication link;

[0013] Step S4: Discretize the optimization problem corresponding to the weighted sum minimization model of the interference intensity suffered by the downlink communication links of all CUAVs and the interference intensity caused by the uplink communication links, and map the discretized optimization problem into a partially observable Markov decision process to construct the state space, action space, and reward function of the CUAV in each time slot;

[0014] Step S5: designing a UCB-QMIX path planning algorithm that combines the UCB exploration strategy with QMIX deep reinforcement learning, initializing the parameters of the neural network corresponding to the UCB-QMIX path planning algorithm, and iteratively optimizing the parameters of the neural network corresponding to the UCB-QMIX path planning algorithm;

[0015] Step S6: Generate the optimal path plan for the CUAV based on the trained UCB-QMIX path planning algorithm.

[0016] A further solution is that the step S3 of establishing the transmission signal model according to the base station sector information, CUAV information and external interference source information specifically includes:

[0017] Get the m CUAV to the first k Equivalent channel coefficients between base station sectors;

[0018] Where: , and Respectively expressed in t The base station antenna gain, large-scale path loss, and small-scale shadow fading at each moment; Indicates the total number of sectors providing communication services to CUAV; Indicates the number of CUAVs, Indicates the m Flight time of the CUAV;

[0019] Get the external interference source on the ground to m Equivalent channel coefficient of CUAV;

[0020] ;

[0021] Where: and Respectively expressed in tThe external interference source at the moment m CUAV large-scale path loss and small-scale shadow fading;

[0022] ;

[0023] Where, Indicates the m The Euclidean distance between the CUAV and the external interference source, is the carrier frequency of the base station communicating with the CUAV, H u Indicates the flight altitude of CUAV; Indicates t Moment m The space-time coordinates of the CUAV.

[0024] A further solution is that the step S3 of establishing the uplink and downlink interference signal model according to the base station sector information, CUAV information and external interference source information specifically includes:

[0025] Get the m CUAV downlink communication received the first k The signal-to-interference-and-noise ratio of the base station sector signal;

[0026] ;

[0027] Where: , Indicates the transmission power of the external interference source; is the base station transmit power, For the m The mean value received by the CUAV is 0 and the variance is Additive Gaussian white noise; represents the interference sum of other base station sectors, namely:

[0028] ;

[0029] Where: Indicates that the downlink is m The resource blocks allocated by the CUAV are Whether there are users in each base station sector that use the same frequency band for downlink communication with the connected drone; express Belong to other sectors in the sector set except the k-th sector; express t Time downlink m CUAV to the first Equivalent channel coefficients between base station sectors;

[0030] Get them The CUAV interferes with the uplink communication link of the ground co-channel users;

[0031] ;

[0032] Where: Indicates that the uplink is m The frequency band allocated to the CUAV is The probability that a base station sector is occupied; express t Time in uplink m CUAV to the first The equivalent channel coefficient between base station sectors.

[0033] A further solution is that the weighted sum minimization model of the interference intensity suffered by the downlink communication link of all CUAVs and the interference intensity caused by the uplink communication link is constructed based on the transmission signal model and the uplink and downlink interference signal model in step S3 specifically includes:

[0034] Definition m The duration of communication interruption of the CUAV is:

[0035] ;

[0036] Where: Indicates the m The last time the CUAV was connected to the ground base station, represents the SINR threshold of the CUAV downlink. When m CUAV and ground base station sectors k The downlink communication link is in a connected state;

[0037] Definition m CUAV in t The base station sector connected at the moment is ,and , then the optimization problem of minimizing the weighted sum of the interference intensity suffered by the downlink communication links of all CUAVs and the interference intensity caused by the uplink communication links can be written as:

[0038] ;

[0039] in, are the weights of the interference received by all CUAV downlink communications and the interference caused by all CUAV uplink communication links; constraint C2 ensures the flight direction of CUAV The module length is unit 1; C3-C5 is the mThe flight start point, end point and flight area constraints of the CUAV, Indicates the position tolerance for reaching a given target point; C6 constrains the duration of CUAV communication interruption; C7 ensures that CUAV is connected to a base station sector at all times; C8 ensures that there will be no collision between CUAVs. Indicates the minimum safe distance between CUAVs, is the preset time threshold; Indicates the coordinates of the CUAV's flight starting point; Indicates the m CUAV mission destination coordinates.

[0040] A further solution is that the optimization problem corresponding to the weighted sum minimization model of the discretization of the interference intensity suffered by the downlink communication links of all CUAVs and the interference intensity caused by the uplink communication links in step S4 specifically includes:

[0041] The first m CUAV flight time Discretized into time slots, i.e. , the optimization problem after discretization is:

[0042] ;

[0043] in ; The first m The communication interruption duration of the CUAV is Indicates that the allowed CUAV interrupt duration is constrained by express; No. m CUAV in the n The position coordinates of each time slot.

[0044] A further solution is to map the discretized optimization problem in step S4 into a partially observable Markov decision process to construct the state space, action space, and reward function of the CUAV in each time slot. Specifically, the following are included:

[0045] Definition m CUAV in the first n The local observation space of time slots ,but M Global observation space of CUAV ; CUAV flight area size ;

[0046] Definition m CUAV in the first n action space of time slots , Respectively indicate movement to the east, west, north, northeast, northwest, southeast, southwest and stop movement; M The action space of the CUAV is ;

[0047] definition M CUAV in the global observation space When selecting the action space Global rewards obtained when ;

[0048] Where: Represents the reward for all CUAVs flying to the finish line; Indicates the penalty for a CUAV flying out of the restricted area and the duration of communication interruption. Exceed punishment; represents the penalty for collision between CUAVs; CUAV does not reach the target point but chooses to perform the action punishment, of which , , and are all constants; Indicates the m CUAV in the first n The interference suffered by a time slot in the downlink communication link and the penalty caused by the uplink communication link to its co-channel users, that is,

[0049] ;

[0050] in and Respectively represent m The interference caused by the downlink communication between the CUAV and the ground base station and the uplink communication, and The weight for balancing uplink and downlink interference.

[0051] A further solution is that step S5 specifically includes:

[0052] Step S501: Acquire M The historical observation sequences of the CUAV are used as training samples;

[0053] Step S502: Initialize the parameters of the QMIX network structure corresponding to the UCB-QMIX path planning algorithm, wherein the QMIX network structure includes individual networks and hybrid networks;

[0054] Step S503: Input the historical observation sequence of each CUAV into the corresponding individual network, and use the corresponding individual network to calculate the original Q value of each action. An action counter is also maintained to record the number of times each action was selected in historical decisions. A confidence interval for each action is calculated based on the number of times each action was selected in historical decisions. Based on the original Q value and confidence interval of each action, a local Q value for each action is calculated, and the action with the largest local Q value is set as the action to be executed in the current time slot.

[0055] Step S504: Input the maximum value of the local Q value corresponding to each action calculated by the individual networks corresponding to all CUAVs into the hybrid network to calculate the global Q value; at the same time, use the target network and the global observation space corresponding to the next time slot and the global reward of the current time slot to calculate the target Q value, and construct a loss function based on the global Q value and the target Q value;

[0056] Step S505: Update the parameters of the individual network and the hybrid network by using the gradient descent method until the loss value of the constructed loss function is less than a preset value.

[0057] A further solution is to calculate the local Q value of each action based on the original Q value and confidence interval of each action in step S503, and set the action with the largest local Q value as the action to be executed in the current time slot. The calculation formula is as follows:

[0058] ;

[0059] Where: ; is the exploration coefficient that decays with time slots, represents the local Q value of each action, represents the original Q value of each action, Indicates the number of times each action is selected in historical decisions; total number of decisions ;

[0060] in .

[0061] According to a second aspect of the present invention, there is provided an electronic device, comprising: a memory and a processor;

[0062] The memory is used to store programs;

[0063] The processor is used to call the program stored in the memory to execute the intelligent interference avoidance method for networked drone communications as described above.

[0064] According to a third aspect of the present invention, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it implements a method for avoiding intelligent interference in networked drone communications as described above.

[0065] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention is aimed at a system for urban inspections of multiple CUAVs in the presence of external malicious interference sources, in order to effectively reduce the interference of downlink communication links and uplink communication links to ground users during the execution of tasks, improve communication quality and reliability, and ensure the efficient execution of inspection tasks. Under the multi-constraint conditions of multiple CUAVs reaching the designated destination, not flying out of the specified area, the communication interruption duration not exceeding the set threshold, and avoiding collisions between drones, a UCB-QMIX intelligent path planning algorithm combining an upper confidence interval exploration strategy and a value decomposition network framework in multi-agent reinforcement learning is proposed to achieve collaborative decision-making and path optimization of multiple CUAVs in complex environments, improve the convergence speed and stability of the algorithm, and provide an efficient, intelligent, and reliable solution for avoiding communication interference of multiple CUAVs in urban inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0067] Figure 1 This is a model of a multi-CUAV urban inspection system with external interference sources provided by the first embodiment of the present invention;

[0068] Figure 2 This is a flow chart of a method for avoiding intelligent interference in networked UAV communications provided by the first embodiment of the present invention;

[0069] Figure 3 It is the network framework of the UCB-QMIX algorithm provided by the first embodiment of the present invention;

[0070] Figure 4 This is a distribution map of buildings, base stations, and interference sources in an urban scene provided by the first embodiment of the present invention;

[0071] Figure 5 The interference suffered by the CUAV in downlink communication and the interference caused by the CUAV in uplink communication at different position coordinates provided by the first embodiment of the present invention, as well as the change in communication interruption;

[0072] Figure 6The interference suffered by the downlink communication links, the interference caused by the uplink communication links, the cumulative rewards, and the flight path diagram of the four CUAVs provided by the first embodiment of the present invention are shown;

[0073] Figure 7 is a flight path diagram of the CUAV provided by the first embodiment of the present invention when a target point is given but the location of the interference source is different;

[0074] Figure 8 1 is a flight path diagram of a CUAV under different interference powers but the same target location provided by the first embodiment of the present invention;

[0075] Figure 9 This is a performance comparison chart of the UCB-QMIX algorithm provided by the first embodiment of the present invention and other algorithms. DETAILED DESCRIPTION

[0076] In order to make the objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0077] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terms used in this specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0078] Example 1

[0079] like Figure 2As shown, the present invention provides a method for intelligent interference avoidance of networked UAV communications, which consists of 6 steps, namely, step S1: method start; step S2: initializing base station sector information, CUAV information, and external interference source information, wherein the base station sector information includes the number of base station sectors and the coordinates of the base station where each sector is located; the CUAV information includes the number of CUAVs and the flight starting point and mission destination of each CUAV, and the external interference source information includes the location coordinates of the external interference source; step S3: establishing a transmission signal model and an uplink and downlink interference signal model based on the base station sector information, CUAV information and external interference source information, and constructing a weighted sum minimization model of the interference intensity suffered by the downlink communication link of all CUAVs and the interference intensity caused by the uplink communication link based on the transmission signal model and the uplink and downlink interference signal model; step S4 : Discretize the optimization problem corresponding to the weighted sum minimization model of the interference intensity suffered by the downlink communication links of all CUAVs and the interference intensity caused by the uplink communication links, and map the discretized optimization problem into a partially observable Markov decision process to construct the state space, action space and reward function of the CUAV in each time slot; Step S5: Design the UCB-QMIX path planning algorithm that combines the UCB exploration strategy with QMIX deep reinforcement learning, initialize the parameters of the neural network corresponding to the UCB-QMIX path planning algorithm, and iteratively optimize the parameters of the neural network corresponding to the UCB-QMIX path planning algorithm; Step S6: Generate the optimal path planning of the CUAV based on the trained UCB-QMIX path planning algorithm. The following is a detailed introduction to these 6 steps:

[0080] Step S1: method starts;

[0081] Step S2: Initialize base station sector information, CUAV information, and external interference source information. The base station sector information includes the number of base station sectors and the coordinates of the base stations where each sector is located. The CUAV information includes the number of CUAVs and the flight starting point and mission destination of each CUAV. The external interference source information includes the location coordinates of the external interference source.

[0082] Specifically, for the intelligent interference avoidance strategy for multiple CUAV communications designed by the present invention, some parameters need to be assigned. First, the base station sector information is initialized:

[0083] Enter the 3D Cartesian coordinate system M The size of the flight area of ​​the CUAV , distributed in this area L Each base station node is divided into three independent radiation sectors through a phased array antenna system, and each sector adopts a 120° symmetrical distribution pattern. Therefore, the number of sectors providing communication services for CUAV is , using a collection Indicates that k The coordinates of the base station where the sector is located are expressed as , where each base station has the same height, and the sectors have the same height as the base stations.

[0084] Next, initialize the CUAV information:

[0085] like Figure 1 As shown, M The flight starting points of the CUAVs are and multiple mission destinations for .make Indicates the m The time-varying coordinates of the CUAV, Indicates the m The flight time of the CUAV. M The flight area of ​​CUAV is a two-dimensional space, and the flight altitude of all CUAVs is fixed at H u . No. m CUAV in t The flight speed at this moment is , and the maximum flight speed is .

[0086] Finally, initialize the external interference source information:

[0087] Initialize the position coordinates of the external interference source to ,in is the antenna height of the interference source.

[0088] Step S3: Establish a transmission signal model and an uplink and downlink interference signal model based on the base station sector information, CUAV information, and external interference source information. Based on the transmission signal model and the uplink and downlink interference signal model, construct a weighted sum minimization model of the interference intensity suffered by the downlink communication link of all CUAVs and the interference intensity caused by the uplink communication link.

[0089] Specifically, establishing a transmission signal model based on base station sector information, CUAV information and external interference source information specifically includes: t Moment, m CUAV to the first k The equivalent channel coefficient between base station sectors is expressed as:

[0090] ;

[0091] Where: , and represent base station antenna gain, large-scale path loss, and small-scale shadow fading, respectively.

[0092] Ground external interference source to the first m The equivalent channel coefficient of a CUAV is expressed as:

[0093] ;

[0094] Where: and Respectively expressed in t The external interference source at the moment m CUAV large-scale path loss and small-scale shadow fading, small-scale shadow fading is a random variable with Rayleigh distribution and mean 1.

[0095] ;

[0096] in , indicating the m The Euclidean distance between the CUAV and the external interference source, is the carrier frequency of the base station communicating with the CUAV.

[0097] Specifically, establishing an uplink and downlink interference signal model based on base station sector information, CUAV information, and external interference source information specifically includes:

[0098] The present invention assumes that each CUAV uses orthogonal time-frequency resource blocks to communicate with the ground base station, and does not consider the same-frequency interference between CUAVs. m If the downlink communication link of a CUAV is interfered by adjacent co-frequency base station sectors, external interference sources, and background noise, t Moment m CUAV downlink communication received by ground sector k The signal-to-interference-and-noise ratio (SINR) of a signal is written as:

[0099] ;

[0100] Where: , Indicates the transmit power of the external interference source. is the base station transmit power, assuming that all sectors transmit power are equal and fixed. For the m The mean value received by the CUAV is 0 and the variance is Additive white Gaussian noise. represents the interference sum of other base station sectors, namely:

[0101] ;

[0102] Where: Indicates that the downlink is m The resource blocks (RBs) allocated by the CUAV are Is there a user in each base station sector that uses the same frequency band for downlink communication with the networked drone? =1 means it exists, otherwise it does not exist; express Belong to other sectors in the sector set except the k-th sector; express t Time downlink m CUAV to the first The calculation method of the equivalent channel coefficient between base station sectors is the same as that of m CUAV to the first k The equivalent channel coefficients between base station sectors are the same.

[0103] Since the base station antenna in the cellular system mainly provides signal coverage for ground users, the signal coverage received by the CUAV as an airborne user is relatively weak. The present invention sets a SINR threshold for the CUAV downlink. To determine whether the CUAV is in a downlink communication link connection with the base station. When m CUAV and ground base station sectors k The downlink communication link between the CUAV and the ground base station is in a connected state, otherwise it means that the downlink communication link between the CUAV and the ground base station is disconnected.

[0104] On the other hand, CUAV needs to transmit information such as video, audio, and CUAV location coordinates to the ground base station at any time during the mission. When CUAV and the ground base station sector use a certain frequency band for uplink communication, the ground users in the radiation area will inevitably use this frequency band for uplink communication with their corresponding service sector base station. Therefore, CUAV will cause serious air-to-ground interference to the communication of a large number of ground users on the same channel in the area, resulting in a significant loss of transmission rate of the ground user uplink communication link. Assuming that CUAV is at a fixed power Transmit signal, then m The uplink communication link interference of a CUAV to the ground co-channel user is:

[0105] ;

[0106] Where: Indicates that the uplink is m The frequency band allocated to the CUAV is The probability that a base station sector is occupied. A rough estimate is the average number of users served by each ground base station in the total number of available RBs in the uplink and downlink communication links in a given time period, so it is assumed that the number of users of each CUAV is same; express t Time in uplink m CUAV to the first The calculation method of the equivalent channel coefficient between base station sectors is the same as that of m CUAV to the first k The equivalent channel coefficients between base station sectors are the same.

[0107] To ensure that the CUAV can simultaneously receive base station control signals, target information, and other signals, and transmit video, audio, and information such as the CUAV's location coordinates to the base station, it is assumed that the CUAV uses frequency division duplex (FDD) mode to communicate with the ground base station. Specifically, the CUAV uses two antennas for uplink and downlink communications on different frequency bands. FDD technology handles uplink and downlink communications separately by allocating two different frequency bands, which are designed to be isolated from each other. Furthermore, modern communication systems often employ self-interference cancellation techniques, further reducing potential self-interference through digital signal processing and filter design. In summary, the present invention does not consider the existence of self-interference in the CUAV uplink and downlink links.

[0108] Specifically, based on the transmission signal model and the uplink and downlink interference signal model, a weighted sum minimization model of the interference intensity suffered by the downlink communication link of all CUAVs and the interference intensity caused by the uplink communication link is constructed, which specifically includes:

[0109] In order to ensure the communication connection requirements of multiple CUAVs, the first m The duration of communication interruption of the CUAV is:

[0110] ;

[0111] Where: Indicates the m The last time a CUAV was connected to a ground base station. To ensure the communication connection requirements of multiple networked drones, each CUAV needs to be constrained by the interruption time, that is, the maximum time that the CUAV is allowed to lose cellular connection does not exceed the preset time threshold. .

[0112] Therefore, the present invention defines m CUAV in t The base station sector connected at the moment is and , then the optimization problem of minimizing the weighted sum of the interference intensity suffered by the downlink communication links of all CUAVs and the interference intensity caused by the uplink communication links can be written as:

[0113] ;

[0114] in, are the weights of the interference received by all CUAV downlink communications and the interference caused by all CUAV uplink communications. Constraint C2 ensures that the flight direction of CUAV The module length is unit 1; C3-C5 is the m The flight start point, end point and flight area constraints of the CUAV, Indicates the position tolerance for reaching a given target point; C6 constrains the duration of CUAV communication interruption; C7 ensures that CUAV is connected to a base station sector at all times; C8 ensures that there will be no collision between CUAVs. Indicates the minimum safe distance between CUAVs; Indicates the coordinates of the CUAV's flight starting point; Indicates the m CUAV mission destination coordinates.

[0115] Step S4: Discretize the optimization problem corresponding to the weighted sum minimization model of the interference intensity suffered by the downlink communication links of all CUAVs and the interference intensity caused by the uplink communication links, and map the discretized optimization problem into a partially observable Markov decision process to construct the state space, action space and reward function of the CUAV in each time slot.

[0116] In this embodiment, the present invention constructs the intelligent interference avoidance problem of multiple CUAV communications into a multi-agent system. First, the optimization problem (P1) is time discretized. m CUAV flight time Discretized into time slots, i.e. , and the CUAV position change in each time slot is very small or even negligible, so the optimization problem after discretization is:

[0117] ;

[0118] The cellular sector association criterion uses CUAV to associate with the sector that provides the maximum channel power gain, that is, ; The first m The communication interruption duration of the CUAV is Indicates that the allowed CUAV interrupt duration is constrained by express.

[0119] Then the optimization problem (P2) is mapped to a partially observable Markov decision process (Dec-POMDP). The specific process of mapping the optimization problem to Dec-POMDP is as follows:

[0120] For the state space: n time slot, m The observation space of a CUAV is its current location coordinates and the duration of communication interruption Composition, that is, m CUAV in the first n The local observation space of time slots , so we can get M The global observation space of a CUAV is .

[0121] Regarding the action space: The setting of the action space needs to comprehensively consider the mission requirements and the motion control of the CUAV. On the one hand, the mission requires that all CUAVs must reach the target point before it is considered complete, so a clear action is needed to indicate that the mission is complete. At the same time, in order to prevent the CUAV from continuing to move near the target point, thereby affecting the judgment of mission completion, and to prevent the CUAV from colliding or interfering with other CUAVs after reaching the target point, affecting the performance and safety of the entire system, the design of the action space must include a stop action. When the CUAV reaches the target point, selecting this action can keep it stationary, thereby completing the urban inspection mission more accurately. Specifically, the flight speed of each CUAV is fixed, and the action space consists of a flight direction and a stop action. The flight direction is discretized into eight main directions, including east, south, west, north, southeast, southwest, northeast and northwest, and each direction corresponds to a discrete action. In addition, a stop action is added. When the CUAV reaches the target point, this action is selected to stop moving, and the flight speed is zero. Therefore, in the nth time slot, the action space of the mth CUAV , , , , , , , , , Therefore, the action space of all CUAVs is .

[0122] For the reward function: the reward function represents the total amount of all CUAVs in the global observation space Select action The global reward obtained when . The reward function designed based on the constraints is:

[0123] ;

[0124] Where: Represents the reward for all CUAVs flying to the finish line; Indicates the penalty for a CUAV flying out of the restricted area and the duration of communication interruption. Exceed punishment; represents the penalty for collision between CUAVs. CUAV does not reach the target point but chooses to perform the action punishment, of which , , and are all constants. Indicates the m CUAV in the first n The interference suffered by a time slot in the downlink communication link and the penalty caused by the uplink communication link to its co-channel users, that is,

[0125] ;

[0126] in and Respectively represent m The interference in downlink communication between the CUAV and the ground base station and the interference in uplink communication are used to force the CUAV to fly to an area with less interference. and The weight for balancing uplink and downlink interference.

[0127] Step S5: Design a UCB-QMIX path planning algorithm that combines the UCB exploration strategy with QMIX deep reinforcement learning, initialize the parameters of the neural network corresponding to the UCB-QMIX path planning algorithm, and iteratively optimize the parameters of the neural network corresponding to the UCB-QMIX path planning algorithm.

[0128] It should be noted that the core idea of ​​QMIX is to learn a hybrid network through centralized training, and to transform the local Q value function of each agent into The nonlinear combination is the global Q value function , while maintaining the independence of individual strategies during decentralized execution. In the traditional QMIX algorithm, the action selection in the decentralized execution phase depends on Strategy, that is, each agent randomly explores non-optimal actions with probability Select the action with the largest local Q value. Although this mechanism is simple, it has the defects of low exploration efficiency and poor sample quality. That is, random exploration may repeatedly select inefficient actions that have been verified, and it cannot quantify the uncertainty of the action value. To this end, this paper proposes a UCB-QMIX algorithm, which embeds the UCB strategy into the action selection process of the decentralized execution stage, and optimizes the learning efficiency of multi-agents through a confidence interval-driven exploration mechanism. The improved network structure is as follows Figure 3 shown.

[0129] Specifically, the UCB-QMIX path planning algorithm training process is as follows:

[0130] Step S501: Acquire M The historical observation sequences of the CUAV are used as training samples;

[0131] Since the local observation space of the current time slot is the local observation space of the previous time slot Execute the action space of the previous time slot Therefore, it can be obtained based on the historical observation sequence Get the historical action sequence .

[0132] Step S502: Initialize the parameters of the QMIX network structure corresponding to the UCB-QMIX path planning algorithm, wherein the QMIX network structure includes individual networks and hybrid networks;

[0133] Among them, the network structure of QMIX includes three core components: individual network, hybrid network and super network.

[0134] Specifically, m The individual network of the CUAV is composed of a recurrent neural network and a multi-layer perceptron. The input of each gated recurrent unit corresponding to the recurrent neural network is the local observation space corresponding to each time slot. And the action space corresponding to the previous time slot The output is the Q value of each action. The cyclic structure of the recurrent neural network enables it to encode historical observation sequences , thereby handling partial observability.

[0135] Hybrid network: The hybrid network receives the local Q values ​​of all agents and the global observation space , output global Q value through multi-layer nonlinear transformation To satisfy the monotonicity constraint, the weights of the hybrid network need to be generated by the super network and remain non-negative.

[0136] Hypernetwork: Hypernetwork uses global observation space As input, generate the weights and biases of each layer of the hybrid network.

[0137] Step S503: Input the historical observation sequence of each CUAV into the corresponding individual network, and use the corresponding individual network to calculate the original Q value of each action. An action counter is also maintained to record the number of times each action was selected in historical decisions. A confidence interval for each action is calculated based on the number of times each action was selected in historical decisions. Based on the original Q value and confidence interval of each action, a local Q value for each action is calculated, and the action with the largest local Q value is set as the action to be executed in the current time slot.

[0138] like Figure 3 As shown, at the beginning of each training time slot, the m CUAV based on historical observation sequence , calculate the original Q value of each action through its own individual Q network Specifically, the local observation space and the action space of the previous time slot After being spliced ​​together, they are input into the multilayer perceptron and the corresponding gated recurrent unit in turn, so that the gated recurrent unit of the recurrent neural network outputs the original Q value of each action At the same time, m CUAV maintains an action counter , record every action The number of times it was selected in historical decisions and the total number of decisions , then the confidence interval of each action is calculated based on the total number of decisions for each action; finally, the local Q value of each action is calculated by combining the original Q value of each action and the introduced confidence interval ,Right now:

[0139] ;

[0140] Where: ; is the exploration coefficient of time slot attenuation. CUAV selection The largest action is its own action in the current time slot, that is, After executing the selected action, the CUAV agent feeds back the action execution result to the server.

[0141] Step S504: Input the maximum value of the local Q value corresponding to each action calculated by the individual networks corresponding to all CUAVs into the hybrid network to calculate the global Q value; at the same time, use the target network and the global observation space corresponding to the next time slot and the global reward of the current time slot to calculate the target Q value, and construct a loss function based on the global Q value and the target Q value;

[0142] After receiving the action execution results sent by each CUAV agent, the server enters the experience collection and storage phase. The experience replay buffer is used to store a certain amount of historical experience data to provide data support for subsequent centralized training.

[0143] When a certain amount of data is accumulated in the experience replay buffer, the server randomly samples a batch of experience data from the buffer for the forward calculation and loss evaluation of the hybrid network. For each sampled experience data, the server inputs the maximum value of the local Q value corresponding to each action calculated by the individual network corresponding to each CUAV into the hybrid network (parameter is ), combined with the global state information, the weights and biases of each layer of the hybrid network are generated through the super network, and nonlinear aggregation operations are performed to calculate the global Q value, that is:

[0144] ;

[0145] Where, They represent the maximum value of the local Q value calculated by each CUAV through the individual network.

[0146] At the same time, using the target hybrid network (parameters are ) and the global observation space corresponding to the next time slot and the global reward for the current time slot , calculate the target Q value, that is:

[0147] ;

[0148] Where: represents the parameters of the target hybrid network, represents the discount factor; Represents the global observation space corresponding to the next time slot The global Q value corresponding to all actions; Indicates the maximum value of the global Q value corresponding to all actions.

[0149] Finally, the loss function is constructed based on the global Q value and the target Q value, namely:

[0150] ;

[0151] Where E[] represents the mean square error.

[0152] Step S505: updating the parameters of the individual network and the hybrid network by using the gradient descent method until the loss value of the constructed loss function is less than a preset value;

[0153] Specifically, the parameters of the individual network and the hybrid network are updated through the gradient descent algorithm:

[0154] ;

[0155] Where: is the learning rate. At the same time, the soft update method is used to update the parameters of the target individual network and the hybrid network:

[0156] ;

[0157] Where: This parameter update mechanism ensures that the network can gradually learn better strategies while maintaining the stability and convergence of the training process. The updated individual network parameters are sent back to each CUAV end to guide subsequent action selection, thus forming a closed-loop learning and optimization process. and Represent the parameters of the target individual network and the mixed network respectively.

[0158] Step S6: Generate the optimal path plan for the CUAV based on the trained UCB-QMIX path planning algorithm.

[0159] As mentioned above, by inputting the historical observation sequence of each CUAV into the corresponding individual network, the local Q value of each action can be calculated, and the action with the largest local Q value is set as the action to be executed in the current time slot; that is, , and the local observation space corresponding to the current time slot is Executing the action space of the current time slot can obtain the local observation space of the next time slot By analogy, the optimal path plan for the CUAV can be generated. Based on the generated optimal path plan for the CUAV, the weighted sum of the interference intensity of the downlink communication link and the interference intensity caused by the uplink communication link of all CUAVs can be obtained, and the process ends here.

[0160] In order to verify the effectiveness of the CUAV communication intelligent interference avoidance strategy based on multi-agent deep reinforcement learning proposed in this paper, Python 3.6, TensorFlow 1.12.0, Keras 2.1.6, and numpy 1.16.6 were used for simulation training. Figure 4 The size of the area shown is The height and location of buildings in the city are set according to the statistical model proposed by the International Telecommunication Union (ITU) standard, where the ratio of the area covered by the building to the total area is , the average number of buildings per unit area , the building height distribution is modeled as There are six ground base stations represented by black hexagons in this urban scene, and their two-dimensional position coordinates are m, m, m, m, m, m, and the antenna height is 25 m. The malicious interference sources used to protect sensitive areas are represented by gray triangles. m, interference antenna height For other relevant parameters, see Table 1 for details unless otherwise specified.

[0161] With this simulated building distribution, we can analyze whether the communication link between the base station and the networked drone is blocked by the building based on the location and height information of the given building, and determine whether the link type is line-of-sight or non-line-of-sight connection. Then, according to the link type and the specific channel model, we can draw the following Figure 5 The figure shows the changes in two types of interference (the interference caused by the downlink communication between the CUAV and the base station sector and the interference caused by the uplink communication of the CUAV to the ground users on the same channel) and the communication interruption situation at different coordinates in a simulated urban environment. The lighter the color, the greater the interference intensity caused or suffered by the CUAV. It can be seen that since the uplink and downlink use different frequency bands, the interference intensity caused and suffered by the CUAV cannot be completely equated. For example, Figure 5 The parts circled by red ellipses in (a) and (b) above need to be weighted to differentiate the importance of the two types of interference. To more intuitively show the degree of change of the two types of interference, the present invention performs linear normalization (maximum-minimum normalization). Figure 5 (c) shows the relationship between the communication connection status and the flight position of the drone. The communication connection status is determined by the SINR of the service sector signal received by the networked drone at the current location. When it is greater than or equal to When the connected drone is in the yellow area, it can maintain communication with the ground base station sector. When it is in the purple area, it loses connection.

[0162] Table 1 Simulation parameter description

[0163]

[0164] Figure 6 (a) and Figure 6Panel (b) shows the interference experienced by the downlink and uplink communication links of the four CUAVs, respectively, and their changing trends over the number of training rounds, with a sliding average window size of 300. It can be seen that the interference experienced by both the downlink and uplink communication links shows a clear downward trend within the first 2500 training rounds, indicating that the algorithm gradually learns how to reduce interference in the communication links during the exploration phase. As the number of training rounds increases, the interference value gradually stabilizes, ultimately remaining at a low level. This verifies that the proposed algorithm can effectively reduce bidirectional interference in the communication links when optimizing the communication interference avoidance strategy. Figure 6 Figure (c) shows the cumulative reward value as the number of training rounds increases. It can be seen that the cumulative reward value gradually increases within the first 2500 training rounds, indicating that the algorithm has gradually mastered how to improve the reward value by optimizing the communication interference avoidance algorithm during the exploration phase. After 2500 rounds, the cumulative reward value tends to stabilize, indicating that the algorithm has converged and can stably output high-quality communication interference avoidance strategies. Figure 6 Figure (d) shows the flight paths of four CUAVs traveling to four target points. As can be seen, all CUAVs were able to reach their target points and perform their inspection missions within the maximum communication interruption duration constraint. Furthermore, the CUAV traveling to the target point with coordinates of (500, 1750) m was able to find a suitable path near the jammer. This validates the effectiveness of the proposed UCB-QMIX-based CUAV communication interference avoidance algorithm.

[0165] Figure 7 The flight paths of four CUAVs trained using the proposed algorithm are shown for interference sources at different locations, with coordinates of (1550, 750) m, (1550, 1550) m, (750, 750) m, and (750, 1550) m, respectively. To better demonstrate the training results, four fixed target points were set near the interference sources, with coordinates of (1750, 500) m, (1750, 1750) m, (500, 500) m, and (500, 1750) m, respectively. It can be seen that the CUAVs evade the interference sources as they approach. When the interference source location changes, the proposed algorithm can also adjust the CUAV flight path to avoid areas of strong interference, further verifying the effectiveness of the proposed UCB-QMIX-based CUAV communication interference avoidance algorithm for interference sources at different locations.

[0166] Figure 8The proposed algorithm generates flight trajectories for four CUAVs under different conditions: no interference, and interference powers of 1dBm, 4dBm, and 7dBm, respectively. Four target points are positioned near four interference sources at distances of (250, 1500) m, (500, 1750) m, (1000, 1750) m, and (1250, 1750) m. As can be seen, as the interference power increases, the CUAVs move away from the interference source, seeking paths with less interference.

[0167] Different interference source locations and different interference power levels can reflect the location of sensitive protection areas and the size of the site. Figure 7 and Figure 8 It can be seen that the proposed algorithm can be trained to obtain different flight paths according to the changes in the interference source position and interference power, which further proves the effectiveness of the proposed algorithm.

[0168] Figure 9 The performance of the proposed UCB-QMIX algorithm is compared with two benchmark multi-agent algorithms: QMIX and VDN. Specifically, Figure 9 The figure shows the trend of the weighted sum of the interference received by the downlink communication link and the interference caused by the uplink communication link with the number of training rounds. It can be seen from the figure that the proposed UCB-QMIX algorithm can more effectively reduce the interference value during the training process and maintain stability in the later stage. For example, in Figure 9 In (a), the weighted sum of the downlink interference value and the uplink interference value of CUAV under the proposed UCB-QMIX algorithm stabilizes at around 5600 in the late training stage, while the weighted sum of the interference values ​​of the QMIX and VDN algorithms stabilizes at around 7000 and 8000, respectively. This shows that the proposed UCB-QMIX communication interference avoidance algorithm can more effectively avoid interference areas than the QMIX and VDN algorithms, thereby reducing communication interference. Figure 9 Panel (b) shows the cumulative reward values ​​of different algorithms as a function of the number of training rounds. It can be seen that the cumulative reward value of the proposed UCB-QMIX algorithm gradually increases during training and stabilizes in subsequent rounds, with a cumulative reward value of around -5000. In contrast, the rewards of QMIX and VDN stabilize at around -6500 and -7000, respectively. This demonstrates that the proposed UCB-QMIX algorithm is more effective in avoiding interference in multi-CUAV communication, achieving higher rewards while satisfying communication constraints.

[0169] Example 2

[0170] The present invention provides an electronic device, comprising: a memory and a processor;

[0171] The memory is used to store programs;

[0172] The processor is used to call the program stored in the memory to execute a method for avoiding intelligent interference in networked drone communications as described in Example 1.

[0173] Example 3

[0174] The present invention provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, a method for avoiding intelligent interference in networked drone communications as described in Example 1 is implemented.

[0175] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the description and implementation methods. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and embodiments shown and described herein.

Claims

1. A method for avoiding intelligent interference in networked UAV communications, characterized in that: The specific steps include: Step S1: method starts; Step S2: Initialize base station sector information, CUAV information, and external interference source information. The base station sector information includes the number of base station sectors and the coordinates of the base stations where each sector is located. The CUAV information includes the number of CUAVs and the flight starting point and mission destination of each CUAV. The external interference source information includes the location coordinates of the external interference source. Step S3: Establish a transmission signal model and an uplink and downlink interference signal model based on the base station sector information, CUAV information, and external interference source information. Based on the transmission signal model and the uplink and downlink interference signal model, construct a weighted sum minimization model of the interference intensity suffered by the downlink communication link of all CUAVs and the interference intensity caused by the uplink communication link; Step S4: Discretize the optimization problem corresponding to the weighted sum minimization model of the interference intensity suffered by the downlink communication links of all CUAVs and the interference intensity caused by the uplink communication links, and map the discretized optimization problem into a partially observable Markov decision process to construct the state space, action space, and reward function of the CUAV in each time slot; Step S5: designing a UCB-QMIX path planning algorithm that combines the UCB exploration strategy with QMIX deep reinforcement learning, initializing the parameters of the neural network corresponding to the UCB-QMIX path planning algorithm, and iteratively optimizing the parameters of the neural network corresponding to the UCB-QMIX path planning algorithm; Step S6: Generate the optimal path plan for the CUAV based on the trained UCB-QMIX path planning algorithm.

2. The method for avoiding intelligent interference in networked UAV communications according to claim 1, characterized in that: The step S3 of establishing a transmission signal model according to the base station sector information, CUAV information and external interference source information specifically includes: Get the m CUAV to the first k Equivalent channel coefficients between base station sectors; Where: , and Respectively expressed in t The base station antenna gain, large-scale path loss, and small-scale shadow fading at each moment; Indicates the total number of sectors providing communication services to CUAV; Indicates the number of CUAVs, Indicates the m Flight time of the CUAV; Get the external interference source on the ground to m Equivalent channel coefficient of CUAV; ； Where: and Respectively expressed in t The external interference source at the moment m CUAV large-scale path loss and small-scale shadow fading; ； Where, Indicates the m The Euclidean distance between the CUAV and the external interference source, is the carrier frequency of the base station communicating with the CUAV, H u Indicates the flight altitude of CUAV; Indicates t Moment m The space-time coordinates of the CUAV.

3. The method for avoiding intelligent interference in networked UAV communications according to claim 2, characterized in that: The step S3 of establishing an uplink and downlink interference signal model according to the base station sector information, CUAV information and external interference source information specifically includes: Get the m CUAV downlink communication received the first k The signal-to-interference-and-noise ratio of the base station sector signal; ； Where: , Indicates the transmission power of the external interference source; is the base station transmit power, For the m The mean value received by the CUAV is 0 and the variance is Additive Gaussian white noise; represents the interference sum of other base station sectors, namely: ； Where: Indicates that the downlink is m The resource blocks allocated by the CUAV are Whether there are users in each base station sector that use the same frequency band for downlink communication with the connected drone; express Belong to other sectors in the sector set except the k-th sector; express t Time downlink m CUAV to the first Equivalent channel coefficients between base station sectors; Get the m The CUAV interferes with the uplink communication link of the ground co-channel users; ； Where: Indicates that the uplink is m The frequency band allocated to the CUAV is The probability that a base station sector is occupied; express t Time in uplink m CUAV to the first The equivalent channel coefficient between base station sectors.

4. The method for avoiding intelligent interference in networked UAV communications according to claim 3, characterized in that: The step S3 constructs a weighted sum minimization model of the interference intensity suffered by the downlink communication links of all CUAVs and the interference intensity caused by the uplink communication links based on the transmission signal model and the uplink and downlink interference signal model, specifically including: Definition m The duration of communication interruption of the CUAV is: ； Where: Indicates the m The last time the CUAV was connected to the ground base station, represents the SINR threshold of the CUAV downlink. When m CUAV and ground base station sectors k The downlink communication link is in a connected state; Definition m CUAV in t The base station sector connected at the moment is ,and , then the optimization problem of minimizing the weighted sum of the interference intensity suffered by the downlink communication links of all CUAVs and the interference intensity caused by the uplink communication links can be written as: ； in, are the weights of the interference received by all CUAV downlink communications and the interference caused by all CUAV uplink communication links; constraint C2 ensures the flight direction of CUAV The module length is unit 1; C3-C5 is the m The flight start point, end point and flight area constraints of the CUAV, Indicates the position tolerance for reaching a given target point; C6 constrains the duration of CUAV communication interruption; C7 ensures that CUAV is connected to a base station sector at all times; C8 ensures that there will be no collision between CUAVs. Indicates the minimum safe distance between CUAVs, is the preset time threshold; Indicates the coordinates of the CUAV's flight starting point; Indicates the m CUAV mission destination coordinates.

5. The method for avoiding intelligent interference in networked UAV communications according to claim 4, characterized in that: The optimization problem corresponding to the weighted sum minimization model of the discretization of the interference intensity suffered by the downlink communication links of all CUAVs and the interference intensity caused by the uplink communication links in step S4 specifically includes: The first m CUAV flight time Discretized into time slots, i.e. , the optimization problem after discretization is: ； in ; The first m The communication interruption duration of the CUAV is Indicates that the allowed CUAV interrupt duration is constrained by express; No. m CUAV in the n The position coordinates of each time slot.

6. The method for avoiding intelligent interference in networked UAV communications according to claim 5, characterized in that: In step S4, the discretized optimization problem is mapped into a partially observable Markov decision process to construct the state space, action space and reward function of the CUAV in each time slot, specifically including: Definition m CUAV in the first n The local observation space of time slots ,but M Global observation space of CUAV ; CUAV flight area size ; Definition m CUAV in the first n action space of time slots , Respectively indicate movement to the east, west, north, northeast, northwest, southeast, southwest and stop movement; M The action space of the CUAV is ; definition M CUAV in the global observation space When selecting the action space Global rewards obtained when ; Where: Represents the reward for all CUAVs flying to the finish line; Indicates the penalty for a CUAV flying out of the restricted area and the duration of communication interruption. Exceed punishment; represents the penalty for collision between CUAVs; CUAV does not reach the target point but chooses to perform the action punishment, of which , , and are all constants; Indicates the m CUAV in the first n The interference suffered by a time slot in the downlink communication link and the penalty caused by the uplink communication link to its co-channel users, that is, ； in and Respectively represent m The interference caused by the downlink communication between the CUAV and the ground base station and the uplink communication, and The weight for balancing uplink and downlink interference.

7. The method for avoiding intelligent interference in networked UAV communications according to claim 1, characterized in that: The step S5 specifically includes: Step S501: Acquire M The historical observation sequences of the CUAV are used as training samples; Step S502: Initialize the parameters of the QMIX network structure corresponding to the UCB-QMIX path planning algorithm, wherein the QMIX network structure includes individual networks and hybrid networks; Step S503: Input the historical observation sequence of each CUAV into the corresponding individual network, and use the corresponding individual network to calculate the original Q value of each action. An action counter is also maintained to record the number of times each action was selected in historical decisions. A confidence interval for each action is calculated based on the number of times each action was selected in historical decisions. Based on the original Q value and confidence interval of each action, a local Q value for each action is calculated, and the action with the largest local Q value is set as the action to be executed in the current time slot. Step S504: Input the maximum value of the local Q value corresponding to each action calculated by the individual networks corresponding to all CUAVs into the hybrid network to calculate the global Q value; at the same time, use the target network and the global observation space corresponding to the next time slot and the global reward of the current time slot to calculate the target Q value, and construct a loss function based on the global Q value and the target Q value; Step S505: Update the parameters of the individual network and the hybrid network by using the gradient descent method until the loss value of the constructed loss function is less than a preset value.

8. The method for avoiding intelligent interference in networked UAV communications according to claim 7, characterized in that: In step S503, the local Q value of each action is calculated based on the original Q value and confidence interval of each action, and the action with the largest local Q value is set as the action to be executed in the current time slot. The calculation formula is as follows: ； Where: ; is the exploration coefficient that decays with time slots, represents the local Q value of each action, represents the original Q value of each action, Indicates the number of times each action is selected in historical decisions; total number of decisions ; in .

9. An electronic device, characterized in that: include: memory and processor; The memory is used to store programs; The processor is used to call the program stored in the memory to execute the intelligent interference avoidance method for networked drone communications as described in any one of claims 1-8.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by the processor, it implements the intelligent interference avoidance method for networked drone communications as described in any one of claims 1 to 8.

Images