Spectrum resource allocation method and system for drone swarm based on cooperative spectrum sensing

Abstract

The present invention provides a spectrum resource allocation method and device for drone swarms based on cooperative spectrum sensing, which belongs to the field of wireless communication technology. The method includes: constructing a dynamic spectrum access network for drone swarms and creating a cooperative spectrum sensing unit; creating a federated deep reinforcement learning environment, initializing task parameters and reward functions; dynamic spectrum access users perform cooperative spectrum sensing to obtain available spectrum resources; training a dynamic spectrum access strategy model for drone swarms based on federated learning and deep reinforcement learning; calling the trained strategy model to complete the dynamic spectrum access of drone swarms. The present invention performs dynamic spectrum access for drone swarms based on cooperative spectrum sensing, realizes efficient use of spectrum resources between drone swarm clusters, alleviates the problem of shortage of spectrum resources for drone swarms, improves channel utilization efficiency and reduces interference levels, and promotes the transformation of drone swarm spectrum access to intelligence and precision.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a spectrum resource allocation method and system for a drone swarm based on cooperative spectrum sensing. Background Art

[0002] During data transmission, drone swarms have a strong demand for spectrum resources, so they need to dynamically access ground authorized frequency bands as dynamic spectrum access users to meet their own spectrum needs.

[0003] Before spectrum access, spectrum sensing is required to obtain spectrum holes and spectrum occupancy information. Energy detection, as a simple and low-cost sensing method, has been widely used in wireless communication networks. However, since path attenuation may cause serious fading of the primary user signal, it is difficult for a single sensing device based on energy detection to accurately detect whether the current channel is occupied. Cooperative spectrum sensing can be achieved through cooperative sensing by decentralized sensing devices, and then the sensing results are sent to the sensing center for collective decision-making, effectively improving the accuracy of spectrum sensing in the authorized frequency band. At present, some studies have applied cooperative spectrum sensing to the fields of Internet of Things and Internet of Vehicles. Xin Liu et al. proposed an integrated joint spectrum sensing and access control model to improve the transmission performance of cognitive industrial Internet of Things (CIIoT), while ensuring the detection probability of spectrum sensing and controlling interference to the primary user (PU). However, the joint spectrum sensing model proposed in the article needs to be jointly optimized, involving multiple variables such as sensing time, number of sensing nodes and transmit power, and has high computational complexity.

[0004] Xiaoying Liu et al. studied the problem of joint spectrum sensing in mobile energy harvesting cognitive radio networks. Auxiliary transmitters with sufficient energy participate in joint spectrum sensing and send local sensing decisions to the fusion center, which uses the general k-out-of-M(k) fusion rule to make the final decision on the spectrum status. However, current research lacks cooperative spectrum sensing for drone swarm networks, especially under the premise of drone swarm clustering. Drones can make full use of line-of-sight channels in the air to avoid being blocked by some low-altitude obstacles. Thanks to the large-scale deployment of drones in drone swarms, more comprehensive spectrum information can be obtained over a larger range.

[0005] Currently, the dynamic spectrum access problem is usually regarded as a Markov decision process or a partially observable Markov decision process. To solve this problem, the mainstream method is to use a reinforcement learning model. In addition:

[0006] Hu Hang designed a distributed architecture based on layered blockchain for spectrum access scenarios between multi-level heterogeneous users, and applied a bilateral auction mechanism to promote efficient transactions of spectrum resources. Although this method achieves user utility optimization and deep spectrum sharing, its application is limited by problems such as opaque bidding strategies and complex bidding processes.

[0007] Stefano Iellamo et al. proposed a distributed spectrum access protocol called SILP for multi-antenna wireless nodes to access multiple frequency channels. Through imitation learning, the nodes decide the number and specific channels to access in each iteration, and finally the network reaches an efficient and fair equilibrium state. However, the nodes in this method need to be able to access and aggregate multiple channels, which is difficult to achieve on some devices. Summary of the invention

[0008] The present invention provides a spectrum resource allocation method and system for a drone swarm based on cooperative spectrum sensing, so as to solve at least one defect in the prior art, namely, low spectrum resource utilization efficiency, insufficient spectrum sensing accuracy and poor dynamic adaptability, and realize efficient, accurate and intelligent spectrum resource allocation and management.

[0009] In a first aspect, the present invention provides a method for allocating spectrum resources of a drone swarm based on cooperative spectrum sensing, comprising:

[0010] Step S1: Building a drone swarm spectrum access network including several drone clusters, wherein each drone cluster consists of a cluster head drone and several cluster member drones, each drone cluster and ground frequency-using equipment together constitute a dynamic spectrum access unit, the ground frequency-using equipment acts as a primary user, and the drone acts as a dynamic spectrum access user to dynamically access the authorized spectrum of the primary user; creating a cooperative spectrum sensing unit in which cluster member drones perform local spectrum sensing and cluster head drones perform comprehensive processing of local spectrum sensing data;

[0011] Step S2: Create a federated learning framework with the air access platform as the aggregation distribution center and the cluster head drones as federation members, where each cluster head drone is equipped with a set of deep reinforcement learning algorithms and performs deep reinforcement learning model training as a cluster head agent;

[0012] Before model training begins, the number of dynamic access channels and channel access strategies are set according to task requirements and the number of authorized channels; the channel access strategies include: filling, underlay and hybrid; when the filling access strategy is adopted, dynamic spectrum access users can only access idle channels; when the underlay access strategy is adopted, dynamic spectrum access users choose to access occupied authorized channels; when the hybrid access strategy is adopted, dynamic spectrum access users choose to access idle channels and occupied authorized channels;

[0013] Set up the optimization problem: The cluster head UAV controls the transmission power by making decisions to access idle channels or use occupied channels to maximize the signal-to-interference-to-noise ratio of users accessing the dynamic spectrum;

[0014] Step S3: Dynamic spectrum access users perform cooperative spectrum sensing, cluster members act as decentralized sensing nodes, and cluster heads act as sensing centers to perform joint spectrum sensing to obtain available spectrum resources;

[0015] Step S4: The cluster head agent uses a deep reinforcement learning algorithm to train the model based on the federated learning framework and makes a channel access strategy based on the current local observation to solve the preset optimization problem. During the training process, if the cluster head agent fails to meet the needs of cluster members or interferes with the primary user, it will be punished.

[0016] Step S5: After the model training is completed, the cluster head UAV downloads the final version of the model from the air access platform and runs it locally to complete the dynamic spectrum access of the UAV swarm.

[0017] In a second aspect, the present invention further provides a spectrum resource allocation device for a drone swarm based on cooperative spectrum sensing, comprising:

[0018] The first processing module is configured to build a drone swarm spectrum access network including a plurality of drone clusters, wherein each drone cluster is composed of a cluster head drone and a plurality of cluster member drones, each drone cluster and ground frequency-using equipment together constitute a dynamic spectrum access unit, the ground frequency-using equipment serves as a primary user, and the drone serves as a dynamic spectrum access user to dynamically access the authorized spectrum of the primary user; a cooperative spectrum sensing unit is created in which the cluster member drones perform local spectrum sensing and the cluster head drone performs comprehensive processing of local spectrum sensing data;

[0019] The second processing module is configured to create a federated learning framework with the air access platform as an aggregation distribution center and the cluster head drones as federation members, wherein each cluster head drone is equipped with a set of deep reinforcement learning algorithms and performs deep reinforcement learning model training as a cluster head agent;

[0020] Before the training begins, the number of dynamic access channels and the channel access strategy are determined according to the task requirements and the number of authorized channels; the channel access strategies include: filling type, underlay type and hybrid type; when the filling type access strategy is adopted, the dynamic spectrum access user can only access the idle channel; when the underlay type access strategy is adopted, the dynamic spectrum access user chooses to access the occupied authorized channel; when the hybrid access strategy is adopted, the dynamic spectrum access user chooses to access the idle channel and the occupied authorized channel;

[0021] Set up the optimization problem: The cluster head UAV controls the transmission power by making decisions to access idle channels or use occupied channels to maximize the signal-to-interference-to-noise ratio of users accessing the dynamic spectrum;

[0022] The third processing module is configured to perform cooperative spectrum sensing for dynamic spectrum access users, with cluster members acting as decentralized sensing nodes and cluster heads acting as sensing centers, to perform joint spectrum sensing and obtain available spectrum resources;

[0023] The fourth processing module is configured as a cluster head agent, which uses a deep reinforcement learning algorithm to perform model training based on a federated learning framework, and makes a channel access strategy based on current local observations to solve a preset optimization problem; during the training process, if the cluster head agent fails to meet the needs of cluster members or interferes with the primary user, it will be punished;

[0024] The fifth processing module is configured such that after the model training is completed, the cluster head UAV downloads the final version of the model from the air access platform and runs it locally to complete the dynamic spectrum access of the UAV swarm.

[0025] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for allocating spectrum resources of a swarm of unmanned aerial vehicles based on cooperative spectrum sensing as described in any one of the above are implemented.

[0026] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described methods for allocating spectrum resources for a swarm of unmanned aerial vehicles based on cooperative spectrum sensing.

[0027] The method and device for allocating spectrum resources of a drone swarm based on cooperative spectrum sensing provided by the present invention have the following beneficial effects compared with the prior art:

[0028] (1) The present invention effectively reduces the false alarm probability and improves the detection probability through intra-cluster cooperative spectrum sensing, thus avoiding channel sensing errors caused by single sensing nodes due to channel fading and other reasons.

[0029] (2) The present invention designs a dynamic spectrum access algorithm based on multi-agent federated deep reinforcement learning, which combines federated learning with reinforcement learning to achieve efficient utilization of spectrum resources among drone clusters, alleviate the problem of shortage of spectrum resources in drone clusters, improve channel utilization efficiency and reduce interference levels, and promote the transformation of drone cluster spectrum access towards intelligence and precision. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 It is a flow chart of a spectrum resource allocation method for a drone swarm based on cooperative spectrum sensing provided by the present invention;

[0032] Figure 2 It is a schematic diagram of the framework of the spectrum access network of the drone swarm in the present invention;

[0033] Figure 3 is a flowchart of federated learning in the present invention;

[0034] Figure 4 This is an example of the simulation results of cooperative spectrum sensing under different numbers of drones in the present invention;

[0035] Figure 5 This is an example of simulation results of the signal to interference and noise ratio of dynamic spectrum access users under different numbers of federation members in the present invention.

[0036] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] It should be noted that in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The orientation or position relationship indicated by the terms "upper", "lower" and the like is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or it can be a connection between two elements. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0039] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0040] Combine the following Figure 1-Figure 6 The present invention describes a method and device for allocating spectrum resources of a drone swarm based on cooperative spectrum sensing provided in an embodiment of the present invention.

[0041] Figure 1 is a flow chart of a spectrum resource allocation method for a drone swarm based on cooperative spectrum sensing provided by the present invention, such as Figure 1 As shown, including but not limited to the following steps:

[0042] Step S1: Build a spectrum access network for drone swarms and create cooperative spectrum sensing units.

[0043] like Figure 2 As shown, build a Each cluster and its corresponding ground frequency-using equipment is called a dynamic spectrum access unit, including a transmitter and a receiver. There is a base station and receiving terminals, collectively referred to as primary users, is the set of primary user receiving terminals. The corresponding air distribution is The cluster member drones and the cluster head drone are collectively referred to as dynamic spectrum access users. It is a collection of dynamic spectrum access user transmitters. In addition, there is an air access platform in the sky. The air access platform establishes a stable communication link with the cluster head drone through a dedicated protection channel to exchange data and conduct command and control. During the mission execution, the cluster member drones need to dynamically access the authorized spectrum resources on the ground to transmit data back to the cluster head drone.

[0044] During the sensing period, cluster member drones perform local spectrum sensing, while cluster head drones perform comprehensive processing of local spectrum sensing data. Cluster member drones use energy detection to detect channel occupancy locally and make decisions, and then send the decision results to cluster head drones for joint decisions, and finally obtain joint spectrum sensing results.

[0045] Step S2: Create a federated deep reinforcement learning environment, determine the spectrum access mode, and set the optimization problem.

[0046] Create a federated learning framework with the air access platform as the aggregation and distribution center and the cluster heads as federation members. Each cluster head is equipped with a set of deep reinforcement learning algorithms and acts as an intelligent agent to train the deep reinforcement learning model. First, the air access platform initializes the deep reinforcement learning model and sends it to each federation member; then, the federation member trains the model locally based on local observations; after the local training is completed, the federation member uploads the local model parameters to the air access platform; after receiving the model parameters uploaded by each member, the air access platform fuses the model parameters to obtain the global model, and sends it to each federation member for training again until the model converges.

[0047] like Figure 3 As shown in the figure, the specific process of the federated learning strategy gradient descent algorithm is as follows:

[0048] 1) Model initialization

[0049] Initialize the global model , local model training learning rate , global model training learning rate , maximum number of iterations , the number of local gradient updates .

[0050] 2) Global model download

[0051] In the In iterations, each agent downloads the global model as the current local model:

[0052] ;

[0053] in, is the set of intelligent agents, represents the current local model, Represents the current global model.

[0054] 3) Local model update

[0055] In the In this update, the agent Based on the current state Take Action Get rewards , and then the experience sample Store to Cache Then, the agent Update the policy network using gradient descent:

[0056] ;

[0057] in, and Respectively indicate that and After the local model is updated, represents the gradient descent function.

[0058] 4) Local model upload

[0059] Model updated locally After that, the local policy network Upload to the central controller.

[0060] 5) Global model update

[0061] Central controller according to Local models are used to update the global model:

[0062] .

[0063] Before the formal training begins, the number of dynamic access channels and the channel access strategy need to be determined according to the mission requirements and the number of authorized channels. To ensure the quality of data transmission, the cluster head UAV needs to assign an orthogonal channel to each cluster member UAV in the cluster to ensure that there is no interference between UAVs in the cluster. Assume that the dynamic spectrum access user, i.e., the UAV, accesses the spectrum of the main user through three channel access strategies: filling, substrate, and hybrid. Among them, when the filling access strategy is adopted, the dynamic spectrum access user can only dynamically access the idle channel for data transmission; when the substrate access strategy is adopted, the dynamic spectrum access user can perform data transmission on the occupied authorized channel, but the transmission power needs to be controlled to ensure that the signal-to-interference-to-noise ratio of the main user is not less than the set threshold; when the hybrid access strategy is adopted, the dynamic spectrum access user can access both idle channels and occupied channels, but the transmission power of the dynamic spectrum access user needs to be controlled to avoid unacceptable interference to the main user.

[0064] In order to measure the spectrum access effect, the present invention uses the signal to interference noise ratio of the dynamic spectrum access user and the primary user as a measurement indicator. , any dynamic spectrum access unit , for any channel , considering dynamic spectrum access users The average signal-to-interference-noise ratio on the assigned channel can be expressed as:

[0065]

[0066] in, represents the assigned channel set, Indicates the primary user Is the channel occupied? , Indicates the primary user The transmitter is in the channel The transmission power on Indicates dynamic spectrum access users The transmitter is in the channel The transmit power on represents the background noise power, For dynamic spectrum access users The channel bandwidth, is the noise power spectral density, Indicates the number of transmitters of dynamic spectrum access users, is the path fading of the communication channel from the base station to the primary user, The channel between the base station and the primary user The channel gain on For dynamic spectrum access users With the main user In the channel The channel gain on Dynamic spectrum access users With the main user The path attenuation between them.

[0067] On the other hand, at the moment , for the primary user receiving end Any user , considering the possible maximum co-channel interference, its signal-to-interference-to-noise ratio can be expressed as:

[0068] ;

[0069] in, Indicates dynamic spectrum access users Is the channel occupied? .

[0070] In the present invention, the cluster head UAV controls the transmission power by making decisions to access idle channels or use occupied channels to maximize the signal-to-interference-to-noise ratio of dynamic spectrum access users. Therefore, the optimization problem can be described as:

[0071] The optimization goal is: ;

[0072] The set of constraints includes:

[0073] ;

[0074] ;

[0075] ;

[0076] in, represents the average signal to interference and noise ratio of the dynamic spectrum access user on the assigned channel in the dynamic spectrum access unit m at time t; represents the channel selection after spectrum decision, where Indicates that a dynamic spectrum access user selects an access channel , Indicates not to access the channel; The number of channels available after spectrum decision; Power decision after spectrum decision; represents the transmission power of the dynamic spectrum access user on the assigned channel at time t; Indicates the maximum transmit power of a user accessing the dynamic spectrum; represents the signal-to-interference-noise ratio of primary user j at time t; represents the minimum signal to interference and noise ratio allowed for primary user j; T represents the time slot; n represents the number of time slots; m represents; M represents the number of dynamic spectrum access units.

[0077] Step S3: Dynamic spectrum access users perform cooperative spectrum sensing, cluster members act as decentralized sensing nodes, and cluster heads act as sensing centers to perform joint spectrum sensing to obtain available spectrum resources.

[0078] In the perception stage, cluster member drones perform spectrum perception on each channel. To reduce the amount of spectrum perception information data transmission, cluster member drones first make a local spectrum occupancy judgment, using 1 to represent that the channel is occupied and 0 to represent that the channel is not occupied. The spectrum perception process of cluster member drones can be regarded as a binary detection problem, namely:

[0079] ;

[0080] in, represents the total sample size, Indicates drone The sampled signal samples are represents the sensing channel, Indicates whether the channel is occupied. is the received primary user signal strength, is the environmental noise. Based on a large amount of signal sampling data, the accumulated signal strength It can be expressed as:

[0081] ;

[0082] In order to reduce the computational complexity, the cluster member drones use the energy detection method to make local channel occupancy judgments, that is, to judge the accumulated signal strength Is it greater than the perception threshold? :

[0083] ;

[0084] Assumptions is large enough, then It roughly obeys the Gaussian distribution, that is:

[0085] ;

[0086] in, is the signal-to-noise ratio of the primary user, is the noise power. On this basis, the UAV The detection probability and false alarm probability can be expressed as:

[0087] ;

[0088] ;

[0089] in, .

[0090] After the cluster member drones make a local channel occupancy judgment, they send the channel perception information to the cluster head drone through a dedicated channel. The cluster head drone then performs data fusion according to the "or" criterion, that is, if at least one cluster member drone judges that the channel If the channel is occupied, the cluster head drone considers the channel to be occupied. Otherwise, if no cluster member drone determines that the channel is occupied, the cluster head drone determines that the channel is idle. Therefore, the detection probability and false alarm probability of intra-cluster cooperative spectrum sensing can be expressed as:

[0091] ;

[0092] ;

[0093] in, express The number of launching drones.

[0094] Step S4: Based on the federated learning framework, the cluster head agent uses a deep reinforcement learning algorithm to perform model training to solve the pre-set optimization problem.

[0095] The present invention adopts the existing SAC algorithm for model training. The model in the present invention includes two fully connected layers and one hidden layer. The model takes the spectrum sensing information of the drone as input and outputs the channel occupancy status and transmission power.

[0096] In the framework of federated learning, cluster head drones, which are dynamic spectrum access users in the local learning process, are equipped with a set of deep reinforcement learning algorithms and are regarded as cluster head agents in deep reinforcement learning training. Cluster head agents need to make spectrum decisions based on current local observations to ensure that they find enough accessible channels. If they do not fully meet the needs of cluster members or cause unacceptable interference to primary users, they will be punished.

[0097] Assume that at time As described in step S1, the dynamic spectrum access unit The space in which you are located Dynamic spectrum access users, The unit has a main user receiving end. The occupancy status of an authorized channel can be expressed as: ,in, Indicates channel is occupied, and Indicates channel In idle state. At this time, the action of the cluster head UAV agent can be expressed as: ,in, represents the channel selection of the agent, Indicates that the agent chooses to occupy the channel ,and Indicates that the agent does not occupy the channel. represents the channel power decision of the agent, , is the maximum transmit power of the UAV. At this point, the cluster head UAV needs to make spectrum decisions based on the current local observations to ensure that an accessible channel is found for each cluster member UAV.

[0098] The three channel access strategies can be specifically described as:

[0099] (1) In the filling access strategy, dynamic spectrum access users can only access idle channels and cannot access occupied channels to avoid affecting the primary user or other dynamic spectrum access users. In this mode, the channel power decision of the agent is not considered, and the maximum power is used by default to ensure that the number of access channels required by cluster member drones is met, and the occupation of busy channels is punished.

[0100] Therefore, the reward function in the filling scenario is designed as:

[0101]

[0102] in is the channel resource difference, which represents the difference between the number of allocated channels and the number of required channels, and encourages the agent to find more idle channels for the drone cluster. represents the channel selection after the agent makes the decision, Indicates that the agent chooses to occupy the channel ,and Indicates that the agent does not occupy the channel. Indicates the conflict between the channel decision made by the agent based on the current channel perception information and the occupied channel. If the current channel is already occupied and the channel is still selected according to the decision made by the agent, then , is a discount factor, which usually takes a negative value to punish the current decision.

[0103] (2) In the underlay access strategy, dynamic spectrum access users can dynamically access occupied channels without causing serious interference to primary users. In this mode, the agent needs to make comprehensive decisions on channel selection and maximum transmit power planning to meet the number of access channels required by dynamic spectrum access users, and while encouraging the maximum signal-to-interference-to-noise ratio, punish potential interference to primary users.

[0104] Therefore, the reward function in the substrate scenario is designed as:

[0105] ;

[0106] in, To penalize the primary user for interference, Signal-to-interference-noise ratio reward for dynamic spectrum access users, , are discount factors, is a step function, specifically:

[0107] ;

[0108] (3) In the hybrid access strategy, dynamic spectrum access users can access any idle channel. At the same time, they can access occupied channels by controlling power without causing serious interference to the primary user. This is equivalent to making power decisions for all channels, encouraging the agent to allocate as much transmit power as possible to all channels.

[0109] Therefore, the reward function in the hybrid scenario is designed as:

[0110] ;

[0111] Step S5: After the model training is completed, the cluster head UAV downloads the final version of the model from the air access platform and runs it locally to complete the dynamic spectrum access of the UAV swarm.

[0112] After training, each drone cluster conducts cooperative spectrum sensing to obtain the current channel occupancy and inputs it into the cluster head agent as environmental observation information. The cluster head drone downloads the final version of the model from the air access platform and runs the intelligent algorithm locally. Based on the spectrum sensing results, it dynamically and autonomously accesses idle spectrum resources to achieve efficient spectrum utilization.

[0113] The effect of the present invention can be further illustrated by simulation:

[0114] 1. Simulation conditions: Consider a spectrum access scenario based on drone swarm clustering, with a total of spectrum access units. For any unit , the length and width are , the number of cluster member drones is , the number of cluster head drones is 1, and the number of primary users is , the number of base stations is 1. The cluster member drones are randomly and evenly distributed relative to the horizontal plane, and the flight heights are randomly distributed in The cluster heads are located at the mass center of the cluster member drones relative to the horizontal plane, and the flight altitude is fixed at , base stations are distributed in units The primary users are randomly and evenly distributed at the center of the base station. The base station transmission power is , the channel bandwidth is The maximum transmission power of the cluster member drone is , the channel bandwidth is The other parameters are set as follows: the background noise spectral density is , .

[0115] 2. Simulation content: The spectrum resource allocation method and system of UAV swarm based on cooperative spectrum sensing of the present invention are adopted. Firstly, the detection probability-false alarm probability ROC curve of the cooperative spectrum sensing algorithm under different numbers of sensing members is analyzed. The results Figure 4 As shown, the present invention verifies the signal-to-interference-to-noise ratio simulation results of dynamic spectrum access users in the substrate access mode. Figure 5 shown.

[0116] On the other hand, the present invention provides a spectrum resource allocation system for a drone swarm based on cooperative spectrum sensing, comprising:

[0117] The first processing module is configured to build a drone swarm spectrum access network including a plurality of drone clusters, wherein each drone cluster is composed of a cluster head drone and a plurality of cluster member drones, each drone cluster and ground frequency-using equipment together constitute a dynamic spectrum access unit, the ground frequency-using equipment serves as a primary user, and the drone serves as a dynamic spectrum access user to dynamically access the authorized spectrum of the primary user; a cooperative spectrum sensing unit is created in which the cluster member drones perform local spectrum sensing and the cluster head drone performs comprehensive processing of local spectrum sensing data;

[0118] The second processing module is configured to create a federated learning framework with the air access platform as an aggregation distribution center and the cluster head drones as federation members, wherein each cluster head drone is equipped with a set of deep reinforcement learning algorithms and performs deep reinforcement learning model training as a cluster head agent;

[0119] Before the training begins, the number of dynamic access channels and the channel access strategy are determined according to the task requirements and the number of authorized channels; the channel access strategies include: filling type, underlay type and hybrid type; when the filling type access strategy is adopted, the dynamic spectrum access user can only access the idle channel; when the underlay type access strategy is adopted, the dynamic spectrum access user chooses to access the occupied authorized channel; when the hybrid access strategy is adopted, the dynamic spectrum access user chooses to access the idle channel and the occupied authorized channel;

[0120] Set up the optimization problem: The cluster head UAV controls the transmission power by making decisions to access idle channels or use occupied channels to maximize the signal-to-interference-to-noise ratio of users accessing the dynamic spectrum;

[0121] The third processing module is configured to perform cooperative spectrum sensing for dynamic spectrum access users, with cluster members acting as decentralized sensing nodes and cluster heads acting as sensing centers, to perform joint spectrum sensing and obtain available spectrum resources;

[0122] The fourth processing module is configured as a cluster head agent, which uses a deep reinforcement learning algorithm to perform model training based on a federated learning framework, and makes a channel access strategy based on current local observations to solve a preset optimization problem; during the training process, if the cluster head agent fails to meet the needs of cluster members or interferes with the primary user, it will be punished;

[0123] The fifth processing module is configured such that after the model training is completed, the cluster head UAV downloads the final version of the model from the air access platform and runs it locally to complete the dynamic spectrum access of the UAV swarm.

[0124] It should be noted that the drone swarm spectrum resource allocation system based on cooperative spectrum sensing provided in an embodiment of the present invention can execute the drone swarm spectrum resource allocation method based on cooperative spectrum sensing described in any of the above embodiments during specific operation, which will not be elaborated in this embodiment.

[0125] Figure 6 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the spectrum resource allocation method for the drone swarm based on cooperative spectrum sensing.

[0126] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the method for allocating spectrum resources for a swarm of drones based on cooperative spectrum sensing provided in the above-mentioned embodiments.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A spectrum resource allocation method for drone swarms based on cooperative spectrum sensing, characterized in that: include: Step S1: Building a drone swarm spectrum access network including several drone clusters, wherein each drone cluster consists of a cluster head drone and several cluster member drones, each drone cluster and ground frequency-using equipment together constitute a dynamic spectrum access unit, the ground frequency-using equipment acts as a primary user, and the drone acts as a dynamic spectrum access user to dynamically access the authorized spectrum of the primary user; creating a cooperative spectrum sensing unit in which cluster member drones perform local spectrum sensing and cluster head drones perform comprehensive processing of local spectrum sensing data; Step S2: Create a federated learning framework with the air access platform as the aggregation distribution center and the cluster head drones as federation members, where each cluster head drone is equipped with a set of deep reinforcement learning algorithms and performs deep reinforcement learning model training as a cluster head agent; Before model training begins, the number of dynamic access channels and channel access strategies are set according to task requirements and the number of authorized channels; the channel access strategies include: filling, underlay and hybrid; when the filling access strategy is adopted, dynamic spectrum access users can only access idle channels; when the underlay access strategy is adopted, dynamic spectrum access users choose to access occupied authorized channels; when the hybrid access strategy is adopted, dynamic spectrum access users choose to access idle channels and occupied authorized channels; Set up the optimization problem: The cluster head UAV controls the transmission power by making decisions to access idle channels or use occupied channels to maximize the signal-to-interference-to-noise ratio of users accessing the dynamic spectrum; Step S3: Dynamic spectrum access users perform cooperative spectrum sensing, cluster members act as decentralized sensing nodes, and cluster heads act as sensing centers to perform joint spectrum sensing to obtain available spectrum resources; Step S4: The cluster head agent uses a deep reinforcement learning algorithm to train the model based on the federated learning framework and makes a channel access strategy based on the current local observation to solve the preset optimization problem. During the training process, if the cluster head agent fails to meet the needs of cluster members or interferes with the primary user, it will be punished. Step S5: After the model training is completed, the cluster head UAV downloads the final version of the model from the air access platform and runs it locally to complete the dynamic spectrum access of the UAV swarm.

2. The method for allocating spectrum resources of drone swarms based on cooperative spectrum sensing according to claim 1 is characterized in that: The model training process based on the federated learning framework includes: The air access platform initializes the deep reinforcement learning model and sends it to each federation member; The federation members train the model locally based on local observations. After the local training is completed, the federation members upload the local model parameters to the air access platform. After receiving the model parameters uploaded by each federation member, the air access platform integrates the model parameters to generate a global model, and then sends it to each federation member for training again until the model converges.

3. According to the method for allocating spectrum resources of drone swarms based on cooperative spectrum sensing according to claim 1, step S1 comprises: Build a A drone swarm of drone clusters; Each drone cluster and its corresponding ground frequency-using equipment is called a dynamic spectrum access unit, including a transmitting end and a receiving end; Among them, any dynamic spectrum access unit There is a base station and receiving terminals, collectively referred to as primary users, is a collection of primary user receiving terminals; The air distribution The cluster member drones and the cluster head drone are collectively referred to as dynamic spectrum access users. It is a collection of dynamic spectrum access user transmitters.

4. The method for allocating spectrum resources of drone swarms based on cooperative spectrum sensing according to claim 3 is characterized in that: Set up the optimization problem, including: The optimization goal is: ; The set of constraints includes: ； ； ； in, Indicates at time t , Dynamic Spectrum Access Unit m , the average signal-to-interference-and-noise ratio of the dynamic spectrum access user on the assigned channel; It is represented as the occupied channel after spectrum decision, where Indicates that a dynamic spectrum access user selects an access channel , Indicates not to access the channel; The number of channels available after spectrum decision; Power decision after spectrum decision; Indicates at time t The transmit power of dynamic spectrum access users on the assigned channels; Indicates the maximum transmit power of a user accessing the dynamic spectrum; Indicates at time t Primary User j Signal-to-interference-noise ratio; Indicates the primary user j Minimum allowed signal-to-interference-noise ratio; T Indicates time slot; n Indicates the number of time slots; m express; M Indicates the number of dynamic spectrum access units.

5. The method for allocating spectrum resources of drone swarms based on cooperative spectrum sensing according to claim 1 is characterized in that: Step S3 includes: Cluster member drones use energy detection to make local channel occupancy judgments and determine whether the channel is occupied based on whether the signal strength exceeds the threshold; If at least one cluster member drone determines that the channel is occupied, the cluster head drone considers that the channel is occupied; otherwise, if none of the cluster member drones determines that the channel is occupied, the cluster head drone determines that the channel is idle.

6. The method for allocating spectrum resources of drone swarms based on cooperative spectrum sensing according to claim 4 is characterized in that: For the filling access strategy, the reward function Designed for: ； in, is the channel resource difference, which represents the difference between the number of allocated channels and the number of required channels, and encourages the agent to find more idle channels for the drone cluster; It indicates the conflict between the channel decision made by the agent based on the current channel perception information and the occupied channel.

7. The method for allocating spectrum resources of a drone swarm based on cooperative spectrum sensing according to claim 4 is characterized in that: For the substrate access strategy, the reward function Designed for: ； in, is the poor channel resource, To penalize the primary user for interference, Signal-to-interference-noise ratio reward for dynamic spectrum access users, , are discount factors, is a step function; Indicates the channel selection of dynamic spectrum access users.

8. The method for allocating spectrum resources of a drone swarm based on cooperative spectrum sensing according to claim 4 is characterized in that: For the hybrid access strategy, the reward function Designed for: ； in, To penalize the primary user for interference, Signal-to-interference-noise ratio reward for dynamic spectrum access users, is the discount factor, is a step function.

9. A spectrum resource allocation system for drone swarms based on cooperative spectrum sensing, characterized in that: include: The first processing module is configured to build a drone swarm spectrum access network including a plurality of drone clusters, wherein each drone cluster is composed of a cluster head drone and a plurality of cluster member drones, each drone cluster and ground frequency-using equipment together constitute a dynamic spectrum access unit, the ground frequency-using equipment serves as a primary user, and the drone serves as a dynamic spectrum access user to dynamically access the authorized spectrum of the primary user; a cooperative spectrum sensing unit is created in which the cluster member drones perform local spectrum sensing and the cluster head drone performs comprehensive processing of local spectrum sensing data; The second processing module is configured to create a federated learning framework with the air access platform as an aggregation distribution center and the cluster head drones as federation members, wherein each cluster head drone is equipped with a set of deep reinforcement learning algorithms and performs deep reinforcement learning model training as a cluster head agent; Before the training begins, the number of dynamic access channels and the channel access strategy are determined according to the task requirements and the number of authorized channels; the channel access strategies include: filling type, underlay type and hybrid type; when the filling type access strategy is adopted, the dynamic spectrum access user can only access the idle channel; when the underlay type access strategy is adopted, the dynamic spectrum access user chooses to access the occupied authorized channel; when the hybrid access strategy is adopted, the dynamic spectrum access user chooses to access the idle channel and the occupied authorized channel; Set up the optimization problem: The cluster head UAV controls the transmission power by making decisions to access idle channels or use occupied channels to maximize the signal-to-interference-to-noise ratio of users accessing the dynamic spectrum; The third processing module is configured to perform cooperative spectrum sensing for dynamic spectrum access users, with cluster members acting as decentralized sensing nodes and cluster heads acting as sensing centers, to perform joint spectrum sensing and obtain available spectrum resources; The fourth processing module is configured as a cluster head agent, which uses a deep reinforcement learning algorithm to perform model training based on a federated learning framework, and makes a channel access strategy based on current local observations to solve a preset optimization problem; during the training process, if the cluster head agent fails to meet the needs of cluster members or interferes with the primary user, it will be punished; The fifth processing module is configured such that after the model training is completed, the cluster head UAV downloads the final version of the model from the air access platform and runs it locally to complete the dynamic spectrum access of the UAV swarm.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method for allocating spectrum resources of a drone swarm based on cooperative spectrum sensing as described in any one of claims 1 to 8 are implemented.

Images