A hybrid intelligent spectrum management method in unmanned system cluster network

Abstract

The present invention discloses a hybrid intelligent spectrum management method in an unmanned system cluster network. The method constructs a weighted interference graph model, and based on the measurement information and communication performance feedback of the nodes, adopts a reinforcement learning method, can autonomously optimize and adjust the working channel of the network, and reduce the frequency conflicts within the system and the adverse effects of interference outside the system. The method uses the communication performance actually perceived by the communication node as feedback to guide spectrum decision-making, can adapt to complex and dynamic spectrum environments and wireless channel environments, and can support large-scale unmanned system networking communication spectrum management. It proposes a hybrid optimization structure that combines centralized optimization (service data network channel selection) with distributed optimization (radio access network channel selection based on reinforcement learning), has learning ability, and has low algorithm complexity.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned system cluster communication, and more specifically, to a hybrid intelligent spectrum management method in an unmanned system cluster network. Background Art

[0002] A large number of unmanned systems will perform various tasks by forming clusters, which will become an important working mode in certain specific fields in the future. In order to successfully complete communication tasks in certain specific fields, unmanned systems need to make reasonable use of wireless spectrum. Their use of spectrum mainly includes three types of frequency-using equipment: First, communication equipment, which supports the transmission of control commands from the control station to the unmanned system, the real-time return of high-speed data reconnaissance information of the unmanned system, and the real-time command and control of the unmanned system. It has the largest spectrum demand and is the key to successfully completing the task. Second, sensor equipment, which provides communication and data transmission support for navigation positioning, target indication, and friend-or-foe identification, mainly including satellite navigation, synthetic aperture radar, mobile target indicator, friend-or-foe identification, and remote sensing detectors. Third, vehicle / aircraft / shipborne electronic countermeasure equipment, which includes jamming equipment and passive jamming equipment to suppress and interfere with enemy electronic systems or conduct electronic deception. On the other hand, unmanned system clusters generally work in harsh environments, with complex terrain and wireless channel characteristics and the threat of enemy interference signals, which makes the use of frequency by unmanned system clusters face many challenges.

[0003] Coordinating frequency conflicts within unmanned system clusters and adapting to complex external spectrum environments requires efficient and reliable spectrum management. However, existing spectrum management methods for unmanned systems are mainly based on simple fixed allocation, which is difficult to adapt to dynamic and complex environments and large-scale cluster communication networks. Some literature has proposed a spectrum allocation method for unmanned aerial vehicles with joint task allocation, which regards unmanned aerial vehicles that complete the same task as the same alliance. UAVs within the alliance share the same channel, while different channels are used between task alliances. In order to improve the utilization of spectrum, some literature has also shared some overlapping channels in cluster-based unmanned aerial vehicle networks. In order to overcome the influence of uncertain state information caused by channels and unmanned aerial vehicle movement, a distributed channel distribution algorithm is designed using fuzzy decision-making and game learning algorithms. Related literature proposes to use millimeter wave frequency bands in unmanned aerial vehicle-based cellular networks. In order to alleviate the interference between base stations and between base stations and wireless backhaul links, a 3D interference graph model is constructed. The spectrum allocation is modeled as a continuous time period optimization problem combined with constraints such as unmanned aerial vehicle mobility, energy consumption and interference. The above-mentioned methods provide references for spectrum management of unmanned system clusters, but they are all based on theoretical analysis, lack actual protocols and network verification, and are difficult to adapt to the application scenarios of large-scale cluster networks. Summary of the invention

[0004] In response to at least one defect or improvement need in the prior art mentioned in the background technology section, the present invention provides a hybrid intelligent spectrum management method in an unmanned system cluster network, so as to overcome the technical defects of the prior art that lacks actual protocols and network verification and is difficult to adapt to application scenarios of large-scale cluster communication networks.

[0005] To achieve the above object, the present invention provides a hybrid intelligent spectrum management method in an unmanned system cluster network, comprising:

[0006] S1. Constructing an external weighted interference relationship matrix to measure the number of non-system WiFi networks on each channel; the external weighted interference relationship matrix includes a matrix for a service data network and a matrix for a wireless access network;

[0007] S2, selecting a channel that minimizes the actual weighted interference received by the service data network on each channel as the channel of the service data network;

[0008] S3, initializing channel selection of the wireless access network;

[0009] S4, constructing a weighted interference relationship matrix within the system to describe the potential interference relationship between wireless switching nodes within the system;

[0010] S5. In the current time slot, the average link quality is updated according to the WiFi signals of each channel detected by the received terminal;

[0011] S6. Update the interference distribution vector and obtain the actual weighted interference of the wireless access network, and update the average link quality estimation value of the wireless access network based on the actual weighted interference of the wireless access network and the average link quality;

[0012] S7. At the beginning of the next time slot, updating the channel selection of the wireless access network according to the probability rule;

[0013] S8. If the channel of the service data network needs to be readjusted or the network status changes, jump to step S1; otherwise, jump to step S4.

[0014] Further, the formula for selecting the channel that minimizes the actual weighted interference received by the service data network on each channel as the channel of the service data network includes:

[0015]

[0016]

[0017] Among them, I 0 (c) represents the actual weighted interference received by the service data network on each channel, a 0represents the channel of the service data network, u n，c Indicates the number of WiFi networks outside the system that exist on channel c detected by the wireless switching node n itself, as shown in δ(a 1 , a 2 ) is the channel interference function, n represents the number of the wireless switching node, and c represents the number of the wireless communication channel available to the system.

[0018] Further, the initializing the channel selection of the wireless access network specifically includes:

[0019] All wireless switching nodes randomly select channels from channels outside the protection channel set of the service data network.

[0020] Furthermore, the specific definition of the protection channel set of the service data network is:

[0021]

[0022] Among them, C 0 represents the protection channel set of the service data network, and the wireless communication channel set available to the system is Indicates the channel used by the business data network.

[0023] Further, the average link quality is obtained by defining the link signal quality to calculate;

[0024] For each single-hop wireless communication link, the link signal quality is defined as quality*(A+signal), where quality and signal represent the current communication link parameters detected by WiFi respectively, quality is a score not exceeding 1, and the larger the score, the better the signal; signal represents the signal strength in dBm, which is usually negative, and the preset positive number A is added to ensure that (A+signal) is a positive value; each wireless switching node calculates the average link quality of its link with the connected terminal, that is, the average link quality of the wireless access network link.

[0025] Furthermore, the formula for updating the interference distribution vector and obtaining the actual weighted interference of the wireless access network specifically includes:

[0026]

[0027] Among them, L n = {l n (c)} is the interference distribution vector of wireless access network n, I n is the actual weighted interference of the wireless access network, a n is the channel used by the wireless access network of wireless switching node n, which is in the form of δ(a 1 , a2 ) is the channel interference function.

[0028] Furthermore, the channel interference function δ(a 1 , a 2 ) is:

[0029]

[0030] Further, the formula for updating the average link quality estimation value of the wireless access network based on the actual weighted interference and the average link quality of the wireless access network specifically includes:

[0031]

[0032] in, represents the average link quality, e n (a n ) represents the average link quality estimation value of the wireless access network, T(a n ) represents the selected channel variable a n The total number of times selected by wireless access network n.

[0033] Furthermore, the formula of the probability rule specifically includes:

[0034] a n for

[0035] Among them, E n ={e n (c)} is the average link quality estimation value when the wireless switching node n uses the channel c, and ε is a preset probability value that decreases as the number of time slots increases.

[0036] Furthermore, the formula for the preset probability value specifically includes:

[0037]

[0038] Among them, B is (0,1) and t is the current number of time slots.

[0039] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0040] The present invention provides a hybrid intelligent spectrum management method in an unmanned system cluster network, which uses the communication performance actually perceived by the communication node as feedback to guide spectrum decision-making, can adapt to complex and dynamic spectrum environments and wireless channel environments, and can support large-scale unmanned system networking communication spectrum management. It proposes a hybrid optimization structure that combines centralized optimization (service data network channel selection) with distributed optimization (radio access network channel selection based on reinforcement learning), has learning capabilities and low algorithm complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 A flow chart of a hybrid intelligent spectrum management method in an unmanned system cluster network provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0044] The terms "including" or "having" and any variations thereof in the specification, claims or drawings of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0045] Aiming at the future large-scale unmanned system cluster application scenario, the present invention proposes a hybrid spectrum management solution based on the layered wireless mesh network architecture. It constructs a weighted interference graph model, based on the measurement information and communication performance feedback of the nodes, and adopts a reinforcement learning method, which can autonomously optimize and adjust the working channel of the network, reduce the frequency conflicts within the system and the adverse effects of interference outside the system. Through the hybrid intelligent spectrum management method of the present invention, the unmanned system cluster can autonomously optimize the working channel and enhance the network's ability to adapt to complex confrontation environments.

[0046] Some prerequisites for the method of the present invention to work:

[0047] (1) Hierarchical wireless mesh network structure. To support the communication needs of large-scale unmanned system clusters, a two-layer network is used to organize various communications of the unmanned system cluster. Some unmanned system nodes are selected as wireless mesh nodes (called wireless switching nodes). The wireless switching nodes work in the same frequency band and can build a wireless backbone network through multi-hop. This layer of network is called the business data network. The wireless switching nodes each provide a wireless local area network. Other unmanned systems in the cluster (called wireless terminal nodes) select the wireless local area network with the strongest signal to access the network and realize wireless networking communication. This layer of network is called the wireless access network.

[0048] (2) Random access mode: Both the service data network and the wireless access network adopt the IEEE802.11 wireless LAN mode based on random multiple access.

[0049] (3) Wireless control link: A physically independent control layer network is built between wireless switching nodes, which has low-speed reliable transmission capabilities, collects and processes wireless communication status information, and sends centralized spectrum management results.

[0050] (4) Controller: The entity that performs centralized spectrum management can be a wireless switching node or a ground control station, which manages the unmanned system cluster communication spectrum through a wireless control link.

[0051] Definitions of some terms and variables:

[0052] (1) Basic variables. The set of M terminal nodes is N wireless switching nodes The set of wireless communication channels available to the system

[0053] (2) Channel selection variables. Indicates the channels used by the business data network, and defines the protection channel set of the business data network as The channel used by the wireless access network of wireless switching node n is denoted as a n In order to reduce the interference between the wireless access network and the service data network, the channel selection of the wireless switching node is after the service data network channel is determined and protected, that is, the channel can only be randomly selected from the protection channel set. That is to ensure sufficient channel spacing. Let the channel set selected by the wireless switching node be vector a = (a 1 , ..., a N ).

[0054] (3) Estimated variables related to link quality. For each single-hop wireless communication link, the link signal quality is defined as quality*(A+signal). When A is 200, it is quality*(200+signal), where quality and signal represent the current communication link parameters detected by WiFi. Quality is a score not exceeding 1, and the larger the score, the better the signal. Signal represents the signal strength in dBm, which is usually negative. The 200 added here ensures that (200+signal) is a positive value. Each wireless switching node calculates the average link quality of its link with the connected terminal, that is, the average link quality of the wireless access network link. Define and initialize the scale as The all-zero vector E n ={e n (c)} is the average link quality estimate when the wireless switching node n uses channel c. Note that since the protection channel set of the service data network is not selected here, it is necessary to subtract Each element e n，c It represents the estimated value of the average link quality when the wireless switching node n adopts the channel c, which is mainly used to guide the update of the service data network channel. The relationship between the two is shown in the following formula (5).

[0055] (4) Interference distribution vector. Define and initialize the scale as The all-zero vector L n = {l n (c)} is the interference distribution vector of wireless access network n, which is used to represent the number of WiFi networks inside and outside the system on each channel detected by the terminal. Define and initialize the scale as The all-zero vector I n is the weighted interference, I n With L n The relationship is shown in the following formula (3).

[0056] (5) Channel selection history variables. Each wireless switching node defines and initializes the scale as The all-zero vector T n =(0, ..., 0), recording the total number of times the node selects each channel in the wireless access network up to the current time slot.

[0057] The present invention takes time T as a cycle, and all wireless terminal nodes update local status information. At the same time, the controller updates all kinds of information collected in the entire network, including topology information, node information, wireless link information and service information, and maintains the information storage of the latest K cycles. Based on the detected wireless link information, a reinforcement learning mechanism is used to periodically adjust the working channels of the service data network and the wireless access network. The specific wireless link information includes: the ID, channel, power, quality, signal strength, bandwidth and delay of the wireless access network currently accessed by the terminal; the access network of other switching node networks detected by the terminal and other identifiable network-related information in the surrounding environment, such as ID, channel, power, quality and signal strength.

[0058] like Figure 1 As shown, in one embodiment, a hybrid intelligent spectrum management method in an unmanned system cluster network is provided, and the method mainly includes the following steps:

[0059] Step 1: Construct a weighted interference matrix outside the system to measure the number of non-system WiFi networks on each channel. There are two weighted interference matrices outside the system: one is U = {u n，c}, the dimension is N*C, and its element u n，c It represents the number of WiFi networks outside the system (non-wireless access network) where the channel c exists detected by the wireless switching node n itself; the other is V for the wireless access network = {v n，c}, dimension is N*C, its element v n，c It indicates the number of WiFi networks outside the system (non-wireless access networks and service data networks) existing in each channel detected by the terminal belonging to the wireless switching node.

[0060] Step 2: Update the channel selection of the service data network. The controller updates the actual weighted interference received by the service data network on each channel. Defined as

[0061]

[0062] The controller selects I 0 (c) The smallest channel is used as the channel of the service data network, that is, the channel with the least number of non-system WiFi networks detected, that is,

[0063]

[0064] Step 3: Initialize the channel selection of the wireless access network. All wireless switching nodes randomly select channels from channels outside the protection channel set of the service data network.

[0065] Step 4: Construct a potential intra-system weighted interference relationship matrix to describe the potential interference relationship between wireless switching nodes in the system. Each terminal node can detect the access network signals of other wireless switching nodes working on each channel. These access network signals can be received and processed. If they use the same channel, they will compete with each other, which is a measure of intra-system network interference. The spectrum management controller constructs a system weighted interference relationship matrix with a dimension of N*N based on the collected status information. Its element w n1→n2 Indicates access to wireless switching node n 2 The terminal can receive the wireless switching node n 1 The number of terminals of the signal, that is, when n 2 With n 1 When working on the same channel, n 2 In the 1 The number of terminals whose signals interfere. Note that this matrix is ​​not necessarily a symmetric matrix because the power of different nodes may be different.

[0066] Step 5: Wireless link quality measurement. In the current time slot, the wireless switching node updates the average link quality based on the WiFi signals of each channel detected by the terminal.

[0067] Step 6: Update the interference state variable. The wireless switching node updates the interference distribution vector L n , and use this to calculate the actual weighted interference I n .

[0068]

[0069] Where δ(a 1 , a 2 ) is the channel interference function, defined as

[0070]

[0071] Step 7: Update the channel selection state variable. Update the selected channel variable a n The total number of times T is selected by wireless access network n n (a n )=T n (a n )+1; Update the average link quality estimate of the wireless access network.

[0072]

[0073] Step 8: Update the wireless access network channel selection. At the beginning of the next time slot, each wireless switching node adjusts the access network channel a according to the following rules: n :

[0074]

[0075] Among them, ε can be initially set to a small number in the range of (0,1), such as 0.3, and decreases as the number of time slots increases. For example, it is inversely proportional to the number of time slots. t is the current time slot number.

[0076] Step 9: If the controller needs to readjust the channel of the service data network or the network status changes, return to step 1; otherwise, return to step 4.

[0077] The hybrid intelligent spectrum management method of the present invention is mainly used in future unmanned confrontation scenarios, and can provide an effective intelligent spectrum management method for large-scale unmanned system cluster networking communications. According to the tasks and communication capabilities undertaken, drones can be divided into wireless switching nodes, spectrum controllers and terminal nodes according to the above requirements. The wireless switching nodes and spectrum controllers work in coordination according to the method scheme of the present invention to achieve autonomous spectrum allocation and adjustment of the network. The present invention is also applicable to spectrum management in large-scale hierarchical wireless mesh networks.

[0078] The present invention discloses a hybrid intelligent spectrum management method in an unmanned system cluster network. A weighted interference graph model is constructed. Based on the measurement information and communication performance feedback of the nodes, a reinforcement learning method is adopted to autonomously optimize and adjust the working channels of the network, reduce the frequency conflicts within the system and the adverse effects of interference outside the system. The method uses the communication performance actually perceived by the communication nodes as feedback to guide spectrum decisions, can adapt to complex and dynamic spectrum environments and wireless channel environments, and can support large-scale unmanned system networking communication spectrum management. It proposes a hybrid optimization structure that combines centralized optimization (service data network channel selection) with distributed optimization (radio access network channel selection based on reinforcement learning), has learning capabilities, and has low algorithm complexity.

[0079] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure here, those skilled in the art will easily think of other embodiments of the present disclosure. The present invention is intended to cover any variation, use or adaptive change of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not recorded in the present disclosure. The description and examples are regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

[0080] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0081] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A hybrid intelligent spectrum management method in an unmanned system cluster network, characterized in that: include: S1. Construct a weighted interference relationship matrix outside the system to measure the number of non-system WiFi networks on each channel; The external system weighted interference relationship matrix includes a matrix for a service data network and a matrix for a wireless access network; S2, selecting a channel that minimizes the actual weighted interference received by the service data network on each channel as the channel of the service data network; S3, initializing channel selection of the wireless access network; S4, constructing a weighted interference relationship matrix within the system to describe the potential interference relationship between wireless switching nodes within the system; S5. In the current time slot, the average link quality is updated according to the WiFi signals of each channel detected by the received terminal; S6. Update the interference distribution vector and use it to obtain the actual weighted interference of the wireless access network, and update the average link quality estimation value of the wireless access network based on the actual weighted interference of the wireless access network and the average link quality; S7. At the beginning of the next time slot, updating the channel selection of the wireless access network according to the probability rule; S8. If the channel of the service data network needs to be readjusted or the network status changes, jump to step S1; otherwise, jump to step S4.

2. The hybrid intelligent spectrum management method according to claim 1, characterized in that: The formula for selecting the channel that minimizes the actual weighted interference received by the service data network on each channel as the channel of the service data network includes: Wherein, I0(c) represents the actual weighted interference received by the service data network on each channel, a0 represents the channel of the service data network, and u n，c It indicates the number of WiFi networks outside the system where channel c exists and detected by wireless switching node n itself. δ(a1, a2) is the channel interference function, n indicates the number of wireless switching node, and c indicates the number of wireless communication channels available to the system.

3. The hybrid intelligent spectrum management method according to claim 1, characterized in that: The initializing the channel selection of the wireless access network specifically includes: All wireless switching nodes randomly select channels from channels outside the protection channel set of the service data network.

4. The hybrid intelligent spectrum management method according to claim 3, characterized in that: The specific definition of the protection channel set of the service data network is: Among them, C0 represents the protection channel set of the service data network, and the wireless communication channel set available to the system is Indicates the channel used by the business data network.

5. The hybrid intelligent spectrum management method according to claim 1, characterized in that: The average link quality is obtained by calculating by defining the link signal quality; For each single-hop wireless communication link, the link signal quality is defined as quality*(A+signal), where quality and signal represent the current communication link parameters detected by WiFi respectively, quality is a score not exceeding 1, and the larger the score, the better the signal; signal represents the signal strength in dBm, which is a negative value, and the preset positive number A is added to ensure that (A+signal) is a positive value; each wireless switching node calculates the average link quality of the link with the connected terminal, that is, the average link quality of the wireless access network link.

6. The hybrid intelligent spectrum management method according to claim 1, characterized in that: The formula for updating the interference distribution vector and obtaining the actual weighted interference of the wireless access network specifically includes: Among them, L n = {l n (c)} is the interference distribution vector of wireless access network n, I n is the actual weighted interference of the wireless access network, a n is the channel used by the wireless access network of the wireless switching node n, and δ(a1, a2) is the channel interference function.

7. The hybrid intelligent spectrum management method according to claim 2 or 6, characterized in that: The expression of the channel interference function δ(a1, a2) is:

8. The hybrid intelligent spectrum management method according to claim 6, characterized in that: The formula for updating the average link quality estimation value of the wireless access network based on the actual weighted interference and the average link quality of the wireless access network specifically includes: in, represents the average link quality, e n (a n ) represents the average link quality estimation value of the wireless access network, T(a n ) represents the selected channel variable a n The total number of times selected by wireless access network n.

9. The hybrid intelligent spectrum management method according to claim 1, characterized in that: The formula of the probability rule specifically includes: Among them, E n ={e n (c)} is the average link quality estimation value when the wireless switching node n uses the channel c, and ε is a preset probability value that decreases as the number of time slots increases.

10. The hybrid intelligent spectrum management method according to claim 9, characterized in that: The formula for the preset probability value specifically includes: Among them, B is (0,1) and t is the current number of time slots.

Images