Self-adaptive MAC protocol design method based on dynamic hierarchical authorization system in UASNs

Abstract

The invention discloses a self-adaptive MAC protocol design method based on a dynamic hierarchical authorization system in a UASNs environment. Comprising the following steps: firstly, layering all underwater agent nodes according to self delay and time slot length to form a plurality of sub-networks; secondly, all the agent nodes broadcast self-related information in the whole agent network, so that other agent nodes can record the information conveniently; then, a node is selected from the intelligent agent network to serve as a gateway node, a deep Q network (DQN) is used for training, an idle time slot occupation strategy at a water surface aggregation node is learned, and after training is completed, the gateway node shares the strategy in the network; then, the other agent nodes adjust the strategy according to own information, so that the strategy can be successfully used; and finally, all agent nodes interact with each other by using a multi-level authorization system, and a proper node is selected to occupy the free time slot. By using the dynamic hierarchical authorization system, all agent nodes can utilize idle time slots which are not fully utilized by other nodes in a conflict-free manner, so that the overall throughput of the network is improved, the energy consumption among the nodes is effectively balanced, and the overall service life of the network is prolonged.

Description

Technical Field

[0001] The present invention belongs to the field of underwater communication and underwater sensor network, and in particular relates to an adaptive MAC protocol based on a dynamic hierarchical authorization system for heterogeneous networks in a UASNs environment. Background Art

[0002] Underwater acoustic wireless sensor networks have broad application prospects in environmental monitoring, resource exploration, and assisted navigation. However, unlike ground-based wireless sensor networks, UASNs face challenges such as high propagation delay, limited bandwidth, and channel fading. In addition, high energy consumption and energy limitation are also significant. Therefore, when designing underwater acoustic wireless sensor networks, these factors need to be considered comprehensively to optimize system performance.

[0003] The underwater acoustic medium access control (MAC) protocol is an important component of underwater wireless communication networks. Its goal is to effectively manage the medium access between multiple sensor nodes, thereby optimizing network performance, such as throughput, latency, and energy consumption. However, due to the high difficulty in designing the underwater acoustic MAC protocol and the increasing use of underwater acoustic sensor networks, the existing traditional underwater MAC protocol has become difficult to adapt to the complex and changing underwater environment. Summary of the invention

[0004] In view of the above problems, the present invention designs an adaptive MAC protocol based on a dynamic hierarchical authorization system for heterogeneous networks in UASNs environments. By utilizing long propagation delays, the throughput of underwater acoustic sensor networks is effectively improved; by using dynamic hierarchical and multi-level authorization systems, the energy consumption between nodes is balanced, extending the overall service life of the network.

[0005] To achieve the above objectives and the above technical effects, the present invention is implemented through the following technical solutions: an adaptive MAC protocol based on a dynamic hierarchical authorization system for heterogeneous networks in a UASNs environment, comprising the following steps:

[0006] (1) During network initialization, all underwater intelligent agent nodes are stratified according to their own delays and time slot lengths to form multiple sub-networks; each intelligent agent node broadcasts its own relevant information in the intelligent agent network and stores information from other intelligent agent nodes.

[0007] (2) A node is selected from the intelligent agent network as the gateway node and trained using the DQN algorithm to learn the idle time slot occupation strategy at the water surface aggregation node.

[0008] (3) After the gateway node is trained, it shares the strategy in the agent network. The remaining agent nodes adjust the strategy based on their own information so that it can be successfully used.

[0009] (4) All intelligent agent nodes interact using a multi-level authorization system to select appropriate nodes to occupy the idle time slots at the surface aggregation node.

[0010] All underwater intelligent agent nodes in the above step (1) are layered according to their own delay and time slot length, and each node broadcasts its own information and stores other node information, which specifically includes the following steps:

[0011] (1.1) Assume that the length of a single time slot is t. t is set to the packet length t d , protection time t g and buffer time t b The sum is:

[0012] t=t d +t g +t b (1)

[0013] (1.2) For any agent node, whose delay is τ, the subnetwork level n to which the node belongs can be calculated as follows:

[0014]

[0015] Where [] is a rounding symbol; the delay of the nodes in the subnetwork of level n satisfies the condition (n-1)t≤τ≤nt.

[0016] (1.3) After the network layering is completed, all intelligent agent nodes broadcast their own location, delay information, node ID and the level of the sub-network to which they belong in the intelligent agent network; each node stores the received broadcast information and specially marks other nodes in its own sub-network for subsequent communication.

[0017] Preferably, the above step (2) of selecting a gateway node from the agent network, using the DQN algorithm for training, and learning the idle time slot occupation strategy at the surface aggregation node specifically includes the following steps:

[0018] (2.1) Since the gateway node needs to bear the additional energy overhead caused by training, the node with the smallest sending energy consumption is selected as the gateway node for training.

[0019] (2.2) There are three types of nodes using different MAC protocols in the underwater acoustic sensor network: TDMA nodes, ALOHA nodes, and intelligent nodes using the DHAS-MAC protocol. Among them, TDMA nodes and ALOHA nodes are uniformly deployed, while intelligent nodes are randomly deployed. During data transmission, each node uses its own MAC protocol to transmit at the beginning of the time slot.

[0020] (2.3) Since the intelligent nodes are randomly deployed, they may not meet the condition of conflict-free transmission, that is, the data packet sent at the beginning of the time slot is successfully received by the sink node within the length of a single time slot. Therefore, it is necessary to set the advance transmission time dp for the intelligent node to meet this condition. Assuming that the delay of the intelligent node is τ and the level of the subnetwork to which it belongs is n, dp is calculated as follows:

[0021] dp=τ-(n-1)t (3)

[0022] Obviously τ≥(n-1)t, so dp always satisfies dp≥0.

[0023] (2.4) The gateway node uses the DQN algorithm for training. The specific algorithm framework is as follows:

[0024] Action space: At the beginning of each time slot, the agent node can independently decide whether to occupy the time slot for data transmission, so the action space A is defined as:

[0025] A={Transmit,Wait} (4)

[0026] State: When the agent node selects action a at time t t After that, we get the observed value o t ∈{Success,Collision,Idle}. However, due to the long propagation delay and the presence of advance sending time, the observed value o t For action a t-2[τ+1] , so action a t The observation value of needs to be obtained at time slot t+2[τ+1]. t =Transmit, and the DLMA node receives the ACK packet at time slot t+2[τ+1], then o t+2[τ+1] =Success, otherwise o t+2[τ+1] =Collision; if a t =Wait, and the DLMA node receives the ACK packet corresponding to other nodes at time slot t+2[τ+1], then o t+2[τ+1] =Success, otherwise o t+2[τ+1] =Idle.

[0027] The actions and observations of a node can form action-observation pairs:

[0028] z t+1 ={a t-2[τ+1] ,o t} (5)

[0029] Since DLMA nodes can only observe part of the network state in each time slot, the state of the network consists of multiple action-observation pairs, defined as:

[0030] s t+1 ={z t-M+2 ,...,z t+1} (6)

[0031] Where M represents the length of the historical state. The larger the M value, the more historical information the agent can obtain. However, as M increases, the state space will also expand accordingly.

[0032] Reward: In order to maximize the overall throughput of the network, the agent's reward needs to be combined with the observation value, which is defined as follows:

[0033]

[0034] Due to the long propagation delay underwater, the reward obtained by the agent node for taking action is not immediate. When the slot length is less than 2[τ+1], that is, t<2[τ+1], the agent node takes action a t After that, no reward can be obtained from the environment. When t≥2[τ+1], the agent node takes action a t After that, you can receive the reward r t+1 , the environment state becomes s t+1 However, the reward t+1 It is aimed at the action at time t-2[τ+1], that is, a t-2[τ+1] , not a t Therefore, the agent node needs to wait for 2[τ+1] time slots to receive the reward corresponding to the action. Need to sample multiple experiences t Reorganize. t =(s t ,a t ,r t+1 ,s t+1 )、e t+2[τ+1] =(s t+2[τ+1] ,a t+2[τ+1] ,r t+2[τ+1]+1 ,s t+2[τ+1]+1 ), e t With e t+2[τ+1] Reorganization can gain real experience The real experience obtained by reorganizing each time slot Put it into the real experience buffer and train the weight parameter θ of the evaluation network by minimizing the following loss function:

[0035]

[0036] Among them, N E is the size of random sampling; y = ri+2[τ+1]+1 +γmax a' Q(s i+2[τ+1]+1 ,a';θ - ), θ - is the weight parameter of the target network, which is set by setting θ every F time slots - =θ to obtain.

[0037] Preferably, in the above step (3), the remaining intelligent agent nodes share the strategy learned by the gateway node and adjust the strategy according to their own information, which specifically includes the following steps:

[0038] (3.1) When the gateway node training is completed, it broadcasts the trained model parameters in the entire intelligent agent network.

[0039] (3.2) The remaining nodes receive the parameter and update it into their own neural network.

[0040] (3.3) Since the delay of each intelligent agent node is different, the strategy of the gateway node cannot be used directly and needs to be adjusted according to its own delay. Set the advance / delay sending time ds for each node, calculated as follows:

[0041] ds=τ-τ g (9)

[0042] Where τ is the delay of the current node, τ g is the latency of the gateway node.

[0043] If ds>0, the current node sends the data packet ds seconds in advance based on the sharing strategy; if ds<0, the node delays sending by |ds| seconds based on the sharing strategy.

[0044] Preferably, in the above step (4), all intelligent nodes interact using a multi-level authorization system to select appropriate nodes to occupy the idle time slots at the surface convergence node. Multi-level authorization is divided into two levels, both of which are transmitted between intelligent nodes in the form of control packets. The first-level authorization is transmitted between sub-networks of different levels to determine which node in the sub-network occupies the current idle time slot; the second-level authorization is transmitted between nodes in the same-level sub-network to determine which node occupies the time slot. Each layer of sub-network has its own second-level authorization, and the entire intelligent network has only one first-level authorization. The control packets of the multi-level authorization are transmitted after the node sends the data packet. The multi-level authorization specifically includes the following contents:

[0045] (4.1) Level 1 authorization: This authorization is used between subnetworks and is passed from the nodes in the upper subnetwork to all nodes in the lower subnetwork. It is used to activate the nodes in the subnetwork to use idle time slots. Level 1 authorization is first passed from the lower subnetwork to the higher subnetwork, and then from the higher subnetwork to the lower subnetwork. During the level 1 authorization transmission process, each subnetwork has P opportunities to use the level 1 authorization, i.e., the number of times the level 1 authorization is used. The value of P is determined by the score score1 of the current subnetwork (assuming it is n). n Decision, and score1 n The size of depends on two parameters, namely the sum of the remaining energy of the nodes in the sub-network α n , the number of nodes in the sub-network β n and the packet stacking amount S n . score1 n The calculation of is as follows:

[0046] score1 n =k1α n +k2β n +k3S n (10)

[0047] Among them, k1, k2, and k3 are weight parameters, which normalize the three indicators to the same order of magnitude and then allocate the weight of each indicator according to its importance; α n It can be calculated through the energy consumption model; the energy consumption model mainly considers the energy consumption of the communication module. The average working power consumption of the communication module is expressed as:

[0048] E=N tx ·[P tx ·(T tx +T st )+P out ·T tx ]+N rx ·[P rx ·(T rx +T st )] (11)

[0049] Among them, N tx / rx is the number of times the sending / receiving unit works; P tx / rx is the average energy consumption of the circuit of the sending / receiving unit; P out is the output power; T sw is the state switching time, assuming that the state switching time is the same; T tx / rx It is the working time of the sending / receiving unit, which is mainly determined by the data packet length, transmission rate, and coding rate, that is:

[0050] T tx / rx =l / R·R code(12)

[0051] Where l is the packet length, R is the bit rate, and R code is the coding rate.

[0052] When the remaining energy of a node in a sub-network is too low or there is no task to send temporarily, the node will send a control signal to the remaining nodes in the current sub-network to exit the score calculation. The score of the sub-network is updated after the network has used up the number of times the first-level authorization is used, and the information is stored in the control package of the first-level authorization and passed to the next-level sub-network for subsequent calculation of the sub-network.

[0053] The scores of each layer of sub-network are standardized, and the standardized score Q of the nth layer of sub-network is n for:

[0054]

[0055] Where m is the total number of sub-network layers; min(score11, score12, ..., score1 m )≠0, that is, the sub-network with a score of 0 is not included in the calculation of the number of times the first-level authorization is used.

[0056] The number of times P the first-level authorization is used by the n-th layer sub-network n The definition is as follows:

[0057]

[0058] (4.2) Secondary authorization: This authorization is used between nodes in the sub-network to determine which node occupies the idle time slot. The order of using the secondary authorization depends on the score score2 of each node in the network. The score is determined by two parameters, namely the remaining energy R of the current node and the amount of data packet stacking S, which is calculated as follows:

[0059]

[0060] Among them, w1 and w2 are weight parameters used to balance the energy consumption and throughput of the node; R threshold is the node remaining energy threshold, and if it is less than this threshold, the node will no longer send data packets. The score of each node is updated after each use of the secondary authorization, and the score information is stored in the control packet of the secondary authorization and passed to the next node.

[0061] Each node in the sub-network uses the secondary authorization in turn according to the score until the primary authorization usage times are exhausted. If the primary authorization usage times of the sub-network is greater than the number of nodes in the network, that is, P>η, the secondary authorization will continue to be used cyclically according to the score until the primary authorization usage times are exhausted.

[0062] (4.3) Each node will send a small control packet after sending the data packet. The control packet contains an information matrix. The specific content of the information matrix is ​​as follows:

[0063]

[0064] The control packet contains the ID of the current sending node sender , Remaining energy R sender , Data packet stacking amount S sender and the IDs of all other nodes in the agent network i (i≠sender), the corresponding remaining energy R i and the packet stacking amount S i The control packet sent by each node contains the energy and data packet stacking information of other nodes. The purpose of including this redundant information is to prevent the loss of control packets from causing untimely updates of information of each node.

[0065] Each intelligent node has a local information table, which contains the ID of the current sending node. sender , the subnetwork level, the level of the current subnetwork n sender , the number of times the first-level authorization is used P sender , the score of the current node Score2 sender and the IDs of all other nodes i (i≠sender), the sub-network level n i , the corresponding first-level authorized usage times P i With score Score2 i The node updates the relevant information in the local information table through the obtained control packet, and calculates the corresponding first-level authorization usage times and second-level authorization usage order. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A flowchart of an embodiment of this invention;

[0067] Figure 2 It is the application scenario diagram;

[0068] Figure 3 Transmit the scene graph for conflict-free transmission;

[0069] Figure 4 Schematic diagram for setting advance\delay sending time; DETAILED DESCRIPTION

[0070] In order to enable people in the technical field to better understand the technical solutions in this application, the technical solutions in the embodiments of this application are clearly and completely described below. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0071] The application principle of the present invention is described in detail below in conjunction with the accompanying drawings.

[0072] like Figure 2 As shown in the figure, TDMA nodes and ALOHA nodes are evenly distributed at different depths underwater to form a TDMA+ALOHA node network; intelligent agent nodes are randomly distributed at different depths underwater to form an intelligent agent network. Each sensor node collects underwater data and transmits data packets directly to the surface aggregation node through single-hop transmission. TDMA and ALOHA nodes use TDMA and ALOHA protocols respectively, and intelligent agent nodes use DHASA-MAC protocol. TDMA nodes and ALOHA nodes do not interact with other nodes, and intelligent agent nodes interact using a multi-level authorization system.

[0073] An adaptive MAC protocol based on a dynamic hierarchical authorization system in UASNs environment, such as Figure 1 As shown, the specific steps include:

[0074] (1) Agent Node Network Layering

[0075] (1.1) Assume that the length of a single time slot is t. t is set to the packet length t d , protection time t g and buffer time t b The sum is:

[0076] t=t d +t g +t b (1)

[0077] (1.2) For any agent node, whose delay is τ, the subnetwork level n to which the node belongs can be calculated as follows:

[0078]

[0079] Where [] is a rounding symbol; the delay of the nodes in the subnetwork of level n satisfies the condition (n-1)t≤τ≤nt.

[0080] (1.3) After the network layering is completed, all intelligent agent nodes broadcast their own location, delay information, node ID and the level of the sub-network to which they belong in the intelligent agent network; each node stores the received broadcast information and specially marks other nodes in its own sub-network for subsequent communication.

[0081] (2) Selecting and training gateway nodes

[0082] Use the DQN algorithm for training and learn the idle time slot occupation strategy at the water surface aggregation node, which includes the following steps:

[0083] (2.1) Since the gateway node needs to bear the additional energy overhead caused by training, the node with the smallest sending energy consumption is selected as the gateway node for training.

[0084] (2.2) There are three types of nodes using different MAC protocols in the underwater acoustic sensor network: TDMA nodes, ALOHA nodes, and intelligent nodes using the DHAS-MAC protocol. Among them, TDMA nodes and ALOHA nodes are uniformly deployed, while intelligent nodes are randomly deployed. During data transmission, each node uses its own MAC protocol to transmit at the beginning of the time slot.

[0085] (2.3) Since the intelligent nodes are randomly deployed, they may not meet the condition of conflict-free transmission, that is, the data packet sent at the beginning of the time slot is successfully received by the sink node within the length of a single time slot. Therefore, it is necessary to set the advance transmission time dp for the intelligent node to meet this condition. Assuming that the delay of the intelligent node is τ and the level of the subnetwork to which it belongs is n, then dp The calculation is as follows:

[0086] dp=τ-(n-1)t (3)

[0087] Obviously, τ≥(n-1)t, so dp always satisfies dp≥0. The conditions for conflict-free transmission are as follows Figure 3 As shown in (a), the data packet sent by the agent node at the beginning of the time slot is received by the sink node in the middle of the time slot. If this condition cannot be met, the following will occur: Figure 3 In the scenario shown in (b), the data packet sent by the intelligent node 1 collides with the data packet sent by the TDMA node in time slot t=3, resulting in a decrease in the network throughput. After setting the advance transmission time dp for the intelligent node, conflict-free transmission between nodes can be achieved. The specific scenario is as follows Figure 3 (c) as shown.

[0088] (2.4) The gateway node uses the DQN algorithm for training. The specific algorithm framework is as follows:

[0089] Action space: At the beginning of each time slot, the agent node can independently decide whether to occupy the time slot for data transmission, so the action space A is defined as:

[0090] A={Transmit,Wait} (4)

[0091] State: When the agent node selects action a at time t t After that, we get the observed value o t ∈{Success,Collision,Idle}. However, due to the long propagation delay and the presence of advance sending time, the observed value o t For action a t-2[τ+1] , so action a t The observation value of needs to be obtained at time slot t+2[τ+1]. t =Transmit, and the DLMA node receives the ACK packet at time slot t+2[τ+1], then o t+2[τ+1] =Success, otherwise o t+2[τ+1] =Collision; if a t =Wait, and the DLMA node receives the ACK packet corresponding to other nodes at time slot t+2[τ+1], then o t+2[τ+1] =Success, otherwise o t+2[τ+1] =Idle.

[0092] The actions and observations of a node can form action-observation pairs:

[0093] z t+1 ={a t-2[τ+1] ,o t} (5)

[0094] Since DLMA nodes can only observe part of the network state in each time slot, the state of the network consists of multiple action-observation pairs, defined as:

[0095] s t+1 ={z t-M+2 ,...,z t+1} (6)

[0096] Where M represents the length of the historical state. The larger the M value, the more historical information the agent can obtain. However, as M increases, the state space will also expand accordingly.

[0097] Reward: In order to maximize the overall throughput of the network, the agent's reward needs to be combined with the observation value, which is defined as follows:

[0098]

[0099] Due to the long propagation delay underwater, the reward obtained by the agent node for taking action is not immediate. When the slot length is less than 2[τ+1], that is, t<2[τ+1], the agent node takes action a t After that, no reward can be obtained from the environment. When t≥2[τ+1], the agent node takes action a t After that, you can receive the reward r t+1 , the environment state becomes s t+1 However, the reward t+1 It is aimed at the action at time t-2[τ+1], that is, a t-2[τ+1] , not a t Therefore, the agent node needs to wait for 2[τ+1] time slots to receive the reward corresponding to the action. Need to sample multiple experiences t Reorganize. t =(s t ,a t ,r t+1 ,s t+1 ), e t+2[τ+1] =(s t+2[τ+1] ,a t+2[τ+1] ,r t+2[τ+1]+1 ,s t+2[τ+1]+1 ), e t With e t+2[τ+1] Reorganization can gain real experience The real experience obtained by reorganizing each time slot Put it into the real experience buffer and train the weight parameter θ of the evaluation network by minimizing the following loss function:

[0100]

[0101] Among them, N E is the size of random sampling; y = r i+2[τ+1]+1 +γmax a' Q(s i+2[τ+1]+1 ,a';θ - ), θ - is the weight parameter of the target network, which is set by setting θ every F time slots - =θ to obtain.

[0102] (3) Agent Network Strategy Sharing

[0103] (3.1) When the gateway node training is completed, it broadcasts the trained model parameters in the entire intelligent agent network.

[0104] (3.2) The remaining nodes receive the parameter and update it into their own neural network.

[0105] (3.3) Since the delay of each intelligent agent node is different, the strategy of the gateway node cannot be used directly and needs to be adjusted according to its own delay. Set the advance / delay sending time ds for each node, calculated as follows:

[0106] ds=τ-τ g (9)

[0107] Where τ is the delay of the current node, τ g is the latency of the gateway node.

[0108] If ds>0, the current node sends the data packet ds seconds in advance based on the sharing strategy; if ds<0, the node delays sending by |ds| seconds based on the sharing strategy.

[0109] (4) Use multi-level authorization for interaction

[0110] Multi-level authorization is divided into two levels, both of which are transmitted between intelligent agent nodes in the form of control packets. The first-level authorization is transmitted between sub-networks of different levels to determine which node in the current idle time slot is occupied by the node in the sub-network; the second-level authorization is transmitted between nodes in the same level of sub-network to determine which node occupies the time slot. It should be noted that each layer of sub-network has its own second-level authorization, while the entire intelligent agent network has only one first-level authorization. The control packets of multi-level authorization are transmitted after the node sends the data packet.

[0111] The following is a detailed introduction to primary and secondary authorization:

[0112] Level 1 authorization: This authorization is used between sub-networks and is passed from the nodes in the upper sub-network to all nodes in the lower sub-network. It is used to activate the nodes in the sub-network to use the idle time slots. Level 1 authorization is first passed from the lower sub-network to the higher sub-network, and then from the higher sub-network to the lower sub-network. During the level 1 authorization transmission process, each sub-network has P opportunities to use the level 1 authorization, that is, the number of times the level 1 authorization is used. The value of P is determined by the score score1 of the current sub-network (assuming it is n). n Decision, and score1 n The size of depends on two parameters, namely the sum of the remaining energy of the nodes in the sub-network α n , the number of nodes in the sub-network β n and the packet stacking amount S n . score1 n The calculation of is as follows:

[0113] score1 n =k1α n +k2β n +k3S n(10)

[0114] Among them, k1, k2, and k3 are weight parameters, which normalize the three indicators to the same order of magnitude and then allocate the weight of each indicator according to its importance; α n It can be calculated through the energy consumption model; the energy consumption model mainly considers the energy consumption of the communication module. The average working power consumption of the communication module is expressed as:

[0115] E=N tx ·[P tx ·(T tx +T st )+P out ·T tx ] +N rx ·[P rx ·(T rx +T st )] (11)

[0116] Among them, N tx / rx is the number of times the sending / receiving unit works; P tx / rx is the average energy consumption of the circuit of the sending / receiving unit; P out is the output power; T sw is the state switching time, assuming that the state switching time is the same; T tx / rx It is the working time of the sending / receiving unit, which is mainly determined by the data packet length, transmission rate, and coding rate, that is:

[0117] T tx / rx =l / R·R code (12)

[0118] Where l is the packet length, R is the bit rate, and R code is the coding rate.

[0119] When the remaining energy of a node in a sub-network is too low or there is no task to send temporarily, the node will send a control signal to the remaining nodes in the current sub-network to exit the score calculation. The score of the sub-network is updated after the network has used up the number of times the first-level authorization is used, and the information is stored in the control package of the first-level authorization and passed to the next-level sub-network for subsequent calculation of the sub-network.

[0120] The scores of each layer of sub-network are standardized, and the standardized score Q of the nth layer of sub-network is n for:

[0121]

[0122] Where m is the total number of sub-network layers; min(score11, score12, ..., score1 m)≠0, that is, the sub-network with a score of 0 is not included in the calculation of the number of times the first-level authorization is used.

[0123] The number of times P the first-level authorization is used by the n-th layer sub-network n The definition is as follows:

[0124]

[0125] (4.2) Secondary authorization: This authorization is used between nodes in the sub-network to determine which node occupies the idle time slot. The order of using the secondary authorization depends on the score score2 of each node in the network. The score is determined by two parameters, namely the remaining energy R of the current node and the amount of data packet stacking S, which is calculated as follows:

[0126]

[0127] Among them, w1 and w2 are weight parameters used to balance the energy consumption and throughput of the node; R threshold is the node remaining energy threshold, and if it is less than this threshold, the node will no longer send data packets. The score of each node is updated after each use of the secondary authorization, and the score information is stored in the control packet of the secondary authorization and passed to the next node.

[0128] Each node in the sub-network uses the secondary authorization in turn according to the score until the primary authorization usage times are exhausted. If the primary authorization usage times of the sub-network is greater than the number of nodes in the network, that is, P>η, the secondary authorization will continue to be used cyclically according to the score until the primary authorization usage times are exhausted.

[0129] (4.3) Each node will send a small control packet after sending the data packet. The control packet contains an information matrix. The specific content of the information matrix is ​​as follows:

[0130]

[0131] The control packet contains the ID of the current sending node sender , Remaining energy R sender , Data packet stacking amount S sender and the IDs of all other nodes in the agent network i (i≠sender), the corresponding remaining energy R i and the packet stacking amount S i The control packet sent by each node contains the energy and data packet stacking information of other nodes. The purpose of including this redundant information is to prevent the loss of control packets from causing untimely updates of information of each node.

[0132] Each intelligent agent node has a local information table, the specific information is shown in Table 1. The information table contains the ID of the current sending node sender , belongs to the sub-network level n sender , the number of times the first-level authorization is used in the current sub-network P sender , the score of the current node Score2 sender and the IDs of all other nodes i (i≠sender), the sub-network level n i , the corresponding first-level authorized usage times P i With score Score2 i The node updates the relevant information in the local information table through the obtained control packet, and calculates the corresponding first-level authorization usage times and second-level authorization usage order.

[0133] Table 1. Local information table of agent node

[0134] <![CDATA[ID sender ]]> <![CDATA[n sender ]]> <![CDATA[P sender ]]> <![CDATA[Score2 sender ]]> <![CDATA[ID i ]]> <![CDATA[n i ]]> <![CDATA[P i ]]> <![CDATA[Score2 i ]]> ... ... ... ...

[0135] Finally, it should be noted that the method steps described in the embodiments disclosed in the present invention can be configured to build a programming environment for implementation. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for designing an adaptive MAC protocol based on a dynamic hierarchical authorization system in a UASNs environment, characterized in that: The following steps are involved: (1) During network initialization, all underwater intelligent agent nodes are layered according to their own delays and time slot lengths to form multiple sub-networks; each intelligent agent node broadcasts its own relevant information in the intelligent agent network and stores information from other intelligent agent nodes; (2) A node is selected from the agent network as the gateway node and trained using the DQN algorithm to learn the idle time slot occupation strategy at the water surface aggregation node; (3) After the gateway node is trained, the strategy described in step (2) is shared in the intelligent agent network, and the remaining intelligent agent nodes adjust the strategy described in step (2) according to their own information so that it can be successfully used; (4) All intelligent agent nodes interact using a multi-level authorization system to select appropriate nodes to occupy the idle time slots at the surface aggregation node.

2. The method for designing an adaptive MAC protocol based on a dynamic hierarchical authorization system in UASNs according to claim 1, characterized in that: The step (1) specifically comprises the following steps: (2.1) Assume that the length of a single time slot is t, and t is set to the packet length t d , protection time t g and buffer time t b The sum is: t=t d +t g +t b (1) (2.2) For any agent node, its delay is τ, then the subnetwork level n to which the node belongs is calculated as follows: Where [] is a rounding symbol; the delay of the nodes in the subnetwork of level n satisfies the condition (n-1)t≤τ≤nt; (2.3) After the network layering is completed, all intelligent agent nodes broadcast their own location, delay information, node ID and the level of the sub-network to which they belong in the intelligent agent network; each node stores the received broadcast information and specially marks other nodes in its own sub-network for subsequent communication.

3. The method for designing an adaptive MAC protocol based on a dynamic hierarchical authorization system in UASNs according to claim 1, characterized in that: The step (2) specifically comprises the following steps: (3.1) Since the gateway node needs to bear the additional energy overhead caused by training, the node with the smallest sending energy consumption is selected as the gateway node for training; (3.2) There are three types of nodes using different MAC protocols in the underwater acoustic sensor network, namely: TDMA nodes, ALOHA nodes and intelligent nodes using DHAS-MAC protocol. Among them, TDMA nodes and ALOHA nodes are uniformly deployed, while intelligent nodes are randomly deployed. When transmitting data, each node uses its own MAC protocol to transmit at the beginning of the time slot; (3.3) Since the intelligent nodes are randomly deployed, it is necessary to set the advance sending time dp for the intelligent nodes to ensure that the data packets sent at the beginning of the time slot are successfully received by the sink node within the length of a single time slot. Assuming that the delay of the intelligent node is τ and the level of the subnetwork to which it belongs is n, dp is calculated as follows: dp=τ-(n-1)t (3) Obviously τ≥(n-1)t, so dp always satisfies dp≥0; (3.4) The gateway node uses the DQN algorithm for training. The specific algorithm framework is as follows: Action space: At the beginning of each time slot, the agent node can independently decide whether to occupy the time slot for data transmission, so the action space A is defined as: A={Transmit,Wait}(4) State: When the agent node selects action a at time t t After that, we get the observed value o t ∈{Success,Collision,Idle}, however, due to the long propagation delay and the presence of advance sending time, the observed value o t For action a t-2[τ+1] , so action a t The observation value of needs to be obtained at time slot t+2[τ+1]. If a t =Transmit, and the DLMA node receives the ACK packet at time slot t+2[τ+1], then o t+2[τ+1] =Success, otherwise o t+2[τ+1] =Collision; if a t =Wait, and the DLMA node receives the ACK packet corresponding to other nodes at time slot t+2[τ+1], then o t+2[τ+1] =Success, otherwise o t+2[τ+1] =Idle; The actions and observations of a node constitute an action-observation pair: from t+1 ={a t-2[τ+1] ,o t } (5) Since DLMA nodes can only observe part of the network state in each time slot, the state of the network consists of multiple action-observation pairs, defined as: s t+1 ={z t-M+2 ,...,z t+1 } (6) Among them, M represents the length of the historical state. The larger the M value, the more historical information the agent can obtain. However, as M increases, the state space will also expand accordingly. Reward: In order to maximize the overall throughput of the network, the agent's reward needs to be combined with the observation value, which is defined as follows: When the slot length is less than 2[τ+1], that is, t<2[τ+1], the agent node takes action a t After that, no reward can be obtained from the environment. When t≥2[τ+1], the agent node takes action a t After that, you can receive the reward t+1 , the environment state becomes s t+1 ; However, the reward r t+1 It is aimed at the action at time t-2[τ+1], that is, a t-2[τ+1] , not a t Therefore, the agent node needs to wait for 2[τ+1] time slots to receive the reward corresponding to the action. In order to obtain real and usable training experience Need to sample multiple experiences t to reorganize; Reorganize as follows: e t =(s t ,a t ,r t+1 ,s t+1 )、e t+2[τ+1] =(s t+2[τ+1] ,a t+2[τ+1] ,r t+2[τ+1]+1 ,s t+2[τ+1]+1 ) E t With e t+2[τ+1] Reorganize and gain real experience The real experience obtained by reorganizing each time slot Put it into the real experience buffer and train the weight parameter θ of the evaluation network by minimizing the following loss function: Among them, N E is the size of random sampling; y = r i+2[τ+1]+1 +γmax a' Q(s i+2[τ+1]+1 ,a';θ - ), θ - is the weight parameter of the target network, which is set by setting θ every F time slots - =θ to obtain.

4. The method for designing an adaptive MAC protocol based on a dynamic hierarchical authorization system in UASNs according to claim 1, characterized in that: The step (3) specifically comprises the following steps: (4.1) When the gateway node is trained, it broadcasts the trained model parameters in the entire agent network; (4.2) The remaining nodes receive the parameter and update it into their own neural network; (4.3) Since the delay of each intelligent agent node is different, the strategy of the gateway node cannot be used directly. It needs to be adjusted according to its own delay and set the advance / delay sending time ds for each node. The calculation is as follows: ds=τ-τ g (9) Where τ is the delay of the current node, τ g is the latency of the gateway node; If ds>0, the current node sends the data packet ds seconds in advance based on the sharing strategy; if ds<0, the node delays sending by |ds| seconds based on the sharing strategy.

5. The method for designing an adaptive MAC protocol based on a dynamic hierarchical authorization system in UASNs according to claim 1, characterized in that: The multi-level authorization in step (4) is divided into two levels, both of which are transmitted between intelligent agent nodes in the form of control packets, wherein the first-level authorization is transmitted between sub-networks of different levels, and its function is to determine which node in the sub-network occupies the current idle time slot; the second-level authorization is transmitted between nodes in the sub-network of the same level, and its function is to determine which node occupies the time slot; each layer of sub-network has its own second-level authorization, and the entire intelligent agent network has only one first-level authorization, and the control packets of the multi-level authorization are transmitted after the node sends the data packet; Multi-level authorization specifically includes the following: (5.1) Level 1 authorization: This authorization is used between sub-networks and is passed from the nodes in the upper sub-network to all nodes in the lower sub-network. It is used to activate the nodes in the sub-network to use the idle time slots. Level 1 authorization is first passed from the lower sub-network to the higher sub-network, and then from the higher sub-network to the lower sub-network. During the level 1 authorization transmission process, each sub-network has P opportunities to use the level 1 authorization, i.e., the number of times the level 1 authorization is used. The value of P is determined by the score of the current sub-network, score1. n Determine that the current subnetwork is assumed to be n; and score1 n The size of depends on two parameters, namely the sum of the remaining energy of the nodes in the sub-network α n , the number of nodes in the sub-network β n and the packet stacking amount S n , score1 n The calculation of is as follows: score1 n =k1α n +k2β n +k3S n (10) Among them, k1, k2, and k3 are weight parameters, which normalize the three indicators to the same order of magnitude and then allocate the weight of each indicator according to its importance; α n Calculated by the energy consumption model; the energy consumption model takes into account the energy consumption of the communication module. The average power consumption of the communication module is expressed as: E=N tx ·[P tx ·(T tx +T st )+P out ·T tx ]+N rx ·[P rx ·(T rx +T st )] (11) Among them, N tx / rx is the number of times the sending / receiving unit works; P tx / rx is the average energy consumption of the circuit of the sending / receiving unit; P out is the output power; T sw is the state switching time, assuming that the state switching time is the same; T tx / rx It is the working time of the sending / receiving unit, which is determined by the data packet length, transmission rate, and coding rate, that is: T tx / rx =l / R·R code (12) Where l is the packet length, R is the bit rate, and R code is the coding rate; When the remaining energy of a node in a sub-network is too low or there is no task to send temporarily, the node will send a control signal to the remaining nodes in the current sub-network to exit the score calculation. The score of the sub-network will be updated after the network has used up the number of times the first-level authorization is used, and the information will be stored in the control package of the first-level authorization and passed to the next-level sub-network for subsequent calculation of the sub-network; The scores of each layer of sub-network are standardized, and the standardized score Q of the nth layer of sub-network is n for: Where m is the total number of sub-network layers; min(score11, score12, ..., score1 m )≠0, that is, the sub-network with a score of 0 is not included in the calculation of the number of times the primary authorization is used; The number of times P the first-level authorization is used by the n-th layer sub-network n The definition is as follows: (5.2) Secondary authorization: This authorization is used between nodes in the sub-network to determine which node occupies the idle time slot. The order of using the secondary authorization depends on the score score2 of each node in the network. The score is determined by two parameters, namely the remaining energy R of the current node and the amount of data packet stacking S, and is calculated as follows: Among them, w1 and w2 are weight parameters used to balance the energy consumption and throughput of the node; R threshold is the node's remaining energy threshold. If it is less than this threshold, the node will no longer send data packets. The score of each node is updated after each use of the secondary authorization, and the score information is stored in the control packet of the secondary authorization and passed to the next node. ; Each node in the sub-network uses the secondary authorization in turn according to the score until the primary authorization is used up. If the primary authorization usage of the sub-network is greater than the number of nodes in the network, that is, P>η, the secondary authorization will continue to be used cyclically according to the score until the primary authorization is used up. (5.3) Each node will send a small control packet after sending the data packet. The control packet contains an information matrix. The specific content of the information matrix is ​​as follows: The control packet contains the ID of the current sending node sender , Remaining energy R sender , Data packet stacking amount S sender and the IDs of all other nodes in the agent network i , where i≠sender, the corresponding residual energy R i and the packet stacking amount S i ; The control packet sent by each node contains the energy and data packet stacking information of other nodes. The purpose of including this redundant information is to prevent the loss of control packets from causing untimely updates of information of each node; Each intelligent node has a local information table, which contains the ID of the current sending node. sender , the subnetwork level, the level of the current subnetwork n sender , the number of times the first-level authorization is used P sender , the score of the current node Score2 sender and the IDs of all other nodes i , where i≠sender, belongs to the sub-network level n i , the corresponding first-level authorized usage times P i With score Score2 i The node updates the relevant information in the local information table through the obtained control packet, and calculates the corresponding first-level authorization usage times and second-level authorization usage order.

Images