Intelligent CSMA / CA backoff method based on IEEE 802.15.4 protocol for differentiated services

Abstract

The invention belongs to the technical field of wireless sensor networks, and specifically relates to an intelligent CSMA / CA backoff method based on the IEEE802.15.4 protocol of differentiated services, comprising dividing data services arriving at a node into high priority and low priority, and maintaining two queues, a high priority queue and a low priority queue, at each node; setting a first greediness rate, selecting services from the low priority queue according to the first greediness rate to enter the high priority queue for channel access; constructing a reinforcement learning model, taking the wireless sensor as an intelligent agent, and constructing a reward value table according to rewards obtained from actions performed by the intelligent agent in various states; and performing channel access by using the constructed reinforcement learning model. The invention comprehensively considers factors such as data of different priorities, channel access conditions, and selection of backoff parameters, and can effectively improve the channel access efficiency of data of different priorities and reduce network delay.

Description

Technical Field

[0001] The invention belongs to the technical field of wireless sensor networks, and in particular relates to an intelligent carrier sense multiple access with collision avoidance (CSMA / CA) backoff method based on IEEE802.15.4 protocol for differentiated services. Background Art

[0002] At present, IoT devices have been widely used in various industries, such as medical rescue, precision agriculture, environmental monitoring and other fields, and the large-scale use of IoT devices mainly depends on the development of wireless sensor networks (WSN). WSN uses wireless communication technologies such as WIFI, Bluetooth, LoRa, ZigBee, etc. to conveniently transmit the data collected by IoT devices. Among them, ZigBee technology has been widely used in IoT scenarios such as smart cities and smart agriculture due to its advantages such as self-organizing networks, low power consumption, low cost and low complexity. The implementation of ZigBee technology is based on the IEEE802.15.4 standard protocol, which defines the specifications of the physical layer (PHY) and the medium access control layer (MAC), and is suitable for short-distance, low-rate and low-power WSN networks. In the IEEE802.15.4 protocol, the MAC layer uses a superframe structure to organize the time allocation of channel occupancy. The superframe period is the time between two beacon frames and consists of an active period and an inactive period. The communication of nodes is performed by using carrier sense multiple access / collision avoidance mechanism or guaranteed time slot (GTS) mechanism for channel access during the active period, and the nodes enter the dormant state during the inactive period.

[0003] In WSN networks, event-triggered data and periodically collected data have significantly different requirements for transmission delay, and it is necessary to ensure the real-time and reliability of emergency data transmission. In order to provide real-time communication services for emergency services, the IEEE 802.15.4 protocol adopts the GTS mechanism to access the channel in a contention-free manner, but the GTS mechanism has the following limitations: First, the GTS request and GTS revocation command sent by the node need to use the CSMA / CA mechanism and data frames to compete for the right to use the channel during the contention access period (CAP). For the transmission of time-sensitive events, the coordinator cannot receive the GTS request of the node in time, thereby increasing the delay of the node using the GTS time slot; second, there are only 7 GTS time slots in each superframe, which limits the use of the GTS mechanism in large-scale networks.

[0004] In short, how to provide better differentiated services for IEEE 802.15.4 networks and implement an efficient channel access mechanism to reduce network latency and improve reliability has become an urgent problem to be solved. Researchers are currently studying this issue mainly from the perspective of improving the GST mechanism and CSMA / CA mechanism. The improved GTS mechanism algorithms proposed by many scholars seek better GTS allocation methods in limited GTS resources, and have achieved certain advantages in network performance. However, due to the limitations of the 802.15.4 protocol itself, the GTS mechanism can only provide real-time communication for a small number of nodes and cannot meet the requirements of most network nodes for providing differentiated services. Therefore, the contention access period with a larger time slot range has also become a research focus.

[0005] The contention access period mainly uses the CSMA / CA mechanism for channel access. Since the channel only allows one node to transmit data at the same time, a binary backoff algorithm is used in the CSMAC / CA mechanism to avoid conflicts. When a device uses the CSMA / CA mechanism to access, it first maintains three variables: the number of backoffs (NB), the contention window length (CW), and the backoff index (BE). The backoff number will increase by 1 and retransmit each time the channel access fails, until the maximum backoff number macMaxCSMABackoff defined by the system is reached; the length of the contention window indicates the number of times the device needs to perform a clear channel assessment (CCA) before accessing the channel; the backoff index indicates the duration of each backoff from (0, 2 BE -1) Randomly delay a backoff cycle within the range.

[0006] Each time the CSMA / CA mechanism monitors that the channel is busy, it will increase the size of the backoff window in a binary exponential manner. As the network load continues to increase, the binary backoff algorithm of the CSMA / CA mechanism does not show good robustness. The CSMA / CA mechanism adopts a blind backoff strategy. The blind increase of the backoff exponent BE and the contention window CW will affect network access, increase the probability of network conflicts and retransmissions, and cause node packet failures, resulting in network congestion. In addition, the CSMA / CA mechanism cannot respond to business needs with special delay requirements in a timely manner. Emergency data (GTS request frames, command frames, data frames carrying alarm information, etc.) need to compete with ordinary data frames for time slots for transmission, resulting in increased delays for emergency business data and failure to deliver them in a timely manner. Summary of the invention

[0007] Aiming at the problems that the backoff parameter of the improved algorithm is not adaptive enough at the current research stage, the channel access of priority services has high delay and low channel utilization, the CSMA / CA algorithm in the IEEE 802.15.4 protocol is improved, and an intelligent CSMA / CA backoff method based on the IEEE 802.15.4 protocol for differentiated services is proposed, which includes the following steps:

[0008] S1: Divide the data services arriving at the node into high priority and low priority. Each node maintains two queues: a high priority queue and a low priority queue.

[0009] S2, setting a first greedy rate, selecting services from the low priority queue according to the first greedy rate to enter the high priority queue for channel access;

[0010] S3, build a reinforcement learning model, take the wireless sensor as an intelligent agent, and build a reward value table based on the rewards obtained by the actions performed by the intelligent agent in each state;

[0011] S4. Use the constructed reinforcement learning model to access the channel.

[0012] Furthermore, when initializing the backoff number and the contention window length, if it is a high priority queue task, the backoff number is initialized to the high priority backoff number, and the contention window length is initialized to 2; if it is a low priority queue task, the backoff number is initialized to the low priority backoff number, and the contention window length is initialized to 3, where the high priority backoff number is less than the low priority backoff number.

[0013] Furthermore, channel access using the constructed reinforcement learning model includes the following steps:

[0014] Initialize the reward value table, learning efficiency, greed rate, reward value and backoff times of the sensor node;

[0015] Determine the priority of the node data packet and initialize the backoff index and contention window length of the data packet according to the priority;

[0016] Positioning with backoff period, if the current period is the exploration period, the sensor node accesses the channel according to the CSMA / CA algorithm; if the current period is the utilization period, the node adopts the ε-greedy strategy for channel access, and selects the backoff index corresponding to the maximum reward value from the reward value table with the second greedy rate ε to perform backoff.

[0017] Furthermore, channel access is performed according to the CSMA / CA algorithm, that is, in (0,2 BE -1) randomly delays a backoff period in the range of BE, where BE represents the value of the backoff exponent.

[0018] Furthermore, the ε-greedy strategy is used for channel access, that is, a random number n is generated at (0,1). When n is greater than the value of the second greedy rate ε, a backoff index corresponding to the maximum reward value is selected from the reward value table as the current backoff index, and then the backoff index is generated at (0,2 BE -1) in the range of a random delay of one backoff period; where BE represents the value of the backoff exponent.

[0019] Furthermore, the update of the reward value table includes:

[0020] Perform CCA at the backoff period boundary and determine the channel status;

[0021] If the channel is empty, the contention window length is reduced by 1. If the contention window length is 0, the reward value table is updated according to the current access status and the value of the backoff index, and the backoff ends. If the contention window length is not 0, the CCA judgment continues.

[0022] If the channel is not empty, the contention window length is updated according to the priority of the data packet, and the reward value table is updated according to the current channel status and the value of the backoff index;

[0023] Update the backoff index of the data packet and determine whether the backoff index is within the maximum backoff range. If so, increment the backoff index by 1 and continue to perform the CCA judgment; otherwise, update the reward value table according to the current access status and the value of the backoff index;

[0024] Among them, when the contention window length is 0, the access state of the current channel is access success; when the channel is not empty, the access state of the current channel is busy; when the channel is not empty and the backoff index exceeds the maximum backoff range, the channel state is access failure.

[0025] Furthermore, the reward value after backoff is performed according to the backoff index is expressed as:

[0026] Q t(BE) = Q t-1 (BE)+α[r t -Q t-1 (BE)];

[0027] Among them, Q t (BE) represents the reward value after backoff according to the backoff index BE at the current time t, α is the learning rate; r t Represents the reward value obtained by channel access at the current time t.

[0028] Furthermore, the reward value r obtained by channel access at the current time t is t It is expressed as:

[0029]

[0030] Among them, when the channel access is successful, the reward value is 1, when the channel access fails, the reward value is -1, and when the channel access is busy, the reward value is -0.5.

[0031] The present invention is an intelligent time slot CSMA / CA backoff method for the IEEE802.15.4 protocol in a wireless sensor network. The method comprehensively considers factors such as data of different priorities, channel access conditions, and the selection of backoff parameters to form a reinforcement learning CSMA / CA backoff algorithm based on differentiated services. The algorithm can effectively improve the channel access efficiency of data of different priorities and reduce network delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic diagram of an IEEE 802.15.4 network based on a priority queue according to the present invention;

[0033] Figure 2 It is a schematic diagram of the CSMA / CA algorithm based on the IEEE 802.15.4 standard;

[0034] Figure 3 Design of the priority queue queuing process of the present invention;

[0035] Figure 4 It is a schematic diagram of the reinforcement learning model of the present invention;

[0036] Figure 5 It is a flow chart of the implementation of the CSMA / CA intelligent backoff algorithm based on differentiated services of the present invention;

[0037] Figure 6 This is a comparison chart of the network delay between the original protocol (Original) and the algorithm of the present invention (QCSMA) simulated by OMNET++;

[0038] Figure 7It is a comparison chart of the delivery rate between the original protocol and the algorithm of the present invention simulated by OMNET++;

[0039] Figure 8 It is a comparison chart of average energy consumption of the terminal and coordinator of the original protocol simulated by OMNET++ and the algorithm of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] The present invention proposes an intelligent CSMA / CA backoff method based on the IEEE 802.15.4 protocol for differentiated services, comprising the following steps:

[0042] S1: Divide the data services arriving at the node into high priority and low priority. Each node maintains two queues: a high priority queue and a low priority queue.

[0043] S2, setting a first greedy rate, selecting services from the low priority queue according to the first greedy rate to enter the high priority queue for channel access;

[0044] S3, build a reinforcement learning model, take the wireless sensor as an intelligent agent, and build a reward value table based on the rewards obtained by the actions performed by the intelligent agent in each state;

[0045] S4. Use the constructed reinforcement learning model to access the channel.

[0046] The present invention is mainly used in the channel access process of time-sensitive services. Since the time-sensitive services still need to compete with ordinary data frames for access to the channel in the CAP stage, the transmission delay is high and the channel utilization rate is low. The purpose of the present invention is to improve the CSMA / CA algorithm in the IEEE 802.15.4 protocol, and propose an intelligent CSMA / CA backoff method based on the IEEE 802.15.4 protocol for differentiated services, so as to improve the channel access efficiency of the priority services and reduce the network transmission delay.

[0047] Figure 1The present invention is a schematic diagram of an IEEE 802.15.4 network based on a priority queue, which includes an upper layer (Upper Layers), an IEEE 802.15.4 link control sublayer (logical link control, LLC), a physical medium (Physical Medium), and a service specific convergence sublayer (service specific convergence sublayer, SSCS). The MAC sublayer provides two services: the MAC Common Part Sublayer (MAC Common Part Sublayer, MCPS) provides data services and the MAC Subplayer Management Entity (MAC Subplayer Management Entity, MLME) provides management services. MCPS and MLME are connected to the MAC layer through their access points, namely MCPS-SAP and MLME-SAP.

[0048] In this embodiment, the channel access efficiency of node priority services is improved from the perspectives of service priority allocation, data packet queuing strategy, and adaptive optimization of backoff parameters. First, in view of the fact that emergency services need to compete with data frames for time slots during the CAP phase, a service priority queuing strategy is designed to provide services for data of different priorities; on this basis, in order to further improve the channel access efficiency, the existing CSMA / CA improved algorithm is studied, and then the reinforcement learning method is applied to the node channel access scenario, which specifically includes the following steps:

[0049] (1) In order to enable nodes to differentiate services during the CAP contention access phase, in this embodiment, different services generated by the nodes are first differentiated, and data services are divided into two priorities, namely, high priority and low priority. Each node will maintain two priority queues, namely, a high priority queue and a low priority queue.

[0050] (2) According to a certain probability, low-priority services are selected to enter the high-priority queue for channel access.

[0051] (3) Define the node backoff period decision problem in wireless sensor networks and build a reinforcement learning model. In the node reinforcement learning model, the agent represents the wireless sensor node with independent decision-making ability, the action represents the backoff parameter BE value selected by the node in the CSMA / CA algorithm, the environment represents the wireless sensor network channel environment information, the state represents the wireless sensor network environment information perceived by the sensor node, and the changes caused by its own actions, the policy represents the way the sensor node selects the backoff parameter BE value in the current state, and the reward represents the reward obtained by the sensor node after selecting and executing the backoff parameter.

[0052] (4) In reinforcement learning, each agent performs a specific action in a specific state. Each action has a reward. Based on the reward obtained, the agent will move to the next state. For each state-action pair, the agent will have a unique state value. The state value function is used to represent the long-term cumulative reward of the sensor node and the wireless sensor network channel environment through trial and error. It represents how good the agent is to take a certain action in a state. In this way, the node backoff parameter decision problem is transformed into the node learning optimal strategy problem, that is, how to choose the best backoff parameter value in the current state.

[0053] (5) The node uses the reinforcement learning model to access the channel. First, the reinforcement learning parameters (Q table, learning rate α, second greedy rate ε, reward value r) and back-off times of the terminal sensor node are initialized.

[0054] (6) During the channel access process, the priority of the node data packet is determined, and the initial backoff index BE and contention window length CW are determined based on the priority information.

[0055] (7) When locating the backoff period, if the current period is the exploration period, the node accesses the channel according to the CSMA / CA algorithm; if the current period is the utilization period, the node adopts the ε-greedy strategy for channel access and selects the BE value with the maximum Q value for backoff with the second greedy rate ε.

[0056] (8) A reward r is given according to the result of channel access, and the Q table is updated according to the current backoff index BE.

[0057] (9) Update the NB, CW, and BE values ​​to determine whether the channel access is successful. If the channel access fails but the NB does not exceed the maximum backoff range RangeBackoff, then go to (7). If the channel access fails and the NB exceeds the maximum backoff range RangeBackoff, then go to (6). If the channel access is successful, the backoff ends.

[0058] As an implementation method, this embodiment provides Figure 5 The flowchart shown provides an implementation method based on distinguishing data of different priorities and using reinforcement learning for intelligent backoff, which specifically includes the following steps:

[0059] Step 1: As an optional implementation, this embodiment distinguishes data of different priorities by adding a priority field to the header of the application layer data, for example, using the field PH to represent high priority data and the field PL to represent low priority data.

[0060] Step 2: Set up two priority queues on the MAC upper layer. High-priority data enters the high-priority queue. At the same time, the high-priority queue selects low-priority services according to a certain probability to enter the high-priority queue for channel access. The priority queue queuing process is as follows: Figure 3 As shown, in the process of processing data, data in high priority queues are processed first, and then data in low priority queues are processed until all queues complete transmission.

[0061] Step 3: Construct a node reinforcement learning model based on the characteristics of the wireless sensor network. Figure 4 As shown. There is only one state for a node, and it learns through continuous "trial and error" and accumulates more "experience". The node reinforcement learning model needs to design action sets, rewards, Q value updates, and strategy selection:

[0062] (1) Action Set

[0063] The action set A of the wireless node is the selection range of the backoff parameter BE [macMinBE, macMaxBE]. In order to distinguish different services, different priority data are divided into different action sets. The action set of high priority data is expressed as [macMinBE PH ,macMaxBE], the action set of low priority data is expressed as [macMinBE PL ,macMaxBE], the action set selection is shown in formula (1):

[0064]

[0065] Among them, data indicates the upper layer data received by the MAC layer, PH indicates that the data is of high priority, and PL indicates that the data is of low priority; macMinBE PH Indicates the minimum backoff index value of high priority data, macMinBE PL Indicates the minimum backoff index value for low priority data, and macMinBE PH The value is lower than macMinBE PL , providing a wider backoff period selection for high-priority data.

[0066] (2) Rewards

[0067] Reward is the feedback given by the environment, which indicates the direct benefit of taking a certain action in a specific state. In wireless sensor network channel access, the ultimate goal of the node is to successfully compete for the right to use the channel. At the same time, the backoff index BE when the channel is busy must also be punished. Therefore, the specific expression of reward is as follows:

[0068]

[0069] Among them, success means that the data transmission node successfully competes for the channel use right, fail means that the competition fails, and busy means that CCA detects that the current channel is busy.

[0070] (3) Q value update

[0071] The Q value, i.e. the reward value, indicates how much reward can be obtained in the future by executing the action in the current state. Since the wireless sensor node only defines one state, each action (backoff index BE) corresponds to an expected value of the reward, which is expressed as formula (3):

[0072] Q t (BE) = Q t-1 (BE)+α[r t -Q t-1 (BE)] (3)

[0073] Among them, the learning rate α indicates how fast the update rate is, r t Indicates the reward obtained by the node's current channel access.

[0074] (4) Strategy selection

[0075] In order to balance the exploration and utilization of nodes in IEEE 802.15.4 wireless sensor networks, this algorithm adopts the ε-greedy strategy, as shown in formula (4). That is, in each stage, a backoff action a is selected. t When , the backoff parameter BE with the maximum Q value is selected with the second greedy rate ε, and the BE value is selected according to the CSMA / CA mechanism with probability (1-ε):

[0076]

[0077] Among them, a t Indicates the backoff strategy. When the strategy value is When , it means that the backoff parameter BE with the maximum Q value is selected according to the second greedy rate ε to perform the backoff operation; when the policy value is BE, it means that the BE value is selected according to the CSMA / CA mechanism with probability (1-ε).

[0078] Step 4: The node has data to send in the CAP phase and enters the initialization phase. Initialize the reinforcement learning parameters (Q table, learning rate α, second greed rate ε, reward value r) and NB value. If the data is high priority, the initial value of the backoff index BE is macMinBE PH , the contention window length CW is 2; if the data is of low priority, the initial value of the backoff index BE is macMinBE PL , the value of the contention window length CW is 3.

[0079] Step 5: Locate the backoff period boundary and determine the current period. If the current period is the exploration period, then in (0,2 BE -1) randomly delays a backoff period; if the current period is the utilization period, first generate a random number n in (0,1), when n is greater than the second greedy rate value, select the action with the largest Q value in the Q table and assign it to the backoff index BE, then in (0,2 BE -1) Randomly delay a backoff cycle within the range.

[0080] Step 6: Perform CCA at the backoff period boundary. If the channel is empty, the contention window length CW is reduced by 1. Then determine whether the contention window length CW is 0. If the contention window length CW is 0, it means that the access is successful, the network feedback reward r=1, and the Q table of the current BE value is updated, and the backoff ends. If CW is not 0, continue to perform CCA judgment; if the channel is not empty, then determine the priority of the data. If the data is high priority, update the contention window length CW to 2. Otherwise, update the contention window length CW to 3. At the same time, the network feedback reward r=-0.5 and update the Q table of the current BE value. Continue to judge downward.

[0081] Step 7: BE selects the smallest one in (BE+1, macMaxBE), NB increases by 1, and determines the NB value. If NB does not exceed the maximum backoff range RangeBackoff, it proceeds to step 5; otherwise, the node fails to access the channel, and the network feedback reward r=-1, and updates the Q table of the current BE value. This backoff ends.

[0082] The intelligent CSMA / CA backoff method based on the IEEE 802.15.4 protocol of differentiated services proposed in the present invention uses reinforcement learning to create Q values ​​for the backoff parameters BE of different priority data to effectively improve the CSMA / CA backoff strategy. The algorithm can adaptively adjust the backoff parameters according to different channel access environments, so that nodes can efficiently access the channel. In order to verify the effectiveness of the algorithm proposed in this embodiment, the IEEE802.15.4 protocol model provided by the OMNET++ network simulation software is used to simulate and analyze the improved protocol. According to the technical solution steps proposed in the present invention, OMNET++ is used for simulation, and the simulation scenario is as follows:

[0083] The simulation uses a star network enabled by IEEE 802.15.4 beacons for communication;

[0084] A total of 10 wireless nodes were tested. The network center is the coordinator node, which is located in the center of the (200m×200m) site. Other terminal nodes are randomly distributed, so hidden terminals may appear. At the same time, two data with different priorities are randomly generated at the terminal nodes with probability ω.

[0085] The simulation parameters are shown in Table 1.

[0086] Table 1 802.15.4 simulation parameters

[0087]

[0088] End-to-end delay is the time required for a data packet to travel from source to destination. Data packets may be delayed due to different duty cycles, collisions in the transmission channel caused by the CSMA / CA backoff mechanism, and affected by external interference, CCA mechanism, etc. In this paper, the duty cycle (SO, BO) of the superframe structure is fixed, and priority queuing and QCSMA are used to adjust the backoff parameters for adaptive backoff to improve the efficient access of nodes in the contention channel. Figure 6 The simulation results of the end-to-end delay of two priority data under different traffic conditions using the original 802.15.4 protocol and the QCSMA algorithm of the present invention are shown. Compared with the standard protocol, the average end-to-end delay of high priority data of QCSMA is reduced by 13.35%. It is worth noting that this embodiment uses a 50% probability to randomly generate priority data, so the difference in delay between low priority data and high priority data is larger due to the increase in high priority data.

[0089] The delivery rate mainly refers to the ratio of the number of data frames sent by the sending node to the number of data frames successfully received by the receiving data node. The delivery rate is mainly affected by the contention access channel. Data packets will be discarded after multiple retransmission failures. At the same time, due to the problem of hidden terminals, data packets transmitted by nodes in the wireless channel will collide, resulting in the coordinator being unable to successfully receive data packets. This paper mainly uses the QCSMA algorithm to improve the access performance of the CSMA / CA mechanism. The QCSMA algorithm adjusts the backoff parameter BE for adaptive backoff according to different network environments, and provides differentiated services by setting priority action selection sets and different contention windows CW. Figure 7 The simulation results of the delivery rate of two priority data under different traffic conditions are shown. Compared with the standard protocol, the packet delivery rate of the QCSMA algorithm is improved by 32.79%. The improvement in delivery rate is mainly due to the adaptive selection of the backoff parameter BE by the QCSMA algorithm according to different network environments. Under different CBR traffic conditions, the QCSMA algorithm rewards and punishes the backoff parameter BE according to the channel access situation, thereby optimizing the selection of the backoff value BE, so that it selects the best BE value for channel access, and reduces node competition.

[0090] The energy consumption of nodes is affected by the number of nodes, duty cycle, efficiency of channel access mechanism, hidden terminal problem and collision when transmitting data packets. The reason for the influence of channel access mechanism is that the node adopts binary blind backoff strategy. When the channel is detected to be busy before frame transmission, the backoff index BE and contention window CW will increase blindly. Figure 8 The average energy consumption per packet per node shown shows that the proposed QCSMA algorithm improves the energy efficiency of nodes in the network. Compared with the original protocol, the average energy consumption per packet of the terminal node in the QCSMA algorithm is reduced by 23.8%. The main reason for the reduction in energy consumption per packet is that the QCSMA algorithm improves the node delivery rate without increasing the total energy consumption of the node, reduces the number of packet collisions, and enables the node to access the channel as much as possible and send data successfully.

[0091] After distinguishing different priority data, the present invention uses priority queues to provide different priority services; on this basis, a QCSMA adaptive backoff algorithm is proposed. The QCSMA algorithm can adaptively adjust the backoff parameters according to different channel access environments, so that nodes can efficiently access the channel. Through OMNET++ simulation verification, the scheme is superior to the original 802.15.4 protocol in terms of latency, delivery rate, node energy efficiency, etc., ensuring the reliable transmission of wireless sensor networks.

[0092] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent CSMA / CA backoff method based on the IEEE 802.15.4 protocol for differentiated services, characterized in that: The following steps are involved: S1: Divide the data services arriving at the node into high priority and low priority. Each node maintains two queues: a high priority queue and a low priority queue. S2, setting a first greedy rate, selecting services from the low priority queue according to the first greedy rate to enter the high priority queue for channel access; S3, build a reinforcement learning model, take the wireless sensor as an intelligent agent, and build a reward value table based on the rewards obtained by the actions performed by the intelligent agent in each state; S4, using the constructed reinforcement learning model for channel access, that is, in (0,2 BE -1) randomly delays a backoff period within the range of BE, where BE represents the value of the backoff index, including the following steps: Initialize the reward value table, learning efficiency, greed rate, and backoff times of the sensor node; Determine the priority of the node data packet, initialize the backoff index and contention window length of the data packet according to the priority, and when initializing the backoff times and contention window length, if it is a high priority queue task, the backoff times are initialized to the high priority backoff times, and the contention window length is initialized to 2; if it is a low priority queue task, the backoff times are initialized to the low priority backoff times, and the contention window length is initialized to 3, where the high priority backoff times are less than the low priority backoff times; Positioning with backoff cycle, if the current period is the exploration period, the sensor node accesses the channel according to the CSMA / CA algorithm; if the current period is the utilization period, the node adopts the ε-greedy strategy for channel access, and selects the backoff index corresponding to the maximum reward value from the reward value table with the second greedy rate ε to perform backoff, that is, a random number n is generated in (0,1), when n is greater than the value of the set second greedy rate ε, a backoff index corresponding to the maximum reward value is selected from the reward value table as the current backoff index, and then in (0,2 BE -1) Randomly delay a backoff cycle within the range; Updates to the Reward Value table include: Perform CCA at the backoff period boundary and determine the channel status; If the channel is empty, the contention window length is reduced by 1. If the contention window length is 0, the reward value table is updated according to the current access status and the value of the backoff index, and the backoff ends. If the contention window length is not 0, the CCA judgment continues. If the channel is not empty, the contention window length is updated according to the priority of the data packet, and the reward value table is updated according to the current channel status and the value of the backoff index; Update the backoff index of the data packet and determine whether the backoff index is within the maximum backoff range. If so, increment the backoff index by 1 and continue to perform the CCA judgment; otherwise, update the reward value table according to the current access status and the value of the backoff index; Among them, when the contention window length is 0, the access status of the current channel is access success; when the channel is not empty, the access status of the current channel is busy; when the channel is not empty and the backoff index exceeds the maximum backoff range, the channel status is access failure; The reward value after backoff is expressed as: Q t (BE)=Q t-1 (BE)+α[r t -Q t-1 (BE)]; Among them, Q t (BE) represents the reward value after backoff according to the backoff index BE at the current time t, α is the learning rate; r t It represents the reward value obtained by channel access at the current time t, that is: Among them, when the channel access is successful, the reward value is 1, when the channel access fails, the reward value is -1, and when the channel access is busy, the reward value is -0.5.

Images