Wireless sensor network conflict avoidance and service guarantee channel allocation method and system

Abstract

The invention discloses a wireless sensor network conflict avoidance and service guarantee channel allocation method and system, and the method comprises the steps: predicting the electromagnetic interference intensity of each channel, and obtaining an electromagnetic interference intensity value; the method comprises the following steps: constructing a channel allocation model based on multi-agent reinforcement learning by taking a sink node as an agent, interacting with a wireless sensor network environment through the agent, and performing iterative training on the channel allocation model by adopting a composite reward function to obtain output channel allocation strategies under different electromagnetic interference intensities and different data transmission requirements; and the aggregation node distributes an optimal channel to the sensing node in real time based on a channel distribution strategy output by a channel distribution model according to a received data transmission request sent by the sensing node and in combination with an electromagnetic interference intensity value and a current channel distribution condition. According to the invention, wireless channel allocation in scenes such as wide-narrow band wireless hybrid networking and the like can be more effectively guaranteed, channel conflicts of the same system and different systems are reduced, the frequency efficiency is improved, and the maximum transmission delay of services is reduced.

Description

Technical Field

[0001] The present application belongs to the technical field of wireless sensor network channel allocation for power systems, and specifically relates to a method and system for wireless sensor network conflict avoidance and service guarantee channel allocation. Background Art

[0002] In wireless sensor networks, channel allocation mechanism is an important means to reduce interference and ensure efficient and reliable data transmission. For wireless sensor networks with many sensor nodes and multiple sub-networks, how to reasonably allocate limited channel resources to reduce co-channel interference and improve spectrum efficiency is one of the important issues that need to be considered urgently. Especially in the substation scenario with a more complex electromagnetic environment, the incompletely known electromagnetic environment poses a greater challenge to the reliable and efficient transmission of business data.

[0003] The current channel allocation methods in wireless sensor networks are mainly divided into two categories. One is the allocation method for time-invariant channels, such as distributed channel allocation methods. In this type of method, each user or node selects a suitable channel or resource block based on its own channel quality and needs. The channel state in the substation scenario is affected by electromagnetic interference and generally presents dynamic changes, which makes it difficult for such methods to cope with. The other is the allocation method for time-varying channels, such as using feedback neural networks to predict time-varying channels or dynamically allocating channels based on the time-varying characteristics of channels to optimize the utilization of spectrum resources.

[0004] However, the above-mentioned prior arts do not take into account the electromagnetic interference of the substation environment, and are not suitable for the substation environment where electromagnetic interference exists. Summary of the invention

[0005] In order to solve the deficiencies in the prior art, the present invention provides a wireless sensor network conflict avoidance and service guarantee channel allocation method, which solves the problem that the existing channel allocation method is not suitable for substation environments with electromagnetic interference and cannot guarantee service transmission delay.

[0006] The present invention adopts the following technical solution.

[0007] A first aspect of the present invention provides a method for allocating conflict avoidance and service guarantee channels in a wireless sensor network, which is applied to a wireless sensor network in a substation environment. The wireless sensor network includes sensor nodes, aggregation nodes, and access nodes. The sensor nodes collect operating data of substation equipment, transmit the data to the aggregation node of the subnetwork where the data is located through the allocated channel, and the aggregation node forwards the data to the access node. The method includes the following contents:

[0008] Predict the electromagnetic interference intensity of each channel to obtain the electromagnetic interference intensity value;

[0009] Taking the sink node as the intelligent agent, a channel allocation model is constructed based on multi-agent reinforcement learning. Through the interaction between the intelligent agent and the wireless sensor network environment, the channel allocation model is iteratively trained to obtain the channel allocation strategy under different electromagnetic interference intensities and different data transmission requirements.

[0010] The sink node allocates the optimal channel to the sensor node in real time based on the channel allocation strategy output by the channel allocation model according to the data transmission request sent by the sensor node, the electromagnetic interference intensity value and the current channel allocation situation.

[0011] Optionally, the electromagnetic interference intensity of each channel is predicted based on a pre-trained electromagnetic interference intensity prediction model to obtain an electromagnetic interference intensity value.

[0012] Optionally predicting the electromagnetic interference intensity of each channel, obtaining the electromagnetic interference intensity value includes:

[0013] All sensor nodes are disconnected from the aggregation node to obtain complete electromagnetic interference intensity data in continuous time slots of each channel;

[0014] After normalizing the electromagnetic interference intensity data, the data are grouped into a training data set and a prediction data set, and the electromagnetic interference intensity data in each data set are processed into a unified format;

[0015] The electromagnetic interference intensity prediction model is trained using the complete electromagnetic interference intensity data in each channel's continuous time slots. During the training process, the batch is set to 1 and the epoch is set to 4.

[0016] All sensor nodes are connected to the aggregation node as needed. When the channel is idle, the measured value is obtained as the electromagnetic interference intensity value of the channel. When the channel is occupied, the predicted value output by the electromagnetic interference intensity prediction model is used as the electromagnetic interference intensity value of the channel.

[0017] Optionally, the compound reward function is expressed as follows:

[0018]

[0019] in,

[0020] r c (t) is the transmission rate reward for the tth time slot,

[0021] r f (t) is the delay penalty for the tth time slot,

[0022] is the weight coefficient of the transmission rate reward,

[0023] is the weight coefficient of delay penalty.

[0024] Optionally, the weight coefficient of the transmission rate reward and the weight coefficient of the delay penalty are calculated according to the following formula:

[0025]

[0026]

[0027] in,

[0028] is the electromagnetic interference threshold of channel l,

[0029] is the electromagnetic radiation interference of channel l in the substation environment at the tth time slot.

[0030] Optionally, the transfer rate reward is calculated according to the following formula:

[0031]

[0032]

[0033] in,

[0034] r c (t) is the transmission rate reward for the tth time slot,

[0035] K l (t) is the co-channel interference of the sub-network where the adjacent sink node in the tth time slot is located,

[0036] is the electromagnetic radiation interference of channel l in the substation environment at the tth time slot,

[0037] The sink node numbered n in the tth time slot is numbered n j The decision of the sensor node to allocate channel l,

[0038] σ is the ambient noise power,

[0039] P is the transmission power of the sensor node.

[0040] Optionally, the sensor node determines whether the task is completed within the maximum transmission delay of the service. If completed, the delay penalty is 0, otherwise, the delay penalty is a preset value, the preset value is greater than 0, and when the size of the remaining service data packets decreases to a negative number, the number of the remaining service data packets is set to 0;

[0041] The delay penalty is calculated according to the following formula:

[0042]

[0043] in,

[0044] For number n j The maximum service transmission delay required by the sensor node transmission task,

[0045] For number n j The remaining service data packet size of the sensor node,

[0046] For number n j The amount of data transmitted from the sensor node to the sink node numbered n through channel l.

[0047] Optionally, the agent's loss is calculated as follows:

[0048]

[0049] in,

[0050] Q tot (s(t), a(t); θ) is the value function of the agent under the global environment observation state at the tth time slot and the channel allocation decision at the tth time slot,

[0051] s(t) is the global environment observation state at the tth time slot,

[0052] a(t) is the channel allocation decision for the tth time slot, a′(t+1) is the channel allocation decision for the t+1th time slot,

[0053] θ is the network parameter, θ′ is the updated value of the network parameter in the t+1 time slot,

[0054] γ is the reward discount,

[0055] is the sample set,

[0056] r(t) is the compound reward function.

[0057] A second aspect of the present invention provides a wireless sensor network conflict avoidance and service guarantee channel allocation system, which is applied to a wireless sensor network in a substation environment, wherein the wireless sensor network includes a sensor node, a sink node and an access node. The sensor node collects operation data of the substation equipment, transmits the data to the sink node of the subnetwork where the data is located through the allocated channel, and the sink node forwards the data to the access node. The system includes:

[0058] The electromagnetic interference prediction module is used to predict the electromagnetic interference intensity of each channel and obtain the electromagnetic interference intensity value;

[0059] The channel allocation learning module is used to construct a channel allocation model based on multi-agent reinforcement learning with the sink node as the intelligent agent. The channel allocation model interacts with the wireless sensor network environment, learns the experience of channel allocation of each sink node under different electromagnetic interference and data transmission requirements, and outputs a channel allocation strategy.

[0060] The channel allocation module is used to allocate the optimal channel to the sensor node in real time based on the channel allocation model according to the data transmission request sent by the sensor node, the electromagnetic interference intensity value and the current channel allocation situation.

[0061] The third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when loaded into the processor, implements the above-mentioned wireless sensor network conflict avoidance and service guarantee channel allocation method.

[0062] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned wireless sensor network conflict avoidance and service assurance channel allocation method.

[0063] Compared with the prior art, the beneficial effects of the present invention include at least:

[0064] The present invention predicts the electromagnetic interference intensity and allocates channels based on the predicted value. It also takes into account the slow-time-varying electromagnetic interference and the same-frequency interference between AP subnets, more effectively guarantees the wireless channel allocation in scenarios such as wide- and narrow-band wireless hybrid networking, can improve data transmission quality, reduce channel conflicts between the same system and different systems, improve spectrum efficiency, and is more suitable for use in substation environments.

[0065] The present invention is based on a multi-agent reinforcement learning method, which introduces the service transmission delay and service completion status of the sensor node into the utility function, effectively reduces the maximum transmission delay of the service and optimizes the fairness of the service delay guarantee. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0067] Figure 1 A schematic diagram of a wireless sensor network architecture in a substation environment provided by an embodiment of the present invention;

[0068] Figure 2A schematic diagram of a channel allocation process based on multi-agent reinforcement learning provided by an embodiment of the present invention;

[0069] Figure 3 A schematic diagram showing a comparison of average completion times of a data service provided by an embodiment of the present invention;

[0070] Figure 4 A schematic diagram of comparing the maximum completion time of a data service provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The embodiments described in this application are only embodiments of a part of the present invention, rather than all embodiments. Based on the spirit of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the protection scope of the present invention.

[0072] Considering the wireless sensor network in the substation environment, the architecture diagram is as follows Figure 1 As shown. The network contains three types of nodes: SN (sensor node), AP (access point) and GW (gateway). The sensor node collects the operating data of the substation equipment and periodically sends the data to the aggregation node of its subnet, which then forwards it to the access node. On the one hand, when the substation equipment is densely distributed, the number of APs and subnets deployed is large, but the channel resources are limited. Different APs are prone to conflict in allocating channels to sensor nodes requesting access, resulting in high co-frequency interference. On the other hand, there is slow time-varying electromagnetic interference in the substation environment. Both of these factors will affect the effective transmission of data. It is necessary to design a channel allocation method that can avoid channel conflicts, avoid electromagnetic interference, and ensure service transmission delay.

[0073] Embodiment 1 of the present invention provides a method for conflict avoidance and service guarantee channel allocation in a wireless sensor network, comprising the following steps:

[0074] Step 1: Predict the electromagnetic interference intensity of each channel to obtain the electromagnetic interference intensity value.

[0075] Optionally, in step 1, the electromagnetic interference intensity of the substation is predicted based on a pre-trained electromagnetic interference intensity prediction model to obtain an electromagnetic interference intensity value.

[0076] Optionally, the electromagnetic interference intensity of each channel is predicted to obtain the electromagnetic interference intensity value, including: disconnecting all sensor nodes from the aggregation node to obtain complete electromagnetic interference intensity data in continuous time slots of each channel; normalizing the electromagnetic interference intensity data, grouping them into a training data set and a prediction data set, and processing the electromagnetic interference intensity data in each data set into a unified format; inputting the training data set into an electromagnetic interference intensity prediction model for iterative training to obtain a trained electromagnetic interference intensity prediction model; inputting the prediction data set into the trained electromagnetic interference intensity prediction model, and comparing the output value of the electromagnetic interference intensity prediction model with the actual obtained value, and completing the training when the comparison result meets the preset conditions; connecting all sensor nodes to the aggregation node as needed, and when the channel is idle, obtaining the measured value as the electromagnetic interference intensity value of the channel, and when the channel is occupied, using the predicted value output by the electromagnetic interference intensity prediction model as the electromagnetic interference intensity value of the channel.

[0077] Specifically, the preset condition includes that the error between the model output value and the actual acquired value is less than the error threshold, and the error threshold is 1% or 3% or 5%. It is understandable that those skilled in the art can set the specific value of the error threshold according to actual needs.

[0078] In step 1, by normalizing and processing the electromagnetic interference intensity data in each data set into a unified format, the data processing speed can be improved, thereby further improving the subsequent channel allocation efficiency.

[0079] Preferably, the electromagnetic interference intensity of the substation is predicted based on a time series neural network to obtain an electromagnetic interference intensity value. Specifically, it includes:

[0080] Data acquisition phase: First, obtain the training data for electromagnetic interference prediction. In the data acquisition phase, all SNs are not connected to the AP, and the AP uses UHF (Ultra High Frequency) sensors to obtain complete electromagnetic interference data in a continuous time slot on each channel: in It indicates the electromagnetic interference intensity in the 1st to n-1th time slots on the lth channel.

[0081] Training phase: Use LSTM (Long Short Term Memory) time series neural network to predict the electromagnetic interference intensity of the nth time slot First, data preprocessing, preprocessing and normalizing the data obtained in the data acquisition phase, and then grouping the obtained data into training data sets and prediction data sets, accounting for 80% and 20% of the total data set respectively, and then processing them into [x, y, z] data format, where x represents the time step, that is, the length of each sequence, y represents the number of sequences, and z represents the input data dimension, which is set to [100, 1, 1] here. Secondly, model training, input the training data set into the model for training. Due to the small amount of data, batch is set to 1 and epoch is set to 4. Epoch is a unit of the model training process, which controls the number of iterations of the entire training. Batch is a large-scale data set divided into several small blocks for training in the middle of an epoch. Each batch contains a certain number of sample data, which determines the number of samples processed in each iteration, thereby controlling the speed of training. When iterating to the fourth round, the loss value tends to be stable. Finally, model prediction, input the predicted data set into the trained network, and compare the output value with the actual acquisition value. The results are not much different.

[0082] Execution phase: SN accesses AP as needed. AP obtains electromagnetic interference data on idle time and frequency resources. For occupied channels, predicted values ​​are used instead of measured values.

[0083] Step 2: Taking the aggregation node as the intelligent agent, a channel allocation model is constructed based on multi-agent reinforcement learning. The channel allocation model is iteratively trained through the interaction between the intelligent agent and the wireless sensor network environment to obtain the channel allocation strategy under different electromagnetic interference intensities and different data transmission requirements.

[0084] In step 2, the channel allocation model based on multi-agent reinforcement learning takes reducing the service transmission delay as the starting point, and through interactive learning between multiple APs, the learning results of each AP are integrated to finally generate the allocation strategy of the spectrum allocation system.

[0085] The overall process is as follows Figure 2 As shown in the figure, at the beginning of each time slot, all APs observe the local channel environment and the service demand status in the sub-network, obtain the SN channel allocation strategy under this state through exploration and learning, execute the strategy to interact with the substation environment to obtain rewards as rewards to update network parameters, and iterate the training process until the network converges stably.

[0086] Step 2 specifically includes:

[0087] Step 201, the aggregation node acts as an intelligent agent, observes the local channel environment and the service demand status in the sub-network where the intelligent agent is located, and obtains the global environment observation status at the current moment, wherein the global environment observation status at the current moment includes the local information status and electromagnetic interference intensity value observed by the intelligent agent.

[0088] State observation phase: At the initial moment of time slot t, obtain the global environment observation state in is the prediction result of the electromagnetic interference prediction module for the time slot interference, s n (t) is the local information state observed by APn as an independent agent. AP local information Including the location of AP n (t), number n, and the set of SN states of all SNs in the subnet that need to provide channel access SN j Local Status Including the remaining service data packet size in each time slot SN's location No. j , Maximum transmission delay of the service Channel allocation decision at the last moment

[0089] Step 202: Based on the local information state and electromagnetic interference intensity value observed by the agent, a channel allocation decision is made for the sensor node that sends a data transmission request in the sub-network, and the channel allocation decision is executed.

[0090] Decision acquisition phase: Each APn responds to the SNn that makes an access request in the subnet. j Make channel allocation decisions through the formula Assign channels, where is number n j The channel allocation strategy, θ n Represents the network parameters, which are subsequently updated based on the environmental feedback reward. After executing the action, the state is transferred to the next moment.

[0091] Optionally, a gradient descent algorithm is used to update the network parameters.

[0092]

[0093] Among them, α is the learning rate, which is fixed at 0.001 here. is the gradient of the loss function with respect to the network parameters θ.

[0094] Step 203: The intelligent agent interacts with the wireless sensor network environment in the substation environment to obtain a composite reward value corresponding to the execution channel allocation decision.

[0095] Reward feedback stage: The channel decisions obtained by all APs are executed simultaneously and interact with the environment to obtain the compound reward value corresponding to the execution of the channel allocation decision.

[0096] Preferably, the compound reward function is expressed as follows:

[0097]

[0098] in,

[0099] r c (t) is the transmission rate reward for the tth time slot,

[0100] r f (t) is the delay penalty for the tth time slot,

[0101] is the weight coefficient of the transmission rate reward,

[0102] is the weight coefficient of delay penalty.

[0103] Specifically, Initial Setup Adjust the weight coefficient according to the strength of electromagnetic interference in the environment.

[0104] More specifically, the weight coefficient of the transmission rate reward is calculated as follows:

[0105]

[0106] in, is the electromagnetic interference threshold of channel l, is the electromagnetic radiation interference of channel l in the substation environment at the tth time slot.

[0107] In this way, the electromagnetic interference in the environment can be adjusted according to its strength. When the interference is high, the When the interference is low, increase the weight appropriately. In this way, by setting different weight coefficients, the compound reward is suitable for different electromagnetic interference situations, thereby improving the efficiency of the channel allocation model.

[0108] Preferably, the transmission rate reward is calculated according to the following formula:

[0109]

[0110] in,

[0111] r c (t) is the transmission rate reward for the tth time slot,

[0112] K l (t) is the co-channel interference of the sub-network where the adjacent sink node in the tth time slot is located,

[0113] is the electromagnetic radiation interference of channel l in the substation environment at the tth time slot,

[0114] The sink node numbered n in the tth time slot is numbered n j The decision of the sensor node to allocate channel l,

[0115] σ is the ambient noise power,

[0116] P is the transmission power of the sensor node.

[0117] Specifically, the value of σ is -80dBm, and the value of P is 0.1w.

[0118] SN j Transmission to APn through channel l mainly receives two types of interference. One is the co-channel interference of the adjacent APm sub-network. The other is electromagnetic radiation interference from the substation environment.

[0119] Preferably, the sensor node determines whether the task is completed within the maximum transmission delay of the service. If completed, the delay penalty is 0, otherwise, the delay penalty is a preset value, the preset value is greater than 0, and when the size of the remaining service data packets decreases to a negative number, the number of remaining service data packets is set to 0;

[0120] The delay penalty is calculated according to the following formula:

[0121]

[0122] in,

[0123] For number n j The maximum service transmission delay required by the sensor node transmission task,

[0124] For number n j The remaining service data packet size of the sensor node,

[0125] For number n j The amount of data transmitted from the sensor node to the sink node numbered n through channel l.

[0126] Specifically, each time slot SN n j Determine whether the service has the maximum transmission delay The completed task is transferred within the time frame, and the completed task is recorded as 0, and the uncompleted task is recorded as a larger number (for example, 1000). j Remaining service data packet size When the packet size decreases to a negative number, the remaining packet size is set to 0.

[0127] Step 204, repeating steps 201 to 203 until the network loss value is minimized, thereby obtaining a trained channel allocation model.

[0128] Preferably, the loss value of the network is calculated according to the following formula:

[0129]

[0130] in,

[0131] Q tot (s(t), a(t); θ) is the value function of the agent under the global environment observation state at the tth time slot and the channel allocation decision at the tth time slot,

[0132] s(t) is the global environment observation state at the tth time slot,

[0133] a(t) is the channel allocation decision for the tth time slot, a′(t+1) is the channel allocation decision for the t+1th time slot,

[0134] θ is the network parameter, θ′ is the updated value of the network parameter in the t+1 time slot,

[0135] γ is the reward discount,

[0136] is the sample set,

[0137] r(t) is the compound reward function.

[0138] Optionally, γ takes the value of 0.9 or 0.8. The value is 50.

[0139] Specifically, in the iterative training phase: when the network is learning, the AP performs joint training to ensure cooperation. First, randomly sample the set of <state, action, reward, next moment state> stored in the cache After training, the loss value of the network is calculated according to the formula and the QMIX network is updated. The QMIX network can handle multiple agents and can calculate the Q value function of each agent in parallel, thereby improving computational efficiency.

[0140] Step 3: The sink node allocates the optimal channel to the sensor node in real time based on the channel allocation strategy output by the channel allocation model according to the data transmission request sent by the sensor node, the electromagnetic interference intensity prediction result and the current channel allocation situation.

[0141] Step 3 specifically includes:

[0142] Step 301, (task generation phase) In multiple AP subnetworks, at the beginning of a certain time slot, an SN randomly generates a delay-sensitive service that needs to be transmitted to the AP and transferred to the GW, and submits an access request to the AP.

[0143] Step 302, (deployment model decision phase) When each AP receives an SN request from a subnetwork, it first allocates a channel with a high transmission rate and avoids conflicts as much as possible to the SN that initiates the request based on the current SN location, service data size, service delay requirements, current channel allocation, and current channel electromagnetic interference.

[0144] Step 303: (Execution decision phase) SN receives the channel scheduling assigned by AP and selects the assigned channel for transmission.

[0145] In this embodiment, in response to the electromagnetic interference problem in the substation environment, an interference prediction module based on a time series neural network is designed to predict the electromagnetic interference intensity based on the monitored historical data for subsequent channel allocation strategy training and learning. In response to the problem of co-frequency interference between multiple AP subnets due to channel conflicts, a channel allocation method based on multi-agent reinforcement learning is designed to train each AP agent to learn the optimal channel allocation strategy under different electromagnetic interference and business requirements. Compared with the traditional channel allocation method, this method, on the one hand, takes into account both the slow-varying electromagnetic interference and the co-frequency interference between AP subnets, and is more suitable for substation environments; on the other hand, the utility function of this method not only considers the real-time transmission rate, but also the data transmission delay, which can effectively reduce the maximum transmission delay of the service and optimize the fairness of the service delay guarantee.

[0146] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0147] Embodiment 2 of the present invention provides a wireless sensor network conflict avoidance and service guarantee channel allocation system, which runs the wireless sensor network conflict avoidance and service guarantee channel allocation method as described in Embodiment 1. The system includes: an electromagnetic interference prediction module, a channel allocation learning module and a channel allocation module.

[0148] The electromagnetic interference prediction module is used to predict the electromagnetic interference intensity of each channel and obtain the electromagnetic interference intensity value. It is responsible for predicting the slow-changing electromagnetic interference intensity of the substation.

[0149] The channel allocation learning module is used to build a channel allocation model based on multi-agent reinforcement learning with the aggregation node as the intelligent agent. The channel allocation model is iteratively trained through the interaction between the intelligent agent and the wireless sensor network environment to obtain the channel allocation strategy under different electromagnetic interference intensities and different data transmission requirements.

[0150] The channel allocation module is used to allocate the optimal channel to the sensor node in real time based on the channel allocation strategy output by the channel allocation model according to the data transmission request sent by the sensor node, the electromagnetic interference intensity prediction result and the current channel allocation situation. That is, real-time channel allocation is performed based on the learning result of the channel allocation learning module.

[0151] The optimal channel is the channel with the smallest combined result of conflict, electromagnetic interference impact and service transmission delay.

[0152] Embodiment 3 of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, the wireless sensor network conflict avoidance and service guarantee channel allocation method described in Embodiment 1 is implemented.

[0153] Embodiment 4 of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for wireless sensor network conflict avoidance and service guarantee channel allocation according to embodiment 1 is implemented.

[0154] Embodiment 5 of the present invention is a verification example of the wireless sensor network conflict avoidance and service guarantee channel allocation method provided in Embodiment 1 of the present invention. In order to verify the effectiveness of the proposed method, a numerical simulation experiment is performed on the method. The simulation scenario is set to deploy 1 access node and 5-25 wireless sensor network aggregation nodes in a substation network environment, and 5 sensor nodes are randomly distributed around each aggregation node. Comparing the proposed method with the traditional channel allocation method, the simulation results are as follows: Figure 3 and Figure 4 shown.

[0155] Figure 3 and Figure 4 The average completion time and maximum completion time of the data service are shown respectively. It can be seen that when the number of sensor nodes is small, the average completion time and maximum completion time of the proposed method and the comparison method are not much different; as the number of sensor nodes increases, the probability of co-frequency interference between networks increases, and the average completion time and maximum completion time of the comparison method are significantly higher than the proposed method. This shows that the proposed method can effectively avoid channel conflicts and ensure service transmission. In addition, the comparison Figure 3 and Figure 4It can be seen that the proposed method reduces the maximum completion time more significantly, which shows that the proposed method improves the fairness of service transmission.

[0156] Compared with the prior art, the beneficial effects of the present invention include at least:

[0157] The present invention predicts the electromagnetic interference intensity and allocates channels based on the predicted value. It also takes into account the slow-time-varying electromagnetic interference and the same-frequency interference between AP subnets, more effectively guarantees the wireless channel allocation in scenarios such as wide- and narrow-band wireless hybrid networking, can improve data transmission quality, reduce channel conflicts between the same system and different systems, improve spectrum efficiency, and is more suitable for use in substation environments.

[0158] The present invention is based on a multi-agent reinforcement learning method, which introduces the service transmission delay and service completion status of the sensor node into the utility function, effectively reduces the maximum transmission delay of the service and optimizes the fairness of the service delay guarantee.

[0159] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0160] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0161] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0162] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for allocating channels for conflict avoidance and service assurance in a wireless sensor network, applied to a wireless sensor network in a substation environment, wherein the wireless sensor network comprises sensor nodes, aggregation nodes and access nodes, wherein the sensor nodes collect operating data of substation equipment, transmit the data to the aggregation nodes of the subnetwork where the data is located through the allocated channels, and the aggregation nodes forward the data to the access nodes, characterized in that: The method comprises the following: Predict the electromagnetic interference intensity of each channel to obtain the electromagnetic interference intensity value; Taking the sink node as the intelligent agent, a channel allocation model is constructed based on multi-agent reinforcement learning. Through the interaction between the intelligent agent and the wireless sensor network environment, the channel allocation model is iteratively trained using a composite reward function to obtain the channel allocation strategy under different electromagnetic interference intensities and different data transmission requirements. The sink node allocates the optimal channel to the sensor node in real time based on the channel allocation strategy output by the channel allocation model according to the data transmission request sent by the sensor node, the electromagnetic interference intensity value and the current channel allocation situation.

2. The method for conflict avoidance and service assurance channel allocation in a wireless sensor network according to claim 1, characterized in that: The electromagnetic interference intensity of each channel is predicted based on a pre-trained electromagnetic interference intensity prediction model to obtain an electromagnetic interference intensity value; the electromagnetic interference intensity prediction model is constructed based on an LSTM time series neural network.

3. The method for allocating conflict avoidance and service assurance channels in a wireless sensor network according to claim 2, characterized in that: The electromagnetic interference intensity of each channel is predicted, and the electromagnetic interference intensity value obtained includes: All sensor nodes are disconnected from the aggregation node to obtain complete electromagnetic interference intensity data in continuous time slots of each channel; The electromagnetic interference intensity prediction model is trained using the complete electromagnetic interference intensity data in each channel's continuous time slots. During the training process, the batch is set to 1 and the epoch is set to 4. All sensor nodes are connected to the aggregation node as needed. When the channel is idle, the measured value is obtained as the electromagnetic interference intensity value of the channel. When the channel is occupied, the predicted value output by the electromagnetic interference intensity prediction model is used as the electromagnetic interference intensity value of the channel.

4. The method for conflict avoidance and service assurance channel allocation in a wireless sensor network according to claim 1, characterized in that: The compound reward function is expressed as follows: in, r c (t) is the transmission rate reward for the tth time slot, r f (t) is the delay penalty for the tth time slot, is the weight coefficient of the transmission rate reward, is the weight coefficient of delay penalty.

5. The method for allocating conflict avoidance and service assurance channels in a wireless sensor network according to claim 4, characterized in that: The weight coefficient of the transmission rate reward and the weight coefficient of the delay penalty are calculated according to the following formula: in, is the electromagnetic interference threshold of channel l, is the electromagnetic radiation interference of channel l in the substation environment at the tth time slot.

6. The method for allocating conflict avoidance and service assurance channels in a wireless sensor network according to claim 4, characterized in that: The transfer rate reward is calculated according to the following formula: r c (t) is the transmission rate reward for the tth time slot, K l (t) is the co-channel interference of the sub-network where the adjacent sink node in the tth time slot is located, is the electromagnetic radiation interference of channel l in the substation environment at the tth time slot, The sink node numbered n in the tth time slot is numbered n j The decision of the sensor node to allocate channel l, σ is the ambient noise power, P is the transmission power of the sensor node.

7. The method for conflict avoidance and service assurance channel allocation in a wireless sensor network according to claim 4, characterized in that: The sensor node determines whether the task is completed within the maximum transmission delay of the service. If completed, the delay penalty is 0. Otherwise, the delay penalty is a preset value, which is greater than 0. When the size of the remaining service data packets decreases to a negative number, the number of remaining service data packets is set to 0. The delay penalty is calculated according to the following formula: in, For number n j The maximum service transmission delay required by the sensor node transmission task, For number n j The remaining service data packet size of the sensor node, For number n j The amount of data transmitted from the sensor node to the sink node numbered n through channel l.

8. The method for conflict avoidance and service assurance channel allocation in a wireless sensor network according to any one of claims 1 to 7, characterized in that: The loss value of the agent is calculated according to the following formula: in, Q tot (s(t), a(t); θ) is the value function of the agent under the global environment observation state at the tth time slot and the channel allocation decision at the tth time slot, s(t) is the global environment observation state at the tth time slot, a(t) is the channel allocation decision for the tth time slot, a′(t+1) is the channel allocation decision for the t+1th time slot, θ is the network parameter, θ′ is the updated value of the network parameter in the t+1 time slot, γ is the reward discount, is the sample set, r(t) is the compound reward function.

9. A wireless sensor network conflict avoidance and service assurance channel allocation system using the wireless sensor network conflict avoidance and service assurance channel allocation method according to any one of claims 1 to 8, applied to a wireless sensor network in a substation environment, wherein the wireless sensor network comprises sensor nodes, aggregation nodes and access nodes, the sensor nodes collect operating data of substation equipment, transmit the data to the aggregation node of the subnet to which it belongs through the allocated channel, and the aggregation node forwards the data to the access node, characterized in that: The system comprises: The electromagnetic interference prediction module is used to predict the electromagnetic interference intensity of each channel and obtain the electromagnetic interference intensity value; The channel allocation learning module is used to build a channel allocation model based on multi-agent reinforcement learning with the aggregation node as the intelligent agent. Through the interaction between the intelligent agent and the wireless sensor network environment, the channel allocation model is iteratively trained using a composite reward function to obtain the channel allocation strategy under different electromagnetic interference intensities and different data transmission requirements; The channel allocation module is used to allocate the optimal channel to the sensor node in real time based on the channel allocation strategy output by the channel allocation model according to the data transmission request sent by the sensor node, the electromagnetic interference intensity value and the current channel allocation situation.

10. An electronic device, comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the wireless sensor network conflict avoidance and service guarantee channel allocation method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the wireless sensor network conflict avoidance and service guarantee channel allocation method described in any one of claims 1 to 8 are implemented.

Images