A method and system for collision avoidance and traffic guarantee channel allocation in wireless sensor networks

Abstract

The application discloses a wireless sensor network conflict avoidance and service guarantee channel allocation method and system, the electromagnetic interference intensity of each channel is predicted, and an electromagnetic interference intensity value is obtained; a sink node is taken as an intelligent agent, a channel allocation model is constructed based on multi-agent reinforcement learning, the intelligent agent and the wireless sensor network environment are interacted, a composite reward function is used for iterative training of the channel allocation model, and an output channel allocation strategy under different electromagnetic interference intensities and different data transmission requirements is obtained; the sink node combines the electromagnetic interference intensity value and the current channel allocation condition according to the data transmission request sent by the received sensor node, and allocates an optimal channel for the sensor node in real time based on the channel allocation strategy output by the channel allocation model. The application can more effectively guarantee wireless channel allocation in wide / narrow band wireless hybrid networking scenes, reduce channel conflicts of the same system and different systems, improve frequency efficiency, and reduce the maximum transmission delay of services.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of wireless sensor network channel allocation in power systems, and particularly relates to a wireless sensor network conflict avoidance and service guarantee channel allocation method and system. BACKGROUND

[0002] In a wireless sensor network, a channel allocation mechanism is an important means to reduce interference and ensure efficient and reliable data transmission. For a wireless sensor network with a large number of sensor nodes and multiple sub-networks, how to reasonably allocate limited channel resources to reduce co-channel interference and improve spectral efficiency is one of the important problems that need to be considered. Especially in the substation scenario where the electromagnetic environment is more complex, the incompletely known electromagnetic environment brings greater challenges to the reliable and efficient transmission of service data.

[0003] Current channel allocation methods in wireless sensor networks are mainly divided into two categories. One is the allocation method for time-invariant channels, such as distributed channel allocation methods. In this category of methods, each user or node selects a suitable channel or resource block according to its own channel quality and demand. The channel state in the substation scenario is affected by electromagnetic interference and generally presents dynamic changes. This method is difficult to cope with. The other category is the allocation method for time-varying channels, such as using feedback neural networks to predict time-varying channels or dynamically allocating channels according to the time-varying characteristics of the channel to optimize the utilization of spectral resources.

[0004] However, the above existing technologies do not consider electromagnetic interference in the substation environment and are not suitable for the substation environment with electromagnetic interference. SUMMARY

[0005] To solve the problems in the prior art, the application provides a wireless sensor network conflict avoidance and service guarantee channel allocation method, which solves the problems that the existing channel allocation method is not suitable for the substation environment with electromagnetic interference and cannot guarantee the transmission delay of services.

[0006] The application adopts the following technical solutions.

[0007] The first aspect of the application provides a wireless sensor network conflict avoidance and service guarantee channel allocation method applied to a wireless sensor network in a substation environment. The wireless sensor network includes sensor nodes, sink nodes, and access nodes. The sensor nodes collect operation data of substation equipment, transmit the data to the sink nodes of the subnetworks through allocated channels, and forward the data to the access nodes by the sink nodes. The method includes the following contents:

[0008] The electromagnetic interference intensity of each channel is predicted to obtain an electromagnetic interference intensity value.

[0009] The sink node is taken as an intelligent agent, a channel allocation model is constructed based on multi-agent reinforcement learning, the channel allocation model is iteratively trained through interaction between the intelligent agent and the wireless sensor network environment, and a channel allocation strategy under different electromagnetic interference intensities and different data transmission demands is obtained;

[0010] The sink node allocates an optimal channel for the sensor node in real time according to the data transmission request sent by the sensor node, in combination with the electromagnetic interference intensity value and the current channel allocation situation, and based on the channel allocation strategy output by the channel allocation model.

[0011] Optionally, the electromagnetic interference intensities of the channels are predicted based on a pre-trained electromagnetic interference intensity prediction model to obtain the electromagnetic interference intensity values.

[0012] Optionally, the electromagnetic interference intensities of the channels are predicted to obtain the electromagnetic interference intensity values, including:

[0013] All the sensor nodes are not connected to the sink node, and complete electromagnetic interference intensity data in continuous time slots of the channels are obtained;

[0014] After the electromagnetic interference intensity data are normalized, the electromagnetic interference intensity data are grouped into a training data set and a prediction data set, and the electromagnetic interference intensity data in each data set are processed into a unified format;

[0015] The electromagnetic interference intensity prediction model is trained by using the complete electromagnetic interference intensity data in continuous time slots of the channels, wherein the batch is set to 1 and the epoch is set to 4 during the training process.

[0016] All the sensor nodes are connected to the sink node according to the demand, when the channel is idle, the measured value is taken as the electromagnetic interference intensity value of the channel, and when the channel is occupied, the prediction value output by the electromagnetic interference intensity prediction model is taken as the electromagnetic interference intensity value of the channel.

[0017] Optionally, the composite reward function is expressed according to the following formula:

[0018]

[0019] wherein,

[0020] r c (t) is a transmission rate reward in the tth time slot,

[0021] r f (t) is a delay penalty in the tth time slot,

[0022] is a weight coefficient of the transmission rate reward,

[0023] is a weight coefficient of the delay penalty.

[0024] Optionally, the weight coefficient of the transmission rate reward and the weight coefficient of the time delay penalty are calculated according to the following formula:

[0025]

[0026] wherein,

[0027] is an electromagnetic interference threshold value of the channel l,

[0028] is the electromagnetic radiation interference of the channel l in the substation environment in the tth time slot.

[0029] Optionally, the transmission rate reward is calculated according to the following formula:

[0030]

[0031] wherein,

[0032] r c is the transmission rate reward in the tth time slot,

[0033] K l is the co-frequency interference of the sub-network where the adjacent sink node is located in the tth time slot,

[0034] is the electromagnetic radiation interference of the channel l in the substation environment in the tth time slot,

[0035] is the decision of the sink node numbered n in the tth time slot to allocate the channel l for the sensor node numbered n j .

[0036] σ is the environmental noise power,

[0037] P is the transmission power of the sensor node.

[0038] Optionally, it is judged by the sensor node whether the task is completed within the maximum transmission time delay of the service, if yes, the time delay penalty is 0, otherwise, the time delay penalty is a preset value, the preset value is greater than 0, and when the remaining service data packet size is reduced to a negative number, the number of the remaining service data packets is set to 0;

[0039] The time delay penalty is calculated according to the following formula:

[0040]

[0041] wherein,

[0042] is the maximum service transmission time delay required by the sensor node numbered n j for transmitting the task,

[0043] a residual traffic packet size of a sensor node numbered n j

[0044] a data transmission amount of a sensor node numbered n through a channel l to a sink node numbered n j

[0045] Optionally, the loss value of the agent is calculated according to the following formula:

[0046]

[0047] wherein,

[0048] Q tot (s(t),a(t);θ) is a value function of the agent under the global environment observation state at the t time slot and the channel allocation decision at the t time slot,

[0049] s(t) is the global environment observation state at the t time slot,

[0050] a(t) is the channel allocation decision at the t time slot, and a'(t+1) is the channel allocation decision at the t+1 time slot,

[0051] θ is a network parameter, and θ' is the value of the network parameter updated at the t+1 time slot,

[0052] γ is a reward discount,

[0053] is a sample set,

[0054] r(t) is a composite reward function.

[0055] A second aspect of the present application provides a wireless sensor network conflict avoidance and traffic guarantee channel allocation system, applied to a wireless sensor network in a substation environment, the wireless sensor network comprising sensor nodes, sink nodes and access nodes, the sensor nodes collect operation data of the substation equipment, transmit the data to the sink nodes of the substation through the allocated channels, and the sink nodes forward the data to the access nodes, the system comprising:

[0056] An electromagnetic interference prediction module is configured to predict the electromagnetic interference intensity of each channel and obtain an electromagnetic interference intensity value.

[0057] A channel allocation learning module is configured to take the sink node as an agent, construct a channel allocation model based on multi-agent reinforcement learning, interact with the wireless sensor network environment, learn the experience of channel allocation of each sink node under different electromagnetic interference and data transmission requirements, and output a channel allocation strategy. ​​

[0058] The channel allocation module is configured to allocate an optimal channel for the sensor node in real time based on a channel allocation model according to the received data transmission request sent by the sensor node, in combination with the electromagnetic interference intensity value and the current channel allocation situation.

[0059] The third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when loaded into the processor, implements the wireless sensor network conflict avoidance and service guarantee channel allocation method.

[0060] The fourth aspect of the present application provides a computer readable storage medium, which stores a computer program, wherein the computer program, when executed by a processor, implements the wireless sensor network conflict avoidance and service guarantee channel allocation method.

[0061] Compared with the prior art, the present application has at least the following beneficial effects:

[0062] The present application predicts the electromagnetic interference intensity and allocates channels based on the predicted value. It takes into account the slow time-varying electromagnetic interference and the same frequency interference between AP sub-networks, more effectively guarantees the wireless channel allocation in the wide and narrow band wireless hybrid networking scene, can improve the data transmission quality, reduce the channel conflict of the same system and different systems, improve the spectrum efficiency, and is more suitable for use in substation environment.

[0063] The present application introduces the service transmission delay and service completion of the sensor node in the utility function based on the multi-agent reinforcement learning method, effectively reduces the maximum transmission delay of the service, and optimizes the fairness of the service delay guarantee. BRIEF DESCRIPTION OF DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. Among them:

[0065] Figure 1 A wireless sensor network architecture considering the substation environment is provided for the embodiments of the present application;

[0066] Figure 2 A channel allocation flowchart based on multi-agent reinforcement learning is provided for the embodiments of the present application;

[0067] Figure 3 A comparison diagram of the average completion time of data service is provided for the embodiments of the present application;

[0068] Figure 4 This is a schematic diagram comparing the maximum completion time of a data service provided in an embodiment of the present invention. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The embodiments described in this application are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this invention.

[0070] Considering a wireless sensor network in a substation environment, the architecture diagram is as follows: Figure 1 As shown, the network contains three types of nodes: SN (sensor node), AP (access point), and GW (gateway). Sensor nodes collect operational data from substation equipment and periodically deliver this data to the access point of their respective subnetwork. The access point then forwards the data to the access point. On one hand, when substation equipment is densely distributed, the number of APs and subnetworks deployed is large, but channel resources are limited. Different APs may conflict when allocating channels to requesting access from different sensor nodes, leading to high levels of co-channel interference. On the other hand, the substation environment experiences slow-time-varying electromagnetic interference. Both of these factors affect effective data transmission, necessitating the design of a channel allocation method that minimizes channel conflicts, avoids electromagnetic interference, and ensures reliable service transmission latency.

[0071] Embodiment 1 of the present invention provides a method for collision avoidance and service guarantee channel allocation in wireless sensor networks, comprising the following steps:

[0072] Step 1: Predict the electromagnetic interference intensity of each channel and obtain the electromagnetic interference intensity value.

[0073] Optionally, in step 1, the electromagnetic interference intensity of the substation is predicted based on a pre-trained electromagnetic interference intensity prediction model to obtain the electromagnetic interference intensity value.

[0074] Optionally, the electromagnetic interference strength of each channel is predicted, and obtaining the electromagnetic interference strength value comprises: disconnecting all the sensor nodes from the sink node to obtain complete electromagnetic interference strength data in each channel in a continuous time slot; after normalizing the electromagnetic interference strength data, grouping, training data set and prediction data set are obtained, and the electromagnetic interference strength data in each data set is processed into a unified format; the training data set is input into the electromagnetic interference strength prediction model for iterative training to obtain the trained electromagnetic interference strength prediction model; the prediction data set is input into the trained electromagnetic interference strength prediction model, and the output value of the electromagnetic interference strength prediction model is compared with the actual obtained value, and the training is completed when the comparison result meets the preset condition; all sensor nodes are connected to the sink node as required, and when the channel is idle, the measured value is obtained as the electromagnetic interference strength value of the channel, and when the channel is occupied, the predicted value output by the electromagnetic interference strength prediction model is used as the electromagnetic interference strength value of the channel.

[0075] Specifically, the preset condition includes that the error between the model output value and the actual obtained value is less than an error threshold, and the error threshold is 1% or 3% or 5%. It can be understood that the specific value of the error threshold can be set by the person skilled in the art according to the actual demand.

[0076] In step 1, the normalization processing and the processing of the electromagnetic interference strength data in each data set into a unified format can improve the data processing speed, thereby further improving the subsequent channel allocation efficiency.

[0077] Preferably, the electromagnetic interference strength of the substation is predicted based on a time sequence neural network to obtain the electromagnetic interference strength value. Specifically, it comprises:

[0078] Data acquisition stage: first, the training data for electromagnetic interference prediction is obtained. In the data acquisition stage, all SNs are disconnected from the AP, and the AP uses the UHF (Ultra High Frequency) sensor to obtain complete electromagnetic interference data in a continuous time slot on each channel: Wherein represents the electromagnetic interference strength of the first to the n-1 time slot on the first channel.

[0079] Training stage: using the LSTM (Long Short Term Memory) time sequence neural network, the electromagnetic interference strength of the n time slot is predicted First, data preprocessing, the data obtained in the data acquisition stage is preprocessed and normalized, then the obtained data is grouped into training data set and prediction data set, which accounts for 8 / 10 and 2 / 10 of the total data set respectively, and then it is processed into [x, y, z] data format, where x represents the time step, that is, the length of each sequence, y represents the number of sequences, and z represents the input data dimension, which is set to [100, 1, 1] here. Second, model training, the training data set is input to the model for training. Due to the small amount of data, the batch is set to 1 and the epoch is set to 4. Epoch is a unit of model training process, which controls the number of iterations of the entire training. Batch is a large-scale data set divided into several small blocks for training, each batch contains a certain number of sample data, which determines the number of samples processed in each iteration, thereby controlling the training speed. When the iteration reaches the fourth round, the loss value tends to be stable. Finally, model prediction, the prediction data set is input to the trained network, and the output value is compared with the true acquisition value, and the difference is not large.

[0080] Execution phase: SN normally accesses AP on demand, AP obtains electromagnetic interference data on idle time-frequency resources, and for occupied channels, uses predicted values instead of measured values.

[0081] Step 2, taking the sink node as an agent, a channel allocation model is constructed based on multi-agent reinforcement learning, the channel allocation model is iteratively trained by interacting with the wireless sensor network environment, and the channel allocation strategy under different electromagnetic interference intensities and different data transmission demands is obtained.

[0082] In step 2, the channel allocation model based on multi-agent reinforcement learning takes reducing business transmission delay as the starting point, learns through interaction between multiple APs, integrates the learning results of each AP, and finally generates the allocation strategy of the spectrum allocation system.

[0083] The overall process is shown in Figure 2 At the beginning of each time slot, all APs observe the local channel environment and business demand state in the sub-network, obtain the SN channel allocation strategy under this state through exploration learning, execute the strategy and interact with the substation environment to obtain rewards as reward to update network parameters, and iterate the training process until the network converges stably.

[0084] Step 2 specifically includes:

[0085] Step 201, the sink node as an agent, observes the local channel environment and business demand state in the sub-network, obtains the global environment observation state at the current time, and the global environment observation state at the current time includes the local information state observed by the agent and the electromagnetic interference intensity value.

[0086] State observation phase: At the initial moment of time slot t, the global environment observation state is obtained wherein is the prediction result of the electromagnetic interference prediction module for the interference of this time slot, s n (t) is the local information state observed by APn as an independent intelligent agent. The AP local information includes the position q n (t) of the AP, the number n, and the state set of all SNs in the subnetwork that need to provide channel access SN n j Local state includes the remaining service data packet size of each time slot The position of the SN Number n j , the maximum transmission delay of the service Channel allocation decision at the last moment

[0087] Step 202, based on the local information state observed by the intelligent agent and the electromagnetic interference strength value, make a channel allocation decision for the sensor nodes in the subnetwork that send data transmission requests, and execute the channel allocation decision.

[0088] Decision acquisition phase: each APn makes a channel allocation decision for the SN n j in the subnetwork that makes an access request, through the formula allocates a channel, wherein is the channel allocation strategy of the number n j , θ n represents the network parameters, which are updated according to the environmental feedback reward later. After the action is executed, the state is transferred to the next moment.

[0089] Optionally, the gradient descent algorithm is used to update the network parameters.

[0090]

[0091] wherein α is the learning rate, which is fixed at 0.001 here, is the gradient of the loss function with respect to the network parameters θ.

[0092] Step 203, the intelligent agent interacts with the wireless sensor network environment in the substation environment to obtain a composite reward value corresponding to the execution of the channel allocation decision.

[0093] Reward feedback phase: the channel decisions made by all APs are executed simultaneously and interact with the environment to obtain a composite reward value corresponding to the execution of the channel allocation decision.

[0094] Preferably, the composite reward function is expressed as follows:

[0095]

[0096] wherein,

[0097] r c (t) is the transmission rate reward of the tth time slot,

[0098] r f (t) is the delay penalty of the tth time slot,

[0099] is the weight coefficient of the transmission rate reward,

[0100] is the weight coefficient of the delay penalty.

[0101] Specifically, Initial setting The weight coefficient is adjusted according to the strength of electromagnetic interference in the environment.

[0102] More specifically, the weight coefficient of the transmission rate reward is calculated as follows:

[0103]

[0104] wherein, is the electromagnetic interference threshold of the channel l, is the electromagnetic radiation interference of the channel l in the tth time slot in the substation environment.

[0105] In this way, according to the strength of electromagnetic interference in the environment, the weight of is appropriately increased in high interference, and the weight of is appropriately increased in low interference. In this way, by setting different weight coefficients, the composite reward is suitable for different electromagnetic interference situations, thereby improving the efficiency of the channel allocation model.

[0106] Preferably, the transmission rate reward is calculated as follows:

[0107]

[0108] wherein,

[0109] r c (t) is the transmission rate reward of the tth time slot,

[0110] K l (t) is the co-frequency interference of the tth time slot in the subnetwork where the adjacent sink node is located,

[0111] is the electromagnetic radiation interference of the channel l in the tth time slot in the substation environment,

[0112] The decision of assigning channel l for the sink node numbered n for the tth time slot to the sensor node numbered n j

[0113] σ is the ambient noise power,

[0114] P is the transmit power of the sensor node.

[0115] Specifically, σ takes the value of -80dBm, and P takes the value of 0.1w.

[0116] SN n j The APn transmits through channel l mainly receives two kinds of interference, one is the adjacent APm subnetwork's same frequency interference The other is the electromagnetic radiation interference of the substation environment

[0117] Preferably, the sensor node judges whether the task is completed within the maximum traffic transmission delay, if yes, the delay penalty is 0, otherwise, the delay penalty is a preset value, the preset value is greater than 0, and when the remaining traffic packet size is reduced to a negative number, the number of remaining traffic packets is set to 0;

[0118] The delay penalty is calculated according to the following formula:

[0119]

[0120] Wherein,

[0121] The maximum traffic transmission delay required for the sensor node numbered n j to transmit the task,

[0122] The remaining traffic packet size of the sensor node numbered n j

[0123] The data transmission amount of the sensor node numbered n j to the sink node numbered n through channel l.

[0124] Specifically, the SN n j judges whether the task transmission is completed within the maximum traffic transmission delay , and if yes, 0 is recorded, otherwise, a large number (for example, 1000) is recorded. The SN n j remaining traffic packet size When the packet size is reduced to a negative number, the number of remaining packets is set to 0.

[0125] ​​Step 204, repeat the iteration of steps 201 to 203 until the loss value of the network is minimum, and obtain the trained channel allocation model.

[0126] Preferably, the loss value of the network is calculated according to the following formula:

[0127]

[0128] wherein,

[0129] Q tot (s(t),a(t);θ) is the value function of the agent under the global environment observation state at the t time slot and the channel allocation decision at the t time slot,

[0130] s(t) is the global environment observation state at the t time slot,

[0131] a(t) is the channel allocation decision at the t time slot, and a'(t+1) is the channel allocation decision at the t+1 time slot,

[0132] θ is the network parameter, and θ' is the value of the network parameter updated at the t+1 time slot,

[0133] γ is the reward discount,

[0134] is the sampled sample set,

[0135] r(t) is the composite reward function.

[0136] Optionally, γ takes the value of 0.9 or 0.8, and the value of 50.

[0137] Specifically, in the iteration training phase: when learning, the AP performs joint training to ensure the cooperation capability. First, randomly sample the set of <state, action, reward, next time action state> stored in the cache and then calculate the loss value of the network according to the formula and update the QMIX network. The QMIX network can process multiple agents and can calculate the Q value function of each agent in parallel, thereby improving the calculation efficiency.

[0138] Step 3, the sink node combines the electromagnetic interference strength prediction result and the current channel allocation situation according to the data transmission request sent by the sensor node, and allocates the optimal channel for the sensor node in real time based on the channel allocation strategy output by the channel allocation model.

[0139] Step 3 specifically includes:

[0140] Step 301, (task generation stage) in a plurality of AP sub-networks, at the beginning of a time slot, a SN randomly generates a delay-sensitive service needing to be transmitted to the AP for transfer to the GW, and submits an access request to the AP.

[0141] Step 302, (deployment model decision stage) when each AP obtains a SN request from the sub-network, first allocates a transmission rate to the SN initiating the request according to the current SN location, service data size, service delay requirement, current channel allocation situation, and current channel electromagnetic interference situation to avoid conflicts as much as possible.

[0142] Step 303, (execution decision stage) the SN receives the channel allocation of the AP and selects the allocated channel for transmission.

[0143] In this embodiment, an interference prediction module based on a timing neural network is designed for the electromagnetic interference problem in the substation environment, the electromagnetic interference strength is predicted based on the monitored historical data, and is used for subsequent training and learning of the channel allocation strategy. A channel allocation method based on multi-agent reinforcement learning is designed for the problem of same frequency interference between multiple AP sub-networks due to channel conflict, and each AP agent learns the optimal channel allocation strategy under different electromagnetic interference and service requirements. Compared with the traditional channel allocation method, on the one hand, the slow time-varying electromagnetic interference and the same frequency interference between the AP sub-networks are considered at the same time, which is more suitable for the substation environment; on the other hand, the utility function of the method not only considers the real-time transmission rate, but also considers the data transmission delay, which can effectively reduce the maximum transmission delay of the service and optimize the fairness of the service delay guarantee.

[0144] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0145] Embodiment 2 of the application provides a wireless sensor network conflict avoidance and service guarantee channel allocation system, which runs the wireless sensor network conflict avoidance and service guarantee channel allocation method as described in embodiment 1, and the system comprises an electromagnetic interference prediction module, a channel allocation learning module and a channel allocation module.

[0146] The electromagnetic interference prediction module is used for predicting the electromagnetic interference strength of each channel and obtaining the electromagnetic interference strength value. It is responsible for predicting the slow time-varying electromagnetic interference strength of the substation.

[0147] The channel allocation learning module is configured to take the sink node as an agent, construct a channel allocation model based on multi-agent reinforcement learning, interact with the wireless sensor network environment through the agent, iteratively train the channel allocation model, and obtain a channel allocation strategy under different electromagnetic interference intensities and different data transmission demands.

[0148] The channel allocation module is configured to allocate an optimal channel for the sensor node in real time based on the channel allocation strategy output by the channel allocation model according to the received data transmission request sent by the sensor node, in combination with the electromagnetic interference intensity prediction result and the current channel allocation situation.

[0149] The optimal channel is a channel with a minimum joint result of conflict, electromagnetic interference influence and service transmission delay.

[0150] Embodiment 3 of the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when loaded into the processor, implements the wireless sensor network conflict avoidance and service guarantee channel allocation method described in embodiment 1.

[0151] Embodiment 4 of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the wireless sensor network conflict avoidance and service guarantee channel allocation method according to embodiment 1.

[0152] Embodiment 5 of the present application provides a verification example of the wireless sensor network conflict avoidance and service guarantee channel allocation method provided in embodiment 1 of the present application. In order to verify the effectiveness of the method, numerical simulation experiments are performed on the method, and the simulation scenario is set as deploying 1 access node and 5-25 wireless sensor network sink nodes in a substation network environment, and 5 sensor nodes are randomly distributed around each sink node. Compared with the traditional channel allocation method, the simulation results are shown in Figure 3 and Figure 4 .

[0153] Figure 3 and Figure 4 respectively show the average completion time and the maximum completion time of the data service. It can be seen that when the number of sensor nodes is small, the average completion time and the maximum completion time of the proposed method and the comparison method are not much different; as the number of sensor nodes increases, the probability of same frequency interference between self-networks increases, and the average completion time and the maximum completion time of the comparison method are significantly higher than those of the proposed method. This shows that the proposed method can effectively avoid channel conflict and guarantee service transmission. In addition, compared with Figure 3 and Figure 4It can be seen that the proposed method is more obvious for reducing the maximum completion time, which shows that the proposed method improves the service transmission fairness.

[0154] Compared with the prior art, the present application has at least the following beneficial effects:

[0155] The present application predicts the electromagnetic interference intensity, and performs channel allocation based on the predicted value.

[0156] The present application introduces the service transmission delay and service completion of the sensor node in the utility function based on the multi-agent reinforcement learning method, effectively reduces the maximum transmission delay of the service, and optimizes the fairness of the service delay guarantee.

[0157] The present application can be a system, a method and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions stored therein for causing a processor to implement various aspects of the present application.

[0158] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device such as a punch card or a hole-in-punch structure having instructions stored thereon, and any suitable combination of the foregoing. The computer readable storage medium used here is not to be interpreted as a transitory signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (for example, an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0159] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0160] Computer readable program instructions for carrying out operations of the present disclosure can be assembly instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, for example, through the Internet using an Internet Service Provider. In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0161] Finally, it should be noted that the above-mentioned embodiments are merely intended for describing and illustrating, not limiting the technical solutions of the present application. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the specific embodiments of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the present application, and any modification or equivalent replacement without departing from the spirit and scope of the present application should be covered within the protection scope of the claims of the present application.

Claims

1. A wireless sensor network collision avoidance and traffic guarantee channel allocation method applied to a wireless sensor network in a substation environment, the wireless sensor network comprising sensor nodes, sink nodes and access nodes, the sensor nodes collecting operation data of substation equipment, transmitting the data to the sink nodes of the subnetworks where the sensor nodes are located through allocated channels, and forwarding the data to the access nodes by the sink nodes, characterized in that, The method comprises the following contents: The electromagnetic interference intensity of each channel is predicted to obtain an electromagnetic interference intensity value; The sink node is taken as an intelligent agent, a channel allocation model is constructed based on multi-agent reinforcement learning, the intelligent agent interacts with the wireless sensor network environment, a composite reward function is used to iteratively train the channel allocation model, and a channel allocation strategy under different electromagnetic interference intensities and different data transmission demands is obtained; The sink node allocates an optimal channel for the sensor node in real time according to the data transmission request sent by the sensor node, in combination with the electromagnetic interference intensity value and the current channel allocation situation, and based on the channel allocation strategy output by the channel allocation model; The composite reward function is expressed according to the following formula: wherein, r c (t) is the tth time slot transmission rate reward, r f (t) is a tth time slot delay penalty, a weight factor for a transmission rate reward, The weighting coefficients for the delay penalty; The weight coefficient of the transmission rate reward and the weight coefficient of the delay penalty are calculated according to the following formula: wherein, EMI threshold for channel 1, is the electromagnetic radiation interference for the channel / in the substation environment at the tth time slot.

2. The wireless sensor network conflict avoidance and service guarantee channel allocation method according to claim 1, characterized in that: The electromagnetic interference intensity of each channel is predicted based on a pre-trained electromagnetic interference intensity prediction model to obtain an electromagnetic interference intensity value; the electromagnetic interference intensity prediction model is constructed based on an LSTM time sequence neural network.

3. The wireless sensor network conflict avoidance and service guarantee channel allocation method according to claim 2, characterized in that: The electromagnetic interference intensity of each channel is predicted to obtain an electromagnetic interference intensity value, which comprises: All the sensor nodes are not connected to the sink node, and complete electromagnetic interference intensity data in a continuous time slot of each channel is obtained; The electromagnetic interference intensity prediction model is trained by using the complete electromagnetic interference intensity data in the continuous time slot of each channel, wherein the batch is set to 1 and the epoch is set to 4 during the training process; All the sensor nodes are connected to the sink node according to the demand, when the channel is idle, the measured value is taken as the electromagnetic interference intensity value of the channel, and when the channel is occupied, the prediction value output by the electromagnetic interference intensity prediction model is taken as the electromagnetic interference intensity value of the channel.

4. The wireless sensor network conflict avoidance and service guarantee channel allocation method according to claim 1, characterized in that: The transmission rate reward is calculated according to the following formula: wherein, r c (t) is the tth time slot transmission rate reward, K l (t) is the co-channel interference from the subnetwork of the adjacent sink node in the tth time slot, is the electromagnetic radiation interference of the channel l in the substation environment for the tth time slot, The decision for the sink node numbered n for time slot number n to assign channel l to the sensor node numbered n j of the sensor node numbered n σ is the environmental noise power, P is the transmission power of the sensor node.

5. The wireless sensor network conflict avoidance and service guarantee channel allocation method according to claim 1, characterized in that: Whether the task is completed within the maximum transmission delay of the service is judged by the sensor node, if yes, the delay penalty is 0, otherwise, the delay penalty is a preset value, the preset value is greater than 0, and when the remaining business data packet size is reduced to a negative number, the number of the remaining business data packets is set to 0; The delay penalty is calculated according to the following formula: wherein, the maximum traffic transmission latency required for the transmission task of the sensor node numbered n j the maximum traffic transmission latency required for the transmission task of the sensor node numbered n For the sensing node with the number n j remaining service data packet size, Data transmission volume from a sensor node numbered n to a sink node numbered n through a channel l. j Data transmission volume from a sensor node numbered n to a sink node numbered n through a channel l.

6. The wireless sensor network conflict avoidance and service guarantee channel allocation method according to any one of claims 1 to 5, characterized in that: The loss value of the intelligent agent is calculated according to the following formula: wherein, Q tot Q(s(t), a(t); θ) is the value function of the agent under the global environment observation state at the tth time slot and the channel allocation decision at the tth time slot, s(t) is the global environment observation state in the t time slot, a(t) is the channel allocation decision in the t time slot, and a'(t+1) is the channel allocation decision in the t+1 time slot, θ is the network parameter, θ' is the value of the network parameter updated in the t+1 time slot, γ is the reward discount, for the sampled set of samples, r(t) is a compound reward function.

7. A wireless sensor network collision avoidance and traffic guaranteed channel allocation system using the method of any one of claims 1 to 6, applied to a wireless sensor network in a substation environment, the wireless sensor network comprising sensor nodes, sink nodes and access nodes, the sensor nodes collecting operation data of substation equipment, transmitting the data to the sink nodes of the subnetworks in which they are located via the allocated channels, and the sink nodes forwarding the data to the access nodes, characterized in that, The system comprises: An electromagnetic interference prediction module is configured to predict the electromagnetic interference intensity of each channel and obtain an electromagnetic interference intensity value; A channel allocation learning module is configured to take the sink node as an agent, construct a channel allocation model based on multi-agent reinforcement learning, interact with the wireless sensor network environment through the agent, use a compound reward function to iteratively train the channel allocation model, and obtain a channel allocation strategy under different electromagnetic interference intensities and different data transmission requirements; A channel allocation module is configured to allocate an optimal channel to a sensor node in real time according to a data transmission request sent by the sensor node, in combination with the electromagnetic interference intensity value and the current channel allocation situation, and based on the channel allocation strategy output by the channel allocation model. 8.An electronic device comprising a processor and a storage medium; characterized in that: The storage medium is configured to store instructions; The processor is configured to operate according to the instructions to perform the steps of the wireless sensor network conflict avoidance and service guarantee channel allocation method according to any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the wireless sensor network conflict avoidance and service guarantee channel allocation method according to any one of claims 1 to 6.

Images