Method and device for allocating resources in electric power sensor network

Abstract

The invention relates to a resource allocation method and device in an electric power sensor network, and belongs to the technical field of electric power system communication, and the method comprises the following steps: building a wireless sensor network model of an electric power system, obtaining data streams of all tasks generated by all wireless sensor nodes, and abstracting the data streams into different tasks; determining a weighted sum of time delay and energy consumption for transmitting each data stream, taking the weighted sum of time delay and energy consumption for transmitting each data stream as feedback of a wireless sensor network model, taking task unloading as a decision of the wireless sensor network model, taking a task unloading decision as an optimization problem, and modeling the optimization problem as a Markov decision process; and solving the Markov decision process by using a D3QN algorithm to obtain an optimal unloading strategy, and carrying out resource allocation according to the optimal unloading strategy. According to the method, the wireless sensor network technology is combined with reinforcement learning, multi-service-oriented resource allocation is realized by solving the optimal unloading strategy, and the time delay and the energy consumption of the system are effectively reduced.

Description

Technical Field

[0001] The invention relates to a method and a device for allocating resources in a power sensor network, belonging to the technical field of power system communications. Background Art

[0002] In the power system, wireless sensor networks are responsible for global information monitoring, management and control of equipment and users, and are also a part of the construction of new smart grids that urgently needs to be updated and upgraded. In the past, power grid systems usually adopted centralized power supply, and power energy and information were transmitted one-way from power plants to users. In today's smart grids, power can be generated on the user side, which is a distributed architecture, so the power information generated on the user side needs to interact with the power plant in a two-way manner. Smart grids require wireless sensor networks to support the two-way flow of information, and use this information to flexibly allocate and efficiently transmit power energy. The power sensor network is mainly a self-organizing network composed of a large number of static sensors in a multi-hop manner. Environmental data is collected through large-scale deployed wireless nodes and transmitted to base stations or servers for processing and analysis. Edge computing servers are usually deployed on the base station side and have large computing, storage, and network resources. When the information of the sensor node is unloaded to the edge server, it not only reduces the energy consumption of the edge node itself, but also reduces the overall delay of the entire system and the communication pressure transmitted to the power plant system. In addition, the server can also allocate different computing resources to wireless sensor nodes according to different task requirements to improve computing efficiency and energy utilization.

[0003] The problem of task offloading and resource allocation in power sensor networks is a hot and difficult problem. It is essentially an optimization problem, and different allocation strategies are adopted for different data volumes and different numbers of sensor nodes. In the prior art, traditional reinforcement learning methods are used to allocate resources in power sensor networks. The disadvantage of this method is that it is limited to situations where both the action space and the sample space are small and discrete, and it is difficult to handle complex tasks with large state space and continuous action space. Summary of the invention

[0004] In order to solve the above problems, the present invention proposes a method and device for resource allocation in a power sensor network, which can minimize the total delay and total energy consumption by optimizing the unloading of node data and the allocation of edge server computing resources.

[0005] The technical solution adopted by the present invention to solve the technical problem is: In a first aspect, an embodiment of the present invention provides a method for allocating resources in a power sensor network, comprising the following steps: Establishing a wireless sensor network model of the power system, obtaining data streams of all tasks generated by all wireless sensor nodes, and abstracting the data streams into different tasks, wherein the nodes in the wireless sensor network model communicate with each other by transmitting data streams; Determine the weighted sum of the time delay and energy consumption of each data stream, use the weighted sum of the time delay and energy consumption of each data stream as feedback of the wireless sensor network model, use task offloading as the decision of the wireless sensor network model, use the task offloading decision as an optimization problem, and model the optimization problem as a Markov decision process; The D3QN algorithm is used to solve the Markov decision process, obtain the optimal offloading strategy, and allocate resources according to the optimal offloading strategy.

[0006] As a possible implementation of this embodiment, the step of establishing a wireless sensor network model of the power system includes: Considering coverage, signal strength and network connectivity, multiple wireless sensor nodes are reasonably distributed around the base station of the power system to collect data; An edge server is deployed on the base station side, and the edge server is used to receive data streams from wireless sensor nodes. The data streams are calculated by the wireless sensor nodes themselves or offloaded to the edge server for calculation.

[0007] As a possible implementation of this embodiment, the step of acquiring data streams of all tasks generated by all wireless sensor nodes and abstracting the data streams into different tasks includes: The data streams generated by all wireless sensor nodes are obtained. The data streams generated by different nodes are used as different tasks. The size of the data stream is used as the task size. The data stream of each wireless sensor node is abstracted as an independent task. The data stream includes task data generated by the wireless sensor node when sensing changes in the external environment.

[0008] As a possible implementation manner of this embodiment, the power system includes a photovoltaic power grid, wireless sensor nodes deployed at different positions in the photovoltaic power grid, and all the wireless sensor nodes form a wireless sensor network.

[0009] As a possible implementation manner of this embodiment, the weighted sum of delay and energy consumption includes a first weighted sum of delay and energy consumption and a second weighted sum of delay and energy consumption, and determining the weighted sum of delay and energy consumption of each transmission data stream includes: According to the distribution of each wireless sensor node, the upload rate of each wireless sensor node is calculated using the channel model; Based on the upload rate of the wireless sensor node, and according to the energy consumption and delay generated when the wireless sensor node uploads the generated task to the edge server for calculation, a first delay and a weighted sum of energy consumption are obtained; According to the energy consumption and delay generated when the wireless sensor node calculates the task itself, a second delay and a weighted sum of energy consumption are obtained.

[0010] As a possible implementation of this embodiment, the Markov decision process is a tuple, and the tuple includes a state space, an action space, and a reward function; The state space is: ,include The weighted sum of the latency and energy consumption of each task transmission; The action space is: , including the offloading actions of all wireless sensor nodes, where ,0 means the task is uploaded to the edge server for calculation, and 1 means it is calculated by the wireless sensor node itself; The reward function is: ,in Represents the total cost of the system, which is the weighted sum of the total latency and energy consumption of the system.

[0011] As a possible implementation of this embodiment, using the D3QN algorithm to solve the Markov decision process, obtaining the optimal offloading strategy, and allocating resources according to the optimal offloading strategy includes: Initialize the parameters of the neural network in D3QN, including the state, network parameters and playback experience pool parameters. The state consists of two variables, namely the delay of each sample task and the energy consumption of each sample task; Set the agent's reward to the weighted sum of the delay and energy consumption of each sample task, and receive the initial state And input the estimated Q neural network to output the value function value and action function value, add the value function value and action function value to get , according to the initial state passed into D3QN use Policy gets uninstall action And enter the state , based on the current state of the agent , Actions performed and the new state to which it is transferred Get rewarded ; The status Input the target Q neural network, according to the state of the input D3QN use Policy gets uninstall action and get ; Constructing experience pool parameters , calculate the loss function: ,in, is the discount factor; Update the estimated network parameters of the Q neural network by gradient descent to minimize the loss function; Multiple iterations to update the network parameters of the estimated Q neural network and the target Q neural network; When the number of training rounds reaches a certain number, samples are extracted from the experience pool parameters, and the rewards, value function values, and action function values ​​are calculated from these samples. The network parameters are updated by minimizing the loss function to obtain the final estimated Q neural network and target Q neural network; The weighted sum of latency and energy consumption is input into the trained agent to obtain the optimal offloading strategy for the current task.

[0012] In a second aspect, an embodiment of the present invention provides a device for allocating resources in a power sensor network, including: A data acquisition module is used to establish a wireless sensor network model of the power system, obtain the data streams of all tasks generated by all wireless sensor nodes, and abstract the data streams into different tasks. The nodes in the wireless sensor network model communicate with each other by transmitting data streams; A problem conversion module is used to determine the weighted sum of the delay and energy consumption of each data stream, use the weighted sum of the delay and energy consumption of each data stream as feedback of the wireless sensor network model, use task offloading as the decision of the wireless sensor network model, use the task offloading decision as an optimization problem, and model the optimization problem as a Markov decision process; The problem solving module is used to solve the Markov decision process using the D3QN algorithm, obtain the optimal offloading strategy, and allocate resources according to the optimal offloading strategy.

[0013] As a possible implementation of this embodiment, the step of establishing a wireless sensor network model of the power system includes: Considering coverage, signal strength and network connectivity, multiple wireless sensor nodes are reasonably distributed around the base station of the power system to collect data; An edge server is deployed on the base station side, and the edge server is used to receive data streams from wireless sensor nodes. The data streams are calculated by the wireless sensor nodes themselves or offloaded to the edge server for calculation.

[0014] As a possible implementation of this embodiment, the step of acquiring data streams of all tasks generated by all wireless sensor nodes and abstracting the data streams into different tasks includes: The data streams generated by all wireless sensor nodes are obtained. The data streams generated by different nodes are used as different tasks. The size of the data stream is used as the task size. The data stream of each wireless sensor node is abstracted as an independent task. The data stream includes task data generated by the wireless sensor node when sensing changes in the external environment.

[0015] As a possible implementation manner of this embodiment, the power system includes a photovoltaic power grid, wireless sensor nodes deployed at different positions in the photovoltaic power grid, and all the wireless sensor nodes form a wireless sensor network.

[0016] As a possible implementation manner of this embodiment, the weighted sum of delay and energy consumption includes a first weighted sum of delay and energy consumption and a second weighted sum of delay and energy consumption, and determining the weighted sum of delay and energy consumption of each transmission data stream includes: According to the distribution of each wireless sensor node, the upload rate of each wireless sensor node is calculated using the channel model; Based on the upload rate of the wireless sensor node, and according to the energy consumption and delay generated when the wireless sensor node uploads the generated task to the edge server for calculation, a first delay and a weighted sum of energy consumption are obtained; According to the energy consumption and delay generated when the wireless sensor node calculates the task itself, a second delay and a weighted sum of energy consumption are obtained.

[0017] As a possible implementation of this embodiment, the Markov decision process is a tuple, and the tuple includes a state space, an action space, and a reward function; The state space is: ,include The weighted sum of the latency and energy consumption of each task transmission; The action space is: , including the offloading actions of all wireless sensor nodes, where ,0 means the task is uploaded to the edge server for calculation, and 1 means the wireless sensor node calculates it by itself; The reward function is: ,in Represents the total cost of the system, which is the weighted sum of the total latency and energy consumption of the system.

[0018] As a possible implementation of this embodiment, the problem solving module is specifically used to: Initialize the parameters of the neural network in D3QN, including the state, network parameters, and playback experience pool parameters; The state consists of two variables, namely the delay of each sample task and the energy consumption of each sample task. The reward is set as the weighted sum of the delay and energy consumption of each sample task. The initial state is received. And input the estimated Q neural network to output the value function value and action function value, add the value function value and action function value to get , according to the initial state passed into D3QN use Policy gets uninstall action And enter the state , based on the current state of the agent , Actions performed and the new state to which it is transferred Get rewarded ; The status Input the target Q neural network, according to the state of the input D3QN use Policy gets uninstall action and get ; Constructing experience pool parameters , calculate the loss function: ,in, is the discount factor; Update the estimated network parameters of the Q neural network by gradient descent to minimize the loss function; Multiple iterations to update the network parameters of the estimated Q neural network and the target Q neural network; When the number of training rounds reaches a certain number, samples are extracted from the experience pool parameters, and the rewards, value function values, and action function values ​​are calculated from these samples. The network parameters are updated by minimizing the loss function to obtain the final estimated Q neural network and target Q neural network; The agent obtains the weighted sum of the delay and energy consumption of the current task as a reward, and inputs the weighted sum of the delay and energy consumption into the trained agent to obtain the optimal offloading strategy for the current task.

[0019] In a third aspect, an embodiment of the present invention provides a computer device, including: one or more processors; The processor is used to store one or more programs; When the one or more programs are executed by the one or more processors, the method for allocating resources in a power sensor network as described above is implemented.

[0020] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed, the method for resource allocation in a power sensor network as described above is implemented.

[0021] The beneficial effects of the technical solution of the embodiment of the present invention are as follows: A method for resource allocation in an electric power sensor network according to a technical solution of an embodiment of the present invention comprises the following steps: establishing a wireless sensor network model of an electric power system, obtaining data streams of all tasks generated by all wireless sensor nodes, and abstracting the data streams into different tasks, wherein the nodes in the wireless sensor network model communicate with each other by transmitting data streams; determining the weighted sum of the delay and energy consumption of transmitting each data stream, taking the weighted sum of the delay and energy consumption of transmitting each data stream as feedback of the wireless sensor network model, taking task offloading as a decision of the wireless sensor network model, taking the task offloading decision as an optimization problem, and modeling the optimization problem as a Markov decision process; using the D3QN algorithm to solve the Markov decision process, obtaining an optimal offloading strategy, and allocating resources according to the optimal offloading strategy, thereby reducing delay and energy consumption. The present invention combines wireless sensor network technology with reinforcement learning, models the system as a Markov decision process (MDP) based on the process of task offloading and resource allocation, uses the D3QN algorithm to solve the MDP problem, and realizes flexible multi-service-oriented resource allocation by giving an optimal offloading strategy, thereby effectively reducing the delay and energy consumption of the system.

[0022] The device for allocating resources in a power sensor network according to the technical solution of an embodiment of the present invention has the same beneficial effects as the method for allocating resources in a power sensor network according to the technical solution of an embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of a method for resource allocation in a power sensor network according to an exemplary embodiment; Figure 2 is a schematic diagram of a wireless sensor network model based on an edge server according to an exemplary embodiment; Figure 3 It is a structural schematic diagram of a device for allocating resources in a power sensor network according to an exemplary embodiment. DETAILED DESCRIPTION

[0024] In order to more clearly illustrate the technical features of the solution of the present invention, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0025] like Figure 1 As shown, a method for allocating resources in a power sensor network provided by an embodiment of the present invention includes the following steps: Establishing a wireless sensor network model of the power system, obtaining data streams of all tasks generated by all wireless sensor nodes, and abstracting the data streams into different tasks, wherein the nodes in the wireless sensor network model communicate with each other by transmitting data streams; Determine the weighted sum of the time delay and energy consumption of each data stream, use the weighted sum of the time delay and energy consumption of each data stream as feedback of the wireless sensor network model, use task offloading as the decision of the wireless sensor network model, use the task offloading decision as an optimization problem, and model the optimization problem as a Markov decision process; The D3QN algorithm is used to solve the Markov decision process, obtain the optimal offloading strategy, and allocate resources according to the optimal offloading strategy.

[0026] As a possible implementation of this embodiment, the step of establishing a wireless sensor network model of the power system includes: Considering coverage, signal strength and network connectivity, multiple wireless sensor nodes are reasonably distributed around the base station of the power system to collect data; An edge server is deployed on the base station side, and the edge server is used to receive data streams from wireless sensor nodes. The data streams are calculated by the wireless sensor nodes themselves or offloaded to the edge server for calculation.

[0027] As a possible implementation of this embodiment, the step of acquiring data streams of all tasks generated by all wireless sensor nodes and abstracting the data streams into different tasks includes: The data streams generated by all wireless sensor nodes are obtained. The data streams generated by different nodes are used as different tasks. The size of the data stream is used as the task size. The data stream of each wireless sensor node is abstracted as an independent task. The data stream includes task data generated by the wireless sensor node when sensing changes in the external environment.

[0028] As a possible implementation manner of this embodiment, the power system includes a photovoltaic power grid, wireless sensor nodes deployed at different positions in the photovoltaic power grid, and all the wireless sensor nodes form a wireless sensor network.

[0029] like Figure 2 As shown, the wireless sensor network model of the present invention is composed of multiple wireless sensor nodes, a base station and an edge server. Multiple sensor nodes such as wireless sensor node 1, wireless sensor node 2...wireless sensor node N are responsible for receiving environmental signals and selecting wireless sensor nodes to calculate by themselves or upload to the edge server on the base station side for calculation. The base station is responsible for communicating with the wireless sensor nodes through the wireless air interface, and the edge server calculates and processes the data collected by the base station side.

[0030] As a possible implementation manner of this embodiment, the weighted sum of delay and energy consumption includes a first weighted sum of delay and energy consumption and a second weighted sum of delay and energy consumption, and determining the weighted sum of delay and energy consumption of each transmission data stream includes: According to the distribution of each wireless sensor node, the upload rate of each wireless sensor node is calculated using the channel model; Based on the upload rate of the wireless sensor node, and according to the energy consumption and delay generated when the wireless sensor node uploads the generated task to the edge server for calculation, a first delay and a weighted sum of energy consumption are obtained; According to the energy consumption and delay generated when the wireless sensor node calculates the task itself, a second delay and a weighted sum of energy consumption are obtained.

[0031] As a possible implementation of this embodiment, the Markov decision process is a tuple, and the tuple includes a state space, an action space, and a reward function; The state space is: ,include The weighted sum of the latency and energy consumption of each task transmission; The action space is: , including the offloading actions of all wireless sensor nodes, where ,0 means the task is uploaded to the edge server for calculation, and 1 means it is calculated by the wireless sensor node itself; The reward function is: ,in Represents the total cost of the system, which is the weighted sum of the total latency and energy consumption of the system.

[0032] As a possible implementation of this embodiment, the specific process of calculating the energy consumption and delay of the task generated by the wireless sensor node by the wireless sensor node itself is as follows: Assume A wireless sensor node obtains data flow tasks through external interaction. , is the number of wireless sensor nodes and is calculated by the wireless sensor nodes themselves; The wireless sensor node calculates the delay by itself as: , in, The wireless sensor node's own computing capability, is the amount of data to be processed by the wireless sensor node, is the complexity of the data; The energy consumption calculated by the wireless sensor node itself is: , in, The power consumption calculated for the sensor node; The total cost is: , in, and is the weighting coefficient, and To simplify the calculation, the weighting coefficients of all sensor nodes are the same.

[0033] As a possible implementation of this embodiment, the specific process of uploading the tasks generated by the wireless sensor nodes to the edge server for calculating the energy consumption and delay is as follows: Assume Each wireless sensor node obtains data flow tasks by interacting with the outside world and offloads them to the edge server for calculation; Unload upload delay: , , in is the upload rate, is the wireless channel bandwidth, Transmit power to sensor nodes, is the channel fading coefficient, is the distance between the sensor node and the base station, is the path fading factor, is the Gaussian white noise power spectral density, Takes 0 or 1, When it is 0, it means that the sensor node is unloaded to the edge server strategy. When it is 1, it means that the sensor node chooses to calculate the strategy by the wireless sensor node itself; After being offloaded to the edge server, the edge server's data processing delay is: , , in, is the total computing capacity of the edge computing server; When the wireless sensor node selects the offloading strategy, the total delay is: , The total energy consumption is: , in, , , Upload power to sensor nodes, Calculate power for edge servers; The total cost is: , The total cost of the entire system is: , In this scenario, the research goal is to minimize the total cost of the entire system; build a system total cost optimization model: , in, Indicates the uninstallation policy. Indicates that the data processing delay should meet the maximum tolerable delay of the data.

[0034] As a possible implementation of this embodiment, using the D3QN algorithm to solve the Markov decision process, obtaining the optimal offloading strategy, and allocating resources according to the optimal offloading strategy includes: Initialize the parameters of the neural network in D3QN, including the state, network parameters and playback experience pool parameters. The state consists of two variables, namely the delay of each sample task and the energy consumption of each sample task; Set the agent's reward to the weighted sum of the delay and energy consumption of each sample task, and receive the initial state And input the estimated Q neural network to output the value function value and action function value, add the value function value and action function value to get , according to the initial state passed into D3QN use Policy gets uninstall action And enter the state , based on the current state of the agent , Actions performed and the new state to which it is transferred Get rewarded ; The status Input the target Q neural network, according to the state of the input D3QN use Policy gets uninstall action and get ; Constructing experience pool parameters , calculate the loss function: ,in, is the discount factor; Update the estimated network parameters of the Q neural network by gradient descent to minimize the loss function; Multiple iterations to update the network parameters of the estimated Q neural network and the target Q neural network; When the number of training rounds reaches a certain number, samples are extracted from the experience pool parameters, and the rewards, value function values, and action function values ​​are calculated from these samples. The network parameters are updated by minimizing the loss function to obtain the final estimated Q neural network and target Q neural network; The weighted sum of latency and energy consumption is input into the trained agent to obtain the optimal offloading strategy for the current task.

[0035] Specifically, in the present invention, the Markov decision process is described as an MDP problem, and is thus solved using the D3QN algorithm.

[0036] The MDP problem is a tuple , each element can be described as follows: 1) Status : , in, Indicates The delay of each wireless sensor network node, Indicates Energy consumption of wireless sensor network nodes; 2) Action : , in, Indicates whether to uninstall; 3) Rewards : .

[0037] In the present invention, D3QN combines the characteristics of the dual DQN network and the adversarial DQN network. The D3QN algorithm is as follows: First, initialize an experience pool and initialize the estimated Q network parameters and target Q network parameters ; The round begins, the environment and parameters are initialized, and the current state is the initial state; To estimate the Q network The strategy selects the best action and gets rewarded. value and next state; Record the feature vector and state, action, reward and next state into the experience pool; For the target Q network, the greedy strategy is used to select the optimal action, and we get ; Update the estimated Q network by establishing a minimized loss function; After multiple rounds, update the target Q network parameters; After multiple rounds, multiple experiences are selected from the experience pool to retrain the network and update the estimated Q network parameters; Until the algorithm converges or the round ends.

[0038] like Figure 3 As shown, an embodiment of the present invention provides a device for allocating resources in a power sensor network, including: A data acquisition module is used to establish a wireless sensor network model of the power system, obtain the data streams of all tasks generated by all wireless sensor nodes, and abstract the data streams into different tasks. The nodes in the wireless sensor network model communicate with each other by transmitting data streams; A problem conversion module is used to determine the weighted sum of the delay and energy consumption of each data stream, use the weighted sum of the delay and energy consumption of each data stream as feedback of the wireless sensor network model, use task offloading as the decision of the wireless sensor network model, use the task offloading decision as an optimization problem, and model the optimization problem as a Markov decision process; The problem solving module is used to solve the Markov decision process using the D3QN algorithm, obtain the optimal offloading strategy, and allocate resources according to the optimal offloading strategy.

[0039] As a possible implementation of this embodiment, the step of establishing a wireless sensor network model of the power system includes: Considering coverage, signal strength and network connectivity, multiple wireless sensor nodes are reasonably distributed around the base station of the power system to collect data; An edge server is deployed on the base station side, and the edge server is used to receive data streams from wireless sensor nodes. The data streams are calculated by the wireless sensor nodes themselves or offloaded to the edge server for calculation.

[0040] As a possible implementation of this embodiment, the step of acquiring data streams of all tasks generated by all wireless sensor nodes and abstracting the data streams into different tasks includes: The data streams generated by all wireless sensor nodes are obtained. The data streams generated by different nodes are used as different tasks. The size of the data stream is used as the task size. The data stream of each wireless sensor node is abstracted as an independent task. The data stream includes task data generated by the wireless sensor node when sensing changes in the external environment.

[0041] The abstracted tasks are mapped to the available computing and communication resources to determine which tasks are calculated by the wireless sensor nodes themselves and which tasks need to be offloaded to the edge server for processing, thereby simulating the communication and computing process of each node in the wireless sensor network.

[0042] As a possible implementation manner of this embodiment, the power system includes a photovoltaic power grid, wireless sensor nodes deployed at different positions in the photovoltaic power grid, and all the wireless sensor nodes form a wireless sensor network.

[0043] As a possible implementation manner of this embodiment, the weighted sum of delay and energy consumption includes a first weighted sum of delay and energy consumption and a second weighted sum of delay and energy consumption, and determining the weighted sum of delay and energy consumption of each transmission data stream includes: According to the distribution of each wireless sensor node, the upload rate of each wireless sensor node is calculated using the channel model; For each wireless sensor node, based on the upload rate of the wireless sensor node, calculate the energy consumption and delay generated when the wireless sensor node uploads the generated task to the edge server for calculation, as well as the energy consumption and delay generated when the wireless sensor node calculates the generated task by itself; Obtaining a first delay and a weighted sum of energy consumption according to energy consumption and delay generated when the wireless sensor node uploads the generated task to the edge server for calculation; According to the energy consumption and delay generated when the wireless sensor node calculates the task itself, a second delay and a weighted sum of energy consumption are obtained.

[0044] As a possible implementation of this embodiment, the Markov decision process is a tuple, and the tuple includes a state space, an action space, and a reward function; The state space is: ,include The weighted sum of the latency and energy consumption of each task transmission; The action space is: , including the offloading actions of all wireless sensor nodes, where ,0 means the task is uploaded to the edge server for calculation, and 1 means the wireless sensor node calculates it by itself; The reward function is: ,in Represents the total cost of the system, which is the weighted sum of the total latency and energy consumption of the system.

[0045] The task offloading decision determines the size of the weighted sum of delay and energy consumption. The task offloading problem is regarded as an optimization problem, and the weighted sum of delay and energy consumption is used as the optimization target. The optimization problem is modeled as a Markov decision process for resource allocation. In order to obtain the task offloading strategy, the data streams generated by all wireless sensor nodes in the wireless sensor network of the power system communicating with the same base station are obtained, and the data streams of different tasks generated by different wireless sensor nodes are used to simulate the communication process of each node under the wireless sensor network model.

[0046] As a possible implementation of this embodiment, the problem solving module is specifically used to: Initialize the parameters of the neural network in D3QN, including the state, network parameters, and playback experience pool parameters; The state consists of two variables, namely the delay of each sample task and the energy consumption of each sample task. The reward is set as the weighted sum of the delay and energy consumption of each sample task. The initial state is received. And input the estimated Q neural network to output the value function value and action function value, add the value function value and action function value to get , according to the initial state passed into D3QN use Policy gets uninstall action And enter the state , based on the current state of the agent , Actions performed and the new state to which it is transferred Get rewarded ; The status Input the target Q neural network, according to the state of the input D3QN use Policy gets uninstall action and get ; Constructing experience pool parameters , calculate the loss function: ,in, is the discount factor; Update the estimated network parameters of the Q neural network by gradient descent to minimize the loss function; Multiple iterations to update the network parameters of the estimated Q neural network and the target Q neural network; When the number of training rounds reaches a certain number, samples are extracted from the experience pool parameters, and the rewards, value function values, and action function values ​​are calculated from these samples. The network parameters are updated by minimizing the loss function to obtain the final estimated Q neural network and target Q neural network; The agent obtains the weighted sum of the delay and energy consumption of the current task as a reward, and inputs the weighted sum of the delay and energy consumption into the trained agent to obtain the optimal offloading strategy for the current task.

[0047] The present invention combines wireless sensor network technology with reinforcement learning, minimizes total latency and total energy consumption by optimizing the unloading of node data and the allocation of edge server computing resources, and realizes multi-service resource allocation by solving the optimal unloading strategy, effectively reducing the latency and energy consumption of the system. The present invention also simulates scenarios with different numbers of wireless sensor nodes, and the results are better than those using the DQN algorithm.

[0048] An embodiment of the present invention provides a computer device, comprising: one or more processors; The processor is used to store one or more programs; When the one or more programs are executed by the one or more processors, the method for allocating resources in a power sensor network as described above is implemented.

[0049] Among them, in the embodiment of the present application, the processor is connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits together, which are all well known in the art, and therefore, are not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The business data processed by the processor is transmitted on the wireless medium through the antenna, and further, the antenna also receives the business data and transmits the business data to the processor. The processor is responsible for managing the bus and normal processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions.

[0050] Corresponding to the above-mentioned method for starting the application, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed, the above-mentioned method for allocating resources in the power sensor network is implemented.

[0051] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a computer-readable storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned computer-readable storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for resource allocation in a power sensor network, characterized in that: The steps include: Establishing a wireless sensor network model of the power system, obtaining data streams of all tasks generated by all wireless sensor nodes, and abstracting the data streams into different tasks, wherein the nodes in the wireless sensor network model communicate with each other by transmitting data streams; Determine the weighted sum of the time delay and energy consumption of each data stream, use the weighted sum of the time delay and energy consumption of each data stream as feedback of the wireless sensor network model, use task offloading as the decision of the wireless sensor network model, use the task offloading decision as an optimization problem, and model the optimization problem as a Markov decision process; The D3QN algorithm is used to solve the Markov decision process, obtain the optimal offloading strategy, and allocate resources according to the optimal offloading strategy.

2. The method for resource allocation in a power sensor network according to claim 1, characterized in that: The wireless sensor network model of the power system is established, including: Considering coverage, signal strength and network connectivity, multiple wireless sensor nodes are reasonably distributed around the base station of the power system to collect data; An edge server is deployed on the base station side, and the edge server is used to receive data streams from wireless sensor nodes. The data streams are calculated by the wireless sensor nodes themselves or offloaded to the edge server for calculation.

3. The method for resource allocation in a power sensor network according to claim 2, characterized in that: The method of obtaining the data streams of all tasks generated by all wireless sensor nodes and abstracting the data streams into different tasks includes: The data streams generated by all wireless sensor nodes are obtained. The data streams generated by different nodes are used as different tasks. The size of the data stream is used as the task size. The data stream of each wireless sensor node is abstracted as an independent task. The data stream includes task data generated by the wireless sensor node when sensing changes in the external environment.

4. The method for resource allocation in a power sensor network according to claim 1, characterized in that: The power system includes a photovoltaic power grid, wireless sensor nodes deployed at different positions in the photovoltaic power grid, and all the wireless sensor nodes form a wireless sensor network.

5. The method for resource allocation in a power sensor network according to claim 1, characterized in that: The weighted sum of delay and energy consumption includes a first weighted sum of delay and energy consumption and a second weighted sum of delay and energy consumption. Determining the weighted sum of delay and energy consumption of each transmission data stream includes: According to the distribution of each wireless sensor node, the upload rate of each wireless sensor node is calculated using the channel model; Based on the upload rate of the wireless sensor node, and according to the energy consumption and delay generated when the wireless sensor node uploads the generated task to the edge server for calculation, a first delay and a weighted sum of energy consumption are obtained; According to the energy consumption and delay generated when the wireless sensor node calculates the task itself, a second delay and a weighted sum of energy consumption are obtained.

6. The method for resource allocation in a power sensor network according to claim 5, characterized in that: The Markov decision process is a tuple, which includes a state space, an action space and a reward function; The state space is: ,include The weighted sum of the latency and energy consumption of each task transmission; The action space is: , including the offloading actions of all wireless sensor nodes, where ,0 means the task is uploaded to the edge server for calculation, and 1 means it is calculated by the wireless sensor node itself; The reward function is: ,in Represents the total cost of the system, which is the weighted sum of the total latency and energy consumption of the system.

7. The method for resource allocation in a power sensor network according to any one of claims 1 to 6, characterized in that: The method of using the D3QN algorithm to solve the Markov decision process, obtaining the optimal offloading strategy, and allocating resources according to the optimal offloading strategy includes: Initialize the parameters of the neural network in D3QN, including the state, network parameters and playback experience pool parameters. The state consists of two variables, namely the delay of each sample task and the energy consumption of each sample task; Set the agent's reward to the weighted sum of the delay and energy consumption of each sample task, and receive the initial state And input the estimated Q neural network to output the value function value and action function value, add the value function value and action function value to get , according to the initial state passed into D3QN use Policy gets uninstall action And enter the state , based on the current state of the agent , Actions performed and the new state to which it is transferred Get rewarded ; The status Input the target Q neural network, according to the state of the input D3QN use Policy gets uninstall action and get ; Constructing experience pool parameters , calculate the loss function: ,in, is the discount factor; Update the estimated network parameters of the Q neural network by gradient descent to minimize the loss function; Multiple iterations to update the network parameters of the estimated Q neural network and the target Q neural network; When the number of training rounds reaches a certain number, samples are extracted from the experience pool parameters, and the rewards, value function values, and action function values ​​are calculated from these samples. The network parameters are updated by minimizing the loss function to obtain the final estimated Q neural network and target Q neural network; The weighted sum of latency and energy consumption is input into the trained agent to obtain the optimal offloading strategy for the current task.

8. A device for resource allocation in a power sensor network, characterized in that: include: A data acquisition module is used to establish a wireless sensor network model of the power system, obtain the data streams of all tasks generated by all wireless sensor nodes, and abstract the data streams into different tasks. The nodes in the wireless sensor network model communicate with each other by transmitting data streams; A problem conversion module is used to determine the weighted sum of the delay and energy consumption of each data stream, use the weighted sum of the delay and energy consumption of each data stream as feedback of the wireless sensor network model, use task offloading as the decision of the wireless sensor network model, use the task offloading decision as an optimization problem, and model the optimization problem as a Markov decision process; The problem solving module is used to solve the Markov decision process using the D3QN algorithm, obtain the optimal offloading strategy, and allocate resources according to the optimal offloading strategy.

9. The device for allocating resources in a power sensor network according to claim 8, characterized in that: The wireless sensor network model of the power system is established, including: Considering coverage, signal strength and network connectivity, multiple wireless sensor nodes are reasonably distributed around the base station of the power system to collect data; An edge server is deployed on the base station side, and the edge server is used to receive data streams from wireless sensor nodes. The data streams are calculated by the wireless sensor nodes themselves or offloaded to the edge server for calculation.

10. The device for allocating resources in a power sensor network according to claim 9, characterized in that: The method of obtaining the data streams of all tasks generated by all wireless sensor nodes and abstracting the data streams into different tasks includes: The data streams generated by all wireless sensor nodes are obtained. The data streams generated by different nodes are used as different tasks. The size of the data stream is used as the task size. The data stream of each wireless sensor node is abstracted as an independent task. The data stream includes task data generated by the wireless sensor node when sensing changes in the external environment.

11. The device for allocating resources in a power sensor network according to claim 8, characterized in that: The power system includes a photovoltaic power grid, wireless sensor nodes deployed at different positions in the photovoltaic power grid, and all the wireless sensor nodes form a wireless sensor network.

12. The device for allocating resources in a power sensor network according to claim 8, characterized in that: The weighted sum of delay and energy consumption includes a first weighted sum of delay and energy consumption and a second weighted sum of delay and energy consumption. Determining the weighted sum of delay and energy consumption of each transmission data stream includes: According to the distribution of each wireless sensor node, the upload rate of each wireless sensor node is calculated using the channel model; Based on the upload rate of the wireless sensor node, and according to the energy consumption and delay generated when the wireless sensor node uploads the generated task to the edge server for calculation, a first delay and a weighted sum of energy consumption are obtained; According to the energy consumption and delay generated when the wireless sensor node calculates the task itself, a second delay and a weighted sum of energy consumption are obtained.

13. The device for allocating resources in a power sensor network according to claim 12, characterized in that: The Markov decision process is a tuple, which includes a state space, an action space and a reward function; The state space is: ,include The weighted sum of the latency and energy consumption of each task transmission; The action space is: , including the offloading actions of all wireless sensor nodes, where ,0 means the task is uploaded to the edge server for calculation, and 1 means the wireless sensor node calculates it by itself; The reward function is: ,in Represents the total cost of the system, which is the weighted sum of the total latency and energy consumption of the system.

14. The device for allocating resources in a power sensor network according to any one of claims 8 to 13, characterized in that: The problem-solving module is specifically used for: Initialize the parameters of the neural network in D3QN, including the state, network parameters, and playback experience pool parameters; The state consists of two variables, namely the delay of each sample task and the energy consumption of each sample task. The reward is set as the weighted sum of the delay and energy consumption of each sample task. The initial state is received. And input the estimated Q neural network to output the value function value and action function value, add the value function value and action function value to get , according to the initial state passed into D3QN use Policy gets uninstall action And enter the state , based on the current state of the agent , Actions performed and the new state to which it is transferred Get rewarded ; The status Input the target Q neural network, according to the state of the input D3QN use Policy gets uninstall action and get ; Constructing experience pool parameters , calculate the loss function: ,in, is the discount factor; Update the estimated network parameters of the Q neural network by gradient descent to minimize the loss function; Multiple iterations to update the network parameters of the estimated Q neural network and the target Q neural network; When the number of training rounds reaches a certain number, samples are extracted from the experience pool parameters, and the rewards, value function values, and action function values ​​are calculated from these samples. The network parameters are updated by minimizing the loss function to obtain the final estimated Q neural network and target Q neural network; The agent obtains the weighted sum of the delay and energy consumption of the current task as a reward, and inputs the weighted sum of the delay and energy consumption into the trained agent to obtain the optimal offloading strategy for the current task.

15. A computer device, characterized in that: include: one or more processors; The processor is used to store one or more programs; When the one or more programs are executed by the one or more processors, the method for resource allocation in a power sensor network according to any one of claims 1 to 7 is implemented.

16. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed, the method for allocating resources in a power sensor network as claimed in any one of claims 1 to 7 is implemented.

Images