Edge device control method and device, equipment and medium

Abstract

The invention relates to the technical field of power supply and distribution, in particular to an edge device control method and device, equipment and a medium. The method comprises the following steps: acquiring state information and edge-end coordination action information of the power Internet of Things; based on the state information and the edge-end coordination action information, establishing an edge-end coordination strategy model; converting the edge-end collaborative strategy model into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and training the edge-end collaborative strategy Q learning model; and based on the trained edge-end coordination strategy Q learning model, obtaining target edge-end coordination action information corresponding to the state information of the current power Internet of Things and the terminal node service data volume, and based on the target edge-end coordination action information, controlling the edge device. According to the scheme, the edge device and the terminal node can cooperatively process the service on the terminal node with relatively high efficiency and success rate, so that resources in the electric power Internet of Things are accurately allocated, and the operation efficiency of the electric power Internet of Things is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of power supply and distribution technology, and in particular to an edge device control method, device, equipment and medium. Background Art

[0002] The power Internet of Things is the application of the Internet of Things in the smart grid. It can effectively integrate communication infrastructure resources and power system infrastructure resources, improve the level of informatization of the power system, improve the utilization efficiency of the existing infrastructure of the power system, and provide important technical support for the power grid's generation, transmission, transformation, distribution, and consumption.

[0003] In recent years, with the rapid development of the power Internet of Things, the explosive growth in data volume has far exceeded the capacity of network bandwidth, and the emergence of a large number of new intelligent applications has put forward higher requirements on system latency performance.

[0004] As a new computing model, edge computing can enable data to be processed promptly and effectively at the edge network near the source. The power Internet of Things based on the edge computing model can enable the terminal node to offload tasks to nearby edge devices for execution, effectively alleviating the network congestion and high latency problems caused by the terminal node transmitting all data to the data center in the traditional solution.

[0005] However, the applicant found that when the number of terminal nodes and the scale of data in the power Internet of Things further increase, the resources in the power Internet of Things often cannot be accurately allocated, resulting in problems in the coordination between edge devices and terminal nodes in the power Internet of Things, resulting in reduced operating efficiency of the power Internet of Things. Summary of the invention

[0006] In order to solve the problems in the related art, the embodiments of the present disclosure provide an edge device control method, apparatus, device and medium.

[0007] In a first aspect, an edge device control method is provided in an embodiment of the present disclosure. The method is applied to an edge device in an electric power Internet of Things. The method includes:

[0008] Obtain the status information of the power Internet of Things and the edge-end collaborative action information; wherein the status information includes the uplink bandwidth of the link, the proportion of the bandwidth allocated by the link to the business of the terminal node, the bandwidth of the terminal node, and the link passed by the terminal node to the edge device and the task data volume of the terminal node; the edge-end collaborative action information includes the computing power resources allocated by the edge device to the corresponding terminal node, the computing power resources allocated by the corresponding container in the edge device to the corresponding terminal node, and the computing power resources allocated by the edge device to the corresponding container;

[0009] Based on the state information and the edge-end collaborative action information, an edge-end collaborative strategy model is established. The edge-end collaborative strategy model is used to indicate the relationship between the state information, the edge-end collaborative action information, and the probability of successful collaborative processing of services by the edge device and the terminal node;

[0010] Convert the edge-end collaborative strategy model into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and train the edge-end collaborative strategy Q learning model;

[0011] Based on the trained edge-end collaborative strategy Q learning model, the target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the business data volume of the terminal node is obtained, and the edge device is controlled based on the target edge-end collaborative action information.

[0012] In one implementation of the present disclosure, an edge-end collaboration strategy model is established based on state information and edge-end collaboration action information, including:

[0013] Based on the status information, a communication model is established. The communication model includes:

[0014]

[0015] Among them, B i is the bandwidth of the ith terminal node, ω j is the ratio of bandwidth allocated to the service of the i-th terminal node by the j-th link, B j is the uplink bandwidth of the jth link, R i is the set of links from the i-th terminal node to the edge device, T tr (i) is the transmission delay from the i-th terminal node to the edge device, z i is the business data volume of the i-th terminal node;

[0016] At least an edge-end collaboration strategy model is established based on the communication model.

[0017] In one implementation of the present disclosure, an edge-end collaboration strategy model is established based on the state information and the edge-end collaboration action information, further comprising:

[0018] Based on the edge-end collaborative action information, a computing model is established. The computing model includes:

[0019]

[0020] Among them, c edge,i The computing resources allocated to the i-th terminal node by the edge device, is the ratio of computing resources allocated from the kth container in the edge device to the ith terminal node, E kThe computing resources T allocated to the kth container by the edge device c (i) The time required for the edge device to process the service of the i-th terminal node;

[0021] At least establish an edge-end collaboration strategy model based on the communication model, including:

[0022] An edge-end collaboration strategy model is established based at least on the computing model and the communication model.

[0023] In one implementation of the present disclosure, the edge-end collaboration strategy model includes:

[0024]

[0025] Among them, R S is the probability of successful business processing between edge devices and terminal nodes, T is the cycle length of business processing between edge devices and terminal nodes, N is the number of terminal nodes, M is the number of containers in the edge device, is the success rate of the coordination between the i-th terminal node and the edge device to process the service at time t, where It means that the i-th terminal node and the edge device successfully coordinate and process the business at time t. Indicates that the i-th terminal node and the edge device fail to coordinate and process the service at time t, T total (i) = T tr (i)+T c (i), T task It is the preset duration threshold.

[0026] In one implementation of the present disclosure, the edge-end collaborative strategy model is converted into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, including:

[0027] Get state space data s (t) And the action space data a (t) ,in in is the business data volume of the Nth terminal node at time t, is the ratio of bandwidth allocated to the Nth terminal node by the link corresponding to time t, is the ratio of computing resources allocated by the corresponding container in the edge device to the Nth terminal node at time t;

[0028] Obtain the state of the distribution network to satisfy the state space data s( t ), based on the action space data a( t ) The updated value function Q of the control distribution network new (s (t) ,a (t));

[0029] in,

[0030] R (t+1) The number of successful services coordinated by the terminal nodes and edge devices at time t+1;

[0031] Q old (s (t) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t) When, based on the action space data a (t) The value function before the update of the control distribution network;

[0032] Q old (s (t+1) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t+1) When, based on the action space data a (t) The value function before the update of the control distribution network is: is the business data volume of the Nth terminal node at time t+1;

[0033] g is the learning rate, β is the discount rate, and A is the set of action space data;

[0034] A Q-table is constructed based on the updated value function, and an edge-end collaborative strategy Q-learning model is constructed based on the Q-table.

[0035] In a second aspect, an edge device control device is provided in an embodiment of the present disclosure. The device is located on an edge device in the power Internet of Things. The device includes:

[0036] An information acquisition module is configured to acquire status information and edge-end collaborative action information of the electric power Internet of Things; wherein the status information includes the uplink bandwidth of the link, the proportion of the bandwidth allocated by the link to the business of the terminal node, the bandwidth of the terminal node, and the link passed by the terminal node to the edge device and the task data volume of the terminal node; the edge-end collaborative action information includes the computing power resources allocated by the edge device to the corresponding terminal node, the computing power resources allocated by the corresponding container in the edge device to the corresponding terminal node, and the computing power resources allocated by the edge device to the corresponding container;

[0037] A model building module is configured to build an edge-end collaborative strategy model based on the state information and the edge-end collaborative action information, where the edge-end collaborative strategy model is used to indicate the relationship between the state information, the edge-end collaborative action information, and the probability of successful collaborative processing of services by the edge device and the terminal node;

[0038] A model training module is configured to convert the edge-end collaborative strategy model into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and train the edge-end collaborative strategy Q learning model;

[0039] The device control module is configured to obtain target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the amount of terminal node business data based on the trained edge-end collaborative strategy Q learning model, and control the edge device based on the target edge-end collaborative action information.

[0040] In one implementation of the present disclosure, the model building module is specifically configured as follows:

[0041] Based on the status information, a communication model is established. The communication model includes:

[0042]

[0043] Among them, B i is the bandwidth of the ith terminal node, ω j is the ratio of bandwidth allocated to the service of the i-th terminal node by the j-th link, B j is the uplink bandwidth of the jth link, R i is the set of links from the i-th terminal node to the edge device, T tr (i) is the transmission delay from the i-th terminal node to the edge device, z i is the business data volume of the i-th terminal node;

[0044] At least an edge-end collaboration strategy model is established based on the communication model.

[0045] In one implementation of the present disclosure, the model building module is specifically configured as follows:

[0046] Based on the edge-end collaborative action information, a computing model is established. The computing model includes:

[0047]

[0048] Among them, c edge,i The computing resources allocated to the i-th terminal node by the edge device, is the ratio of computing resources allocated from the kth container in the edge device to the ith terminal node, E k The computing resources T allocated to the kth container by the edge device c (i) The time required for the edge device to process the service of the i-th terminal node;

[0049] An edge-end collaboration strategy model is established based at least on the computing model and the communication model.

[0050] In one implementation of the present disclosure, the edge-end collaboration strategy model includes:

[0051]

[0052] Among them, R S is the probability of successful business processing between edge devices and terminal nodes, T is the cycle length of business processing between edge devices and terminal nodes, N is the number of terminal nodes, M is the number of containers in the edge device, is the success rate of the coordination between the i-th terminal node and the edge device to process the service at time t, where It means that the i-th terminal node and the edge device successfully coordinate and process the business at time t. Indicates that the i-th terminal node and the edge device fail to coordinate and process the service at time t, T total (i) = T tr (i)+T c (i), T task It is the preset duration threshold.

[0053] In one implementation of the present disclosure, the model training module is specifically configured as follows:

[0054] Get state space data s (t) And the action space data a (t) ,in in is the business data volume of the Nth terminal node at time t, is the ratio of bandwidth allocated to the Nth terminal node by the link corresponding to time t, is the ratio of computing resources allocated by the corresponding container in the edge device to the Nth terminal node at time t;

[0055] Obtain the state of the distribution network to satisfy the state space data s (t) When, based on the action space data a (t) The updated value function Q of the control distribution network new (s (t) ,a (t) );

[0056] in,

[0057] R (t+1) The number of successful services coordinated by the terminal nodes and edge devices at time t+1;

[0058] Q old (s (t) ,a (t) ) is the state of the distribution network that satisfies the state space data s(t) When, based on the action space data a (t) The value function before the update of the control distribution network;

[0059] Q old (s (t+1) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t+1) When, based on the action space data a (t) The value function before the update of the control distribution network is: is the business data volume of the Nth terminal node at time t+1;

[0060] g is the learning rate, β is the discount rate, and A is the set of action space data;

[0061] A Q-table is constructed based on the updated value function, and an edge-end collaborative strategy Q-learning model is constructed based on the Q-table.

[0062] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, and wherein the one or more computer instructions are executed by the processor to implement a method as described in any one of the first aspects.

[0063] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, a method as described in any one of the first aspects is implemented.

[0064] According to the technical solution provided by the embodiments of the present disclosure, the state information and edge-end collaborative action information of the power Internet of Things are obtained by the edge device in the power Internet of Things; based on the state information and edge-end collaborative action information, an edge-end collaborative strategy model is established, and the edge-end collaborative strategy model is used to indicate the relationship between the state information, the edge-end collaborative action information, and the probability of successful collaborative processing of services by the edge device and the terminal node; the edge-end collaborative strategy model is converted into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and the edge-end collaborative strategy Q learning model is trained; based on the trained edge-end collaborative strategy Q learning model, the target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the amount of terminal node business data is obtained, and the edge device is controlled based on the target edge-end collaborative action information. In this scheme, the edge device can be controlled based on the target edge-end collaborative action information, under the premise of fully utilizing the uplink bandwidth of the link, the bandwidth allocated to the business of the terminal node, and the bandwidth of the terminal node, so that the edge device and the terminal node can collaboratively process the business on the terminal node with higher efficiency, and improve the success rate of collaborative processing of business, thereby accurately allocating resources in the power Internet of Things and improving the operating efficiency of the power Internet of Things.

[0065] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Other features, objectives and advantages of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0067] Figure 1 A flow chart of an edge device control method according to an embodiment of the present disclosure is shown.

[0068] Figure 2 A structural block diagram of an edge device control apparatus according to an embodiment of the present disclosure is shown.

[0069] Figure 3 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0070] Figure 4 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0071] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0072] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, behaviors, components, parts, or a combination thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or a combination thereof exist or are added.

[0073] It should also be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0074] In the present disclosure, if it involves operations of obtaining user information or user data or displaying user information or user data to others, the operations are all authorized and confirmed by the user, or actively selected by the user.

[0075] Figure 1 The flowchart of the edge device control method according to the embodiment of the present disclosure is shown, and the method is applied to the edge device in the power Internet of Things. Figure 1 As shown, the edge device control method includes the following steps S101-S104:

[0076] In step S101, the state information of the power Internet of Things and the edge-end collaborative action information are obtained;

[0077] Among them, the status information includes the uplink bandwidth of the link, the proportion of the bandwidth allocated by the link to the business of the terminal node, the bandwidth of the terminal node, the link passed by the terminal node to the edge device, and the task data volume of the terminal node. The edge-end collaborative action information includes the computing power resources allocated by the edge device to the corresponding terminal node, the computing power resources allocated by the corresponding container in the edge device to the corresponding terminal node, and the computing power resources allocated by the edge device to the corresponding container;

[0078] In step S102, an edge-end collaboration strategy model is established based on the state information and the edge-end collaboration action information;

[0079] Among them, the edge-end collaboration strategy model is used to indicate the relationship between state information, edge-end collaboration action information, and the probability of successful collaboration between edge devices and terminal nodes to process services;

[0080] In step S103, the edge-end collaborative strategy model is converted into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and the edge-end collaborative strategy Q learning model is trained;

[0081] In step S104, based on the trained edge-end collaborative strategy Q learning model, the target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the amount of terminal node business data is obtained, and the edge device is controlled based on the target edge-end collaborative action information.

[0082] In one implementation of the present disclosure, the terminal node can be understood as an electric meter equipped with a sensor, a smart switch, etc. in the power Internet of Things. The terminal nodes can be locally networked through high-speed power line carrier communication (High Power Line Carrie, HPLC) or micro-power wireless communication. The edge device can perform routing calculation and bandwidth allocation for the terminal node network composed of multiple terminal nodes through the southbound communication interface.

[0083] According to the technical solution provided by the embodiments of the present disclosure, the state information and edge-end collaborative action information of the power Internet of Things are obtained by the edge device in the power Internet of Things; based on the state information and edge-end collaborative action information, an edge-end collaborative strategy model is established, and the edge-end collaborative strategy model is used to indicate the relationship between the state information, the edge-end collaborative action information, and the probability of successful collaborative processing of services by the edge device and the terminal node; the edge-end collaborative strategy model is converted into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and the edge-end collaborative strategy Q learning model is trained; based on the trained edge-end collaborative strategy Q learning model, the target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the amount of terminal node business data is obtained, and the edge device is controlled based on the target edge-end collaborative action information. In this scheme, the edge device can be controlled based on the target edge-end collaborative action information, under the premise of fully utilizing the uplink bandwidth of the link, the bandwidth allocated to the business of the terminal node, and the bandwidth of the terminal node, so that the edge device and the terminal node can collaboratively process the business on the terminal node with higher efficiency, and improve the success rate of collaborative processing of business, thereby accurately allocating resources in the power Internet of Things and improving the operating efficiency of the power Internet of Things.

[0084] In one implementation of the present disclosure, an edge-end collaboration strategy model is established based on state information and edge-end collaboration action information, including:

[0085] Based on the status information, a communication model is established. The communication model includes:

[0086]

[0087] Among them, B i is the bandwidth of the ith terminal node, ω j is the ratio of bandwidth allocated to the service of the i-th terminal node by the j-th link, B jis the uplink bandwidth of the jth link, R i is the set of links from the i-th terminal node to the edge device, T tr (i) is the transmission delay from the i-th terminal node to the edge device, z i is the business data volume of the i-th terminal node;

[0088] At least an edge-end collaboration strategy model is established based on the communication model.

[0089] According to the technical solution provided by the embodiments of the present disclosure, by establishing a communication model based on state information, and establishing an edge-end collaborative strategy model at least based on the communication model, the efficiency of establishing the edge-end collaborative strategy model can be improved.

[0090] In one implementation of the present disclosure, an edge-end collaboration strategy model is established based on the state information and the edge-end collaboration action information, further comprising:

[0091] Based on the edge-end collaborative action information, a computing model is established. The computing model includes:

[0092]

[0093] Among them, c edge,i The computing resources allocated to the i-th terminal node by the edge device, is the ratio of computing resources allocated from the kth container in the edge device to the ith terminal node, E k The computing resources T allocated to the kth container by the edge device c (i) The time required for the edge device to process the service of the i-th terminal node;

[0094] At least establish an edge-end collaboration strategy model based on the communication model, including:

[0095] An edge-end collaboration strategy model is established based at least on the computing model and the communication model.

[0096] According to the technical solution provided by the embodiments of the present disclosure, by establishing a computing model based on edge-end collaborative action information, and establishing an edge-end collaborative strategy model based at least on the computing model and the communication model, the reliability of the established edge-end collaborative strategy model can be improved.

[0097] In one implementation of the present disclosure, the edge-end collaboration strategy model includes:

[0098]

[0099] Among them, R Sis the probability of successful business processing between edge devices and terminal nodes, T is the cycle length of business processing between edge devices and terminal nodes, N is the number of terminal nodes, M is the number of containers in the edge device, is the success rate of the coordination between the i-th terminal node and the edge device to process the service at time t, where It means that the i-th terminal node and the edge device successfully coordinate and process the business at time t. Indicates that the i-th terminal node and the edge device fail to coordinate and process the service at time t, T total (i) = T tr (i)+T c (i).

[0100] In one implementation of the present disclosure, the edge-end collaborative strategy model is converted into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, including:

[0101] Get state space data s (t) And the action space data a (t) ,in in is the business data volume of the Nth terminal node at time t, is the ratio of bandwidth allocated to the Nth terminal node by the link corresponding to time t, is the ratio of computing resources allocated by the corresponding container in the edge device to the Nth terminal node at time t;

[0102] Obtain the state of the distribution network to satisfy the state space data s( t ), based on the action space data a( t ) The updated value function Q of the control distribution network new (s (t) ,a (t) );

[0103] in,

[0104] R (t+1) The number of successful services coordinated by the terminal nodes and edge devices at time t+1;

[0105] Q old (s (t) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t) When, based on the action space data a (t) The value function before the update of the control distribution network;

[0106] Q old (s(t+1) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t+1) When, based on the action space data a (t) The value function before the update of the control distribution network is: is the business data volume of the Nth terminal node at time t+1;

[0107] g is the learning rate, β is the discount rate, and A is the set of action space data;

[0108] A Q-table is constructed based on the updated value function, and an edge-end collaborative strategy Q-learning model is constructed based on the Q-table.

[0109] In one implementation of the present disclosure, training the edge-end collaborative strategy Q learning model can be implemented through the following steps:

[0110] Initialize the learning rate α, discount rate β, and Q table. For example, the values ​​of the learning rate α, discount rate β, and Q table can all be initialized to 0;

[0111] Get the state space data s( t );

[0112] According to the greedy strategy, obtain the current action space data a( t );

[0113] Based on the current state space data s( t ) and action space data a( t ), calculate R( t+1 );

[0114] based on Calculate the state space data s for the next time slot (t+1) ,in, is the business data volume of the i-th terminal node at time t+1, is the business data volume of the i-th terminal node at time t, is the proportion of computing resources allocated by the Kth container in the edge device to the i-th terminal node at time t;

[0115] Get the state space data s( t+1 );

[0116] based on Update the Q table in the edge-end collaborative strategy Q learning model.

[0117] Figure 2The structural block diagram of the edge device control device according to the embodiment of the present disclosure is shown. The device is located on the edge device in the power Internet of Things, wherein the device can be implemented as part or all of the electronic device through software, hardware or a combination of both.

[0118] like Figure 2 As shown, the edge device control device 200 includes:

[0119] The information acquisition module 201 is configured to acquire the status information of the electric power Internet of Things and the edge-end collaborative action information; wherein the status information includes the uplink bandwidth of the link, the ratio of the bandwidth allocated by the link to the business of the terminal node, the bandwidth of the terminal node, and the link passed by the terminal node to the edge device and the task data volume of the terminal node; the edge-end collaborative action information includes the computing power resources allocated by the edge device to the corresponding terminal node, the computing power resources allocated by the corresponding container in the edge device to the corresponding terminal node, and the computing power resources allocated by the edge device to the corresponding container;

[0120] A model building module 202 is configured to build an edge-end collaborative strategy model based on the state information and the edge-end collaborative action information, where the edge-end collaborative strategy model is used to indicate the relationship between the state information, the edge-end collaborative action information, and the probability of successful collaborative processing of services by the edge device and the terminal node;

[0121] A model training module 203 is configured to convert the edge-end collaborative strategy model into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and train the edge-end collaborative strategy Q learning model;

[0122] The device control module 204 is configured to obtain target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the amount of terminal node business data based on the trained edge-end collaborative strategy Q learning model, and control the edge device based on the target edge-end collaborative action information.

[0123] In one implementation of the present disclosure, the model building module 202 is specifically configured as follows:

[0124] Based on the status information, a communication model is established. The communication model includes:

[0125]

[0126] Among them, B i is the bandwidth of the ith terminal node, ω j is the ratio of bandwidth allocated to the service of the i-th terminal node by the j-th link, B j is the uplink bandwidth of the jth link, R i is the set of links from the i-th terminal node to the edge device, Ttr (i) is the transmission delay from the i-th terminal node to the edge device, z i is the business data volume of the i-th terminal node;

[0127] At least an edge-end collaboration strategy model is established based on the communication model.

[0128] In one implementation of the present disclosure, the model building module 202 is specifically configured as follows:

[0129] Based on the edge-end collaborative action information, a computing model is established. The computing model includes:

[0130]

[0131] Among them, c edge,i The computing resources allocated to the i-th terminal node by the edge device, is the ratio of computing resources allocated from the kth container in the edge device to the ith terminal node, E k The computing resources T allocated to the kth container by the edge device c (i) The time required for the edge device to process the service of the i-th terminal node;

[0132] An edge-end collaboration strategy model is established based at least on the computing model and the communication model.

[0133] In one implementation of the present disclosure, the edge-end collaboration strategy model includes:

[0134]

[0135] Among them, R S is the probability of successful business processing between edge devices and terminal nodes, T is the cycle length of business processing between edge devices and terminal nodes, N is the number of terminal nodes, M is the number of containers in the edge device, is the success rate of the coordination between the i-th terminal node and the edge device to process the service at time t, where It means that the i-th terminal node and the edge device successfully coordinate and process the business at time t. Indicates that the i-th terminal node and the edge device fail to coordinate and process the service at time t, T total (i) = T tr (i)+T c (i), T task It is the preset duration threshold.

[0136] In one implementation of the present disclosure, the model training module 203 is specifically configured as follows:

[0137] Get state space data s (t)And the action space data a (t) ,in in is the business data volume of the Nth terminal node at time t, is the ratio of bandwidth allocated to the Nth terminal node by the link corresponding to time t, is the ratio of computing resources allocated by the corresponding container in the edge device to the Nth terminal node at time t;

[0138] Obtain the state of the distribution network to satisfy the state space data s( t ), based on the action space data a( t ) The updated value function Q of the control distribution network new (s (t) ,a (t) );

[0139] in,

[0140] R (t+1) The number of successful services coordinated by the terminal nodes and edge devices at time t+1;

[0141] Q old (s (t) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t) When, based on the action space data a (t) The value function before the update of the control distribution network;

[0142] Q old (s (t+1) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t+1) When, based on the action space data a (t) The value function before the update of the control distribution network is: is the business data volume of the Nth terminal node at time t+1;

[0143] g is the learning rate, β is the discount rate, and A is the set of action space data;

[0144] A Q-table is constructed based on the updated value function, and an edge-end collaborative strategy Q-learning model is constructed based on the Q-table.

[0145] According to the technical solution provided by the embodiments of the present disclosure, the state information and edge-end collaborative action information of the power Internet of Things are obtained by the edge device in the power Internet of Things; based on the state information and edge-end collaborative action information, an edge-end collaborative strategy model is established, and the edge-end collaborative strategy model is used to indicate the relationship between the state information, the edge-end collaborative action information, and the probability of successful collaborative processing of services by the edge device and the terminal node; the edge-end collaborative strategy model is converted into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and the edge-end collaborative strategy Q learning model is trained; based on the trained edge-end collaborative strategy Q learning model, the target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the amount of terminal node business data is obtained, and the edge device is controlled based on the target edge-end collaborative action information. In this scheme, the edge device can be controlled based on the target edge-end collaborative action information, under the premise of fully utilizing the uplink bandwidth of the link, the bandwidth allocated to the business of the terminal node, and the bandwidth of the terminal node, so that the edge device and the terminal node can collaboratively process the business on the terminal node with higher efficiency, and improve the success rate of collaborative processing of business, thereby accurately allocating resources in the power Internet of Things and improving the operating efficiency of the power Internet of Things.

[0146] The present disclosure also discloses an electronic device, Figure 3 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0147] like Figure 3 As shown, the electronic device includes a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to an embodiment of the present disclosure.

[0148] An edge device control method is provided in an embodiment of the present disclosure. The method is applied to an edge device in an electric power Internet of Things. The method includes:

[0149] Obtain the status information of the power Internet of Things and the edge-end collaborative action information; wherein the status information includes the uplink bandwidth of the link, the proportion of the bandwidth allocated by the link to the business of the terminal node, the bandwidth of the terminal node, and the link passed by the terminal node to the edge device and the task data volume of the terminal node; the edge-end collaborative action information includes the computing power resources allocated by the edge device to the corresponding terminal node, the computing power resources allocated by the corresponding container in the edge device to the corresponding terminal node, and the computing power resources allocated by the edge device to the corresponding container;

[0150] Based on the state information and the edge-end collaborative action information, an edge-end collaborative strategy model is established. The edge-end collaborative strategy model is used to indicate the relationship between the state information, the edge-end collaborative action information, and the probability of successful collaborative processing of services by the edge device and the terminal node;

[0151] Convert the edge-end collaborative strategy model into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and train the edge-end collaborative strategy Q learning model;

[0152] Based on the trained edge-end collaborative strategy Q learning model, the target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the business data volume of the terminal node is obtained, and the edge device is controlled based on the target edge-end collaborative action information.

[0153] In one implementation of the present disclosure, an edge-end collaboration strategy model is established based on state information and edge-end collaboration action information, including:

[0154] Based on the status information, a communication model is established. The communication model includes:

[0155]

[0156] Among them, B i is the bandwidth of the ith terminal node, ω j is the ratio of bandwidth allocated to the service of the i-th terminal node by the j-th link, B j is the uplink bandwidth of the jth link, R i is the set of links from the i-th terminal node to the edge device, T tr (i) is the transmission delay from the i-th terminal node to the edge device, z i is the business data volume of the i-th terminal node;

[0157] At least an edge-end collaboration strategy model is established based on the communication model.

[0158] In one implementation of the present disclosure, an edge-end collaboration strategy model is established based on the state information and the edge-end collaboration action information, further comprising:

[0159] Based on the edge-end collaborative action information, a computing model is established. The computing model includes:

[0160]

[0161] Among them, c edge,i The computing resources allocated to the i-th terminal node by the edge device, is the ratio of computing resources allocated from the kth container in the edge device to the ith terminal node, E k The computing resources T allocated to the kth container by the edge device c (i) The time required for the edge device to process the service of the i-th terminal node;

[0162] At least establish an edge-end collaboration strategy model based on the communication model, including:

[0163] An edge-end collaboration strategy model is established based at least on the computing model and the communication model.

[0164] In one implementation of the present disclosure, the edge-end collaboration strategy model includes:

[0165]

[0166] Among them, R S is the probability of successful business processing between edge devices and terminal nodes, T is the cycle length of business processing between edge devices and terminal nodes, N is the number of terminal nodes, M is the number of containers in the edge device, is the success rate of the coordination between the i-th terminal node and the edge device to process the service at time t, where It means that the i-th terminal node and the edge device successfully coordinate and process the business at time t. Indicates that the i-th terminal node and the edge device fail to coordinate and process the service at time t, T total (i) = T tr (i)+T c (i), T task It is the preset duration threshold.

[0167] In one implementation of the present disclosure, the edge-end collaborative strategy model is converted into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, including:

[0168] Get state space data s (t) And the action space data a (t) ,in in is the business data volume of the Nth terminal node at time t, is the ratio of bandwidth allocated to the Nth terminal node by the link corresponding to time t, is the ratio of computing resources allocated by the corresponding container in the edge device to the Nth terminal node at time t;

[0169] Obtain the state of the distribution network to satisfy the state space data s( t ), based on the action space data a( t ) The updated value function Q of the control distribution network new (s (t) ,a (t) );

[0170] in,

[0171] R (t+1) The number of successful services coordinated by the terminal nodes and edge devices at time t+1;

[0172] Q old (s (t) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t) When, based on the action space data a (t) The value function before the update of the control distribution network;

[0173] Q old (s (t+1) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t+1) When, based on the action space data a (t) The value function before the update of the control distribution network is: is the business data volume of the Nth terminal node at time t+1;

[0174] g is the learning rate, β is the discount rate, and A is the set of action space data;

[0175] A Q-table is constructed based on the updated value function, and an edge-end collaborative strategy Q-learning model is constructed based on the Q-table.

[0176] Figure 4 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown.

[0177] like Figure 4 As shown, the computer system includes a processing unit, which can perform the various methods in the above-mentioned embodiments according to the program stored in the read-only memory (ROM) or the program loaded from the storage part into the random access memory (RAM). In the RAM, various programs and data required for the operation of the computer system are also stored. The processing unit, ROM and RAM are connected to each other through a bus. The input / output (I / O) interface is also connected to the bus.

[0178] The following components are connected to the I / O interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a LAN card, a modem, etc. The communication part performs a communication process via a network such as the Internet. The drive is also connected to the I / O interface as needed. Removable media, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed so that the computer program read therefrom is installed into the storage part as needed. Among them, the processing unit can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0179] In particular, according to an embodiment of the present disclosure, the method described above can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a machine-readable medium, and the computer program includes a program code for executing the above method. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium.

[0180] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0181] The units or modules involved in the embodiments described in the present disclosure may be implemented by software or programmable hardware. The units or modules described may also be set in a processor, and the names of these units or modules do not constitute limitations on the units or modules themselves in some cases.

[0182] As another aspect, the present disclosure further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system in the above embodiment; or a computer-readable storage medium that exists independently and is not assembled into a device. The computer-readable storage medium stores one or more programs, and the programs are used by one or more processors to execute the method described in the present disclosure.

[0183] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

Claims

1. A method for controlling an edge device, characterized in that: The method is applied to an edge device in the electric power Internet of Things, and the method comprises: Obtaining the status information and edge-end collaborative action information of the electric power Internet of Things; wherein the status information includes the uplink bandwidth of the link, the proportion of the bandwidth allocated to the services of the terminal node, the bandwidth of the terminal node, and the link passed from the terminal node to the edge device, and the task data volume of the terminal node; the edge-end collaborative action information includes the computing power resources allocated by the edge device to the corresponding terminal node, the computing power resources allocated by the corresponding container in the edge device to the corresponding terminal node, and the computing power resources allocated by the edge device to the corresponding container; Based on the state information and the edge-end collaborative action information, an edge-end collaborative strategy model is established, wherein the edge-end collaborative strategy model is used to indicate the relationship between the state information, the edge-end collaborative action information, and the probability of successful collaborative processing of services by the edge device and the terminal node; Converting the edge-end collaborative strategy model into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and training the edge-end collaborative strategy Q learning model; Based on the trained edge-end collaborative strategy Q learning model, the target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the amount of terminal node business data is obtained, and the edge device is controlled based on the target edge-end collaborative action information.

2. The edge device control method according to claim 1, characterized in that: The establishing of an edge-end collaborative strategy model based on the state information and the edge-end collaborative action information includes: Based on the state information, a communication model is established, wherein the communication model includes: Among them, B i is the bandwidth of the ith terminal node, ω j is the ratio of bandwidth allocated to the service of the i-th terminal node by the j-th link, B j is the uplink bandwidth of the jth link, R i is the set of links from the i-th terminal node to the edge device, T tr (i) is the transmission delay from the i-th terminal node to the edge device, z i is the business data volume of the i-th terminal node; The edge-end collaboration strategy model is established based at least on the communication model.

3. The edge device control method according to claim 2, characterized in that: The establishing of an edge-end coordination strategy model based on the state information and the edge-end coordination action information further includes: Based on the edge-end collaborative action information, a calculation model is established, wherein the calculation model includes: Among them, c edge,i The computing resources allocated to the i-th terminal node by the edge device, is the ratio of computing resources allocated from the kth container in the edge device to the ith terminal node, E k The computing resources allocated to the kth container by the edge device, T c (i) The time required for the edge device to process the service of the i-th terminal node; The establishing the edge-end collaboration strategy model at least based on the communication model includes: The edge-end collaboration strategy model is established based at least on the computing model and the communication model.

4. The edge device control method according to claim 3, characterized in that: The edge-end collaboration strategy model includes: Among them, R S is the probability of successful business processing between edge devices and terminal nodes, T is the cycle length of business processing between edge devices and terminal nodes, N is the number of terminal nodes, M is the number of containers in the edge device, is the success rate of the coordination between the i-th terminal node and the edge device to process the service at time t, where It means that the i-th terminal node and the edge device successfully coordinate and process the business at time t. Indicates that the i-th terminal node and the edge device fail to coordinate and process the service at time t, T total (i) = T tr (i)+T c (i), T task It is the preset duration threshold.

5. The edge device control method according to claim 4, characterized in that: The step of converting the edge-end collaborative strategy model into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model includes: Get state space data s (t) And the action space data a (t) ,in in is the business data volume of the Nth terminal node at time t, is the ratio of bandwidth allocated to the Nth terminal node by the link corresponding to time t, is the ratio of computing resources allocated by the corresponding container in the edge device to the Nth terminal node at time t; Obtain the state of the distribution network to satisfy the state space data s (t) When, based on the action space data a (t) The updated value function Q of controlling the distribution network new (s (t) ,a (t) ); in, R (t+1) The number of successful services coordinated by the terminal nodes and edge devices at time t+1; Q old (s (t) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t) When, based on the action space data a (t) a pre-updated value function for controlling the power distribution network; Q old (s (t+1) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t+1) When, based on the action space data a (t) The value function before the update of the control distribution network, where is the business data volume of the Nth terminal node at time t+1; g is the learning rate, β is the discount rate, and A is the set of action space data; A Q-table is constructed based on the updated value function, and an edge-end collaborative strategy Q-learning model is constructed based on the Q-table.

6. An edge device control device, characterized in that: The device is located on an edge device in the electric power Internet of Things, and the device includes: An information acquisition module is configured to acquire status information and edge-end collaborative action information of the electric power Internet of Things; wherein the status information includes the uplink bandwidth of the link, the ratio of the bandwidth allocated by the link to the business of the terminal node, the bandwidth of the terminal node, and the link passed by the terminal node to the edge device and the task data volume of the terminal node; the edge-end collaborative action information includes the computing power resources allocated by the edge device to the corresponding terminal node, the computing power resources allocated by the corresponding container in the edge device to the corresponding terminal node, and the computing power resources allocated by the edge device to the corresponding container; A model building module is configured to build an edge-end collaborative strategy model based on the state information and the edge-end collaborative action information, wherein the edge-end collaborative strategy model is used to indicate the relationship between the state information, the edge-end collaborative action information, and the probability of successful collaborative processing of services by the edge device and the terminal node; A model training module is configured to convert the edge-end collaborative strategy model into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and train the edge-end collaborative strategy Q learning model; The device control module is configured to obtain target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the amount of terminal node business data based on the trained edge-end collaborative strategy Q learning model, and control the edge device based on the target edge-end collaborative action information.

7. The edge device control device according to claim 6, characterized in that: The model building module is specifically configured as follows: Based on the state information, a communication model is established, wherein the communication model includes: Among them, B i is the bandwidth of the ith terminal node, ω j is the ratio of bandwidth allocated to the service of the i-th terminal node by the j-th link, B j is the uplink bandwidth of the jth link, R i is the set of links from the i-th terminal node to the edge device, T tr (i) is the transmission delay from the i-th terminal node to the edge device, z i is the business data volume of the i-th terminal node; The edge-end collaboration strategy model is established based at least on the communication model.

8. The edge device control device according to claim 7, characterized in that: The model building module is specifically configured as follows: Based on the edge-end collaborative action information, a calculation model is established, wherein the calculation model includes: Among them, c edge,i The computing resources allocated to the i-th terminal node by the edge device, is the ratio of computing resources allocated from the kth container in the edge device to the ith terminal node, E k The computing resources T allocated to the kth container by the edge device c (i) The time required for the edge device to process the service of the i-th terminal node; The edge-end collaboration strategy model is established based at least on the computing model and the communication model.

9. The edge device control device according to claim 8, characterized in that: The edge-end collaboration strategy model includes: Among them, R S is the probability of successful business processing between edge devices and terminal nodes, T is the cycle length of business processing between edge devices and terminal nodes, N is the number of terminal nodes, M is the number of containers in the edge device, is the success rate of the coordination between the i-th terminal node and the edge device to process the service at time t, where It means that the i-th terminal node and the edge device successfully coordinate and process the business at time t. Indicates that the i-th terminal node and the edge device fail to coordinate and process the service at time t, T total (i) = T tr (i)+T c (i), T task It is the preset duration threshold.

10. The edge device control device according to claim 9, characterized in that: The model training module is specifically configured as follows: Get state space data s (t) And the action space data a (t) ,in in is the business data volume of the Nth terminal node at time t, is the ratio of bandwidth allocated to the Nth terminal node by the link corresponding to time t, is the ratio of computing resources allocated by the corresponding container in the edge device to the Nth terminal node at time t; Obtain the state of the distribution network to satisfy the state space data s( t ), based on the action space data a( t ) controls the updated value function Q of the distribution network new (s (t) ,a (t) ); in, R (t+1) The number of successful services coordinated by the terminal nodes and edge devices at time t+1; Q old (s (t) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t) When, based on the action space data a (t) a pre-updated value function for controlling the power distribution network; Q old (s (t+1) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t+1) When, based on the action space data a (t) The value function before the update of the control distribution network, where is the business data volume of the Nth terminal node at time t+1; g is the learning rate, β is the discount rate, and A is the set of action space data; A Q-table is constructed based on the updated value function, and an edge-end collaborative strategy Q-learning model is constructed based on the Q-table.

11. An electronic device, characterized in that: The method comprises a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps described in any one of claims 1 to 5.

12. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the method steps described in any one of claims 1 to 5 are implemented.

Images