Edge device control method, device, equipment and medium

Abstract

The present disclosure relates to the field of power supply and distribution technology, and specifically to an edge device control method, device, equipment and medium. The method includes: obtaining the status information of the power Internet of Things and the edge-end collaborative action information; establishing an edge-end collaborative strategy model based on the status information and the edge-end collaborative action information; converting the edge-end collaborative strategy model into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model and train it; based on the trained edge-end collaborative strategy Q learning model, obtaining the target edge-end collaborative action information corresponding to the current status information of the power Internet of Things and the amount of terminal node business data, and controlling the edge device based on the target edge-end collaborative action information. This solution can enable the edge device and the terminal node to collaboratively process the business on the terminal node with high efficiency and success rate, thereby accurately allocating resources in the power Internet of Things and improving the operating efficiency of the power Internet of Things.

Description

Technical Field

[0001] The present disclosure relates to the field of power supply and distribution technology, and in particular to an edge device control method, apparatus, equipment, and medium. Background Art

[0002] The Power Internet of Things is the application of the Internet of Things in smart grids. It can effectively integrate communication infrastructure resources and power system infrastructure resources, improve the level of informatization of the power system, improve the utilization efficiency of the existing infrastructure of the power system, and provide important technical support for the power grid's generation, transmission, transformation, distribution, and consumption.

[0003] In recent years, with the rapid development of the power Internet of Things, the explosive growth in data volume has far exceeded the capacity of network bandwidth. The emergence of a large number of new intelligent applications has put forward higher requirements on system latency performance.

[0004] Edge computing, as a new computing model, enables timely and efficient data processing at the edge of the network near its source. The power Internet of Things (IoT) based on this edge computing model enables end nodes to offload tasks to nearby edge devices, effectively alleviating the network congestion and high latency caused by traditional solutions where end nodes transmit all data to data centers.

[0005] However, the applicant found that when the number of terminal nodes and data scale in the power Internet of Things further increased, the resources in the power Internet of Things often could not be accurately allocated, resulting in problems in the coordination between edge devices and terminal nodes in the power Internet of Things, resulting in reduced operating efficiency of the power Internet of Things. Summary of the Invention

[0006] In order to solve the problems in the related art, the embodiments of the present disclosure provide an edge device control method, apparatus, device and medium.

[0007] In a first aspect, an embodiment of the present disclosure provides an edge device control method, which is applied to an edge device in a power Internet of Things. The method includes:

[0008] Obtain the status information of the power Internet of Things and edge-end collaborative action information; the status information includes the uplink bandwidth of the link, the proportion of bandwidth allocated to the terminal node's business, the bandwidth of the terminal node, the link from the terminal node to the edge device, and the task data volume of the terminal node. The edge-end collaborative action information includes the computing power resources allocated by the edge device to the corresponding terminal node, the computing power resources allocated by the corresponding container in the edge device to the corresponding terminal node, and the computing power resources allocated by the edge device to the corresponding container;

[0009] Based on the status information and edge-end collaborative action information, an edge-end collaborative strategy model is established. The edge-end collaborative strategy model is used to indicate the relationship between the status information, the edge-end collaborative action information, and the probability of successful collaborative processing of services by edge devices and terminal nodes.

[0010] Convert the edge-end collaborative strategy model into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model and train the edge-end collaborative strategy Q learning model;

[0011] Based on the trained edge-end collaborative strategy Q learning model, the target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the business data volume of the terminal node is obtained, and the edge device is controlled based on the target edge-end collaborative action information.

[0012] In one implementation of the present disclosure, an edge-end collaboration strategy model is established based on state information and edge-end collaboration action information, including:

[0013] Based on the status information, a communication model is established. The communication model includes:

[0014]

[0015] Among them, B i is the bandwidth of the i-th terminal node, ω j is the ratio of bandwidth allocated to the service of the i-th terminal node by the j-th link, B j is the uplink bandwidth of the jth link, R i is the set of links from the i-th terminal node to the edge device, T tr (i) is the transmission delay from the i-th terminal node to the edge device, z i is the business data volume of the i-th terminal node;

[0016] At least establish an edge-end collaboration strategy model based on the communication model.

[0017] In one implementation of the present disclosure, an edge-end collaboration strategy model is established based on state information and edge-end collaboration action information, further comprising:

[0018] Based on the edge-end collaborative action information, a computational model is established. The computational model includes:

[0019]

[0020] Among them, c edge,i The computing resources allocated to the i-th terminal node by the edge device, is the ratio of computing resources allocated from the kth container to the ith terminal node in the edge device, E kThe computing resources T allocated to the kth container by the edge device c (i) The time required for the edge device to process the service of the i-th terminal node;

[0021] At least establish an edge-end collaboration strategy model based on the communication model, including:

[0022] An edge-end collaboration strategy model is established based at least on the computing model and the communication model.

[0023] In one implementation of the present disclosure, the edge-end collaboration strategy model includes:

[0024]

[0025] Among them, R S is the probability of successful collaborative processing of services by edge devices and terminal nodes, T is the cycle length of collaborative processing of services by edge devices and terminal nodes, N is the number of terminal nodes, and M is the number of containers in the edge device. is the success rate of the coordination between the i-th terminal node and the edge device to process the business at time t, where Indicates that the i-th terminal node and the edge device successfully coordinate and process the business at time t. Indicates that the i-th terminal node and the edge device fail to coordinate and process the service at time t, T total (i) = T tr (i)+T c (i), T task It is the preset duration threshold.

[0026] In one implementation of the present disclosure, the edge-end collaborative strategy model is converted into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, including:

[0027] Get state space data s (t) And the action space data a (t) ,in in is the business data volume of the Nth terminal node at time t, is the ratio of bandwidth allocated to the Nth terminal node by the link corresponding to time t, is the ratio of computing resources allocated by the corresponding container in the edge device to the Nth terminal node at time t;

[0028] Obtain the state of the distribution network to meet the state space data s( t ), based on the action space data a( t ) The updated value function Q of the control distribution network new (s (t) ,a (t));

[0029] in,

[0030] R (t+1) The number of successful business transactions coordinated between the terminal nodes and edge devices at time t+1;

[0031] Q old (s (t) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t) When, based on the action space data a (t) The value function before the update of the control distribution network;

[0032] Q old (s (t+1) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t+1) When, based on the action space data a (t) The value function before the update of the control distribution network, where is the business data volume of the Nth terminal node at time t+1;

[0033] g is the learning rate, β is the discount rate, and A is the set of action space data;

[0034] A Q-table is constructed based on the updated value function, and an edge-end collaborative strategy Q-learning model is constructed based on the Q-table.

[0035] In a second aspect, an embodiment of the present disclosure provides an edge device control device, which is located on an edge device in the power Internet of Things. The device includes:

[0036] An information acquisition module is configured to acquire status information of the power Internet of Things and edge-end collaborative action information; wherein the status information includes the uplink bandwidth of the link, the proportion of bandwidth allocated by the link to the services of the terminal node, the bandwidth of the terminal node, the link traversed by the terminal node to the edge device, and the task data volume of the terminal node; the edge-end collaborative action information includes the computing power resources allocated by the edge device to the corresponding terminal node, the computing power resources allocated by the corresponding container in the edge device to the corresponding terminal node, and the computing power resources allocated by the edge device to the corresponding container;

[0037] A model building module is configured to establish an edge-end collaboration strategy model based on the state information and the edge-end collaboration action information, where the edge-end collaboration strategy model is used to indicate the relationship between the state information, the edge-end collaboration action information, and the probability of successful collaborative processing of services by the edge device and the terminal node;

[0038] a model training module, configured to convert the edge-end collaborative strategy model into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and train the edge-end collaborative strategy Q learning model;

[0039] The device control module is configured to obtain target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the amount of terminal node business data based on the trained edge-end collaborative strategy Q learning model, and control the edge device based on the target edge-end collaborative action information.

[0040] In one implementation of the present disclosure, the model building module is specifically configured to:

[0041] Based on the status information, a communication model is established. The communication model includes:

[0042]

[0043] Among them, B i is the bandwidth of the i-th terminal node, ω j is the ratio of bandwidth allocated to the service of the i-th terminal node by the j-th link, B j is the uplink bandwidth of the jth link, R i is the set of links from the i-th terminal node to the edge device, T tr (i) is the transmission delay from the i-th terminal node to the edge device, z i is the business data volume of the i-th terminal node;

[0044] At least establish an edge-end collaboration strategy model based on the communication model.

[0045] In one implementation of the present disclosure, the model building module is specifically configured to:

[0046] Based on the edge-end collaborative action information, a computational model is established. The computational model includes:

[0047]

[0048] Among them, c edge,i The computing resources allocated to the i-th terminal node by the edge device, is the ratio of computing resources allocated from the kth container to the ith terminal node in the edge device, E k The computing resources T allocated to the kth container by the edge device c (i) The time required for the edge device to process the service of the i-th terminal node;

[0049] An edge-end collaboration strategy model is established based at least on the computing model and the communication model.

[0050] In one implementation of the present disclosure, the edge-end collaboration strategy model includes:

[0051]

[0052] Among them, R S is the probability of successful collaborative processing of services by edge devices and terminal nodes, T is the cycle length of collaborative processing of services by edge devices and terminal nodes, N is the number of terminal nodes, and M is the number of containers in the edge device. is the success rate of the coordination between the i-th terminal node and the edge device to process the business at time t, where Indicates that the i-th terminal node and the edge device successfully coordinate and process the business at time t. Indicates that the i-th terminal node and the edge device fail to coordinate and process the service at time t, T total (i) = T tr (i)+T c (i), T task It is the preset duration threshold.

[0053] In one implementation of the present disclosure, the model training module is specifically configured to:

[0054] Get state space data s (t) And the action space data a (t) ,in in is the business data volume of the Nth terminal node at time t, is the ratio of bandwidth allocated to the Nth terminal node by the link corresponding to time t, is the ratio of computing resources allocated by the corresponding container in the edge device to the Nth terminal node at time t;

[0055] Obtain the state of the distribution network to satisfy the state space data s (t) When, based on the action space data a (t) The updated value function Q of the control distribution network new (s (t) ,a (t) );

[0056] in,

[0057] R (t+1) The number of successful business transactions coordinated between the terminal nodes and edge devices at time t+1;

[0058] Q old (s (t) ,a (t) ) is the state of the distribution network that satisfies the state space data s(t) When, based on the action space data a (t) The value function before the update of the control distribution network;

[0059] Q old (s (t+1) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t+1) When, based on the action space data a (t) The value function before the update of the control distribution network, where is the business data volume of the Nth terminal node at time t+1;

[0060] g is the learning rate, β is the discount rate, and A is the set of action space data;

[0061] A Q-table is constructed based on the updated value function, and an edge-end collaborative strategy Q-learning model is constructed based on the Q-table.

[0062] In a third aspect, an embodiment of the present disclosure provides an electronic device comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, and wherein the one or more computer instructions are executed by the processor to implement a method as described in any one of the first aspects.

[0063] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the method as described in any one of the first aspects.

[0064] According to the technical solution provided by the embodiments of the present disclosure, the status information and edge-end collaborative action information of the power Internet of Things are obtained by the edge devices in the power Internet of Things; based on the status information and edge-end collaborative action information, an edge-end collaborative strategy model is established, and the edge-end collaborative strategy model is used to indicate the relationship between the status information, the edge-end collaborative action information, and the probability of successful collaborative processing of business by the edge device and the terminal node; the edge-end collaborative strategy model is converted into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and the edge-end collaborative strategy Q learning model is trained; based on the trained edge-end collaborative strategy Q learning model, the target edge-end collaborative action information corresponding to the current status information of the power Internet of Things and the amount of terminal node business data is obtained, and the edge device is controlled based on the target edge-end collaborative action information. In this solution, the edge device can be controlled based on the target edge-end collaborative action information, while fully utilizing the uplink bandwidth of the link, the bandwidth allocated by the link to the terminal node's business, and the bandwidth of the terminal node. This enables the edge device and the terminal node to collaboratively process the business on the terminal node with higher efficiency, and improves the success rate of collaborative processing of business, thereby accurately allocating resources in the power Internet of Things and improving the operating efficiency of the power Internet of Things.

[0065] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Other features, objectives and advantages of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0067] Figure 1 A flow chart of an edge device control method according to an embodiment of the present disclosure is shown.

[0068] Figure 2 A structural block diagram of an edge device control apparatus according to an embodiment of the present disclosure is shown.

[0069] Figure 3 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0070] Figure 4 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0071] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0072] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, actions, components, parts, or combinations thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, actions, components, parts, or combinations thereof exist or are added.

[0073] It should also be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0074] In this disclosure, if it involves operations of obtaining user information or user data or displaying user information or user data to others, such operations are all authorized and confirmed by the user, or actively selected by the user.

[0075] Figure 1 The flowchart of the edge device control method according to an embodiment of the present disclosure is shown, and the method is applied to the edge device in the power Internet of Things. Figure 1 As shown, the edge device control method includes the following steps S101-S104:

[0076] In step S101, the state information of the power Internet of Things and the edge-end collaborative action information are obtained;

[0077] The status information includes the uplink bandwidth of the link, the ratio of the bandwidth allocated to the services of the terminal node, the bandwidth of the terminal node, the link from the terminal node to the edge device, and the task data volume of the terminal node. The edge-end collaborative action information includes the computing power resources allocated by the edge device to the corresponding terminal node, the computing power resources allocated by the corresponding container in the edge device to the corresponding terminal node, and the computing power resources allocated by the edge device to the corresponding container.

[0078] In step S102, an edge-end collaboration strategy model is established based on the state information and edge-end collaboration action information;

[0079] The edge-end collaboration strategy model is used to indicate the relationship between state information, edge-end collaboration action information, and the probability of successful collaborative processing of services by edge devices and terminal nodes.

[0080] In step S103, the edge-end collaborative strategy model is converted into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and the edge-end collaborative strategy Q learning model is trained;

[0081] In step S104, based on the trained edge-end collaborative strategy Q learning model, the target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the terminal node business data volume is obtained, and the edge device is controlled based on the target edge-end collaborative action information.

[0082] In one implementation of the present disclosure, terminal nodes can be understood as sensors-equipped meters, smart switches, and the like in the Power Internet of Things. Terminal nodes can be locally networked via high-speed power line carrier (HPLC) or micropower wireless communications. Edge devices can perform routing calculations and bandwidth allocation for a terminal node network consisting of multiple terminal nodes via southbound communication interfaces.

[0083] According to the technical solution provided by the embodiments of the present disclosure, the status information and edge-end collaborative action information of the power Internet of Things are obtained by the edge devices in the power Internet of Things; based on the status information and edge-end collaborative action information, an edge-end collaborative strategy model is established, and the edge-end collaborative strategy model is used to indicate the relationship between the status information, the edge-end collaborative action information, and the probability of successful collaborative processing of business by the edge device and the terminal node; the edge-end collaborative strategy model is converted into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and the edge-end collaborative strategy Q learning model is trained; based on the trained edge-end collaborative strategy Q learning model, the target edge-end collaborative action information corresponding to the current status information of the power Internet of Things and the amount of terminal node business data is obtained, and the edge device is controlled based on the target edge-end collaborative action information. In this solution, the edge device can be controlled based on the target edge-end collaborative action information, while fully utilizing the uplink bandwidth of the link, the bandwidth allocated by the link to the terminal node's business, and the bandwidth of the terminal node. This enables the edge device and the terminal node to collaboratively process the business on the terminal node with higher efficiency, and improves the success rate of collaborative processing of business, thereby accurately allocating resources in the power Internet of Things and improving the operating efficiency of the power Internet of Things.

[0084] In one implementation of the present disclosure, an edge-end collaboration strategy model is established based on state information and edge-end collaboration action information, including:

[0085] Based on the status information, a communication model is established. The communication model includes:

[0086]

[0087] Among them, B i is the bandwidth of the i-th terminal node, ω j is the ratio of bandwidth allocated to the service of the i-th terminal node by the j-th link, B jis the uplink bandwidth of the jth link, R i is the set of links from the i-th terminal node to the edge device, T tr (i) is the transmission delay from the i-th terminal node to the edge device, z i is the business data volume of the i-th terminal node;

[0088] At least establish an edge-end collaboration strategy model based on the communication model.

[0089] According to the technical solution provided by the embodiments of the present disclosure, by establishing a communication model based on state information and establishing an edge-end collaborative strategy model at least based on the communication model, the efficiency of establishing the edge-end collaborative strategy model can be improved.

[0090] In one implementation of the present disclosure, an edge-end collaboration strategy model is established based on state information and edge-end collaboration action information, further comprising:

[0091] Based on the edge-end collaborative action information, a computational model is established. The computational model includes:

[0092]

[0093] Among them, c edge,i The computing resources allocated to the i-th terminal node by the edge device, is the ratio of computing resources allocated from the kth container to the ith terminal node in the edge device, E k The computing resources T allocated to the kth container by the edge device c (i) The time required for the edge device to process the service of the i-th terminal node;

[0094] At least establish an edge-end collaboration strategy model based on the communication model, including:

[0095] An edge-end collaboration strategy model is established based at least on the computing model and the communication model.

[0096] According to the technical solution provided by the embodiments of the present disclosure, by establishing a computing model based on edge-end collaborative action information, and establishing an edge-end collaborative strategy model based at least on the computing model and the communication model, the reliability of the established edge-end collaborative strategy model can be improved.

[0097] In one implementation of the present disclosure, the edge-end collaboration strategy model includes:

[0098]

[0099] Among them, R Sis the probability of successful collaborative processing of services by edge devices and terminal nodes, T is the cycle length of collaborative processing of services by edge devices and terminal nodes, N is the number of terminal nodes, and M is the number of containers in the edge device. is the success rate of the coordination between the i-th terminal node and the edge device to process the business at time t, where Indicates that the i-th terminal node and the edge device successfully coordinate and process the business at time t. Indicates that the i-th terminal node and the edge device fail to coordinate and process the service at time t, T total (i) = T tr (i)+T c (i).

[0100] In one implementation of the present disclosure, the edge-end collaborative strategy model is converted into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, including:

[0101] Get state space data s (t) And the action space data a (t) ,in in is the business data volume of the Nth terminal node at time t, is the ratio of bandwidth allocated to the Nth terminal node by the link corresponding to time t, is the ratio of computing resources allocated by the corresponding container in the edge device to the Nth terminal node at time t;

[0102] Obtain the state of the distribution network to meet the state space data s( t ), based on the action space data a( t ) The updated value function Q of the control distribution network new (s (t) ,a (t) );

[0103] in,

[0104] R (t+1) The number of successful business transactions coordinated between the terminal nodes and edge devices at time t+1;

[0105] Q old (s (t) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t) When, based on the action space data a (t) The value function before the update of the control distribution network;

[0106] Q old (s(t+1) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t+1) When, based on the action space data a (t) The value function before the update of the control distribution network, where is the business data volume of the Nth terminal node at time t+1;

[0107] g is the learning rate, β is the discount rate, and A is the set of action space data;

[0108] A Q-table is constructed based on the updated value function, and an edge-end collaborative strategy Q-learning model is constructed based on the Q-table.

[0109] In one implementation of the present disclosure, training the edge-end collaborative strategy Q learning model can be achieved through the following steps:

[0110] Initialize the learning rate α, discount rate β, and Q table. For example, the values ​​of the learning rate α, discount rate β, and Q table can all be initialized to 0;

[0111] Get the state space data s( t );

[0112] According to the greedy strategy, obtain the current action space data a( t );

[0113] Based on the current state space data s( t ) and action space data a( t ), calculate R( t+1 );

[0114] based on Calculate the state space data s for the next time slot (t+1) ,in, is the business data volume of the i-th terminal node at time t+1, is the business data volume of the i-th terminal node at time t, is the ratio of computing resources allocated by the K-th container in the edge device to the i-th terminal node at time t;

[0115] Get the state space data s( t+1 );

[0116] based on Update the Q table in the edge-end collaborative strategy Q learning model.

[0117] Figure 2A structural block diagram of an edge device control apparatus according to an embodiment of the present disclosure is shown. The apparatus is located on an edge device in the power Internet of Things, wherein the apparatus can be implemented as part or all of an electronic device through software, hardware, or a combination of both.

[0118] like Figure 2 As shown, the edge device control device 200 includes:

[0119] The information acquisition module 201 is configured to obtain status information of the power Internet of Things and edge-end collaborative action information; wherein the status information includes the uplink bandwidth of the link, the proportion of bandwidth allocated to the services of the terminal node, the bandwidth of the terminal node, the link traversed by the terminal node to the edge device, and the task data volume of the terminal node; the edge-end collaborative action information includes the computing power resources allocated by the edge device to the corresponding terminal node, the computing power resources allocated by the corresponding container in the edge device to the corresponding terminal node, and the computing power resources allocated by the edge device to the corresponding container;

[0120] A model building module 202 is configured to build an edge-end collaboration strategy model based on the state information and the edge-end collaboration action information. The edge-end collaboration strategy model is used to indicate the relationship between the state information, the edge-end collaboration action information, and the probability of successful collaborative processing of services by the edge device and the terminal node;

[0121] A model training module 203 is configured to convert the edge-end collaborative strategy model into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and train the edge-end collaborative strategy Q learning model;

[0122] The device control module 204 is configured to obtain target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the amount of terminal node business data based on the trained edge-end collaborative strategy Q learning model, and control the edge device based on the target edge-end collaborative action information.

[0123] In one implementation of the present disclosure, the model building module 202 is specifically configured to:

[0124] Based on the status information, a communication model is established. The communication model includes:

[0125]

[0126] Among them, B i is the bandwidth of the i-th terminal node, ω j is the ratio of bandwidth allocated to the service of the i-th terminal node by the j-th link, B j is the uplink bandwidth of the jth link, R i is the set of links from the i-th terminal node to the edge device, Ttr (i) is the transmission delay from the i-th terminal node to the edge device, z i is the business data volume of the i-th terminal node;

[0127] At least establish an edge-end collaboration strategy model based on the communication model.

[0128] In one implementation of the present disclosure, the model building module 202 is specifically configured to:

[0129] Based on the edge-end collaborative action information, a computational model is established. The computational model includes:

[0130]

[0131] Among them, c edge,i The computing resources allocated to the i-th terminal node by the edge device, is the ratio of computing resources allocated from the kth container to the ith terminal node in the edge device, E k The computing resources T allocated to the kth container by the edge device c (i) The time required for the edge device to process the service of the i-th terminal node;

[0132] An edge-end collaboration strategy model is established based at least on the computing model and the communication model.

[0133] In one implementation of the present disclosure, the edge-end collaboration strategy model includes:

[0134]

[0135] Among them, R S is the probability of successful collaborative processing of services by edge devices and terminal nodes, T is the cycle length of collaborative processing of services by edge devices and terminal nodes, N is the number of terminal nodes, and M is the number of containers in the edge device. is the success rate of the coordination between the i-th terminal node and the edge device to process the business at time t, where Indicates that the i-th terminal node and the edge device successfully coordinate and process the business at time t. Indicates that the i-th terminal node and the edge device fail to coordinate and process the service at time t, T total (i) = T tr (i)+T c (i), T task It is the preset duration threshold.

[0136] In one implementation of the present disclosure, the model training module 203 is specifically configured to:

[0137] Get state space data s (t)And the action space data a (t) ,in in is the business data volume of the Nth terminal node at time t, is the ratio of bandwidth allocated to the Nth terminal node by the link corresponding to time t, is the ratio of computing resources allocated by the corresponding container in the edge device to the Nth terminal node at time t;

[0138] Obtain the state of the distribution network to meet the state space data s( t ), based on the action space data a( t ) The updated value function Q of the control distribution network new (s (t) ,a (t) );

[0139] in,

[0140] R (t+1) The number of successful business transactions coordinated between the terminal nodes and edge devices at time t+1;

[0141] Q old (s (t) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t) When, based on the action space data a (t) The value function before the update of the control distribution network;

[0142] Q old (s (t+1) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t+1) When, based on the action space data a (t) The value function before the update of the control distribution network, where is the business data volume of the Nth terminal node at time t+1;

[0143] g is the learning rate, β is the discount rate, and A is the set of action space data;

[0144] A Q-table is constructed based on the updated value function, and an edge-end collaborative strategy Q-learning model is constructed based on the Q-table.

[0145] According to the technical solution provided by the embodiments of the present disclosure, the status information and edge-end collaborative action information of the power Internet of Things are obtained by the edge devices in the power Internet of Things; based on the status information and edge-end collaborative action information, an edge-end collaborative strategy model is established, and the edge-end collaborative strategy model is used to indicate the relationship between the status information, the edge-end collaborative action information, and the probability of successful collaborative processing of business by the edge device and the terminal node; the edge-end collaborative strategy model is converted into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and the edge-end collaborative strategy Q learning model is trained; based on the trained edge-end collaborative strategy Q learning model, the target edge-end collaborative action information corresponding to the current status information of the power Internet of Things and the amount of terminal node business data is obtained, and the edge device is controlled based on the target edge-end collaborative action information. In this solution, the edge device can be controlled based on the target edge-end collaborative action information, while fully utilizing the uplink bandwidth of the link, the bandwidth allocated by the link to the terminal node's business, and the bandwidth of the terminal node. This enables the edge device and the terminal node to collaboratively process the business on the terminal node with higher efficiency, and improves the success rate of collaborative processing of business, thereby accurately allocating resources in the power Internet of Things and improving the operating efficiency of the power Internet of Things.

[0146] The present disclosure also discloses an electronic device, Figure 3 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0147] like Figure 3 As shown, the electronic device includes a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to an embodiment of the present disclosure.

[0148] An embodiment of the present disclosure provides an edge device control method, which is applied to edge devices in the power Internet of Things. The method includes:

[0149] Obtain the status information of the power Internet of Things and edge-end collaborative action information; the status information includes the uplink bandwidth of the link, the proportion of bandwidth allocated to the terminal node's business, the bandwidth of the terminal node, the link from the terminal node to the edge device, and the task data volume of the terminal node. The edge-end collaborative action information includes the computing power resources allocated by the edge device to the corresponding terminal node, the computing power resources allocated by the corresponding container in the edge device to the corresponding terminal node, and the computing power resources allocated by the edge device to the corresponding container;

[0150] Based on the status information and edge-end collaborative action information, an edge-end collaborative strategy model is established. The edge-end collaborative strategy model is used to indicate the relationship between the status information, the edge-end collaborative action information, and the probability of successful collaborative processing of services by edge devices and terminal nodes.

[0151] Convert the edge-end collaborative strategy model into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model and train the edge-end collaborative strategy Q learning model;

[0152] Based on the trained edge-end collaborative strategy Q learning model, the target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the business data volume of the terminal node is obtained, and the edge device is controlled based on the target edge-end collaborative action information.

[0153] In one implementation of the present disclosure, an edge-end collaboration strategy model is established based on state information and edge-end collaboration action information, including:

[0154] Based on the status information, a communication model is established. The communication model includes:

[0155]

[0156] Among them, B i is the bandwidth of the i-th terminal node, ω j is the ratio of bandwidth allocated to the service of the i-th terminal node by the j-th link, B j is the uplink bandwidth of the jth link, R i is the set of links from the i-th terminal node to the edge device, T tr (i) is the transmission delay from the i-th terminal node to the edge device, z i is the business data volume of the i-th terminal node;

[0157] At least establish an edge-end collaboration strategy model based on the communication model.

[0158] In one implementation of the present disclosure, an edge-end collaboration strategy model is established based on state information and edge-end collaboration action information, further comprising:

[0159] Based on the edge-end collaborative action information, a computational model is established. The computational model includes:

[0160]

[0161] Among them, c edge,i The computing resources allocated to the i-th terminal node by the edge device, is the ratio of computing resources allocated from the kth container to the ith terminal node in the edge device, E k The computing resources T allocated to the kth container by the edge device c (i) The time required for the edge device to process the service of the i-th terminal node;

[0162] At least establish an edge-end collaboration strategy model based on the communication model, including:

[0163] An edge-end collaboration strategy model is established based at least on the computing model and the communication model.

[0164] In one implementation of the present disclosure, the edge-end collaboration strategy model includes:

[0165]

[0166] Among them, R S is the probability of successful collaborative processing of services by edge devices and terminal nodes, T is the cycle length of collaborative processing of services by edge devices and terminal nodes, N is the number of terminal nodes, and M is the number of containers in the edge device. is the success rate of the coordination between the i-th terminal node and the edge device to process the business at time t, where Indicates that the i-th terminal node and the edge device successfully coordinate and process the business at time t. Indicates that the i-th terminal node and the edge device fail to coordinate and process the service at time t, T total (i) = T tr (i)+T c (i), T task It is the preset duration threshold.

[0167] In one implementation of the present disclosure, the edge-end collaborative strategy model is converted into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, including:

[0168] Get state space data s (t) And the action space data a (t) ,in in is the business data volume of the Nth terminal node at time t, is the ratio of bandwidth allocated to the Nth terminal node by the link corresponding to time t, is the ratio of computing resources allocated by the corresponding container in the edge device to the Nth terminal node at time t;

[0169] Obtain the state of the distribution network to meet the state space data s( t ), based on the action space data a( t ) The updated value function Q of the control distribution network new (s (t) ,a (t) );

[0170] in,

[0171] R (t+1) The number of successful business transactions coordinated between the terminal nodes and edge devices at time t+1;

[0172] Q old (s (t) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t) When, based on the action space data a (t) The value function before the update of the control distribution network;

[0173] Q old (s (t+1) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t+1) When, based on the action space data a (t) The value function before the update of the control distribution network, where is the business data volume of the Nth terminal node at time t+1;

[0174] g is the learning rate, β is the discount rate, and A is the set of action space data;

[0175] A Q-table is constructed based on the updated value function, and an edge-end collaborative strategy Q-learning model is constructed based on the Q-table.

[0176] Figure 4 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown.

[0177] like Figure 4 As shown, the computer system includes a processing unit, which can execute the various methods in the above-mentioned embodiments according to a program stored in a read-only memory (ROM) or a program loaded from a storage portion into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer system are also stored. The processing unit, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0178] The following components are connected to the I / O interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), a speaker, etc.; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a LAN card, a modem, etc. The communication part performs a communication process via a network such as the Internet. The drive is also connected to the I / O interface as needed. Removable media, such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed so that the computer program read therefrom is installed into the storage part as needed. Among them, the processing unit can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0179] In particular, according to embodiments of the present disclosure, the methods described above can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for executing the methods described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication component and / or installed from a removable medium.

[0180] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0181] The units or modules involved in the embodiments described in this disclosure may be implemented by software or programmable hardware. The units or modules described may also be provided in a processor, and the names of these units or modules do not, in certain circumstances, constitute limitations on the units or modules themselves.

[0182] As another aspect, the present disclosure further provides a computer-readable storage medium. This computer-readable storage medium may be included in the electronic device or computer system described in the above embodiments, or may be a standalone computer-readable storage medium not incorporated into the device. The computer-readable storage medium stores one or more programs, which are used by one or more processors to execute the methods described in the present disclosure.

[0183] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention herein is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

Claims

1. A method for controlling an edge device, characterized in that: The method is applied to an edge device in the power Internet of Things, and the method includes: Obtaining status information and edge-end collaborative action information of the power Internet of Things; wherein the status information includes the uplink bandwidth of the link, the proportion of bandwidth allocated to the services of the terminal node, the bandwidth of the terminal node, the link traversed by the terminal node to the edge device, and the service data volume of the terminal node; the edge-end collaborative action information includes the computing power resources allocated by the edge device to the corresponding terminal node, the computing power resources allocated by the corresponding container in the edge device to the corresponding terminal node, and the computing power resources allocated by the edge device to the corresponding container; Based on the state information and the edge-end collaborative action information, establishing an edge-end collaborative strategy model, wherein the edge-end collaborative strategy model is used to indicate the relationship between the state information, the edge-end collaborative action information, and the probability of successful collaborative processing of services by the edge device and the terminal node; Converting the edge-end collaborative strategy model into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and training the edge-end collaborative strategy Q learning model; Based on the trained edge-end collaborative strategy Q learning model, the target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the business data volume of the terminal node is obtained, and the edge device is controlled based on the target edge-end collaborative action information.

2. The edge device control method according to claim 1, characterized in that: The establishing of an edge-end collaborative strategy model based on the state information and the edge-end collaborative action information includes: Based on the state information, a communication model is established, wherein the communication model includes: Among them, B i is the bandwidth of the i-th terminal node, ω j is the ratio of bandwidth allocated to the service of the i-th terminal node by the j-th link, B j is the uplink bandwidth of the jth link, R i is the set of links from the i-th terminal node to the edge device, T tr (i) is the transmission delay from the i-th terminal node to the edge device, z i is the business data volume of the i-th terminal node; The edge-end collaboration strategy model is established based at least on the communication model.

3. The edge device control method according to claim 2, characterized in that: The establishing of an edge-end collaborative strategy model based on the state information and the edge-end collaborative action information further includes: Based on the edge-end collaborative action information, a calculation model is established, wherein the calculation model includes: Among them, c edge,i The computing resources allocated to the i-th terminal node by the edge device, is the ratio of computing resources allocated from the kth container to the ith terminal node in the edge device, E k The computing resources allocated to the kth container by the edge device, T c (i) The time required for the edge device to process the service of the i-th terminal node; The establishing the edge-end collaboration strategy model at least based on the communication model includes: The edge-end collaboration strategy model is established based at least on the computing model and the communication model.

4. The edge device control method according to claim 3, characterized in that: The edge-end collaboration strategy model includes: Among them, R S is the probability of successful collaborative processing of services by edge devices and terminal nodes, T is the cycle length of collaborative processing of services by edge devices and terminal nodes, N is the number of terminal nodes, and M is the number of containers in the edge device. is the success rate of the coordination between the i-th terminal node and the edge device to process the business at time t, where Indicates that the i-th terminal node and the edge device successfully coordinate and process the business at time t. Indicates that the i-th terminal node and the edge device fail to coordinate and process the service at time t, T total (i) = T tr (i)+T c (i), T task It is the preset duration threshold.

5. The edge device control method according to claim 4, characterized in that: Converting the edge-end collaborative strategy model into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model includes: Get state space data s (t) And the action space data a (t) ,in in is the business data volume of the Nth terminal node at time t, is the ratio of bandwidth allocated to the Nth terminal node by the link corresponding to time t, is the ratio of computing resources allocated by the corresponding container in the edge device to the Nth terminal node at time t; Obtain the state of the distribution network to satisfy the state space data s (t) When, based on the action space data a (t) The updated value function Q that controls the distribution network new (s (t) ,a (t) ); in, R (t+1) The number of successful business transactions coordinated between the terminal nodes and edge devices at time t+1; Q old (s (t) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t) When, based on the action space data a (t) a pre-updated value function for controlling the power distribution network; Q old (s (t+1) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t+1) When, based on the action space data a (t) The cost function before the update of the control power distribution network, where is the business data volume of the Nth terminal node at time t+1; g is the learning rate, β is the discount rate, and A is the set of action space data; A Q-table is constructed based on the updated value function, and an edge-end collaborative strategy Q-learning model is constructed based on the Q-table.

6. An edge device control device, characterized in that: The device is located on an edge device in the power Internet of Things, and includes: An information acquisition module is configured to acquire status information of the electric power Internet of Things and edge-end collaborative action information; wherein the status information includes the uplink bandwidth of the link, the proportion of bandwidth allocated by the link to the services of the terminal node, the bandwidth of the terminal node, the link traversed by the terminal node to the edge device, and the service data volume of the terminal node; the edge-end collaborative action information includes the computing power resources allocated by the edge device to the corresponding terminal node, the computing power resources allocated by the corresponding container in the edge device to the corresponding terminal node, and the computing power resources allocated by the edge device to the corresponding container; a model building module configured to establish an edge-end collaboration strategy model based on the state information and the edge-end collaboration action information, wherein the edge-end collaboration strategy model is used to indicate a relationship between the state information, the edge-end collaboration action information, and a probability of successful collaborative processing of a service by the edge device and the terminal node; a model training module, configured to convert the edge-end collaborative strategy model into a Markov decision process problem to construct an edge-end collaborative strategy Q learning model, and train the edge-end collaborative strategy Q learning model; The device control module is configured to obtain target edge-end collaborative action information corresponding to the current state information of the power Internet of Things and the business data volume of the terminal node based on the trained edge-end collaborative strategy Q learning model, and control the edge device based on the target edge-end collaborative action information.

7. The edge device control device according to claim 6, characterized in that: The model building module is specifically configured as follows: Based on the state information, a communication model is established, wherein the communication model includes: Among them, B i is the bandwidth of the i-th terminal node, ω j is the ratio of bandwidth allocated to the service of the i-th terminal node by the j-th link, B j is the uplink bandwidth of the jth link, R i is the set of links from the i-th terminal node to the edge device, T tr (i) is the transmission delay from the i-th terminal node to the edge device, z i is the business data volume of the i-th terminal node; The edge-end collaboration strategy model is established based at least on the communication model.

8. The edge device control device according to claim 7, characterized in that: The model building module is specifically configured as follows: Based on the edge-end collaborative action information, a calculation model is established, wherein the calculation model includes: Among them, c edge,i The computing resources allocated to the i-th terminal node by the edge device, is the ratio of computing resources allocated from the kth container to the ith terminal node in the edge device, E k The computing resources allocated to the kth container by the edge device, T c (i) The time required for the edge device to process the service of the i-th terminal node; The edge-end collaboration strategy model is established based at least on the computing model and the communication model.

9. The edge device control device according to claim 8, characterized in that: The edge-end collaboration strategy model includes: Among them, R S is the probability of successful collaborative processing of services by edge devices and terminal nodes, T is the cycle length of collaborative processing of services by edge devices and terminal nodes, N is the number of terminal nodes, and M is the number of containers in the edge device. is the success rate of the coordination between the i-th terminal node and the edge device to process the business at time t, where Indicates that the i-th terminal node and the edge device successfully coordinate and process the business at time t. Indicates that the i-th terminal node and the edge device fail to coordinate and process the service at time t, T total (i) = T tr (i)+T c (i), T task It is the preset duration threshold.

10. The edge device control device according to claim 9, characterized in that: The model training module is specifically configured as follows: Get state space data s (t) And the action space data a (t) ,in in is the business data volume of the Nth terminal node at time t, is the ratio of bandwidth allocated to the Nth terminal node by the link corresponding to time t, is the ratio of computing resources allocated by the corresponding container in the edge device to the Nth terminal node at time t; Obtain the state of the distribution network to satisfy the state space data s (t) When, based on the action space data a (t) The updated value function Q that controls the distribution network new (s (t) ,a (t) ); in, R (t+1) The number of successful business transactions coordinated between the terminal nodes and edge devices at time t+1; Q old (s (t) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t) When, based on the action space data a (t) a pre-updated value function for controlling the power distribution network; Q old (s (t+1) ,a (t) ) is the state of the distribution network that satisfies the state space data s (t+1) When, based on the action space data a (t) The cost function before the update of the control power distribution network, where is the business data volume of the Nth terminal node at time t+1; g is the learning rate, β is the discount rate, and A is the set of action space data; A Q-table is constructed based on the updated value function, and an edge-end collaborative strategy Q-learning model is constructed based on the Q-table.

11. An electronic device, characterized in that: The method comprises a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps according to any one of claims 1 to 5.

12. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the method steps according to any one of claims 1 to 5 are implemented.

Images