Resource optimization method and system based on cloud side-end cooperation of power Internet of Things

Abstract

The invention relates to the technical field of internet-of-things resource allocation, in particular to a resource optimization method and system based on electric power internet-of-things cloud side-end collaboration, and the method comprises the steps: constructing an electric power internet-of-things collaborative computing architecture; constructing a cloud edge-end cooperative computing model; allocating computing resources of the terminal, the edge server and the cloud server; and dynamically accessing the terminal to the server by adopting a task unloading optimization algorithm based on action-critic. According to the method, the power Internet of Things data is dynamically scheduled through the terminal-edge-cloud three-level task queue, the utilization rate of power resources is optimized, and the operation and maintenance cost of the power Internet of Things is reduced.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) resource allocation technology, and in particular to a resource optimization method, system, and computer-readable storage medium based on cloud-edge-device collaboration in the power IoT. Background Technology

[0002] The Internet of Things (IoT) refers to a network that connects various objects to the Internet through information sensing devices such as radio frequency identification (RFID), infrared sensors, global positioning systems (GPS), and laser scanners, according to agreed protocols, to exchange information and communicate, thereby achieving intelligent identification, positioning, tracking, monitoring, and management.

[0003] The diverse types and large quantities of data involved in the power Internet of Things (IoT), along with significantly varying service demands, pose a severe challenge to the data processing capabilities of energy storage power stations with limited local resources. In recent years, the widespread adoption of cloud computing and edge computing technologies has effectively improved the data processing capabilities and system security of the power IoT by deploying service platforms in the cloud and at the edge. However, the massive number of terminals connected to the IoT leads to insufficient computing power, resulting in slow system response times, low power resource utilization, and high operation and maintenance costs.

[0004] Therefore, optimizing the utilization rate of power resources and reducing the operation and maintenance costs of the power Internet of Things are urgent technical problems to be solved in this field. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problem that the massive number of terminals connected to the Internet of Things leads to insufficient computing power of the Internet of Things, resulting in low utilization of power resources and high operation and maintenance costs of the power Internet of Things.

[0006] To address the aforementioned technical problems, this invention provides a resource optimization method based on cloud-edge-device collaboration in the power Internet of Things, comprising:

[0007] Constructing a cloud-edge-device collaborative computing model includes:

[0008] Establish terminal task queue models, edge server task queue models, and cloud server task queue models to calculate the task queue sizes of terminals, edge servers, and cloud servers;

[0009] Calculate the average arrival rate of the terminal task queue, the average arrival rate of the edge server task queue, and the average arrival rate of the cloud server task queue;

[0010] Establish a queuing delay model for the terminal to calculate the actual queuing delay of the terminal;

[0011] An actor-critic-based task offloading optimization algorithm is adopted to minimize the actual queuing latency of terminals, and the dynamic access strategy for each terminal in each time slot is solved, including:

[0012] The state space is defined by the size of the terminal task queue, the size of the edge server task queue, the size of the cloud server task queue, the size of the terminal task data, the average arrival rate of the terminal task queue, the average arrival rate of the edge server task queue, the average arrival rate of the cloud server task queue, and the service availability of the edge server to the terminal.

[0013] The action space is defined by whether to offload the task data of each terminal to the cloud server and whether to offload the task data of each terminal to the edge server.

[0014] The objective function is constructed to minimize the actual queuing delay of the terminal, and the dynamic allocation strategy for each terminal in each time slot is solved.

[0015] Preferably, the terminal task queue model is represented as follows:

[0016] Q i (t+1)=max[Q i (t)-U i (t),0]+A i (t)

[0017] Among them, A i (t) represents the terminal u in time slot t. i The amount of task data generated, U i (t) represents the terminal u in time slot t. i The amount of task data offloaded to edge servers and cloud servers; Q i (t) and Q i (t+1) represent terminals u in time slots t and t+1, respectively. i The size of the task queue.

[0018] Preferably, the edge server task queue model and the cloud server task queue model are represented as follows:

[0019]

[0020] in, and These represent edge servers s in time slots t and t+1, respectively. j With terminal u i The size of the task queue between them Represents time slot t and edge server s j Can be a terminal u i The amount of data processed; Indicates time slot t terminal ui Offload task data to edge servers. j , Indicates time slot t terminal u i Do not offload task data to edge servers j ; Indicates time slot t terminal u i Unload the task data to the cloud server. Indicates time slot t terminal u i Do not offload task data to the cloud server; U i (t) represents the terminal u in time slot t. i The amount of task data offloaded to edge servers and cloud servers; and These represent the cloud server and terminal u in time slots t and t+1, respectively. i The size of the task queue between them This indicates that the time slot t cloud server can provide terminal u i The amount of data processed.

[0021] Preferably, the average arrival rate of the terminal task queue is calculated using the following formula:

[0022]

[0023] in, Indicates time slot t terminal u i The average arrival rate of the task queue, where t represents the time slot, and A i (m) represents the time slot m terminal u i The amount of task data generated.

[0024] Preferably, the average arrival rate of the edge server task queue and the average arrival rate of the cloud server task queue are calculated using the following formula:

[0025]

[0026] in, Represents time slot t and edge server s j With terminal u i The average arrival rate of the task queues between them, where t represents the time slot. Indicates time slot n terminal u i Offload task data to edge servers. j , Indicates time slot n terminal u i Do not offload task data to edge servers j ; Indicates time slot n terminal u i Unload the task data to the cloud server. Indicates time slot n terminal ui Do not offload task data to the cloud server; U i (n) represents time slot n terminal u i The amount of task data offloaded to edge servers and cloud servers; Indicates time slot t for cloud server and terminal u i The average arrival rate of the task queues between them.

[0027] Preferably, a queuing delay model for the terminal is established, with the following formula:

[0028]

[0029] Where, τ i (t) represents the terminal u in time slot t. i The actual queuing delay;

[0030] Indicates time slot t terminal u i The queuing delay during local task unloading is calculated using the following formula:

[0031]

[0032] Among them, Q i (t) represents the terminal u in time slot t. i Task queue size, Indicates time slot t terminal u i Average arrival rate of the task queue;

[0033] Indicates time slot t terminal u i The maximum queuing latency for edge computing and cloud computing is given by the formula:

[0034]

[0035] Indicates time slot t terminal u i With edge servers j The queuing delay for edge computing between them is given by the following formula:

[0036]

[0037] in, Represents time slot t and edge server s j With terminal u i The size of the task queue between them Represents time slot t and edge server s j With terminal u i The average arrival rate of the task queues between them; j = 1, 2, ..., J; J represents the total number of edge servers;

[0038] Indicates time slot t terminal u i The queuing latency for cloud computing between the cloud server and the cloud server is calculated using the following formula:

[0039]

[0040] in, Indicates time slot t for cloud server and terminal u i The size of the task queue between them Indicates time slot t for cloud server and terminal u i The average arrival rate of the task queues between them; Indicates time slot n terminal u i Unload the task data to the cloud server. Indicates time slot t terminal u i Do not unload task data to the cloud server; This represents the additional forwarding latency from the edge server to the cloud server.

[0041] Preferably, the objective function is constructed to minimize the actual queuing delay of the terminal, as shown in the following formula:

[0042]

[0043] Among them, Ψ i (t) represents the objective function. This represents the action space; J represents the total number of edge servers, and t represents the time slot; And represents time slot t edge server s j With terminal u i The size of the task queue between; Indicates time slot n terminal u i Offload task data to edge servers. j , Indicates time slot t terminal u i Do not offload task data to edge servers j ; Indicates time slot n terminal u i Unload the task data to the cloud server. Indicates time slot t terminal u i Do not offload task data to the cloud server; U i (t) represents the terminal u in time slot t. i The amount of task data offloaded to edge servers and cloud servers; Indicates time slot t for cloud server and terminal u i Task queue size between; Q i (t) represents the terminal u in time slot t. i Task queue size; Indicates time slot t+1 terminal u i Average arrival rate of the task queue; V is a non-negative weight; Represents edge server s in time slot t+1 j With terminal u i The average arrival rate of the task queues between them; Indicates the connection between the cloud server and terminal u in time slot t+1. i The average arrival rate of the task queues between them; C1 and C2 are both constraints; Represents a set of terminals. Represents a set of edge servers. Represents a set of time slots.

[0044] Preferably, an actor-critic-based task offloading optimization algorithm is used to solve the dynamic access strategy for each terminal in each time slot, with the goal of minimizing the actual queuing latency of the terminal. The steps include:

[0045] Initialize the global model on the cloud server;

[0046] In each time slot, each terminal downloads the global model from the cloud server as its local model;

[0047] Each terminal selects an action space based on its local model and the state space of the current time slot; each terminal executes a task offloading decision based on the action space, calculates its corresponding cost function value, and obtains the state space of the next time slot; each terminal calculates the timing difference error based on its corresponding cost function value, the state space of the current time slot, and the state space of the next time slot, and performs a local update on the local model of that terminal.

[0048] All locally updated models from all terminals are uploaded to the cloud server. Abnormal local models are removed, and a federated set is constructed. The cloud server performs federated averaging based on the federated set and updates the global model.

[0049] This invention also provides a resource optimization system based on cloud-edge-device collaboration in the power Internet of Things, comprising:

[0050] The computational model building module is used to build cloud-edge-device collaborative computing models, including:

[0051] Establish terminal task queue models, edge server task queue models, and cloud server task queue models to calculate the task queue sizes of terminals, edge servers, and cloud servers; calculate the average arrival rates of terminal task queues, edge server task queues, and cloud server task queues; and establish a terminal queuing latency model to calculate the actual queuing latency of terminals.

[0052] The terminal dynamic access module is used to solve the dynamic access strategy for each terminal in each time slot by employing an actor-critic-based task offloading optimization algorithm, with the goal of minimizing the actual queuing latency of the terminal. This includes:

[0053] The state space definition unit is used to define the terminal task queue size, edge server task queue size, cloud server task queue size, terminal task data volume, average arrival rate of terminal task queue, average arrival rate of edge server task queue, average arrival rate of cloud server task queue, and service availability of edge server to terminal as the state space.

[0054] The action space definition unit is used to define whether to offload the task data of each terminal to the cloud server and whether to offload the task data of each terminal to the edge server as the action space.

[0055] The solver unit is used to construct the objective function by minimizing the actual queuing delay of the terminal and solve the dynamic allocation strategy for each terminal in each time slot.

[0056] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described resource optimization method based on cloud-edge-device collaboration in the power Internet of Things.

[0057] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0058] This invention discloses a resource optimization method based on cloud-edge-device collaboration in the power Internet of Things (IoT). First, a cloud-edge-device collaborative computing model is constructed to simulate the continuous generation of task data by power IoT terminals distributed across different business scenarios. In addition to local computation, each terminal can offload its tasks to edge servers and cloud servers for collaborative processing. The actual queuing latency of each time slot terminal can be calculated using a queuing latency model within the cloud-edge-device collaborative computing model. Furthermore, with minimizing terminal queuing latency as the optimization objective, an actor-critic-based task offloading optimization algorithm is employed to construct a semi-distributed learning-based dynamic terminal access strategy, determining the task offloading direction for each terminal and improving network resource utilization. This invention optimizes power resource utilization and reduces the operation and maintenance costs of the power IoT by dynamically scheduling power IoT data through a three-level task queue system (terminal-edge-cloud). Attached Figure Description

[0059] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0060] Figure 1This is a flowchart of a resource optimization method based on cloud-edge-device collaboration in the power Internet of Things according to the present invention;

[0061] Figure 2 This is a schematic diagram of the collaborative computing architecture for the power Internet of Things;

[0062] Figure 3 This is a schematic diagram of a virtualized network architecture for the power Internet of Things;

[0063] Figure 4 This is a schematic diagram of the feedback iteration mechanism for computing resource allocation;

[0064] Figure 5 This is a structural diagram of the task unloading optimization algorithm. Detailed Implementation

[0065] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0066] Reference Figure 1 As shown, this invention provides a resource optimization method based on cloud-edge-device collaboration in the power Internet of Things, including:

[0067] S1. Construct a collaborative computing architecture for the power Internet of Things (IoT) with a structure consisting of a central cloud server, edge servers, power IoT terminals, and a power IoT network.

[0068] S2. Construct a cloud-edge-device collaborative computing model based on the aforementioned power Internet of Things collaborative computing architecture;

[0069] S3. Allocate computing resources for terminals, edge servers, and cloud servers;

[0070] S4. Employ an actor-critic-based task offloading optimization algorithm to dynamically connect the terminal to the server.

[0071] Specifically, the power Internet of Things collaborative computing architecture built in S1 is a three-layer collaborative computing architecture (refer to...). Figure 2 As shown, the system includes a cloud server, multiple edge servers, and multiple power IoT terminals. This invention utilizes a three-layer collaborative computing architecture for power IoT and a cloud-edge-device collaborative computing model for computation. Power IoT terminals distributed across different business scenarios continuously generate task data. In addition to local computation, the terminals can offload this data to edge servers and cloud servers for collaborative processing, achieving high efficiency in data processing.

[0072] Specifically, the power Internet of Things network includes a sensing layer, a network layer, a platform layer, an application layer, and security protection, which are described in detail below:

[0073] The perception layer should have functions such as data acquisition, local communication, aggregation and forwarding, centralized verification, edge computing, and data storage, including but not limited to acquisition terminals, smart terminals, aggregation nodes, local communication access, and IoT gateways;

[0074] The network layer should have functions such as interconnection and data communication between devices, including private power networks and public networks;

[0075] The platform layer should have functions such as connection management, device management, message processing, application management, operation monitoring, remote maintenance management, edge computing management, and capability openness, including infrastructure, IoT management, data services, and application services;

[0076] The application layer should have functions such as data services and business services, including traditional power Internet of Things applications and cyber-physical Internet of Things applications.

[0077] The security protection should include functions such as terminal access security protection, network communication security protection, and App application security protection, and should extend through the perception layer, network layer, platform layer, and application layer.

[0078] The collaborative computing architecture for the power Internet of Things constructed in this invention takes into account standardization, flexibility and security, and improves efficiency through technologies such as containerized gateways and dual-mode communication, providing reliable support for the digital upgrade of the power system.

[0079] In S2, a cloud-edge-device collaborative computing model is constructed based on the aforementioned power Internet of Things collaborative computing architecture, specifically including:

[0080] S21: This embodiment introduces the concept of time slots to describe the cloud-edge-device collaborative computing model, that is, the overall time is divided into time slots, using sets. This indicates that I power Internet of Things (IoT) terminals are defined as a set. J edge servers are defined as a set.

[0081] S22: The terminal task queue model is represented as follows:

[0082] Q i (t+1)=max[Q i (t)-U i (t),0]+A i (t)

[0083] Among them, A i (t) represents the terminal u in time slot t. i The amount of task data generated, U i (t) represents the terminal u in time slot t. i The amount of task data offloaded to edge servers and cloud servers; Qi (t) and Q i (t+1) represent terminals u in time slots t and t+1, respectively. i The size of the task queue.

[0084] Similarly, the edge server task queue model and the cloud server task queue model are represented as follows:

[0085]

[0086] in, and These represent edge servers s in time slots t and t+1, respectively. j With terminal u i The size of the task queue between them Represents time slot t and edge server s j Can be a terminal u i The amount of data processed; Indicates time slot t terminal u i Offload task data to edge servers. j , Indicates time slot t terminal u i Do not offload task data to edge servers j ; Indicates time slot t terminal u i Unload the task data to the cloud server. Indicates time slot t terminal u i Do not offload task data to the cloud server; U i (t) represents the terminal u in time slot t. i The amount of task data offloaded to edge servers and cloud servers; and These represent the cloud server and terminal u in time slots t and t+1, respectively. i The size of the task queue between them This indicates that the time slot t cloud server can provide terminal u i The amount of data processed.

[0087] S23: Calculate the average arrival rate of the terminal task queue, using the following formula:

[0088]

[0089] in, Indicates time slot t terminal u i The average arrival rate of the task queue, where t represents the time slot, and A i (m) represents the time slot m terminal u i The amount of task data generated.

[0090] The formulas for calculating the average arrival rate of the edge server task queue and the average arrival rate of the cloud server task queue are as follows:

[0091]

[0092] in, Represents time slot t and edge server s j With terminal u i The average arrival rate of the task queues between them, where t represents the time slot. Indicates time slot n terminal u i Offload task data to edge servers. j , Indicates time slot n terminal u i Do not offload task data to edge servers j ; Indicates time slot n terminal u i Unload the task data to the cloud server. Indicates time slot n terminal u i Do not offload task data to the cloud server; U i (n) represents time slot n terminal u i The amount of task data offloaded to edge servers and cloud servers; Indicates time slot t for cloud server and terminal u i The average arrival rate of the task queues between them.

[0093] S24: Establish a queuing delay model for the terminal to calculate the actual queuing delay of the terminal.

[0094] Since queuing latency has a significant impact, the queuing latency during task unloading and computation processing should also be considered. According to Little's Law, queuing latency is directly proportional to the average queue length and inversely proportional to the average data arrival rate. The queuing latency of a terminal during local task unloading can be expressed as:

[0095]

[0096] in, Indicates time slot t terminal u i Queuing latency during local task unloading, Q i (t) represents the terminal u in time slot t. i Task queue size, Indicates time slot t terminal u i Average arrival rate of the task queue.

[0097] Data queue processing on cloud servers introduces additional forwarding latency from the edge calculator to the cloud. The value depends on the type and transmission capacity of the link between the cloud and the edge. (Terminal u)i The queuing latency of edge computing and cloud computing are respectively represented as:

[0098]

[0099] in, Indicates time slot t terminal u i With edge servers j Queuing latency of edge computing between Represents time slot t and edge server s j With terminal u i The size of the task queue between them Represents time slot t and edge server s j With terminal u i The average arrival rate of the task queues between them; Indicates time slot t terminal u i Queuing latency for cloud computing between cloud servers; Indicates time slot t for cloud server and terminal u i The size of the task queue between them Indicates time slot t for cloud server and terminal u i The average arrival rate of the task queues between them; This represents the additional forwarding latency from the edge server to the cloud server.

[0100] The queuing latency for task processing depends on the maximum queuing latency between all edge servers and cloud servers, and can be expressed as:

[0101]

[0102] in, Indicates time slot t terminal u i The maximum queuing latency for edge computing and cloud computing.

[0103] Therefore, the queuing delay model for the terminal is established, and the formula is:

[0104]

[0105] Where, τ i (t) represents the terminal u in time slot t. i The actual queuing delay.

[0106] Preferably, due to the high complexity of cloud, edge, and terminal collaborative networks, this embodiment proposes a method for deploying and configuring virtual network communication and computing resources based on QoS classification. By dividing virtual networks of the same QoS level, the network scale is reduced, and the complexity of resource collaborative deployment and configuration is reduced while ensuring queuing latency for different services.

[0107] The virtualized network architecture for the power Internet of Things constructed in this embodiment refers to... Figure 3 As shown, the hypervisor supports the virtualization of physical networks, abstracting the physical network and dividing it into multiple virtual networks according to service type and QoS requirements, while the virtual network controller is used to realize the dynamic allocation of communication and computing resources.

[0108] This embodiment dynamically adjusts the deployment of communication and computing resource blocks between different virtual networks by employing SDN-controlled feedback and control access mechanisms. Since the number of resource blocks in each time slot is uncertain, and the communication and computing queuing latency of the terminal is not constant, the difference between the actual communication and computing queuing latency up to the current time slot and the expected communication and computing queuing latency can be used as the basis for dynamic resource deployment.

[0109] Reference Figure 4 As shown, in S3, computing resources are allocated to terminals, edge servers, and cloud servers, including:

[0110] An SDN controller is used to allocate computing resources among terminals, edge servers, and cloud servers;

[0111] This allows cloud servers, edge servers, and terminals to randomly access and transmit data.

[0112] The difference between the actual queuing latency and the expected queuing latency of the computing terminal is fed back to the SDN controller, which then redistributes computing resources among the terminal, edge server, and cloud server.

[0113] Existing power IoT systems cannot meet the stringent security and latency requirements of power systems during computational offloading. On the one hand, offloading tasks and computation results are easily tampered with and reversed, thus jeopardizing power system security. On the other hand, limited communication infrastructure and strict resource deployment cannot provide flexible on-demand services. This invention proposes a joint optimization strategy for dynamic terminal access and network resource allocation based on semi-distributed learning. With minimizing network queuing latency as the optimization objective, the dynamic access strategy for terminals is jointly optimized.

[0114] Considering the advantages and disadvantages of distributed and centralized learning, this invention employs an actor-critic-based task offloading optimization algorithm, specifically a Federated deep actor-critic-based electromagnetic interference-awaitable algorithm (FDAC-EMI) based on semi-distributed learning, to dynamically connect the terminal to the server. Its schematic diagram is shown below. Figure 5 As shown.

[0115] Specifically, in S4, the state space, action space, and objective function of the task unloading optimization algorithm are defined, including:

[0116] The state space is defined by the size of the terminal task queue, the size of the edge server task queue, the size of the cloud server task queue, the size of the terminal task data, the average arrival rate of the terminal task queue, the average arrival rate of the edge server task queue, the average arrival rate of the cloud server task queue, and the service availability of the edge server to the terminal. Among them, a i,j (t)∈{0,1} represents the edge server s j For terminal u i Service availability binary indicator variable, a i,j (t) = 1 indicates that the edge server s in time slot t is... j For terminal u i Available, a i,j (t) = 0 indicates that the edge server s in time slot t is... j For terminal u i Unavailable;

[0117] The action space is defined by whether to offload the task data of each terminal to the cloud server and whether to offload the task data of each terminal to the edge server.

[0118] The objective function is constructed to minimize the actual queuing delay of the terminals, and the dynamic allocation strategy for each terminal in each time slot is solved. The expression for the objective function is:

[0119]

[0120]

[0121] Among them, Ψ i (t) represents the objective function, x i (t) represents the action space; J represents the total number of edge servers, and t represents the time slot; And represents time slot t edge server s j With terminal u i The size of the task queue between; Indicates time slot n terminal u i Offload task data to edge servers. j , Indicates time slot t terminal u i Do not offload task data to edge servers j ; Indicates time slot n terminal u i Unload the task data to the cloud server. Indicates time slot t terminal ui Do not offload task data to the cloud server; U i (t) represents the terminal u in time slot t. i The amount of task data offloaded to edge servers and cloud servers; Indicates time slot t for cloud server and terminal u i Task queue size between; Q i (t) represents the terminal u in time slot t. i Task queue size; Indicates time slot t+1 terminal u i Average arrival rate of the task queue; V is a non-negative weight; Represents edge server s in time slot t+1 j With terminal u i The average arrival rate of the task queues between them; Indicates the connection between the cloud server and terminal u in time slot t+1. i The average arrival rate of the task queues between them; C1 and C2 are both constraints, indicating that each device can only choose one architecture and one computing mode in each time slot; Represents a set of terminals. Represents a set of edge servers. Represents a set of time slots.

[0122] In the task unloading optimization algorithm, the cost function is defined as Ψ. i (x i (t)), that is, using x i (t) is the objective function value Ψ when used as a dynamic allocation strategy. i (t); The algorithm utilizes local data from distributed power IoT devices for local model updates, while the cloud server performs periodic global model updates using federated average local models. Edge servers handle model uploads and downloads between power IoT devices and the cloud server. The algorithm comprises one global model and I local models. Each model includes an actor network and a critic network. The actor network takes actions and optimizes the policy, while the critic network critiques the specific situations and guides policy optimization.

[0123] The global model includes the global actor network model θ. G (t) and the global commentator network model ω G (t). The local model includes the local actor model θ. i (t) and the local commentator model ω i (t).

[0124] Specifically, an actor-critic-based task offloading optimization algorithm is adopted to minimize the actual queuing latency of terminals and solve the dynamic access strategy for each terminal in each time slot. The specific steps include:

[0125] S41: Initialization.

[0126] Initialize the global model on the cloud server; and initialize the cost function value to 0.

[0127] S42: Model Download.

[0128] Each terminal downloads the global model from the cloud server as its local model, i.e., sets θ. i (t)=θ G (t), ω i (t)=ω G (t).

[0129] S43: Action execution.

[0130] In each time slot, each terminal u i With the local model and the observation space S of the current time slot i (t), based on policy π(S) i (t)|θ i (t)) Select action space x i (t); each terminal according to u i Action space x i (t) Execute the task unloading decision and calculate its corresponding cost function value Ψ. i (x i (t)), and obtain the observation space S of the next time slot. i (t+1);

[0131] S44: Local model partial update.

[0132] In each time slot, each terminal calculates the time difference (TD) error based on its corresponding cost function value, the observation space of the current time slot, and the observation space of the next time slot:

[0133] κ i (t)=Ψ i (x i (t))+γV(S i (t+1),ω i (t))-V(S i (t),ω i (t))

[0134] Where γ∈{0,1} is the discount factor, and V(·) represents the valuation function.

[0135] κ i The larger the absolute value of (t), the greater the estimation bias of the local model.

[0136] Each terminal's local model is locally updated, as expressed by the formula:

[0137]

[0138] Where, θ i (t) and ω i (t) represent the local actor model and the local critic model for the current time slot, respectively, θ i (t+1) and ω i (t+1) represents the local actor model and the local critic model after local updates, respectively; ξ and ξ′ represent the learning rates of the local actor model and the local critic model, respectively.

[0139] S45: Upload local model.

[0140] Upload the local models of all terminals after partial updates to the cloud server.

[0141] S46: The cloud server builds a federated set after removing the abnormal local model.

[0142] Considering the adverse effects of sudden strong electromagnetic interference on the local model, a federated set of local models without anomalies needs to be determined to ensure the convergence of the global model. This involves defining whether or not there is a terminal u. i When the corresponding local model is involved, the global network model is updated as follows:

[0143]

[0144] in, and They represent terminal u respectively i and terminal u m The size of the global network training batch for the corresponding local model.

[0145] The federation set can be determined as follows:

[0146]

[0147] in, S represents a federation. test This represents the state space used to determine the federated set, and 1 represents the time length of each time slot.

[0148] S47: Federal Average.

[0149] Cloud servers based on federated sets Perform a federated average and update the global model using the following formula:

[0150]

[0151] Where, θ G (t+1) and ω G(t+1) represent the globally updated global actor model and global critic model, respectively; and These represent the local actor model and the local critic model uploaded to the cloud server, respectively. This indicates the size of the global network training batch for the local commentator model.

[0152] In summary, the resource optimization method based on cloud-edge-device collaboration in the power Internet of Things (IoT) of this invention first constructs a cloud-edge-device collaborative computing model to simulate the continuous generation of task data by power IoT terminals distributed across different business scenarios. In addition to local computation, each terminal can offload its tasks to edge servers and cloud servers for collaborative processing. The actual queuing latency of each time slot terminal can be calculated using the terminal queuing latency model within the cloud-edge-device collaborative computing model. Furthermore, with minimizing terminal queuing latency as the optimization objective, an actor-critic-based task offloading optimization algorithm is adopted to construct a semi-distributed learning-based dynamic terminal access strategy, determining the task offloading direction for each terminal and improving network resource utilization. This invention optimizes power resource utilization and reduces the operation and maintenance costs of the power IoT by dynamically scheduling power IoT data through a three-level task queue system (terminal-edge-cloud).

[0153] Based on the above-mentioned resource optimization method based on cloud-edge-device collaboration of the power Internet of Things, the present invention also provides a resource optimization system based on cloud-edge-device collaboration of the power Internet of Things, comprising:

[0154] The computational model building module is used to build cloud-edge-device collaborative computing models, including:

[0155] Establish terminal task queue models, edge server task queue models, and cloud server task queue models to calculate the task queue sizes of terminals, edge servers, and cloud servers; calculate the average arrival rates of terminal task queues, edge server task queues, and cloud server task queues; and establish a terminal queuing latency model to calculate the actual queuing latency of terminals.

[0156] The terminal dynamic access module is used to solve the dynamic access strategy for each terminal in each time slot by employing an actor-critic-based task offloading optimization algorithm, with the goal of minimizing the actual queuing latency of the terminal. This includes:

[0157] The state space definition unit is used to define the terminal task queue size, edge server task queue size, cloud server task queue size, terminal task data volume, average arrival rate of terminal task queue, average arrival rate of edge server task queue, average arrival rate of cloud server task queue, and service availability of edge server to terminal as the state space.

[0158] The action space definition unit is used to define whether to offload the task data of each terminal to the cloud server and whether to offload the task data of each terminal to the edge server as the action space.

[0159] The solver unit is used to construct the objective function by minimizing the actual queuing delay of the terminal and solve the dynamic allocation strategy for each terminal in each time slot.

[0160] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described resource optimization method based on cloud-edge-device collaboration in the power Internet of Things.

[0161] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0163] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0164] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0165] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A resource optimization method based on cloud-edge-device collaboration in the power Internet of Things, characterized in that, include: Constructing a cloud-edge-device collaborative computing model includes: Establish terminal task queue models, edge server task queue models, and cloud server task queue models to calculate the task queue sizes of terminals, edge servers, and cloud servers; Calculate the average arrival rate of the terminal task queue, the average arrival rate of the edge server task queue, and the average arrival rate of the cloud server task queue; Establish a queuing delay model for the terminal to calculate the actual queuing delay of the terminal; An actor-critic-based task offloading optimization algorithm is adopted to minimize the actual queuing latency of terminals, and the dynamic access strategy for each terminal in each time slot is solved, including: The state space is defined by the size of the terminal task queue, the size of the edge server task queue, the size of the cloud server task queue, the size of the terminal task data, the average arrival rate of the terminal task queue, the average arrival rate of the edge server task queue, the average arrival rate of the cloud server task queue, and the service availability of the edge server to the terminal. The action space is defined by whether to offload the task data of each terminal to the cloud server and whether to offload the task data of each terminal to the edge server. The objective function is constructed to minimize the actual queuing delay of the terminal, and the dynamic allocation strategy for each terminal in each time slot is solved.

2. The resource optimization method based on cloud-edge-device collaboration in the power Internet of Things according to claim 1, characterized in that, The terminal task queue model is represented as follows: Q i (t+1)=max[Q i (t)-U i (t),0]+A i (t) Among them, A i (t) represents the terminal u in time slot t. i The amount of task data generated, U i (t) represents the terminal u in time slot t. i The amount of task data offloaded to edge servers and cloud servers; Q i (t) and Q i (t+1) represent terminals u in time slots t and t+1, respectively. i The size of the task queue.

3. The resource optimization method based on cloud-edge-device collaboration in the power Internet of Things according to claim 2, characterized in that, The edge server task queue model and the cloud server task queue model are represented as follows: in, and These represent edge servers s in time slots t and t+1, respectively. j With terminal u i The size of the task queue between them Represents time slot t and edge server s j Can be a terminal u i The amount of data processed; Indicates time slot t terminal u i Offload task data to edge servers. j , Indicates time slot t terminal u i Do not offload task data to edge servers j ; Indicates time slot t terminal u i Unload the task data to the cloud server. Indicates time slot t terminal u i Do not offload task data to the cloud server; U i (t) represents the terminal u in time slot t. i The amount of task data offloaded to edge servers and cloud servers; and These represent the cloud server and terminal u in time slots t and t+1, respectively. i The size of the task queue between them This indicates that the time slot t cloud server can provide terminal u i The amount of data processed.

4. The resource optimization method based on cloud-edge-device collaboration in the power Internet of Things according to claim 3, characterized in that, The average arrival rate of the terminal task queue is calculated using the following formula: in, Indicates time slot t terminal u i The average arrival rate of the task queue, where t represents the time slot, and A i (m) represents the time slot m terminal u i The amount of task data generated.

5. A resource optimization method based on cloud-edge-device collaboration in the power Internet of Things according to claim 4, characterized in that, The formulas for calculating the average arrival rate of the edge server task queue and the average arrival rate of the cloud server task queue are as follows: in, Represents time slot t and edge server s j With terminal u i The average arrival rate of the task queues between them, where t represents the time slot. Indicates time slot n terminal u i Offload task data to edge servers. j , Indicates time slot n terminal u i Do not offload task data to edge servers j ; Indicates time slot n terminal u i Unload the task data to the cloud server. Indicates time slot n terminal u i Do not offload task data to the cloud server; U i (n) represents time slot n terminal u i The amount of task data offloaded to edge servers and cloud servers; Indicates time slot t for cloud server and terminal u i The average arrival rate of the task queues between them.

6. A resource optimization method based on cloud-edge-device collaboration in the power Internet of Things according to claim 5, characterized in that, Establish a queuing delay model for the terminal, with the following formula: Where, τ i (t) represents the terminal u in time slot t. i The actual queuing delay; Indicates time slot t terminal u i The queuing delay during local task unloading is calculated using the following formula: Among them, Q i (t) represents the terminal u in time slot t. i Task queue size, Indicates time slot t terminal u i Average arrival rate of the task queue; Indicates time slot t terminal u i The maximum queuing latency for edge computing and cloud computing is given by the formula: Indicates time slot t terminal u i With edge servers j The queuing delay for edge computing between them is given by the following formula: in, Represents time slot t and edge server s j With terminal u i The size of the task queue between them Represents time slot t and edge server s j With terminal u i The average arrival rate of the task queues between them; j = 1, 2, ..., J; J represents the total number of edge servers; Indicates time slot t terminal u i The queuing latency for cloud computing between the cloud server and the cloud server is calculated using the following formula: in, Indicates time slot t for cloud server and terminal u i The size of the task queue between them Indicates time slot t for cloud server and terminal u i The average arrival rate of the task queues between them; Indicates time slot n terminal u i Unload the task data to the cloud server. Indicates time slot t terminal u i Do not unload task data to the cloud server; This represents the additional forwarding latency from the edge server to the cloud server.

7. A resource optimization method based on cloud-edge-device collaboration in the power Internet of Things according to claim 6, characterized in that, The objective function is constructed to minimize the actual queuing delay of the terminal, and the formula is: Among them, Ψ i (t) represents the objective function. This represents the action space; J represents the total number of edge servers, and t represents the time slot; And represents time slot t edge server s j With terminal u i The size of the task queue between; Indicates time slot n terminal u i Offload task data to edge servers. j , Indicates time slot t terminal u i Do not offload task data to edge servers j ; Indicates time slot n terminal u i Unload the task data to the cloud server. Indicates time slot t terminal u i Do not offload task data to the cloud server; U i (t) represents the terminal u in time slot t. i The amount of task data offloaded to edge servers and cloud servers; Indicates time slot t for cloud server and terminal u i Task queue size between; Q i (t) represents the terminal u in time slot t. i Task queue size; Indicates time slot t+1 terminal u i Average arrival rate of the task queue; V is a non-negative weight; Represents edge server s in time slot t+1 j With terminal u i The average arrival rate of the task queues between them; Indicates the connection between the cloud server and terminal u in time slot t+1. i The average arrival rate of the task queues between them; C1 and C2 are both constraints; Represents a set of terminals. Represents a set of edge servers. Represents a set of time slots.

8. A resource optimization method based on cloud-edge-device collaboration in the power Internet of Things according to claim 1, characterized in that, An actor-critic-based task offloading optimization algorithm is adopted to minimize the actual queuing latency of terminals. The dynamic access strategy for each terminal in each time slot is solved, and the steps include: Initialize the global model on the cloud server; In each time slot, each terminal downloads the global model from the cloud server as its local model; Each terminal selects an action space based on its local model and the state space of the current time slot; each terminal executes a task offloading decision based on the action space, calculates its corresponding cost function value, and obtains the state space of the next time slot; each terminal calculates the timing difference error based on its corresponding cost function value, the state space of the current time slot, and the state space of the next time slot, and performs a local update on the local model of that terminal. All locally updated models from all terminals are uploaded to the cloud server. Abnormal local models are removed, and a federated set is constructed. The cloud server performs federated averaging based on the federated set and updates the global model.

9. A resource optimization system based on cloud-edge-device collaboration in the power Internet of Things, characterized in that, include: The computational model building module is used to build cloud-edge-device collaborative computing models, including: Establish terminal task queue models, edge server task queue models, and cloud server task queue models to calculate the task queue sizes of terminals, edge servers, and cloud servers; calculate the average arrival rates of terminal task queues, edge server task queues, and cloud server task queues; and establish a terminal queuing latency model to calculate the actual queuing latency of terminals. The terminal dynamic access module is used to employ an actor-critic-based task offloading optimization algorithm. With the goal of minimizing the actual queuing delay of terminals, the dynamic access strategy for each terminal in each time slot is solved, including: The state space definition unit is used to define the terminal task queue size, edge server task queue size, cloud server task queue size, terminal task data volume, average arrival rate of terminal task queue, average arrival rate of edge server task queue, average arrival rate of cloud server task queue, and service availability of edge server to terminal as the state space. The action space definition unit is used to define whether to offload the task data of each terminal to the cloud server and whether to offload the task data of each terminal to the edge server as the action space. The solver unit is used to construct the objective function by minimizing the actual queuing delay of the terminal and solve the dynamic allocation strategy for each terminal in each time slot.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the resource optimization method based on cloud-edge-device collaboration in the power Internet of Things as described in any one of claims 1 to 8.

Images