Industrial element universe-oriented cloud edge-end collaborative heterogeneous resource allocation and task scheduling method and system

Abstract

The invention belongs to the technical field of wireless communication, and discloses an industrial meta universe-oriented cloud side-end collaborative heterogeneous resource allocation and task scheduling method and system, and the method comprises the steps: splitting a calculation task generated by an end-side node into a plurality of subtasks, and scheduling each subtask to a node i for processing according to a task scheduling strategy, the node i comprises an end side node where a task is located, an edge node and a cloud; calculating the energy consumption of each sub-task scheduled to the node i based on a resource allocation strategy to obtain the total energy consumption of the calculation task; the task scheduling strategy and the resource allocation strategy are used as optimization objects, the total average energy consumption of minimum system processing tasks is used as a target function, the task processing time limit and the heterogeneous resource limit are used as constraints, an optimization model is constructed and solved, the task scheduling strategy and the resource allocation strategy are obtained, and corresponding resource allocation and task scheduling unloading are carried out. According to the method, the processing efficiency of computing intensive tasks of the industrial element universe system can be improved, the energy consumption of the system is reduced, and the system performance is improved.

Description

Technical Field

[0001] The present invention belongs to the field of wireless communication technology, and more specifically, relates to a cloud-edge-end collaborative heterogeneous resource allocation and task scheduling method and system for the industrial metaverse. Background Art

[0002] While the rapid development of the Industrial Internet has significantly improved production efficiency, it also presents challenges such as task delays, network congestion, and high energy consumption. Furthermore, with the continued development and transformation of the manufacturing market and growing customer demand for highly personalized products, reintegrating human intelligence into manufacturing processes has become a top priority. Since 2021, the metaverse has garnered significant attention and development. Its ability to create immersive human-machine interfaces has made it a strategic channel for integrating human intelligence into cyber-physical systems.

[0003] The industrial metaverse, a key application of the metaverse, integrates data from the virtual and real worlds, enabling full-process visualization, efficient resource scheduling, collaborative production, and rapid fault location and repair, thereby improving production efficiency and significantly reducing operating costs. In light of this, recent proposals have focused on integrating technologies such as 6G, digital twins, and edge computing into the metaverse to provide higher-quality metaverse services. 6G technology enhances network infrastructure capabilities, enabling stable, low-latency transmission. Digital twins provide users with a visual interface and enable algorithm verification and optimization in a virtual environment. Edge computing technology enables flexible allocation of computing resources at the edge of the network to better deliver services.

[0004] However, existing metaverse technologies have yet to achieve support for the underlying architecture of the industrial metaverse and optimize the corresponding energy consumption overhead when dynamically coordinating resource allocation and task scheduling between different layers of the network architecture. In addition, compute-intensive tasks generated in industrial Internet scenarios need to be completed within a specified time limit. In actual production lines, the business loads of different production line equipment vary, and the utilization of heterogeneous resources is uneven, resulting in low task processing efficiency. Therefore, existing resource allocation and scheduling methods may lead to problems such as excessive scheduling overhead, high costs, or low resource utilization. How to effectively utilize heterogeneous network resources, flexibly schedule compute-intensive tasks, and reduce system energy consumption within the constraints of limited resources and task time limits is a very challenging problem. Summary of the Invention

[0005] In response to the above-mentioned defects or improvement needs of the existing technology, the present invention provides a cloud-edge-end collaborative heterogeneous resource allocation and task scheduling method and system for the industrial metaverse, which aims to improve the processing efficiency of computing-intensive tasks in the industrial metaverse system, reduce system energy consumption, and improve the performance of the industrial metaverse system.

[0006] To achieve the above objectives, the present invention provides a cloud-edge-device collaborative heterogeneous resource allocation and task scheduling method for the industrial metaverse, comprising:

[0007] During the current scheduling period, the computing task generated by the end-side node of the industrial metaverse system is split into multiple subtasks, and each subtask is scheduled to node i for processing according to the task scheduling strategy; where node i includes the end-side node that generates the computing task, the edge node associated with the end-side node that generates the computing task, and the cloud; the task scheduling strategy is the proportion of the subtasks scheduled to node i for processing.

[0008] Calculate the energy consumption of scheduling each subtask to node i for processing based on a resource allocation strategy, thereby obtaining the total energy consumption of each computing task; wherein the resource allocation strategy includes the computing resources allocated to node i for task processing and the wireless transmission power allocated to the end-side node;

[0009] Based on the total energy consumption of each computing task, the total average energy consumption E of the industrial metaverse system processing end-side node computing tasks in the current scheduling period is obtained; the scheduling strategy and resource allocation strategy of the task are optimized, minimizing the total average energy consumption E is used as the objective function, and the task processing time limit and heterogeneous resource limit are used as constraints. A heterogeneous resource allocation and task scheduling optimization model based on cloud-edge-end node collaboration is constructed;

[0010] Solve the heterogeneous resource allocation and task scheduling optimization model, obtain the task scheduling strategy and resource allocation strategy in each scheduling period, and perform corresponding resource allocation and task scheduling offloading.

[0011] Furthermore, the heterogeneous resource allocation and task scheduling optimization model is:

[0012]

[0013] Wherein, E is the total average energy consumption; θ m,i is the proportion of subtasks split from the computing task generated by the end-side node m and scheduled to be processed by node i, where i=0 indicates that node i is the end-side node that generates the computing task; Denote the processing delay and queuing delay of the subtask at node i, respectively. m is the processing time limit of the subtask at node i; f i p is the computing resource allocated to node i for task processing; Indicates the upper and lower limits of computing resources allocated to node i for task processing; is the wireless transmission power allocated to the end-side node m; are the upper and lower limits of the wireless transmission power allocated to the end-side node m; For the end-side node cluster, For edge node clusters, {K+1} represents the cloud.

[0014] Furthermore, when node i is the end-side node that generates the computing task, the energy consumption of the subtask processed at node i includes local energy consumption;

[0015] When node i is an edge node associated with the end-side node that generates the computing task, the processing of the subtask at node i includes edge node offloading processing and edge node collaborative offloading processing. The edge node offloading processing refers to offloading the subtask to the edge node i associated with the end-side node for processing. At this time, the energy consumption of scheduling the subtask to node i for processing includes: the processing energy consumption of the subtask at edge node i and the offloading energy consumption; the edge node collaborative offloading processing refers to offloading the subtask to the edge node k associated with the end-side node, and then relaying it to the edge node i for processing, i≠k. At this time, the energy consumption of scheduling the subtask to node i for processing includes: the processing energy consumption of the subtask at edge node i, the offloading energy consumption and the edge node relay energy consumption;

[0016] When node i is in the cloud, processing the subtask at node i means unloading the subtask to the edge node k associated with the end-side node, and then relaying it to the cloud for processing. At this time, the energy consumption of scheduling the subtask to node i for processing includes the processing energy consumption of the subtask in the cloud, the unloading energy consumption, and the cloud relay energy consumption.

[0017] Furthermore, the total average energy consumption E is calculated as follows:

[0018]

[0019] Where M is the edge node cluster The total number of mid-side nodes; m is the computing task generation rate of the end-side node m; α is the energy consumption weighting coefficient; is the local energy consumption of the subtask processed at the end-side node m and the offloading energy consumption; The processing energy consumption of the subtasks processed at the edge node and the cloud and the relay energy consumption; and The sum of is the total energy consumption of a single computing task of the end-side node m, which is calculated as follows:

[0020]

[0021]

[0022] Where, Energy consumption of local processing of computing tasks generated by edge node m; The offloading energy consumption when offloading the computing task to the edge node or the cloud; Relay energy consumption for relaying the computing task from the edge node to other edge nodes or the cloud; is the energy consumption of the computing task processed at the edge node or in the cloud; and Based on the computing resources allocated to node i for task processing and the wireless transmission power allocated to the end-side node m Sure.

[0023] Furthermore, a dual-time-scale algorithm based on Soft Actor-Critic is used to solve the heterogeneous resource allocation and task scheduling optimization model, including:

[0024] The input state of the agent is the long-term state s l = {Q,h,Λ,A}, the corresponding action is the resource allocation strategy a under long time scale l ={f,p}; the input state of the agent is the instantaneous state s s = {Q,h,f,p}, the corresponding action is the task scheduling strategy under short time scale in, Channel state h={h1,...,h M}、The computing task generation rate of each end-side node Λ={λ1,...,λ M}, the computing task size generated by each end node is A={A1,...,A M}, the computing resource allocation of each node Wireless transmission power distribution of end-side nodes

[0025] At the beginning of the current scheduling period, according to the current long-term state s of the agent l Generate the corresponding resource allocation strategy a l ; and the current long-term state s l , resource allocation strategy a l , reward r l The updated long-term state is saved as experience to the experience replay pool;

[0026] During the current scheduling period, when any end-side node generates a computing task, according to the current instantaneous state s s Generate the task scheduling strategy a corresponding to the computing task s And perform task scheduling; the current instantaneous state s s , Task scheduling strategy a s , reward r s and the updated instantaneous state s sSave as experience to the experience replay pool;

[0027] At the end of the current scheduling period, experience is taken from the experience replay pool to evaluate the strategy, and the network loss is calculated based on the evaluation value and the Q value of the corresponding strategy evaluation network to update the network parameters of the intelligent agent; after the parameter update is completed, the resource allocation strategy and task scheduling strategy for the next scheduling period are generated.

[0028] Furthermore, the reward r l and reward r s for:

[0029]

[0030] Among them, γ is the scaling coefficient of the reward function.

[0031] The present invention also provides a cloud-edge-end collaborative heterogeneous resource allocation and task scheduling system for the industrial metaverse, including a computer-readable storage medium and a processor;

[0032] The computer-readable storage medium is used to store executable instructions;

[0033] The processor is used to read the executable instructions stored in the computer-readable storage medium to execute any of the above-mentioned cloud-edge-end collaborative heterogeneous resource allocation and task scheduling methods for the industrial metaverse.

[0034] The present invention also provides an industrial metaverse system based on cloud-edge-end node collaboration, including:

[0035] The physical model layer includes a three-layer network consisting of end-side nodes, edge nodes, and the cloud. Multiple end-side nodes are wirelessly connected to the edge nodes. Deterministic Ethernet low-latency transmission channels are established between edge nodes and between edge nodes and the cloud through TSN time-sensitive network switches.

[0036] The digital twin model layer includes a policy generation engine, wherein the policy generation engine is equipped with any of the above-mentioned cloud-edge-end collaborative heterogeneous resource allocation and task scheduling methods for the industrial metaverse, or the policy generation engine is the above-mentioned cloud-edge-end collaborative heterogeneous resource allocation and task scheduling system for the industrial metaverse;

[0037] The industrial metaverse application layer is used to bind the physical entity model layer with the digital twin model layer to achieve virtual-reality interaction and three-dimensional visualization presentation.

[0038] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cloud-edge-end collaborative heterogeneous resource allocation and task scheduling method for the industrial metaverse as described in any one of the above items.

[0039] The present invention also provides a computer program product, including a computer program. When the computer program runs on a computer, it enables the computer to execute any of the above-mentioned cloud-edge-end collaborative heterogeneous resource allocation and task scheduling methods for the industrial metaverse.

[0040] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0041] (1) The method of the present invention divides the computationally intensive tasks generated by the end-side nodes of the industrial metaverse system into subtasks, and distributes the subtasks proportionally to the local processing of the end-side nodes, the processing of the edge nodes associated with the end-side nodes that generated the computational tasks, and the cloud processing, making full use of the more abundant computing resources of the cloud and edge nodes, as well as the communication resources of the end-side nodes to allocate resources and schedule the task load on demand, thereby realizing collaborative task processing between the cloud, edge nodes, and end-side nodes, improving the balance of resource utilization of cloud-edge-end nodes, realizing the effective utilization of heterogeneous resources of each cloud-edge-end node, and improving the processing efficiency of computationally intensive tasks. Under the premise of better meeting the time limits of different tasks and limited heterogeneous resources, the average processing energy consumption of system tasks is optimized, the performance of the industrial metaverse system is improved, the system energy consumption is effectively reduced, and a feasible idea is provided for the underlying architecture support, resource allocation, and task scheduling in the metaverse.

[0042] (2) The present invention provides a dual-time-scale algorithm based on SAC to solve the heterogeneous resource allocation and task scheduling optimization model of cloud-edge node collaboration. The algorithm combines the characteristics of resource allocation and task scheduling in the industrial metaverse system and divides the state of the intelligent agent into long-term state s within a scheduling period. l and the instantaneous state s s , the corresponding actions are divided into resource allocation strategies a under long time scale l and task scheduling strategies under short time scales a s The resource allocation strategy remains unchanged within a scheduling period. When a computing task is generated by the end-side node within a scheduling period, a corresponding scheduling decision is generated for the computing task, thus realizing real-time policy scheduling with low time computation complexity.

[0043] (3) According to the method of the present invention, the edge nodes and the cloud can cooperate with each other to process tasks based on the task scheduling strategy. When the end-side node unloads part of the task to its associated edge node, the edge node can further split the task into corresponding subtasks according to the task scheduling decision and relay them to other edge nodes or the cloud for processing, thereby realizing the effective utilization of heterogeneous resources of each node at the cloud edge and improving the processing efficiency of computing-intensive tasks, and reducing system energy consumption under the premise of meeting the task time limit and limited heterogeneous resources.

[0044] In general, the present invention utilizes the more abundant computing resources of the cloud and edge nodes, as well as the communication resources of the end-side nodes, to optimize the average processing energy consumption of system tasks through resource allocation and task scheduling, thereby improving the performance of the industrial metaverse system while meeting the task time limit. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flow chart of the cloud-edge-end collaborative heterogeneous resource allocation and task scheduling method for the industrial metaverse provided by an embodiment of the present invention;

[0046] Figure 2 A flow chart of a dual-time-scale algorithm based on SAC for solving resource allocation and task scheduling optimization models provided by an embodiment of the present invention;

[0047] Figure 3 Schematic diagram of an industrial metaverse system based on cloud-edge node collaboration provided by an embodiment of the present invention;

[0048] Figure 4 A schematic diagram of a cloud-edge-end collaborative heterogeneous resource allocation and task scheduling system for the industrial metaverse provided by an embodiment of the present invention for processing computationally intensive tasks;

[0049] Figure 5 A simulation diagram showing the performance of the dual-time-scale algorithm based on SAC provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to illustrate the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0051] Example 1

[0052] like Figure 1 As shown, an embodiment of the present invention provides a cloud-edge-device collaborative heterogeneous resource allocation and task scheduling method for the industrial metaverse, including:

[0053] S1. Split each computing-intensive task generated by the end-side node of the industrial metaverse system during the current scheduling period into multiple subtasks, and schedule each subtask to four situations: local processing, edge node offloading processing, edge node collaborative offloading processing, and cloud offloading processing according to the task scheduling strategy; and calculate the energy consumption of each subtask in the corresponding node processing based on the resource allocation strategy to obtain the total energy consumption of each computing task. Among them, local processing refers to processing the subtask on the end-side node of its corresponding computing task, and its energy consumption includes task processing energy consumption; edge node offloading processing refers to offloading the subtask to the edge node of the industrial metaverse system associated with the end-side node that generates the computing task (the computing task before the subtask is split) for processing, and its energy consumption includes task processing energy consumption and offloading energy consumption; edge node collaborative offloading processing refers to offloading the task to the edge node associated with the end-side node that generates the computing task, and then relaying it to other edge nodes for processing, and its energy consumption includes task processing energy consumption, offloading energy consumption, and edge node relay energy consumption; cloud offloading processing refers to offloading the subtask to the edge node associated with the end-side node that generates the computing task, and then relaying it to the cloud for processing the task, and its energy consumption includes task processing energy consumption, offloading energy consumption, and cloud relay energy consumption. End-side nodes refer to the various devices of the industrial production line in the industrial metaverse system; edge nodes refer to the access layer devices of the industrial metaverse system network, that is, edge servers close to the end-side production line devices (end-side nodes) for managing and controlling multiple production line devices.

[0054] S2. Based on the total energy consumption of each computing task, the total average energy consumption E of the computing tasks of the end-side nodes processed by the industrial metaverse system during the current scheduling period is obtained; the resource allocation strategy of the cloud-edge end nodes and the scheduling strategy of the tasks are used as the optimization objects, the minimization of the total average energy consumption E of the system is used as the objective function, and the processing time limit of the task and the heterogeneous resource limit are used as the constraints to construct a heterogeneous resource allocation and task scheduling optimization model based on the collaboration of cloud-edge end nodes. In an embodiment of the present invention, heterogeneous resources are different types of resources allocated by cloud-edge end nodes, including communication, computing resources, etc. Task collaborative processing refers to splitting the task into subtasks of different proportions and transmitting them to different nodes for processing according to the task scheduling strategy.

[0055] S3. Use a method based on deep reinforcement learning to solve the above optimization model, obtain the resource allocation strategy and task scheduling strategy of the cloud edge node in the current scheduling period, and perform corresponding resource allocation and scheduling offloading.

[0056] As a specific implementation method, in S1, the task scheduling strategy is the ratio of the subtasks generated by the computing task generated by each end-side node to be scheduled to node i for processing. Node i includes the end-side node where the computing task is located, the edge node associated with the end-side node generating the computing task, and the cloud; the resource allocation strategy includes the computing resources f allocated to node i for task processing. ip and the wireless transmission power allocated to the end-side node m, in, Cluster the end-side devices (end-side nodes). Clustering edge nodes, {K+1} represents the cloud; each node i is allocated a certain amount of communication and computing resources to support task transmission and computing. For each end-side device m, a certain communication bandwidth is first allocated to it for transmitting tasks to the associated edge node, and the wireless transmission power is Controlled by resource allocation strategy. For each edge node and cloud, a wired high-capacity low-latency link (a deterministic Ethernet low-latency transmission channel is established between edge nodes and between edge nodes and cloud via TSN time-sensitive network switches) ensures the sequential transmission of tasks. The link transmission rates are r k 、r K+1 , is a known quantity; then for each node i on the cloud edge, a certain amount of computing resources f is allocated to it i p Used to process tasks.

[0057] Each end-side node generates computationally intensive tasks, where the tasks generated on the end-side device m can be represented by a triplet {τ m ,A m ,D m}, τ m ,A m ,D m Represent the task generation time, data volume and time limit respectively. Assume that the task generation rate of the end-side device m is λ m In order to effectively utilize the heterogeneous resources of cloud edge nodes and improve task processing efficiency, it is assumed that θ m,i Indicates the ratio of the subtasks generated by the task generated by the end device m to be dispatched to the device i for processing, f i p represents the computing resources allocated to task execution device i for task processing, Indicates the wireless transmission power allocated to the end-side device, and updates the resource allocation strategy (f i p 、 ) and the scheduling strategy for each task (θ m,i ) to optimize the average energy consumption of the industrial metaverse for processing computationally intensive tasks.

[0058] Task collaboration processing is to split the computationally intensive tasks generated by each end device into different proportions (θ m,i ) size, divide the subtasks into the proportion θ m,iThe data is transferred to the corresponding node for processing. The time limit of each subtask remains the same as that of the original computing task, and the sum of the sizes of all subtasks remains the same as that of the original computing task, so:

[0059]

[0060] Consider a typical edge device m. For any compute-intensive task generated by it and its split subtasks, a task processing energy consumption model is constructed based on the different nodes where the task is processed: local processing, edge node offloading processing, edge node collaborative offloading processing, and cloud offloading processing.

[0061] For tasks processed locally on the end-side device m, the time required for task processing is:

[0062]

[0063] Where η represents the computing resources consumed by processing a task of unit size, which is a known quantity; A m The amount of data generated for the task on the end device m. The energy consumption required to process the entire computing task locally (completely locally) for:

[0064]

[0065] in, It represents the power consumption of the end-side device m when processing a task, which is a known quantity.

[0066] For tasks that need to be offloaded from the end device m to the edge node or the cloud (complete offloading), the wireless offloading transmission time and offloading energy consumption are:

[0067]

[0068] Among them, r m =Blog2(1+SNR) is the wireless transmission rate, B is the system given bandwidth, and SNR is the signal-to-noise ratio.

[0069] For tasks that perform edge node offloading or edge node collaborative offloading at edge node k, the time and energy consumption required for task processing are:

[0070]

[0071] in, represents the power consumption of edge node k when processing a task, which is a known quantity. For tasks that need to be relayed to other edge nodes i for collaborative offloading or relayed to the cloud for offloading, the time and energy consumption required for relaying are:

[0072]

[0073] Among them, P i fwd is the task relay power consumption, which is a known quantity.

[0074] For tasks offloaded to the cloud, the time and energy consumption required for task processing are:

[0075]

[0076] Based on the above analysis, for the multiple subtasks generated by the task generated by the end device m, the energy consumption of processing the subtasks locally and the energy consumption of offloading the subtasks to other nodes (edge nodes and the cloud) via wireless can be expressed as:

[0077]

[0078] Among them, θ m,0 is the proportion of subtasks processed locally; (1-θ m,0 ) is the ratio of offloaded subtasks.

[0079] For multiple subtasks generated by the task generated by the end device m, the subtasks are offloaded to the edge nodes and the cloud for collaborative processing. The energy consumption includes the processing energy consumption of each edge node and the cloud, as well as the energy consumption of the edge node k for relay transmission to other edge nodes i or the cloud, which is expressed as:

[0080]

[0081] in, and The sum of is the energy consumption of a single intensive computing task generated by the end device m.

[0082] Therefore, considering the task generation rate λ m , the total average energy consumption E of the system for processing computationally intensive tasks is calculated as:

[0083]

[0084] Among them, λ m is the computationally intensive task generation rate at the end-side device m, which is a controllable known quantity, and α is the energy consumption weighting coefficient, which is an empirical value.

[0085] For each task execution device, it maintains a first-come, first-served task execution queue. Newly arrived or newly generated tasks that need to be processed by the device need to wait until the previous queue tasks are processed before they can be processed. Therefore, task processing needs to meet the time limit constraint:

[0086]

[0087] in, Denote the processing delay and queuing delay of the subtask on device i, respectively. m is the processing time limit of the subtask on device i, which is a known quantity.

[0088] The computing resources allocated to each cloud edge node cannot exceed the maximum computing resources of its hardware, nor can they be allocated less than the computing resources required to maintain normal operation of the device:

[0089]

[0090] in, Indicates the upper and lower limits of computing resources used by device i for task processing.

[0091] The transmission power allocated to the end-side node cannot exceed the maximum transmission power of its hardware, nor can it be allocated less than the transmission power required to maintain normal wireless communication:

[0092]

[0093] in, Indicates the upper and lower limits of the wireless transmission power allocated to the end-side device m.

[0094] The optimization variables include task scheduling strategy and resource allocation strategy. Therefore, in S2, the problem of heterogeneous resource allocation and task scheduling optimization for cloud-edge nodes can be expressed as:

[0095]

[0096] Where E is the total average energy consumption of the industrial metaverse system for processing computationally intensive tasks (computational tasks generated by each end-side node during the current scheduling period). is the set of end-side nodes, is the edge node cluster, and {K+1} represents the cloud. m,i Indicates the ratio of each subtask dispatched to device i (node i) for processing after the task generated from the end-side device m (end-side node m) is decomposed into multiple subtasks. It is a decision variable, θ m,0 is the proportion of subtasks processed locally; θ m,i The initial value can be randomly assigned, evenly distributed or taken from experience. Denote the processing delay and queuing delay of the subtask on device i, respectively. m is the processing time limit of the subtask on device i, which is a known quantity; f i p It represents the computing resources allocated to task execution device i for task processing, which is a decision variable. Its initial value can be an empirical value. It represents the wireless transmission power allocated to the end-side device m and is a decision variable. Its initial value can be an empirical value.

[0097] In S3, by solving the above optimization problem, the resource allocation and task scheduling strategy that minimizes the average weighted energy consumption of the system is obtained.

[0098] This problem is a complex, non-convex, nonlinear programming problem, typically classified as NP-Hard. Traditional methods often require exponential time complexity to solve. Furthermore, due to the dynamic nature of node operating states, channel environments, and task generation, changes in system state necessitate resolving the optimization problem and generating new strategies.

[0099] To address the above issues, an embodiment of the present invention provides a dual-time-scale algorithm based on SAC (Soft Actor-Critic) to solve the resource allocation and task scheduling optimization model in the embodiment of the present invention. To solve the above optimization problem, the system state is first divided into long-term state and transient state. In the embodiment of the present invention, the long-term state refers to the state that remains unchanged within a scheduling period, corresponding to the unchanged resource allocation policy within a scheduling period; the transient state refers to the state that changes multiple times within a scheduling period, corresponding to each time the end-side node generates a task within a scheduling period, it generates a corresponding scheduling decision for the task, realizing real-time policy scheduling.

[0100] In the embodiment of the present invention, the long-term state s l Including the queue status of each node on the cloud edge Channel state h={h1,...,h M}、The task generation rate of each end-side device Λ={λ1,...,λ M} and generate task size A={A1,...,A M},Right now:

[0101] s l ={Q,h,Λ,A}

[0102] In the embodiment of the present invention, the instantaneous state s s Including the queue status of each node on the cloud edge Channel state h={h1,...,h M}, the computing resource allocation of each node on the cloud edge and the wireless power allocation of the node on the end side Right now:

[0103] s s ={Q,h,f,p}

[0104] The actions of the algorithm are divided into resource allocation strategies under long time scales a l and task scheduling strategies under short time scales as , respectively expressed as:

[0105] a l ={f,p}

[0106]

[0107] The rewards for heterogeneous resource allocation and task scheduling in the algorithm are both based on minimizing the average weighted energy consumption, which can be expressed as:

[0108]

[0109] Among them, γ is the proportional coefficient of the scaling reward function, the experience value.

[0110] The dual-time-scale algorithm based on SAC proposed in the embodiment of the present invention is used to solve the heterogeneous resource allocation and task scheduling optimization model of cloud-edge node collaboration to obtain the resource allocation strategy and task scheduling strategy. The algorithm flow is as follows: Figure 2 As shown, the following steps are included:

[0111] 01. Initialize SAC network parameters, resource allocation, and default task scheduling policies. Set the conditions for the Industrial Metaverse system to terminate. These conditions can include the expiration of a preset scheduling period or manual termination of the system.

[0112] 02. If the system has not finished running, then each time a new scheduling period passes, at the beginning of the current scheduling period, according to the current long-term state s of the system l , update the cloud edge node resource allocation strategy:

[0113] a~π φ (·|s)

[0114] Among them, π φ is the policy generation network of SAC, φ is the parameter of the policy generation network; a here represents the resource allocation strategy a under long time scale l , s represents the current long-term state of the system s l . Set the current long-term state s l 、Action a l , reward r l and the long time scale experience s' of the new state l Save to the experience replay pool.

[0115] 03. During system operation, within the current scheduling period, if any end-side production line (end-side node) generates a task, according to the current instantaneous state s of the system s , generate a scheduling strategy a for the task s And perform task scheduling, the current state s s、Action a s , reward r s and the short-time experience s' of the new state s Save to the experience replay pool. When the current scheduling period has not ended, as long as a new task is generated, the scheduling strategy for the task is generated according to the corresponding instantaneous state and the task scheduling is executed, and the relevant data is saved to the experience replay pool.

[0116] 04. At the end of the current scheduling period, experience is taken from the experience replay pool and the strategy is evaluated:

[0117]

[0118] Where, j = 1, 2 represents the policy evaluation network parameters of SAC, i.e., the evaluation network parameters corresponding to the resource allocation strategy and task scheduling strategy; Q j The Q value of the corresponding strategy evaluation network; y represents the predicted Q value, and the reward value r is r l or r s , the updated action a' is the updated cloud edge node resource allocation strategy a' l Or the updated task scheduling policy a' s , the updated state s' is the updated long-term state s' l or short-term state s' s ; Among them, r l 、a' l 、s' l One-to-one correspondence, r s 、a' s 、s' s One-to-one correspondence; ζ is the discount coefficient, μ is the entropy regularization factor. The SAC network parameters are updated based on the following loss function:

[0119]

[0120] Among them, N B Indicates the number of experiences taken from the experience replay pool. represents the mean square error (MSE) loss function, J(φ) represents the entropy regularized conservative policy loss function; j takes 1 or 2, and the current state s and action a correspond to s respectively. l and a l , s s and a s After the SAC network parameters are updated, step 02 is entered to update the resource allocation strategy, that is, to generate a new resource allocation strategy and corresponding task scheduling strategy in the next scheduling period, so as to realize the real-time generation of resource allocation strategy and task scheduling strategy.

[0121] The dual-time-scale resource allocation and task scheduling solution method based on SAC proposed in the embodiment of the present invention realizes The polynomial complexity of SAC is , where L and U represent the number of layers and the number of neurons in each layer of the SAC neural network, respectively. represents the average task generation rate of each end-side device, and |a| is the complexity of generating resource allocation or task scheduling strategies in the SAC network. The time complexity is low, and the corresponding scheduling strategies can be generated in real time within each scheduling period.

[0122] Example 2

[0123] An embodiment of the present invention provides a cloud-edge-end collaborative heterogeneous resource allocation and task scheduling system for the industrial metaverse, including: a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the cloud-edge-end collaborative heterogeneous resource allocation and task scheduling method for the industrial metaverse in the above-mentioned embodiment 1.

[0124] The relevant technical solutions are the same as above and will not be repeated here.

[0125] Example 3

[0126] like Figure 3 As shown, embodiments of the present invention provide an industrial metaverse system based on cloud-edge-end node collaboration. The industrial metaverse system refers to a complex and advanced new digital industrial system built by combining virtual reality, artificial intelligence, digital twins, 5G, and other technologies. It primarily comprises a physical entity model layer, a digital twin model layer, and an industrial metaverse application layer.

[0127] The physical entity model layer includes a three-layer architecture of cloud, edge, and end, where the end side is the industrial production line equipment, the edge side is the edge server that controls multiple industrial production line equipment, and the cloud is the private cloud platform built by the factory or enterprise. Multiple end-side industrial production line equipment are connected to the edge server through wireless communication modules. Deterministic Ethernet low-latency transmission channels are established between edge nodes and between edge nodes and the cloud through TSN time-sensitive network switches. The private cloud platform is deployed locally in the enterprise to realize local data processing. Each node on the cloud-edge side is allocated a certain amount of computing resources for task processing. The end-side node can control the power consumption of wireless transmission for task offloading. The specific allocation ratio (θ m,i ) is calculated using the heterogeneous resource allocation and task scheduling method based on cloud-edge node collaboration in the above-mentioned embodiment 1; for the generated computationally intensive tasks, their scheduling strategies are also calculated based on the heterogeneous resource allocation and task scheduling method based on cloud-edge node collaboration in the above-mentioned embodiment 1.

[0128] The digital twin model layer is deployed on a private cloud platform and can map the physical entity model layer. It includes a state acquisition module that collects real-time sensor data, task queue status, terminal device and server load parameters, and wireless channel quality indicators of the physical layer; a policy generation engine, which is equipped with the cloud-edge collaborative heterogeneous resource allocation and task scheduling method for the industrial metaverse in Example 1, or the policy generation engine is the cloud-edge collaborative heterogeneous resource allocation and task scheduling system for the industrial metaverse in Example 2. By analyzing historical operation data and real-time context information, it outputs task scheduling and network resource allocation strategies. The relevant technical solutions are the same as above and will not be repeated here; a control instruction distribution module that encodes the strategy into executable commands and transmits them to the corresponding devices via edge node relays.

[0129] The Industrial Metaverse application layer is used to bind physical entities to their corresponding digital twins, enabling virtual-reality interaction and 3D visualization within the Industrial Metaverse environment. This layer allows engineers to manually adjust resource allocation strategies and task scheduling for physical devices, and they can also preview the impact of scheduling plans on production processes within the digital twin space.

[0130] The present invention considers the scenario where a cluster composed of multiple end-side devices randomly generates computationally intensive tasks, such as Figure 4 As shown in the figure, tasks are dispatched to different nodes for processing based on the task scheduling strategy. By scheduling compute-intensive tasks and coordinating task processing between cloud-edge nodes, the heterogeneous resources of each cloud-edge node are effectively utilized. This improves the processing efficiency of compute-intensive tasks, enhances the performance of the industrial metaverse system, and effectively reduces system energy consumption while better meeting the time limits of different tasks and the limited heterogeneous resources.

[0131] The present invention is described in detail below with reference to simulation:

[0132] The simulation parameters were set as follows: the number of edge devices M = 10, the number of edge nodes K = 3, the task generation rate followed a uniform distribution of [5, 15] tasks per scheduling period, the task size followed a uniform distribution of [1, 5] Mbits, and the time limit followed a uniform distribution of [60, 200] ms. The wireless bandwidth of the edge devices was B = 4 MHz, the maximum transmission power was 0.2 W, and the wireless channel followed a Rice distribution. The maximum computing resources of the cloud, edge, and end were 40 GHz, 12 GHz, and 3 GHz, respectively. The transmission rate and power consumption between edge nodes were 1000 Mbps and 1 W, respectively. The transmission rate and power consumption between edge nodes and the cloud were 100 Mbps and 0.3 W, respectively.

[0133] like Figure 5As shown in the figure, the system energy consumption performance of the dual-time-scale SAC-based algorithm provided in the embodiment of the present invention for solving the heterogeneous resource allocation and task scheduling optimization problem based on cloud-edge-end node collaboration is compared with deep reinforcement learning methods based on TD3, DDPG, and PPO. It can be observed that the present invention outperforms other benchmark algorithms, especially when the average task generation rate of a single end-side device is 30 per time slot. Compared with TD3, DDPG, and PPO, the average weighted energy consumption of the system of the present invention is reduced by 15.0%, 26.7%, and 32.5%, respectively.

[0134] Example 4

[0135] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the cloud-edge-end collaborative heterogeneous resource allocation and task scheduling method for the industrial metaverse in the above-mentioned embodiment 1 are implemented.

[0136] Specifically, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0137] The relevant technical solutions are the same as above and will not be repeated here.

[0138] Example 5

[0139] An embodiment of the present application provides a computer program product, including a computer program. When the computer program runs on a computer, it enables the computer to execute the steps of the cloud-edge-end collaborative heterogeneous resource allocation and task scheduling method for the industrial metaverse in the above-mentioned embodiment 1.

[0140] The relevant technical solutions are the same as above and will not be repeated here.

[0141] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A cloud-edge-end collaborative heterogeneous resource allocation and task scheduling method for the industrial metaverse, characterized by: include: During the current scheduling period, the computing task generated by the end-side node of the industrial metaverse system is split into multiple subtasks, and each subtask is scheduled to node i for processing according to the task scheduling strategy; where node i includes the end-side node that generates the computing task, the edge node associated with the end-side node that generates the computing task, and the cloud; the task scheduling strategy is the proportion of the subtasks scheduled to node i for processing. Calculate the energy consumption of scheduling each subtask to node i for processing based on a resource allocation strategy, thereby obtaining the total energy consumption of each computing task; wherein the resource allocation strategy includes the computing resources allocated to node i for task processing and the wireless transmission power allocated to the end-side node; Based on the total energy consumption of each computing task, the total average energy consumption E of the industrial metaverse system processing end-side node computing tasks in the current scheduling period is obtained; the scheduling strategy and resource allocation strategy of the task are optimized, minimizing the total average energy consumption E is used as the objective function, and the task processing time limit and heterogeneous resource limit are used as constraints. A heterogeneous resource allocation and task scheduling optimization model based on cloud-edge-end node collaboration is constructed; Solve the heterogeneous resource allocation and task scheduling optimization model, obtain the task scheduling strategy and resource allocation strategy in each scheduling period, and perform corresponding resource allocation and task scheduling offloading.

2. The cloud-edge-device collaborative heterogeneous resource allocation and task scheduling method according to claim 1 is characterized in that: The heterogeneous resource allocation and task scheduling optimization model is: Wherein, E is the total average energy consumption; θ m,i is the proportion of subtasks split from the computing task generated by the end-side node m and scheduled to be processed by node i, where i=0 indicates that node i is the end-side node that generates the computing task; Denote the processing delay and queuing delay of the subtask at node i, respectively. m is the processing time limit of the subtask at node i; f i p is the computing resource allocated to node i for task processing; Indicates the upper and lower limits of computing resources allocated to node i for task processing; is the wireless transmission power allocated to the end-side node m; are the upper and lower limits of the wireless transmission power allocated to the end-side node m; For the end-side node cluster, For edge node clusters, {K+1} represents the cloud.

3. The cloud-edge-end collaborative heterogeneous resource allocation and task scheduling method according to claim 2 is characterized in that: When node i is the end-side node that generates the computing task, the energy consumption of the subtask processed at node i includes local energy consumption; When node i is an edge node associated with the end-side node that generates the computing task, the processing of the subtask at node i includes edge node offloading processing and edge node collaborative offloading processing. The edge node offloading processing refers to offloading the subtask to the edge node i associated with the end-side node for processing. At this time, the energy consumption of scheduling the subtask to node i for processing includes: the processing energy consumption of the subtask at edge node i and the offloading energy consumption; the edge node collaborative offloading processing refers to offloading the subtask to the edge node k associated with the end-side node, and then relaying it to the edge node i for processing, i≠k. At this time, the energy consumption of scheduling the subtask to node i for processing includes: the processing energy consumption of the subtask at edge node i, the offloading energy consumption and the edge node relay energy consumption; When node i is in the cloud, processing the subtask at node i means unloading the subtask to the edge node k associated with the end-side node, and then relaying it to the cloud for processing. At this time, the energy consumption of scheduling the subtask to node i for processing includes the processing energy consumption of the subtask in the cloud, the unloading energy consumption, and the cloud relay energy consumption.

4. The cloud-edge-end collaborative heterogeneous resource allocation and task scheduling method according to claim 3 is characterized in that: The total average energy consumption E is calculated as follows: Where M is the edge node cluster The total number of mid-side nodes; m is the computing task generation rate of the end-side node m; α is the energy consumption weighting coefficient; is the local energy consumption of the subtask processed at the end-side node m and the offloading energy consumption; The processing energy consumption of the subtasks processed at the edge node and the cloud and the relay energy consumption; and The sum of is the total energy consumption of a single computing task of the end-side node m, which is calculated as follows: Where, Energy consumption of local processing of computing tasks generated by edge node m; The offloading energy consumption when offloading the computing task to the edge node or the cloud; Relay energy consumption for relaying the computing task from the edge node to other edge nodes or the cloud; is the energy consumption of the computing task processed at the edge node or in the cloud; and Based on the computing resources f allocated to node i for task processing i p and the wireless transmission power allocated to the end-side node m Sure.

5. The cloud-edge-device collaborative heterogeneous resource allocation and task scheduling method according to claim 4 is characterized in that: A dual-time-scale algorithm based on Soft Actor-Critic is used to solve the heterogeneous resource allocation and task scheduling optimization model, including: The input state of the agent is the long-term state s l = {Q,h,Λ,A}, the corresponding action is the resource allocation strategy a under long time scale l ={f,p}; the input state of the agent is the instantaneous state s s = {Q,h,f,p}, the corresponding action is the task scheduling strategy under short time scale in, Channel state h={h1,...,h M }、The computing task generation rate of each end-side node Λ={λ1,...,λ M }, the computing task size generated by each end node is A={A1,...,A M }, the computing resource allocation of each node i Wireless transmission power distribution of end-side nodes At the beginning of the current scheduling period, according to the current long-term state s of the agent l Generate the corresponding resource allocation strategy a l ; and the current long-term state s l , resource allocation strategy a l , reward r l The updated long-term state is saved as experience to the experience replay pool; During the current scheduling period, when any end-side node generates a computing task, according to the current instantaneous state s s Generate the task scheduling strategy a corresponding to the computing task s And perform task scheduling; the current instantaneous state s s , Task scheduling strategy a s , reward r s and the updated instantaneous state s s Save as experience to the experience replay pool; At the end of the current scheduling period, experience is taken from the experience replay pool to evaluate the strategy, and the network loss is calculated based on the evaluation value and the Q value of the corresponding strategy evaluation network to update the network parameters of the intelligent agent; after the parameter update is completed, the resource allocation strategy and task scheduling strategy for the next scheduling period are generated.

6. The cloud-edge-device collaborative heterogeneous resource allocation and task scheduling method according to claim 5 is characterized in that: The reward r l and reward r s for: Among them, γ is the scaling coefficient of the reward function.

7. A cloud-edge-end collaborative heterogeneous resource allocation and task scheduling system for the industrial metaverse, characterized by: comprising a computer-readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is used to read the executable instructions stored in the computer-readable storage medium to execute the cloud-edge-end collaborative heterogeneous resource allocation and task scheduling method for the industrial metaverse as described in any one of claims 1-6.

8. An industrial metaverse system based on cloud-edge node collaboration, characterized by: include: The physical model layer includes a three-layer network consisting of end-side nodes, edge nodes, and the cloud. Multiple end-side nodes are wirelessly connected to the edge nodes. Deterministic Ethernet low-latency transmission channels are established between edge nodes and between edge nodes and the cloud through TSN time-sensitive network switches. The digital twin model layer includes a policy generation engine, on which the cloud-edge-end collaborative heterogeneous resource allocation and task scheduling method for the industrial metaverse according to any one of claims 1 to 6 is carried, or the policy generation engine is the cloud-edge-end collaborative heterogeneous resource allocation and task scheduling system for the industrial metaverse according to claim 7; The industrial metaverse application layer is used to bind the physical entity model layer with the digital twin model layer to achieve virtual-reality interaction and three-dimensional visualization presentation.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, the cloud-edge-end collaborative heterogeneous resource allocation and task scheduling method for the industrial metaverse as described in any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that It includes a computer program, which, when running on a computer, enables the computer to execute the cloud-edge-end collaborative heterogeneous resource allocation and task scheduling method for the industrial metaverse as described in any one of claims 1 to 6.

Images