A resource allocation method oriented towards dependent tasks

Abstract

This invention relates to a resource allocation method for dependent tasks, belonging to the field of wireless communication technology. The method includes the following steps: S1: In an industrial internet scenario, an industrial internet system model is constructed, including cloud servers, edge servers, and industrial internet devices, and the dependencies between tasks are modeled as a directed acyclic graph. S2: A weighted sum mathematical model of the total system latency and energy consumption of the industrial internet network is constructed. S3: The problem model is modeled as a Markov decision process, and a SAC algorithm framework based on deep reinforcement learning (DRL) is used to obtain an approximately optimal offloading strategy for dependent tasks in an industrial internet (IIoT) device scenario with multiple edge servers.

Description

Technical Field

[0001] This invention belongs to the field of wireless communication technology and relates to a resource allocation method oriented towards dependent tasks. Background Technology

[0002] With the rapid development of modern information technology, manufacturing, logistics, infrastructure, and other industries are undergoing digital transformation, leading to the rapid development of the Internet of Things (IoT). Industrial Internet of Things (IIoT), as an important research area of ​​IoT, can serve as a key technology for solving the aforementioned problems. IIoT is the application of IoT in industrial manufacturing, where sensors, controllers, actuators, and IIoT devices are interconnected in industrial production environments. Therefore, IIoT has enormous potential to improve the efficiency, productivity, and intelligence of industrial plants.

[0003] IIoT enables the interconnection of heterogeneous IIoT devices, thereby improving manufacturing efficiency and reducing production costs. Since the computing power and battery capacity of these devices are limited, IIoT devices can handle some applications with low computing demands independently. However, handling computationally intensive applications often presents challenges. In traditional cloud computing solutions, when the processing power of IIoT devices is limited, they offload their computing tasks to remote cloud servers for processing and then return the results to the corresponding devices. This alleviates some of the computing power constraints on the devices. With the rapid growth in the number of IIoT devices, the corresponding number of tasks also increases, so offloading tasks to cloud servers is possible. However, for latency-sensitive or computationally intensive applications, continuing to offload tasks to the cloud can result in significant transmission delays and potential network congestion, failing to meet latency requirements and reducing service quality. This poses a significant risk to some operational factories, as the inability to return processing results in a timely manner can lead to unpredictable losses.

[0004] In traditional cloud-based solutions, IIoT devices need to offload their tasks to remote cloud computing servers (CCS). Data traffic increases dramatically with the number of IIoT devices and tasks, leading to network congestion, increased task latency, and reduced Quality of Service (QoS). Fortunately, edge computing, by deploying computing resources near IIoT devices, provides an effective solution to alleviate these problems. This technology allows IIoT devices to offload their tasks to edge computing servers (ECS) instead of remote CCS, meeting the real-time, security, and reliability requirements of computationally expensive and latency-sensitive IIoT applications, while also saving energy and effectively mitigating core network congestion. Although edge servers can handle many tasks, they are still lightweight compared to the cloud, so the computing power of a single edge server is limited. Furthermore, industrial internet networks are heterogeneous, and edge servers have varying workloads. Therefore, leveraging horizontal collaboration among multiple edge servers to balance their workloads is crucial to fully utilize idle computing resources. Collaborative optimization of task offloading and resource allocation can reduce task latency and improve resource utilization.

[0005] Based on the above analysis, it is worth noting that in most application scenarios, computational offloading algorithms are primarily used to handle independent tasks without considering when some tasks depend on others. However, in real-world industrial scenarios, some tasks exhibit complex dependencies and correlations. By considering collaborative edge computing and analyzing the characteristics and dependencies of tasks, the computational efficiency of IIoT networks can be significantly improved. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a resource allocation method oriented towards dependent tasks.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A resource allocation method oriented towards dependent tasks, the method comprising the following steps:

[0009] S1: Construct an industrial internet system model that includes cloud servers, edge servers, and industrial internet devices, and describe the dependencies between industrial device tasks in a real industrial environment using a directed acyclic graph.

[0010] S2: Construct a weighted mathematical model of total latency and energy consumption for an industrial internet system;

[0011] S3: Model the problem as a Markov decision process and propose a SAC (Soft Actor Critic) algorithm framework based on Deep Reinforcement Learning (DRL).

[0012] Furthermore, in S1, based on the characteristics of cloud servers, edge servers, and industrial equipment, an industrial internet network system architecture is constructed, specifically as follows:

[0013] The system consists of a remote cloud server (CS), an SDN controller, and M base stations. N IIoT devices Composition. Each Bs Each IIoT device has a corresponding edge computing server, also denoted as m. If the IIoT device has sufficient computing resources, it will process the entire task locally. Otherwise, the task is offloaded to a local Bs for processing, which can then offload the task to another Bs or a cloud center via wired transmission. Each IIoT device has a compute-intensive application that determines which devices will perform its tasks. During execution, a user can only communicate directly with one B. Edge servers collaborate via fiber optic transmission. Due to competition for computing and communication resources among users, system state changes at different times should be considered during compute offloading.

[0014] Furthermore, in S2, based on the characteristics of cloud servers, edge servers, and industrial internet devices, an industrial internet network communication model and a weighted sum of total system latency and energy consumption are constructed, specifically as follows:

[0015] The waiting time v of the computation task is calculated based on the task's preparation time and the application's arrival time. n,i It is the maximum sum of the completion times and output times of all previous computation tasks. Similar to the computation pattern on edge servers, the output time of computation tasks is negligible. Therefore, based on the dependencies between computation tasks, computation task v n,i The waiting time can be defined as follows:

[0016]

[0017] i'∈pred(i) represents task v n,i The set of prerequisite tasks.

[0018] Based on the above analysis of task processing, IIoT device n completes task v. n,i The total delay is:

[0019]

[0020] Where k = 0 indicates that the task is processed locally. This indicates that the task is processed on an edge server, and k = M + 1 indicates that the task is processed on a cloud server. Then, the task v is calculated. n,i The actual completion time can be obtained by the following formula:

[0021] FT n,i =RT n,i +t n,i

[0022] Definition 1 {λ} It is an index function, meaning that if event λ is true, the function value is 1; otherwise, the value is 0. Similarly, the task v is calculated... n,i The energy consumption is:

[0023]

[0024] The goal of this paper is to find a near-optimal offloading strategy for the applications generated for each IIoT device to minimize the average ETC (Electronic Toll Collection) for all users. The ETC for each device is defined as follows:

[0025]

[0026] This invention, under the constraints of energy consumption and latency of IIoT devices, jointly optimizes offloading strategies, computing resources, and power to minimize the total latency of all device tasks. The scheduling and offloading problem of interdependent computing tasks can be formulated as follows:

[0027]

[0028] Furthermore, in S3, the problem model is modeled as a Markov decision process, and a DRL-based SAC algorithm framework is used to obtain an approximately optimal offloading strategy for dependent tasks in a multi-IIoT device scenario with multiple ECSs. At each decision time t, the scheduling agent observes the state of the IIoT environment and the state of the current decision task to select an offloading action and obtain a reward based on a custom objective. Specifically:

[0029] Markov decision process state space: At decision time t, the SDN controller collects the state of the environment and represents it as s. t s t ∈S. It includes edge server status, device status, and task dependencies. Remaining computing resources, bandwidth, and task waiting queues of the edge servers, as well as the computing power of IIoT devices and currently executing tasks, are all necessary information for decision-making. For task dependencies, since it is graph-related data, GCN (Graph Convolutional Neural Network) is used to extract its task features, and the output is a task-level embedding. Where D is the embedding dimension of each task, and I is the number of tasks generated by IIoT device n.

[0030] At each decision-making moment, the action space includes the current unloading decision, the allocation of computing resources, and the power level.

[0031] In each decision-making step, there is a task that needs to be decided. Given state s t This allows obtaining the ETC status of all devices in the system at time t. After the action is executed, the status changes from s... t Become s t+1 The reward for time step t is defined as the negative increment of the total ETC (Electronic Toll Collection) of all users in the IIoT system, calculated as follows:

[0032]

[0033] Since the MDP proposed in this invention exhibits a high-dimensional and continuous state and action space, as well as multi-modal rewards (i.e., delay and energy consumption), we use the SAC algorithm to make offloading decisions. SAC employs MERL to obtain the optimal strategy that maximizes its expected reward and expected entropy, defined as:

[0034]

[0035] Where E[·] represents the expectation of the random variable, H(π(·|s t )) is state s t The entropy of policy π. α is the weight of the entropy. The soft-Q value function of policy π is given by... This can be expressed as, according to the Bellman equation:

[0036]

[0037] The algorithm proposed in this invention consists of three components: an evaluation network, a target network, and an experience revisit pool. The evaluation network includes a policy network and a value network, and the target network includes a target value network and a target policy network.

[0038] Then, in the soft policy evaluation phase, we define the method used to train D. y The loss function for parameter y:

[0039]

[0040] During the training phase, the network parameters are trained by maximizing the soft-state value function of the policy.

[0041] J(μ)=E[Q y (s t ,a t )-alogp m (a t |s t )]

[0042] Secondly, an adaptive gradient-based method is used to adjust the entropy weight 'a'. Specifically, 'a' must be large enough to encourage exploration when the optimal behavior is uncertain, and gradually decrease as the optimal behavior is determined. Therefore, the weight 'a' can be updated by minimizing the following objective function:

[0043]

[0044] Finally, to enhance learning stability, the parameters of the target network are updated by slowly tracking the parameters of the evaluation network. and It is given by the following formula:

[0045]

[0046] The beneficial effects of this invention are as follows: Due to the complexity of tasks generated by industrial equipment, the focus is on the dependencies between tasks. A multi-dependency task offloading problem for edge-cloud collaboration is established, and the dependencies between tasks are modeled as a directed acyclic graph (DAG). Furthermore, the dependency offloading problem in an IIoT system with multiple IIoT devices and multiple edge servers is modeled as a Markov decision process (MDP), and a SAC algorithm based on DRL is proposed. The method provided by this invention can achieve a significant improvement in service quality and achieves a good trade-off between cost and risk.

[0047] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0048] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0049] Figure 1 This is a network system framework diagram of the present invention;

[0050] Figure 2 This is a flowchart illustrating the task unloading decision and resource allocation process required by the present invention. Detailed Implementation

[0051] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0052] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0053] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0054] Please see Figures 1-2 This is a resource allocation method oriented towards dependent tasks, specifically including the following steps:

[0055] Step 1: Construct an industrial internet system model including cloud servers, edge servers, and industrial internet devices, and describe the dependencies between industrial device tasks in a real industrial environment using a directed acyclic graph. The system under consideration consists of one remote cloud server (CS)c, one SDN controller, and M base stations. N IIoT devices Composition. Each Bs Each IIoT device has a corresponding edge computing server, also denoted as m. If the IIoT device has sufficient computing resources, it will process the entire task locally. Otherwise, the task is offloaded to a local Bs for processing, which can then offload the task to another Bs or a cloud center via wired transmission. Each IIoT device has a compute-intensive application that determines which devices will perform its tasks. During execution, a user can only communicate directly with one B. Edge servers collaborate via fiber optic transmission. Due to competition for computing and communication resources among users, system state changes at different times should be considered during compute offloading.

[0056] The application generated by user n can be modeled as a directed acyclic graph (DAGG). n =(V n E n It is divided into I interdependent tasks, where V n ={v n,i |i=1,2,...,I} represents the task of the application. This represents the dependency relationship between tasks, indicating that a subsequent task must start only after the preceding task has been completed. Each task is represented by a tuple (d...). n,i ,c n,i The symbols () represent the data size of the task and the number of CPU cycles required to complete the task, respectively. Specifically, task V... 1,0 And Task V 1,7 These are defined as entry and exit tasks, respectively. It's important to note that both tasks must be executed locally.

[0057] Step 2: Construct a weighted mathematical model of total latency and energy consumption for the industrial internet system.

[0058] Step 2.1: The IIoT device and Bs communicate with each other using a wireless channel, and use... Let v represent the task of uploading data from IIoT device n to edge server m. n,i The uplink rate. It is assumed that the wireless channels between different edge servers do not interfere with each other, and Depending on the average channel conditions, modulation and coding scheme, and transmit power p n,i According to Shannon's theorem, the wireless communication rate between IIoT device n and edge server m is:

[0059]

[0060] The edge server Bs is connected to the SDN controller via a fiber optic link, with a transmission rate of r. m,s The SDN control connects to the cloud server through the core backbone network, and its transmission rate is assumed to be r. s,c .

[0061] Step 2.2 (Local Calculation):

[0062] The calculation delay is:

[0063]

[0064] The calculated energy consumption is:

[0065]

[0066] Step 2.3 (Local Edge Computing):

[0067] Uplink transmission latency of the task:

[0068]

[0069] Computation latency for tasks processed by the local edge server:

[0070]

[0071] The energy consumption for transmitting tasks offloading from IIoT device n to the local BSM is:

[0072]

[0073] Step 2.3 (Collaborative Edge Computing):

[0074] From the local BS m to the collaborating BS via a wired link The transmission delay is:

[0075]

[0076] Collaborative ECS The computational latency for processing the unloading task is:

[0077]

[0078] The choice of collaborative BS is:

[0079]

[0080] Step 2.4 (Cloud Computing)

[0081] The task transfer latency from local BS m to CS c is:

[0082]

[0083] The latency of the CS computation task is:

[0084]

[0085] Step 2.5:

[0086] Based on the above analysis of task processing, IIoT device n completes task v. n,i The total delay is:

[0087]

[0088] Minimize the average ETC for all users. The ETC for each device is defined as:

[0089]

[0090] Step 2.6:

[0091] The scheduling and unloading problem of interdependent computational tasks can be formulated as follows:

[0092]

[0093] Among them, constraint (C1) represents the range of values ​​for the offloading decision variable. Constraint (C2) indicates that a task can only be computed by one device. Constraint (C3) represents the range of values ​​for the computing resources allocated between the local ECS and collaborating ECS. Constraint (C4) represents the range of values ​​for the power consumption of IIoT devices. C5 constrains the energy consumed by the computing task.

[0094] Step 3:

[0095] The problem model in step 2 is modeled as a Markov decision process, and a SAC algorithm framework based on DRL is proposed to obtain the near-optimal offloading strategy for dependent tasks in multi-IIoT device scenarios with multiple ECSs. At each decision time t, the scheduling agent observes the state of the IIoT environment and the state of the current decision task to select an offloading action and obtain a reward based on a custom objective. The details are as follows:

[0096] Markov decision process state space: At decision time t, the SDN controller collects the state of the environment and represents it as s. t s t ∈S. It includes edge server status, device status, and task dependencies. Remaining computing resources, bandwidth, and task waiting queues of the edge servers, as well as the computing power of IIoT devices and currently executing tasks, are all necessary information for decision-making. For task dependencies, since it is graph-related data, GCN (Graph Convolutional Neural Network) is used to extract its task features, and the output is a task-level embedding. Where D is the embedding dimension of each task, and I is the number of tasks generated by IIoT device n.

[0097] At each decision-making moment, the action space includes the current unloading decision, the allocation of computing resources, and the power level.

[0098] In each decision-making step, there is a task that needs to be decided. Given state s t This allows obtaining the ETC status of all devices in the system at time t. After executing the action, the status changes from s... t Become s t+1 The reward for time step t is defined as the negative increment of the total ETC (Electronic Toll Collection) of all users in the IIoT system, calculated as follows:

[0099]

[0100] Since the MDP proposed in this invention exhibits a high-dimensional and continuous state and action space, as well as multi-modal rewards (i.e., delay and energy consumption), we use the SAC algorithm to make offloading decisions. SAC employs MERL to obtain the optimal strategy that maximizes its expected reward and expected entropy, defined as:

[0101]

[0102] Where E[·] represents the expectation of the random variable, H(π(·|s t )) is state s t The entropy of policy π. α is the weight of the entropy. The soft-Q value function of policy π is given by... This can be expressed as, according to the Bellman equation:

[0103]

[0104] The algorithm proposed in this invention consists of three components: an evaluation network, a target network, and an experience replay pool.

[0105] Then, in the soft policy evaluation phase, we define the method used to train D. y The loss function for parameter y:

[0106]

[0107] During the training phase, the network parameters are trained by maximizing the soft-state value function of the policy.

[0108] J(μ)=E[Q y (s t ,a t )-alogp m (a t |s t )]

[0109] Secondly, an adaptive gradient-based method is used to adjust the entropy weight 'a'. Specifically, 'a' must be large enough to encourage exploration when the optimal behavior is uncertain, and gradually decrease as the optimal behavior is determined. Therefore, the weight 'a' can be updated by minimizing the following objective function:

[0110]

[0111] Finally, to enhance learning stability, the parameters of the target network are updated by slowly tracking the parameters of the evaluation network. and It is given by the following formula:

[0112]

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A resource allocation method oriented towards dependent tasks, characterized in that: The method includes the following steps: S1: Construct an industrial internet system model that includes cloud servers, edge servers, and industrial internet devices, and initialize the number of cloud servers, edge servers, and industrial internet devices in the industrial internet system under consideration, and model the dependencies between tasks as a directed acyclic graph. S2: Considering the computation latency of the dependent tasks of industrial Internet devices in local computing as well as the transmission latency and computation latency in edge servers and cloud servers, construct a weighted mathematical model of the total system latency and energy consumption of the industrial Internet network. S3: The problem model is modeled as a Markov decision process, aiming to maximize the cumulative discount reward in the long run; a DRL-based SAC algorithm framework is used to obtain the near-optimal offloading strategy for dependent tasks in multi-IIoT device scenarios with multiple edge computing servers (ECSs); at each decision time t, the scheduling agent observes the state of the IIoT environment and the state of the current decision task to select an offloading action and obtain a reward based on a custom objective.

2. The resource allocation method for task-dependent tasks according to claim 1, characterized in that: In step S1, the network system architecture and task dependencies are constructed based on cloud servers, edge servers, and industrial internet devices, specifically as follows: The application generated by user n is modeled as a directed acyclic graph (DAGG). n =(V n E n It is divided into I interdependent tasks, where V n ={v n,i |i = 1, 2, ..., I} represents the task of the application; It represents the dependency relationship between tasks, indicating that a subsequent task must start after the preceding task is completed; each task is represented by a tuple (d n,i ,c n,i The numbers represent the data size of the task and the number of CPU cycles required to complete the task, respectively; the start task and end task are defined as the entry and exit tasks, respectively; these two tasks are executed locally.

3. The resource allocation method for task-dependent tasks according to claim 2, characterized in that: In step S2, based on the characteristics of cloud servers, edge servers, and industrial internet devices, an industrial internet network communication model, a latency model, and an energy consumption model are constructed, specifically as follows: use Indicates task v n,i Uninstallation strategy; Indicates task v n,i Execute on device k, where k=0 indicates local processing of the task. This indicates that the task is processed on an edge server, and k = M + 1 indicates that the task is processed on a cloud server. The following definition is proposed: 1)RT n,i : indicates task v n,i The preparation time during execution, i.e., the start time; 2)FT n,i : indicates task v n,i The completion time during execution, i.e., the moment of completion; Based on the dependencies between computation tasks, computation task v n,i The waiting time is defined as follows: i'∈pred(i) represents task v n,i The set of prerequisite tasks; IIoT devices complete tasks. n,i The total delay is: Computation task v n,i The actual completion time is obtained by the following formula: FT n,i =RT n,i +t n,i Definition 1 {λ} It is an indicator function; if event λ is true, the function value is 1; otherwise, the value is 0. The task v is calculated. n,i The energy consumption is: The goal is to find a near-optimal offloading strategy for the applications generated for each IIoT device to minimize the average ETC for all users; the ETC for each device is defined as: in, Under the constraints of energy consumption and latency of IIoT devices, the scheduling and offloading problem of interdependent computing tasks is formulated as follows: This involves joint optimization of offloading strategies, computing resources, and power to minimize the total latency of all device tasks. Among them, constraint C1 is the range of values ​​for the offloading decision variable; constraint C2 indicates that a task can only be computed by one device; constraint C3 indicates the range of values ​​for the computing resources allocated between the local ECS and the collaborating ECS; constraint C4 indicates the range of values ​​for the power of the IIoT device; and constraint C5 is the energy consumed by the computing task.

4. A resource allocation method for task-dependent tasks according to claim 2, characterized in that: In step S2, after obtaining the weighted sum of the total task latency and energy consumption of all devices, the optimization problem is modeled as a Markov decision process and solved using the SAC algorithm based on DRL, specifically: Problem P1 is modeled as an MDP, aiming to maximize the cumulative discounted reward in the long run. A SAC algorithm framework based on DRL is used to obtain an approximately optimal offloading strategy for dependent tasks in a multi-IIoT device scenario with multiple ECSs. At each decision time t, the scheduling agent observes the state of the IIoT environment and the state of the current decision task to select an offloading action and obtain a reward based on a custom objective. The details are as follows: Markov decision process state space: At decision time t, the SDN controller collects the state of the environment and represents it as s. t s t ∈S; The SDN controller collects the state of the environment, including edge server state, device state, and task dependencies; The remaining computing resources, bandwidth, and task queues of edge servers, as well as the computing power of IIoT devices and currently executing tasks, are all essential information for decision-making. The dependencies between tasks are graph-related data; a Graph Convolutional Neural Network (GCN) is used to extract task features, and the output is a task-level embedding. Where D is the embedding dimension of each task, and I is the number of tasks generated by IIoT device n; At each decision-making moment, the action space includes the current unloading decision, the allocation of computing resources, and the power level; In each decision-making step, there is a task that needs to be decided; given state s t , obtain the ETC of all devices in the system at time t.

Images