Control device, control method, and program

Abstract

The purpose of the present disclosure is to achieve efficient task offloading throughout an entire network.　To this end, the present disclosure pertains to a control device that controls the allocation of tasks to a physical network constructed and modeled using nodes, the control device comprising: an observation unit that observes a plurality of tasks requested from terminal devices and a network usage state of the modeled physical network; a calculation unit that, on the basis of observation results of the observation unit, calculates a prescribed weight indicating a ratio of offloading of the plurality of tasks to the respective nodes; and a transfer unit that transfers, to each of the nodes, each task among the plurality of tasks at a ratio corresponding to the prescribed weight.

Description

Control device, control method, and program

[0001] The present disclosure relates to network and cloud control technology, and more particularly to control for allocating tasks.

[0002] With the development of communication technology, a variety of applications have emerged in various fields such as healthcare, smart cities, and manufacturing. Because the computational resources of end devices (EDs) such as PCs, smartphones, IoT (Internet of Things) devices, and automobiles are limited, these applications are offloaded to cloud servers for processing. This mechanism is called Edge Computing (EC) or Cloud Computing (CC).

[0003] The concentration of tasks from multiple terminal devices on a few servers can cause network congestion and server overload, resulting in a degradation of service. Therefore, there is a need to control the offload destination of tasks according to the load status of the network and servers, and to distribute the load efficiently across multiple servers.

[0004] Conventionally, a method has been proposed in which a control device acquires response times, facility information, infrastructure resource metrics, etc. from servers that process applications, detects server or site failures, software process failures, or overloads, and distributes the load on each server depending on the situation (Non-Patent Document 1). In this case, the control device repeatedly monitors various data, generates various metrics based on the monitored data, and performs dynamic control based on the various metrics. In Non-Patent Document 1, metrics refer to indicators that serve as materials for evaluating candidates that can be selected as offload destinations, such as CPU (Central Processing Unit) utilization rate and memory utilization rate.

[0005] "Demonstration Experiment Initiatives and Results - Network Edge Infrastructure," NTT Technical Journal, May 2022, 2022

[0006] However, while the prior art (Non-Patent Document 1) proposes a dynamic task offloading method that repeats monitoring and control, the control algorithm is simple. For example, the control device of the prior art determines the offload destination for each task by determining whether the load on each server exceeds a threshold. While such a simple control method is easy to implement, it is difficult to achieve efficient task offloading across the entire network by achieving optimal task offloading that maximizes network and server utilization and minimizes task delays, etc.

[0007] The present disclosure has been made in consideration of the above-mentioned problems, and aims to achieve efficient task offloading across the entire network.

[0008] In order to solve the above problem, the invention of claim 1 is a control device that controls task allocation to a modeled physical network constructed by each node, and includes an observation unit that observes multiple tasks requested from a terminal device and the network usage status of the modeled physical network, a calculation unit that calculates a predetermined weight indicating the offload ratio of the multiple tasks for each node based on the observation results of the observation unit, and a transfer unit that transfers each of the multiple tasks to each node in a ratio according to the predetermined weight.

[0009] The present invention has the effect of enabling efficient task offloading across the entire network.

[0010] It is a diagram showing an example of the overall configuration of a communication system in an embodiment of the present invention. It is a conceptual diagram showing a physical network of this embodiment. It is a hardware configuration diagram of a control device of this embodiment. It is a flowchart showing the control of a task offload system. It is a diagram showing each equation. It is a diagram showing algorithm 1. It is a diagram showing algorithm 2. It is a diagram showing algorithm 3.

[0011] [Outline of the embodiment] An outline of a communication system that performs task offloading will be described below with reference to Figures 1 and 2. Figure 1 is a diagram showing an example of the overall configuration of a communication system according to an embodiment of the present invention.

[0012] As shown in FIG. 1, the communication system of this embodiment is constructed by a control device 50 and a modeled physical network 140 .

[0013] The control device 50 is a device that controls task allocation for a modeled physical network constructed by multiple nodes. In a real-world physical network 40, an edge server 20 (described later) generally also functions as the control device 50. The control device 50 includes an observation unit 51, a calculation unit 55, and a transfer unit 59. Each of these units has a function that is realized by an instruction from the processor 101 in FIG. 3 based on a program. The processing content of each unit will be described later.

[0014] The control device 50 acquires task information and network usage status information from the modeled physical network 140 and performs task allocation control for the modeled physical network 140 .

[0015] The "task information" includes at least one of information on required computing resource demand and information on required traffic demand.

[0016] "Network usage information" includes information regarding network topology, bandwidth, or metrics.

[0017] "Metrics" are indicators that serve as materials for evaluating candidates that can be selected as offload destinations, such as CPU (Central Processing Unit) usage rate, memory usage rate, and disk usage rate.

[0018] Specifically, the control device 50 enables optimal task offloading by using task offloading algorithms (Algorithms 1 to 3) that calculate optimal control results for a dynamic control system based on task information and network usage information. The decision variables are weights that determine the task offload ratio for each server. For example, if there are three nodes and a total of six tasks requested from each terminal device 10, and the control device 50 calculates the predetermined weights for each of the three nodes as "1, 2, 3," the control device 50 will collectively offload tasks to the three nodes in a ratio of "1:2:3." Furthermore, "optimal task offloading" here refers to control based on predetermined weights that maximize server and link resource utilization efficiency and minimize task delay while satisfying constraints on server capacity, link capacity, and task delay. In this case, the control device 50 calculates the optimal predetermined weights using a trained multi-agent deep reinforcement learning (MADRL) model.

[0019] In the prior art, for example, if there are three nodes and a total of six tasks are requested from each terminal device 10, the control device 50 offloads one of the six tasks to each node.

[0020] As shown in Fig. 1, the modeled physical network 140 is constructed by multiple terminal devices that request tasks, multiple edge nodes 121, 122, and 123, and multiple cloud nodes 131 and 132. Note that due to space limitations, only a limited number of terminal devices, edge nodes, and cloud nodes are shown in Fig. 1, but there may be more of each than the number shown in Fig. 1.

[0021] 2 is a conceptual diagram showing a physical network 40 according to this embodiment. The physical network 40 is constructed by a plurality of terminal devices 11 and 12 that request tasks, a plurality of edge servers 21 and 22, and a plurality of cloud servers 31 and 32.

[0022] Furthermore, the terminal device 11 can be connected to a plurality of edge servers 21 and 22 and a plurality of cloud servers 31 and 32 via an access network an1. Similarly, the terminal device 12 can be connected to a plurality of edge servers 21 and 22 and a plurality of cloud servers 31 and 32 via an access network an2. A core network cn is constructed between the edge server 21 and the edge server 22. The modeled physical network 140 shown in FIG. 1 corresponds to the physical network 40 shown in FIG. 2. Note that, due to space limitations, FIG. 2 shows only a limited number of terminal devices, edge nodes, cloud nodes, access networks, and core networks, but there may be more of each than those shown in FIG. 2.

[0023] Hereinafter, the terminal devices 11 and 12 will be collectively referred to as "terminal device 10." The edge servers 21 and 22 will be collectively referred to as "edge server 20." The cloud servers 31 and 32 will be collectively referred to as "cloud server 30." The edge nodes 121, 122, and 123 will be collectively referred to as "edge node." The cloud nodes 131 and 132 will be collectively referred to as "cloud node." The edge node and cloud node will be collectively referred to as "node." Furthermore, the access networks an1 and an2 will be collectively referred to as "access network an."

[0024] The terminal device 10 may be a personal computer, a smartphone, a smartwatch, an IoT device, a home appliance, a communication device mounted on or installed in a mobile object, etc. Mobile objects include vehicles, aircraft, ships, robots, etc.

[0025] As shown in Fig. 2, every node has computing resources as an edge server 20 or a cloud server 30 that executes tasks on behalf of the terminal device 10. Every node is also connected to routers r1, r2, r3, and r4 that forward traffic to other nodes. Each edge server 20 has a control device 50 (see Fig. 1) for determining the optimal node to offload each task.

[0026] The terminal device 10 is configured by a computer and generates various tasks having a variety of applications.

[0027] Each terminal device 10 can calculate its own tasks within the terminal device 10 or offload tasks to an adjacent edge or cloud.

[0028] [Hardware Configuration of the Embodiment] FIG. 3 is a diagram showing the hardware configuration of the control device of the present embodiment.

[0029] 3, the control device 50 includes a processor 101, a memory 102, an auxiliary storage device 103, a connection device 104, a communication device 105, and a drive device 106. The hardware components constituting the control device 50 are connected to each other via a bus 107.

[0030] The processor 101 serves as a control unit that controls the entire control device 50, and includes various arithmetic devices such as a CPU (Central Processing Unit). The processor 101 reads various programs into the memory 102 and executes them. The processor 101 may also include a GPGPU (General-purpose computing on graphics processing unit).

[0031] The memory 102 has a main storage device such as a read-only memory (ROM) or a random access memory (RAM). The processor 101 and the memory 102 form a so-called computer, and the processor 101 executes various programs read onto the memory 102, thereby enabling the computer to realize various functions.

[0032] The auxiliary storage device 103 stores various programs and various information used when the processor 101 executes the various programs.

[0033] The connection device 104 is a connection device that connects an external device (for example, a display device 108 and an operation device 109 ) to the control device 50 .

[0034] The communication device 105 is a communication device for transmitting and receiving various types of information to and from other devices.

[0035] The drive device 106 is a device for loading a recording medium 106m. The recording medium 106m here includes media that record information optically, electrically, or magnetically, such as a CD-ROM (Compact Disc Read-Only Memory), a flexible disk, or a magneto-optical disk. The recording medium 106m may also include semiconductor memory that records information electrically, such as a ROM (Read Only Memory) or flash memory.

[0036] The various programs to be installed in the auxiliary storage device 103 are installed, for example, by setting the distributed recording medium 106m in the drive device 106 and reading the various programs recorded on the recording medium 106m by the drive device 106. Alternatively, the various programs to be installed in the auxiliary storage device 103 may be installed by being downloaded from a network via the communication device 105.

[0037] The terminal device 10, the edge server 20, and the cloud server 30 have the same hardware configuration as the control device, and therefore, a description thereof will be omitted.

[0038] [Processing of the embodiment] <Control procedure of the task offload system> Next, the control of the task offload system will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the control of the task offload system.

[0039] Now, consider a discrete time step t. Assume that each terminal device 10 has one or more tasks, and consider K tasks during the time step [0, T]. In this state, the following processing is executed.

[0040] Step S11: At the start of each time step t, each task arrives at the edge server 20 closest to each terminal device 10.

[0041] Step S12: The observation unit 51 of each edge server 20 (control device 50) acquires multiple task information and (latest) network usage status information, and observes multiple tasks related to the multiple tasks requested by the terminal device 10 and network usage status indicating the usage status of the physical network 140.

[0042] Step S13: Based on the observation results of step S12, the calculation unit 55 of each edge server 20 (control device 50) calculates the optimal predetermined weight for offloading tasks using the proposed method deployed on each edge server 20 (for details, see [Proposed Method] below).

[0043] Step S14: The transfer unit 59 of each edge server 20 (control device 50) transfers each of the tasks among the multiple tasks to each node in proportions according to a predetermined weight. In this case, it is desirable that each task be transferred over the shortest distance.

[0044] Step S15: Each node to which the task has been transferred executes the task and returns the result to the terminal device 10 that made the request.

[0045] Step S16: If a predetermined termination condition is met (YES), the control of the task offload system shown in Fig. 4 ends. The predetermined termination condition is, for example, when task requests from each terminal device 10 are completed. On the other hand, if the predetermined termination condition is not met (NO), the process returns to step S11 at the next time step t+1 and is repeated.

[0046] <Network Model> Next, Table 1 shows the definitions of the variables in the network model.

[0047] Consider a physical network graph G(N,L) consisting of a set of physical nodes N and a set of physical links L. Assume that each node plays a role as an edge or a cloud. Here, each edge node E is defined as e∈E⊂N, and each cloud node C is defined as c∈C⊂N. Also, the numbers of nodes, edge nodes, and cloud nodes are denoted as |N|, |E|, and |C|, respectively. A terminal device 10 connects to the nearest edge server 20 via an access network an, but in this embodiment, the access network an is not included in G(N,L).

[0048] Also, the node capacity of the i-th node is w N i Let's say.

[0049] "Node capacity" indicates the amount of computing resources a node has. When a task is assigned to a node, the node capacity is consumed by the computing resource demand of each task.

[0050] For example, if you assign one CPU core to each task, N i is equal to the number of CPU cores of the i-th node. The bandwidth capacity of link (i,j) is L ij Let's say.

[0051] <Task Model> Next, Table 2 shows the definitions of the variables in the task model.

[0052] A task model for uniformly expressing various tasks of the terminal device 10 is shown below. Let D = {Dk} be a set of tasks, and let the k-th task be Dk :=[t k ,C k ,B up k ,B down k ] is defined as

[0053] where t k ∈T is the acceptance time (time) of task k, C k indicates the required computing resource demand. up k , B downk denote the traffic demand for uploading and downloading, respectively.

[0054] A task allocates computing and network resources on G(N,L) to k Consume accordingly.

[0055] When a task is assigned to the edge node closest to the terminal device 10, the amount of network resources consumed in G(N, L) is considered to be zero.

[0056] <Formulation of the Optimization Problem> First, Table 3 shows the definitions of the variables in the task offload problem.

[0057] The decision variables in this problem are the weights X t where X := {x en,t} is a variable that represents the weight of the task offloaded from edge node e to node n at time t. We introduce |E|×|N| agents, and the i-th agent g i ∈G is the weight x en,t Learn how to optimize agent g i The observation of i t = [D t ,U L t ,U S t ] is defined as follows.

[0058] where D t denotes the set of tasks accepted at time step t. N t and U L t denotes the maximum utilization of nodes and links at time t, and U N t := max i (U N t ) and U L t := max ij (U L t ) where the utilization rate of the i-th node is defined as u N i,t∈ U N t Let the utilization rate of the i-th link be u L ij,t ∈ U L t Furthermore, the agent's action a i t weight x en,t This allows the agent to learn the optimal action, i.e., the optimal weights. Candidate set of actions A i is defined as a set of positive integers. The reward is designed to return a negative value if the constraint is not satisfied, and a positive value otherwise depending on the value of the objective function.

[0059] (Formulation) The variables of this method are defined in Table 4. This proposed method performs learning and execution.

[0060] Algorithm 1 Figure 6 is a diagram showing Algorithm 1. Algorithm 1 shows the state of intensive learning using MADRL.

[0061] The first line initializes the agent's parameters. A series of procedures (lines 2-15) are repeated until learning is complete. Lines 3-4 generate tasks and initialize the environment parameters. A series of actions is called an episode, and each episode (lines 5-15) is repeated.

[0062] In each episode, the agents <o t ,a t ,r t > training samples are collected, where the time step is t and is reset at the beginning of each episode.

[0063] In line 7, each agent observes information (task information, network usage information).

[0064] In line 8, a random action is chosen with probability ε, otherwise with probability 1-ε, Q i (o i t , a') is selected.

[0065] In the ninth line, a t Task offloading is performed accordingly.

[0066] Line 10 calculates the reward.

[0067] Lines 11-12 indicate the end condition of the episode.

[0068] Line 13 advances to the next t+1.

[0069] The 14th line indicates storage in the Replay memory M.

[0070] In line 15, every agent G is trained with a history of episodes randomly taken from M.

[0071] Algorithm 2 Fig. 7 is a diagram showing Algorithm 2. Algorithm 2 shown in Fig. 7 proposes a task offloading method using MADRL.

[0072] The first line pre-trains G using Algorithm 1.

[0073] Algorithm 2 then continually repeats lines 2-7 as long as controller 50 is willing to accept new tasks.

[0074] In line 5, each agent Q e (o e ,a e ) to maximize a i t is selected.

[0075] (Reward Calculation) A reward function is designed based on the objective function (Equation 1) shown in FIG.

[0076] Also, τ N t and τ L t represents the average node delay and average link delay of the task at time t. Also, λ is a weighting parameter that determines the ratio of importance of each term in the objective function. The reward function is designed based on the objective function (Equation 1).

[0077] Algorithm 3 Fig. 8 is a diagram showing Algorithm 3. Algorithm 3 shows the procedure for calculating the reward of G. Eff(x) represents the efficiency function, and is defined as (Equation 2) in Fig. 5.

[0078] The function in (Equation 2) is designed to be less efficient as x gets larger, and depending on x it returns a positive value if x<1, and a negative value otherwise.

[0079] U N t , U L t The value of C of the running task k , B up k , B down k It can be calculated using the values ​​of , the network capacity wNi, and the server capacity wLij. N t is obtained by measuring the actual mean node delay of the task. L t is obtained by measuring the actual average link delay.

[0080] This concludes the explanation of the proposed method.

[0081] [Major Effects of the Embodiment] The present embodiment has the effect of achieving efficient task offloading across the entire network. Specifically, the control device 50 uses a multi-agent deep reinforcement learning model to learn the relationship between network demand patterns and optimal task offloading, thereby achieving more efficient task offloading (control results) than conventional techniques.

[0082] [Supplementary Note] The present invention is not limited to the above-described embodiment, and may have the following configurations or processes (operations).

[0083] For example, each component of the control device 50 (observation unit 51, calculation unit 55, transfer unit 59) can be realized by a computer and a program, but this program can also be recorded on a (non-temporary) recording medium or provided via a communication network such as the Internet.

[0084] REFERENCE SIGNS LIST 11 Terminal device 12 Terminal device 21 Edge server 22 Edge server 31 Cloud server 32 Cloud server 40 Physical network 50 Control device 51 Observation unit 55 Calculation unit 59 Forwarding unit 121 Edge node 122 Edge node 131 Cloud node 132 Cloud node 133 Cloud node 140 Modeled physical network

Claims

1. A control device that controls task allocation for a modeled physical network constructed by each node, comprising: an observation unit that observes multiple tasks requested by a terminal device and network usage status of the modeled physical network; a calculation unit that calculates a predetermined weight indicating the offload ratio of the multiple tasks for each node based on the observation results of the observation unit; and a transfer unit that transfers each of the multiple tasks to each node in a ratio according to the predetermined weight.

2. The control device according to claim 1, wherein the calculation unit calculates the predetermined weights based on the observation results of the observation unit using a task offloading algorithm based on a trained multi-agent deep reinforcement learning model.

3. The control device according to claim 2, wherein the trained multi-agent deep reinforcement learning model is machine-learned based on the relationship between network demand patterns and optimal task offloading.

4. The control device according to claim 1, wherein each task information indicating each task includes at least one of information on computing resource demand and traffic demand.

5. The control device according to claim 1, wherein the information indicating the network usage status is information regarding network topology, bandwidth, or metrics.

6. A control method executed by a control device that controls task allocation for a modeled physical network constructed by each node, wherein the control device executes an observation process that observes multiple tasks requested by a terminal device and the network usage status of the modeled physical network, a calculation process that calculates a predetermined weight indicating the offload ratio of the multiple tasks for each node based on the observation results of the observation process, and a transfer process that transfers each task of the multiple tasks to each node in a ratio according to the predetermined weight.

7. A program for causing a computer to execute the method according to claim 6.