Joint service placement, task scheduling and resource allocation method in multi-sensor multi-user edge computing network

Abstract

The invention discloses a combined service placement, task scheduling and resource allocation method in a multi-sensor multi-user edge computing network. The method comprises the following steps: constructing a multi-sensor multi-user edge computing system model; establishing a user task processing and sensor data collection delay and energy consumption model, and further establishing a system overhead optimization problem; an optimization problem is decomposed, and a user and sensor transmitting power allocation decision and a computing resource and transmission rate allocation decision are solved based on a numerical optimization algorithm; a service placement and task scheduling sub-problem is constructed into a dual-time-scale Markov decision process, and a combined service placement, task scheduling and resource allocation strategy is trained and applied based on a deep reinforcement learning technology. According to the method, factors such as the user task arrival rate, the sensor data size and the edge server resources are comprehensively considered, service placement, task scheduling and resource allocation decision are jointly optimized, efficient operation of the edge system is guaranteed, and user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of Internet of Things and edge computing technology, and in particular to a method for joint service placement, task scheduling and resource allocation in a multi-sensor multi-user edge computing network. Background Art

[0002] In recent years, data-intensive Internet of Things (IoT) applications have been widely deployed in areas such as smart cities, intelligent transportation, and industrial automation. These applications, based on multi-sensor, multi-user network architectures, employ a task-data decoupled operating model, where tasks originate from users and data originates from sensors. Users generate computing tasks and offload them to servers for processing, while sensors collect data and upload it to servers to support user task processing. Traditional cloud computing architectures, limited by long-distance network transmission and centralized data processing, cannot meet the real-time requirements of user task processing. Consequently, edge computing architectures have been proposed and widely adopted. Edge computing pushes user task computing and sensor data collection functions to edge servers located at the edge of the network, improving the efficiency of both user task processing and sensor data collection.

[0003] To achieve efficient multi-sensor and multi-user edge computing networks, resource management plays a crucial role. It primarily addresses three issues: service placement, task scheduling, and resource allocation. Existing resource management solutions for edge computing networks primarily consider a task-data coupled operating model. This assumes that user-generated tasks consist of both task requests and the data required for task processing, while ignoring the latency and energy consumption of sensor data collection and its impact on system performance. Furthermore, existing solutions assume that service placement decisions, task scheduling decisions, and resource allocation decisions are made simultaneously, which is unfeasible in practical systems. These shortcomings significantly limit the joint optimization of service placement, task scheduling, and resource allocation. Summary of the Invention

[0004] (1) Technical issues to be resolved

[0005] In order to overcome the shortcomings of the above-mentioned existing technologies, the present invention is aimed at multi-sensor multi-user edge computing networks, and comprehensively considers factors such as user task arrival rate, sensor data size, network transmission rate, and edge server computing power to jointly optimize service placement, task scheduling, and resource allocation decisions. The method proposed in the present invention is integrated into the container (or virtual machine) deployment module and resource configuration module of the server management system, and combined with the network status perception and acquisition model to achieve optimal service placement of multiple edge servers, dynamic scheduling of user tasks, and dynamic optimization of edge server resource allocation. It comprehensively considers user task processing and sensor data collection to ensure efficient operation of the edge system and improve user experience.

[0006] (2) Technical solution

[0007] To solve the above technical problems, the present invention proposes a method for joint service placement, task scheduling, and resource allocation in a multi-sensor multi-user edge computing network, comprising the following steps:

[0008] S1: Construct a multi-sensor multi-user edge computing system model, an edge service placement model, and a user task scheduling model;

[0009] S2: Build user task processing latency and energy consumption models, as well as sensor data collection latency and energy consumption models, and then establish a system overhead optimization problem;

[0010] S3: Decompose the system overhead optimization problem established in S2, solve the optimal user and sensor transmit power allocation decision based on the Dinkelbach algorithm, and solve the optimal edge server computing resource allocation decision and edge transmission rate allocation decision based on the Karush-Kuhn-Tucker condition;

[0011] S4: The service placement and task scheduling sub-problems are formulated as a two-time-scale Markov decision process, defining the state, actions, and rewards of the system.

[0012] S5: Based on deep reinforcement learning technology, train and apply joint service placement, task scheduling and resource allocation strategies.

[0013] Furthermore, the step S1 is specifically as follows:

[0014] (1) Construct a multi-sensor multi-user edge computing system model. The edge computing network consists of M edge servers and one cloud server. The edge servers are deployed at base stations. The base stations are connected via a wired access network, while the edge servers and cloud servers are connected via a wired backbone network. There are two types of terminal devices within the coverage of each edge server, namely users and sensors. Users generate various types of tasks and offload them to edge servers or cloud services for processing; sensors collect the data required for the service and upload it to the edge servers and cloud servers where the corresponding services are deployed. The edge server set is represented as The total computing resources of edge server n are given by F n Indicates that the total storage resources are A n Indicates that the total memory resources are B n The index of the cloud server is 0, its total computing resources are represented by F0, its total storage resources are represented by A0, and its total memory resources are represented by B0.

[0015] (2) The user set is represented as , the sensor set is represented as The user set in the coverage area of ​​edge server m is represented by U mIndicates that the sensor set within the coverage of edge server m is represented by S m Represents. Users can generate multiple types of tasks, and each type of task can only be processed by the corresponding type of service. represents all task types and service types. Tasks of type k are represented as <c k ,d k >, where c k represents the computational load required to process k types of tasks, d k represents the data size of type k tasks. The number of tasks generated by users is represented by v uk .

[0016] (3) Service placement model. Define a k The storage resources required to deploy k type services, b k The memory resources required to deploy k types of services. Introduce a binary variable x nk ,in and Used to indicate whether a service of type k is deployed on edge server n. Specifically, if a service of type k is deployed on edge server n, set x nk = 1. Use a collection To indicate that all services place variables.

[0017] (4) Task scheduling model. Introducing binary variables o nuk ,in It is used to represent the scheduling decision of k types of tasks generated by user u. Specifically, if the k types of tasks generated by user u are scheduled to edge server n, set o nuk = 1. Use a collection To represent all task scheduling variables.

[0018] Furthermore, the step S2 is specifically as follows:

[0019] (1) Construct a user task processing delay and energy consumption model.

[0020] Because the service isn't deployed on the user device, user-generated tasks can only be offloaded to edge servers deployed with the corresponding service for processing. User task processing involves three steps: the user offloads the task to its affiliated edge server; the task offloaded to the affiliated edge server can be further offloaded to other edge servers or cloud servers; and the edge servers or cloud servers perform task processing. Since the task result data size is very small compared to the task request data size, the latency in returning the task result is negligible.

[0021] The wireless access system defining each edge server is based on Orthogonal Frequency Division Multiple Access (OFDMA), where multiple users associated with the same edge server transmit over orthogonal channels, thus avoiding inter-channel interference. The bandwidth is given by B user The channel gain is expressed by The noise power spectrum density is represented by N0, and the average channel interference power is represented by χ. Assuming that the maximum user transmission power is Introducing variables represents the transmission power of user u offloading its tasks to the edge server, then the transmission rate from user u to its affiliated edge server can be obtained as:

[0022]

[0023] Therefore, the upload delay of k-type tasks generated by user u to its affiliated edge server can be expressed as:

[0024]

[0025] Then the energy consumption of user u offloading type k task can be expressed as:

[0026]

[0027] If the corresponding service is not deployed on the attached edge server, the task needs to be further offloaded to other edge servers or cloud servers that have the corresponding service deployed. The total transmission rate between edge servers is defined as R mn , introduce variables represents the transmission rate between edge server m and edge server n allocated to the transmission of type k tasks offloaded by user u. Then the transmission delay of type k tasks generated by user u between edge servers m and n can be obtained as:

[0028]

[0029] Introduce variable f nuk represents the computing resources allocated by edge server n to process k types of tasks generated by user u. The processing delay of k types of tasks generated by user u can be expressed as:

[0030]

[0031] (2) Construct a sensor data collection delay and energy consumption model.

[0032] When a specific type of service is deployed on an edge server, sensor data of that type must be transferred to the edge server to facilitate processing of the same type of user tasks offloaded from users. Sensor data collection consists of two processes: uploading sensor data to its affiliated edge server, and transmitting sensor data from the affiliated edge server to other edge servers and cloud servers where the corresponding service is deployed.

[0033] It is also defined that multiple sensors associated with the same edge server transmit through orthogonal channels, and the bandwidth is expressed as B sensor , the channel gain is Assume that the maximum transmit power of the sensor is Introducing variables represents the transmission power of sensor s to upload its data to the affiliated edge server, then the transmission rate from sensor s to its affiliated edge server can be obtained as:

[0034]

[0035] Define the size of k types of data collected by sensor s as l sk , then the delay of uploading sensor data to its affiliated edge server can be expressed as:

[0036]

[0037] The energy consumption of the sensor data uploading process can be expressed as:

[0038]

[0039] Introducing variables represents the transmission rate allocated between edge server m and edge server n for transmitting type k data collected by sensor s. Then, the transmission delay of type k data collected by sensor s between edge servers m and n can be obtained as:

[0040]

[0041] (3) Construct an optimization problem.

[0042] The total system delay includes the delay of the user task processing process and the delay of the sensor data collection process. The total delay of the user task processing process can be expressed as:

[0043]

[0044] The total delay of the sensor data collection process can be expressed as:

[0045]

[0046] The total energy consumption of the system includes the total energy consumption of user devices and the total energy consumption of sensor devices. The total energy consumption of user devices can be expressed as:

[0047]

[0048] The total energy consumption of the sensor device can be expressed as:

[0049]

[0050] We define total system overhead as the weighted sum of total latency and total energy consumption. The weight factor, denoted as λ, represents the trade-off between latency and energy consumption. Combining the latency and energy consumption models, we formulate the optimization problem of joint service placement, task scheduling, and resource allocation in a multi-sensor, multi-user edge computing network:

[0051]

[0052]

[0053] Furthermore, the step S3 is specifically as follows:

[0054] (1) The optimization problem is decomposed into three sub-problems: the user and sensor transmission power allocation sub-problem, the edge server computing resources and transmission rate allocation sub-problem, and the service placement and task scheduling sub-problem.

[0055] (2) Solve the optimal user and sensor transmission power allocation decision based on the Dinkelbach algorithm.

[0056] (3) Based on the KKT condition, the optimal edge server computing resource allocation decision and transmission rate allocation decision are solved under given service placement and task scheduling decisions.

[0057] The optimal transmission rate allocation decision can be expressed as:

[0058]

[0059] The optimal computing resource allocation decision can be expressed as:

[0060]

[0061] Furthermore, the step S4 is specifically as follows:

[0062] (1) Construct a dual-time-scale Markov decision process for service placement and task scheduling. Define the long update period as t i ,i∈{1,2,…}, each long update cycle consists of T short update cycles, and the short update cycle is expressed as A dual-agent reinforcement learning architecture is constructed to update the service placement agent at the beginning of each long update cycle and the task scheduling agent at the beginning of each short update cycle.

[0063] The two-time-scale Markov decision process can be expressed as in represents the state space of the task scheduling agent, represents the state space of the service placement agent, represents the action space of the task scheduling agent, represents the action space of the service placement agent, represents the state transition probability, Represents the reward function. Specifically, at time slot t, the state space of the task scheduling agent is defined as:

[0064]

[0065]

[0066] The state space of the service placement agent is defined as:

[0067]

[0068] Among them, L h Indicates the length of historical information.

[0069] The action space of the task scheduling agent is defined as:

[0070]

[0071] The action space of the service placement agent is defined as:

[0072]

[0073] The reward function of the task scheduling agent is defined as:

[0074] r s (t)=-[T task +T data +λ(E user +E sensor )]

[0075] The reward function of the service placement agent is defined as the sum of rewards over T short time periods:

[0076]

[0077] Furthermore, the step S5 is specifically as follows:

[0078] (1) Training the service placement and task offloading strategy composed of the dual-agent neural network. At each time slot t, according to the current state s s (t),s l (t), decision action a s (t),a l (t), and calculate the reward r s (t),r l (t).

[0079] The specific steps are:

[0080] Step 1: Randomly initialize the parameters of the dual-agent neural network and initialize the time slot t to 1;

[0081] Step 2: Observe the system status l (t), s l (t) Input the service placement agent and obtain the service placement decision a l (t);

[0082] Step 3: Observe the system status s (t), s s (t) and a l (t) Input the task scheduling agent and obtain the task scheduling decision a s (t);

[0083] Step 4: Solve the optimal user and sensor power allocation decision based on the Dinkelbach algorithm

[0084] Step 5: Place decisions based on services a l (t) and task scheduling decision a s (t), using KKT conditions to solve the optimal edge server computing resource allocation decision and transmission rate allocation decisions

[0085] Step 6: Calculate the task scheduling agent reward r based on the results of steps 3, 4, and 5 s (t) and the serving agent reward r l (t);

[0086] Step 7: Use experience to train and update the task scheduling agent and service placement agent;

[0087] Step 8: Increase the value of t by 1 and determine whether the current episode has ended. If so, reset the system state and start a new episode.

[0088] Step 9: Determine whether the training process is finished. If so, end the training process; otherwise, go to step 2.

[0089] (2) After the task scheduling agent and the service placement agent are trained, the dual agent is used to make service placement and task scheduling decisions at each time slot t. The specific steps are as follows:

[0090] Step 1: The edge server senses the number of user tasks arriving, the channel conditions between the user and the edge server, the wired transmission rate between the edge server and the cloud server, and obtains the computing power of the edge server and the cloud server.

[0091] Step 2: The service placement agent outputs a service placement decision based on the system status The task scheduling agent outputs task scheduling decisions based on the system status

[0092] Step 3: Solve the optimal user and sensor transmit power decision based on the Dinkelbach algorithm

[0093] Step 4: Make placement decisions based on services and task scheduling decisions Using KKT conditions to solve the optimal edge server computing resource allocation decision and transmission rate allocation decisions

[0094] Step 5: The user offloads the task according to the task scheduling decision, the sensor uploads the service data according to the service placement decision, the edge server and the cloud server receive the sensor data, process the user task, and return the task result to the user.

[0095] (3) Beneficial effects

[0096] Compared with the prior art, the present invention adopts the above technical solution and has the following beneficial effects:

[0097] 1. A task-data decoupled operation mode is adopted to jointly optimize service placement, task scheduling, and resource allocation. This comprehensively considers the latency and energy consumption of user task processing and sensor data collection, making it more practical in multi-sensor, multi-user edge computing networks.

[0098] 2. When optimizing service placement variables, task scheduling variables, and resource allocation variables, we consider that different variables have different optimization frequencies and construct a dual-time-scale decision model, which is more practical.

[0099] 3. Based on deep reinforcement learning technology, it jointly optimizes service placement, task scheduling, and resource allocation, which can effectively adapt to the dynamic network conditions of the edge environment and the dynamic task requirements of users, and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] Figure 1 is a schematic diagram of a system model of an embodiment;

[0101] Figure 2 is a schematic diagram of the training process of the dual-agent neural network model for joint service placement, task scheduling, and resource allocation;

[0102] Figure 3 It is a schematic diagram of the application process of the dual-agent neural network model for joint service placement, task scheduling and resource allocation;

[0103] Figure 4 It is a graph showing the relationship between the number of users in the edge computing network and the average total cost of the system;

[0104] Figure 5 It is a graph showing the relationship between the number of sensors in the edge computing network and the average total cost of the system;

[0105] Figure 6 It is a graph showing the relationship between the computing resources of the edge server and the average total cost of the system; DETAILED DESCRIPTION

[0106] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0107] The present invention proposes a method for joint service placement, task scheduling, and resource allocation in a multi-sensor multi-user edge computing network. The embodiment includes the following steps:

[0108] Step 1: Reference Figure 1 ,construct a multi-sensor multi-user edge computing system model, an edge service placement model and a user task scheduling model;

[0109] Step 2: Build user task processing delay and energy consumption models, as well as sensor data collection delay and energy consumption models, and then establish a system overhead optimization problem;

[0110] Step 3: Decompose the system overhead optimization problem established in S2, solve the optimal user and sensor transmit power allocation decision based on the Dinkelbach algorithm, and solve the optimal edge server computing resource allocation decision and edge transmission rate allocation decision based on the Karush-Kuhn-Tucker condition given the service placement and task scheduling decisions;

[0111] Step 4: Construct the service placement and task scheduling sub-problems as a two-time-scale Markov decision process, defining the system's states, actions, and rewards.

[0112] Step 5: Reference Figure 2 and Figure 3,Based on deep reinforcement learning technology, in particular, the SD3 algorithm is selected to train and apply the joint service placement, task scheduling and resource allocation strategy.

[0113] Furthermore, the step 1 includes:

[0114] (1) Construct a multi-sensor multi-user edge computing system model. The edge computing network consists of M edge servers and one cloud server. The edge servers are deployed at base stations. The base stations are connected via a wired access network, while the edge servers and cloud servers are connected via a wired backbone network. There are two types of terminal devices within the coverage of each edge server, namely users and sensors. Users generate various types of tasks and offload them to edge servers or cloud services for processing; sensors collect the data required for the service and upload it to the edge servers and cloud servers where the corresponding services are deployed. The edge server set is represented as The total computing resources of edge server n are given by F n Indicates that the total storage resources are A n Indicates that the total memory resources are B n The index of the cloud server is 0, its total computing resources are represented by F0, its total storage resources are represented by A0, and its total memory resources are represented by B0.

[0115] (2) The user set is represented as The sensor set is represented as The user set in the coverage area of ​​edge server m is represented by U m Indicates that the sensor set within the coverage of edge server m is represented by S m Represents. Users can generate multiple types of tasks, and each type of task can only be processed by the corresponding type of service. represents all task types and service types. Tasks of type k are represented as <c k ,d k >, where c k represents the computational load required to process k types of tasks, d k represents the data size of type k tasks. The number of tasks generated by users is represented by v uk .

[0116] (3) Service placement model. Define a k The storage resources required to deploy k type services, b k The memory resources required to deploy k types of services. Introduce a binary variable x nk ,in and Used to indicate whether a service of type k is deployed on edge server n. Specifically, if a service of type k is deployed on edge server n, set x nk= 1. Use a collection To indicate that all services place variables.

[0117] (4) Task scheduling model. Introducing binary variables o nuk ,in It is used to represent the scheduling decision of k types of tasks generated by user u. Specifically, if the k types of tasks generated by user u are scheduled to edge server n, set o nuk = 1. Use a collection To represent all task scheduling variables.

[0118] Furthermore, the step 2 includes:

[0119] (1) Construct a user task processing delay and energy consumption model.

[0120] Because the service isn't deployed on the user device, user-generated tasks can only be offloaded to edge servers deployed with the corresponding service for processing. User task processing involves three steps: the user offloads the task to its affiliated edge server; the task offloaded to the affiliated edge server can be further offloaded to other edge servers or cloud servers; and the edge servers or cloud servers perform task processing. Since the task result data size is very small compared to the task request data size, the latency in returning the task result is negligible.

[0121] The wireless access system defining each edge server is based on Orthogonal Frequency Division Multiple Access (OFDMA), where multiple users associated with the same edge server transmit over orthogonal channels, thus avoiding inter-channel interference. The bandwidth is given by B user The channel gain is expressed by The noise power spectrum density is represented by N0, and the average channel interference power is represented by χ. Assuming that the maximum user transmission power is Introducing variables represents the transmission power of user u offloading its tasks to the edge server, then the transmission rate from user u to its affiliated edge server can be obtained as:

[0122]

[0123] Therefore, the upload delay of k-type tasks generated by user u to its affiliated edge server can be expressed as:

[0124]

[0125] Then the energy consumption of user u offloading type k task can be expressed as:

[0126]

[0127] If the corresponding service is not deployed on the attached edge server, the task needs to be further offloaded to other edge servers or cloud servers that have the corresponding service deployed. The total transmission rate between edge servers is defined as R mn , introduce variables represents the transmission rate between edge server m and edge server n allocated to the transmission of type k tasks offloaded by user u. Then the transmission delay of type k tasks generated by user u between edge servers m and n can be obtained as:

[0128]

[0129] Introduce variable f nuk represents the computing resources allocated by edge server n to process k types of tasks generated by user u. The processing delay of k types of tasks generated by user u can be expressed as:

[0130]

[0131] (2) Construct a sensor data collection delay and energy consumption model.

[0132] When a specific type of service is deployed on an edge server, sensor data of that type must be transferred to the edge server to facilitate processing of the same type of user tasks offloaded from users. Sensor data collection consists of two processes: uploading sensor data to its affiliated edge server, and transmitting sensor data from the affiliated edge server to other edge servers and cloud servers where the corresponding service is deployed.

[0133] It is also defined that multiple sensors associated with the same edge server transmit through orthogonal channels, and the bandwidth is expressed as B sensor , the channel gain is Assume that the maximum transmit power of the sensor is Introducing variables represents the transmission power of sensor s to upload its data to the affiliated edge server, then the transmission rate from sensor s to its affiliated edge server can be obtained as:

[0134]

[0135] Define the size of k types of data collected by sensor s as l sk , then the delay of uploading sensor data to its affiliated edge server can be expressed as:

[0136]

[0137] The energy consumption of the sensor data uploading process can be expressed as:

[0138]

[0139] Introducing variables represents the transmission rate allocated between edge server m and edge server n for transmitting type k data collected by sensor s. Then, the transmission delay of type k data collected by sensor s between edge servers m and n can be obtained as:

[0140]

[0141] (3) Construct an optimization problem.

[0142] The total system delay includes the delay of the user task processing process and the delay of the sensor data collection process. The total delay of the user task processing process can be expressed as:

[0143]

[0144] The total delay of the sensor data collection process can be expressed as:

[0145]

[0146] The total energy consumption of the system includes the total energy consumption of user devices and the total energy consumption of sensor devices. The total energy consumption of user devices can be expressed as:

[0147]

[0148] The total energy consumption of the sensor device can be expressed as:

[0149]

[0150] We define total system overhead as the weighted sum of total latency and total energy consumption. The weight factor, denoted as λ, represents the trade-off between latency and energy consumption. Combining the latency and energy consumption models, we formulate the optimization problem of joint service placement, task scheduling, and resource allocation in a multi-sensor, multi-user edge computing network:

[0151]

[0152] Furthermore, the step three includes:

[0153] (1) The optimization problem is decomposed into three sub-problems: the user and sensor transmission power allocation sub-problem, the edge server computing resources and transmission rate allocation sub-problem, and the service placement and task scheduling sub-problem.

[0154] (2) Solve the optimal user and sensor transmission power allocation decision based on the Dinkelbach algorithm.

[0155] (3) Based on the KKT condition, the optimal edge server computing resource allocation decision and transmission rate allocation decision are solved under given service placement and task scheduling decisions.

[0156] The optimal transmission rate allocation decision can be expressed as:

[0157]

[0158] The optimal computing resource allocation decision can be expressed as:

[0159]

[0160] Furthermore, the step 4 includes:

[0161] (1) Construct a dual-time-scale Markov decision process for service placement and task scheduling. Define the long update period as t i ,i∈{1,2,…}, each long update cycle consists of T short update cycles, and the short update cycle is expressed as A dual-agent reinforcement learning architecture is constructed to update the service placement agent at the beginning of each long update cycle and the task scheduling agent at the beginning of each short update cycle.

[0162] The two-time-scale Markov decision process can be expressed as in represents the state space of the task scheduling agent, represents the state space of the service placement agent, represents the action space of the task scheduling agent, represents the action space of the service placement agent, represents the state transition probability, Represents the reward function. Specifically, at time slot t, the state space of the task scheduling agent is defined as:

[0163]

[0164] The state space of the service placement agent is defined as:

[0165]

[0166] The action space of the task scheduling agent is defined as:

[0167]

[0168] The action space of the service placement agent is defined as:

[0169]

[0170] The reward function of the task scheduling agent is defined as:

[0171] r s (t)=-[T task +T data +λ(E user +E sensor )]

[0172] The reward function of the service placement agent is defined as the sum of rewards over T short time periods:

[0173]

[0174] Furthermore, the step five includes:

[0175] Train the task scheduling agent based on the SD3 algorithm and the service placement agent based on the SD3 algorithm. To this end, six neural networks of the task scheduling agent are constructed: Six neural networks that build the service placement agent:

[0176] Using gradient descent, the parameters of the main critic network of the task scheduling agent and the service placement agent The update rule is:

[0177]

[0178] in,

[0179]

[0180] The learning rate of the Actor network for serving placement agents, is the learning rate of the task scheduling agent Actor network, H l and H s is the sampling batch size, and ε′ is the additional noise.

[0181] Parameters φ of the main actor network of the task scheduling agent and the service placement agent l ,φ s The update rule is:

[0182]

[0183] Using the soft update algorithm, the parameters of the target actor network and target critic network of the service placement agent and task scheduling agent are The update rule is:

[0184]

[0185]

[0186] Among them, ζ is the soft update parameter.

[0187] The specific steps for training the service placement agent and task scheduling agent are:

[0188] Step 1: Randomly initialize the parameters of the dual-agent neural network and initialize the time slot t to 1;

[0189] Step 2: Determine whether the time slot t is a multiple of T. If so, set s l (t) Input the service placement agent and obtain the service placement decision a l (t); if not, keep the service placement decision the same as the previous time slot;

[0190] Step 3: Observe the system status s (t), s s (t) and a l (t) Input the task scheduling agent and obtain the task scheduling decision a s (t);

[0191] Step 4: Solve the optimal user and sensor power allocation decision based on the Dinkelbach algorithm

[0192] Step 5: Place decisions based on services a l (t) and task scheduling decision a s (t), using KKT conditions to solve the optimal edge server computing resource allocation decision and transmission rate allocation decisions

[0193] Step 6: Calculate the task scheduling decision agent reward r based on the results of steps 3, 4, and 5 s (t);

[0194] Step 7: Apply experience to {s s (t),a l (t),a s (t),r s (t),s s (t+1)} is stored in the experience pool of the task scheduling agent;

[0195] Step 8: Take batch experience from the task scheduling agent's experience pool to train and update the task scheduling agent's critic network;

[0196] Step 9: Determine whether the time slot t is a multiple of T-1. If so, calculate the reward r for the service placement agent. l(t), then the experience of {s l (t),a l (t),r l (t),s l (t+T)} is stored in the experience pool of the service placement agent, and batch experience is taken from the experience pool to train and update the critic network of the service placement agent;

[0197] Step 10: Determine whether the time slot t is a multiple of d. If so, train and update the Actor network of the task scheduling agent and the service placement agent based on the experience in the experience pool; update the target network of the two agents based on the soft update algorithm;

[0198] Step 11: Increase the value of t by 1 to determine whether the current episode has ended. If so, reset the system state.

[0199] Step 12: Determine whether t has reached a predetermined maximum value; if so, the training ends; otherwise, go to step 2.

[0200] After the task scheduling agent and service placement agent are trained, the dual agents are used to make service placement and task scheduling decisions at each time slot t. The specific steps are as follows:

[0201] Step 1: The edge server senses the number of user tasks arriving, the channel conditions between it and the users, the wired transmission rate between it and other edge servers, and obtains the computing power of the edge server and the cloud server.

[0202] Step 2: The service placement agent outputs a service placement decision based on the system status The task scheduling agent outputs task scheduling decisions based on the system status

[0203] Step 3: Solve the optimal user and sensor transmit power decision based on the Dinkelbach algorithm

[0204] Step 4: Make placement decisions based on services and task scheduling decisions Using KKT conditions to solve the optimal edge server computing resource allocation decision and transmission rate allocation decisions

[0205] Step 5: The user offloads the task according to the task scheduling decision, the sensor uploads the service data according to the service placement decision, the edge server and the cloud server receive the sensor data, process the user task, and return the task result to the user.

[0206] Figure 4 、 Figure 5and Figure 6 The paper compares the average total system overhead of the disclosed solution with other solutions under different edge computing network user numbers, different edge computing network sensor numbers, and different edge server computing resources. The experimental results show that the disclosed solution can achieve joint optimization of service placement, task scheduling, and resource allocation in dynamic edge network environments and user demands.

[0207] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the claims of the present invention.

Claims

1. A method for joint service placement, task scheduling, and resource allocation in a multi-sensor multi-user edge computing network, characterized in that: The following steps are involved: S1: Construct a multi-sensor multi-user edge computing system model, an edge service placement model, and a user task scheduling model; S2: Build user task processing latency and energy consumption models, as well as sensor data collection latency and energy consumption models, and then establish a system overhead optimization problem; S3: Decompose the system overhead optimization problem established in S2, solve the optimal user and sensor transmit power allocation decision based on the Dinkelbach algorithm, and solve the optimal edge server computing resource allocation decision and edge transmission rate allocation decision based on the Karush-Kuhn-Tucker condition; S4: The service placement and task scheduling sub-problems are formulated as a two-time-scale Markov decision process, defining the state, actions, and rewards of the system. S5: Based on deep reinforcement learning technology, train and apply joint service placement, task scheduling and resource allocation strategies.

2. A method for joint service placement, task scheduling and resource allocation in a multi-sensor multi-user edge computing network according to claim 1, characterized in that: The step S1 comprises: (1) Construct a multi-sensor multi-user edge computing system model. The edge computing network consists of M edge servers and one cloud server. The edge servers are deployed at base stations. The base stations are connected via a wired access network, while the edge servers and cloud servers are connected via a wired backbone network. There are two types of terminal devices within the coverage of each edge server, namely users and sensors. Users generate various types of tasks and offload them to edge servers or cloud services for processing; sensors collect the data required for the service and upload it to the edge servers and cloud servers where the corresponding services are deployed. The edge server set is represented as The total computing resources of edge server n are given by F n Indicates that the total storage resources are A n Indicates that the total memory resources are B n The index of the cloud server is 0, its total computing resources are represented by F0, its total storage resources are represented by A0, and its total memory resources are represented by B0. (2) The user set is represented as The sensor set is represented as The user set within the coverage area of ​​edge server m is represented by U m Indicates that the sensor set within the coverage of edge server m is represented by S m Represents. Users can generate multiple types of tasks, and each type of task can only be processed by the corresponding type of service. represents all task types and service types. Tasks of type k are represented as <c k ,d k >, where c k represents the computational load required to process k types of tasks, d k Represents the data size of k type tasks. The number of tasks generated by the user is represented as v uk . (3) Service placement model. Define a k The storage resources required to deploy k type services, b k The memory resources required to deploy k types of services. Introduce a binary variable x nk ,in and Used to indicate whether a service of type k is deployed on edge server n. Specifically, if a service of type k is deployed on edge server n, set x nk = 1. Use a collection To indicate that all services place variables. (4) Task scheduling model. Introducing binary variables o nuk ,in It is used to represent the scheduling decision of k types of tasks generated by user u. Specifically, if the k types of tasks generated by user u are scheduled to edge server n, set o nuk = 1. Use a collection To represent all task scheduling variables.

3. A method for joint service placement, task scheduling and resource allocation in a multi-sensor multi-user edge computing network according to claim 1, characterized in that: The step S2 comprises: (1) Construct a user task processing delay and energy consumption model. Because the service isn't deployed on the user device, user-generated tasks can only be offloaded to edge servers or cloud servers that have the corresponding service deployed for processing. User task processing involves three steps: the user offloads the task to its affiliated edge server; the task offloaded to the affiliated edge server can be further offloaded to other edge servers or cloud servers; and the edge server or cloud server performs task processing. Since the task result data size is very small compared to the task request data size, the latency in returning the task result is negligible. The wireless access system defining each edge server is based on Orthogonal Frequency Division Multiple Access (OFDMA), where multiple users associated with the same edge server transmit over orthogonal channels, thus avoiding inter-channel interference. The bandwidth is given by B user The channel gain is expressed by The noise power spectrum density is represented by N0, and the average channel interference power is represented by χ. Assuming that the maximum user transmission power is Introducing variables represents the transmission power of user u offloading its tasks to the edge server, then the transmission rate from user u to its affiliated edge server can be obtained as: Therefore, the upload delay of k-type tasks generated by user u to its affiliated edge server can be expressed as: Then the energy consumption of user u offloading type k task can be expressed as: If the corresponding service is not deployed on the attached edge server, the task needs to be further offloaded to other edge servers or cloud servers that have the corresponding service deployed. The total transmission rate between edge servers is defined as R mn , introduce variables represents the transmission rate between edge server m and edge server n allocated to the transmission of type k tasks offloaded by user u. Then the transmission delay of type k tasks generated by user u between edge servers m and n can be obtained as: Introduce variable f nuk represents the computing resources allocated by edge server n to process k types of tasks generated by user u. The processing delay of k types of tasks generated by user u can be expressed as: (2) Construct a sensor data collection delay and energy consumption model. When a specific type of service is deployed on an edge server, sensor data of that type must be transferred to the edge server to facilitate processing of the same type of user tasks offloaded from users. Sensor data collection consists of two processes: uploading sensor data to its affiliated edge server, and transmitting sensor data from the affiliated edge server to other edge servers and cloud servers where the corresponding service is deployed. It is also defined that multiple sensors associated with the same edge server transmit through orthogonal channels, and the bandwidth is expressed as B sensor , the channel gain is Assume that the maximum transmission power of the sensor is Introducing variables represents the transmission power of sensor s to upload its data to the affiliated edge server, then the transmission rate from sensor s to its affiliated edge server can be obtained as: Define the size of k types of data collected by sensor s as l sk , then the delay of uploading sensor data to its affiliated edge server can be expressed as: The energy consumption of the sensor data uploading process can be expressed as: Introducing variables represents the transmission rate allocated between edge server m and edge server n for transmitting type k data collected by sensor s. Then, the transmission delay of type k data collected by sensor s between edge servers m and n can be obtained as: (3) Construct an optimization problem. The total system delay includes the delay of the user task processing process and the delay of the sensor data collection process. The total delay of the user task processing process can be expressed as: The total delay of the sensor data collection process can be expressed as: The total energy consumption of the system includes the total energy consumption of user devices and the total energy consumption of sensor devices. The total energy consumption of user devices can be expressed as: The total energy consumption of the sensor device can be expressed as: We define total system overhead as the weighted sum of total latency and total energy consumption. The weight factor, denoted as λ, represents the trade-off between latency and energy consumption. Combining the latency and energy consumption models, we formulate the optimization problem of joint service placement, task scheduling, and resource allocation in a multi-sensor, multi-user edge computing network:

4. A method for joint service placement, task scheduling and resource allocation in a multi-sensor multi-user edge computing network according to claim 1, characterized in that: The step S3 comprises: (1) The optimization problem is decomposed into three sub-problems: the sensor and user transmission power allocation sub-problem, the edge server computing resources and transmission rate allocation sub-problem, and the service placement and task scheduling sub-problem. (2) Solve the optimal sensor and user transmission power allocation decision based on the Dinkelbach algorithm. (3) Based on the KKT condition, the optimal edge server computing resource allocation decision and transmission rate allocation decision are solved under given service placement and task scheduling decisions. The optimal transmission rate allocation decision can be expressed as: The optimal computing resource allocation decision can be expressed as:

5. A method for joint service placement, task scheduling and resource allocation in a multi-sensor multi-user edge computing network according to claim 1, characterized in that: The step S4 comprises: (1) Construct a dual-time-scale Markov decision process for service placement and task scheduling. Define the long update period as t i ,i∈{1,2,…}, each long update cycle consists of T short update cycles, and the short update cycle is expressed as A dual-agent reinforcement learning architecture is constructed to update the service placement agent at the beginning of each long update cycle and the task scheduling agent at the beginning of each short update cycle. The two-time-scale Markov decision process can be expressed as in represents the state space of the task scheduling agent, represents the state space of the service placement agent, represents the action space of the task scheduling agent, represents the action space of the service placement agent, represents the state transition probability, Represents the reward function. Specifically, at time slot t, the state space of the task scheduling agent is defined as: The state space of the service placement agent is defined as: Among them, L h Indicates the length of historical information. The action space of the task scheduling agent is defined as: The action space of the service placement agent is defined as: The reward function of the task scheduling agent is defined as: r s (t)=-[T task +T data +λ(E user +E sensor )] The reward function of the service placement agent is defined as the sum of rewards over T short time periods:

6. A method for joint service placement, task scheduling and resource allocation in a multi-sensor multi-user edge computing network according to claim 1, characterized in that: The step S5 comprises: (1) Training the service placement and task offloading strategy composed of the dual-agent neural network. At each time slot t, according to the current state s s (t),s l (t), decision action a s (t),a l (t), and calculate the reward r s (t),r l (t). The specific steps are: Step 1: Randomly initialize the parameters of the dual-agent neural network and initialize the time slot t to 1; Step 2: Observe the system status l (t), s l (t) Input the service placement agent and obtain the service placement decision a l (t); Step 3: Observe the system status s (t), s s (t) and a l (t) Input the task scheduling agent and obtain the task scheduling decision a s (t); Step 4: Solve the optimal sensor and user power allocation decision based on the Dinkelbach algorithm Step 5: Place decisions based on services a l (t) and task scheduling decision a s (t), using KKT conditions to solve the optimal edge server computing resource allocation decision and transmission rate allocation decisions Step 6: Calculate the task scheduling agent reward r based on the results of steps 3, 4, and 5 s (t) and the serving agent reward r l (t); Step 7: Use experience to train and update the task scheduling agent and service placement agent; Step 8: Increase the value of t by 1 and determine whether the current episode has ended. If so, reset the system state and start a new episode. Step 9: Determine whether the training process is finished. If so, end the training process; otherwise, go to step 2. (2) After the task scheduling agent and the service placement agent are trained, the dual agent is used to make service placement and task scheduling decisions at each time slot t. The specific steps are as follows: Step 1: The edge server senses the number of user tasks arriving, the channel conditions between the edge server and the user, the wired transmission rate between the edge server and the cloud server, and obtains the computing power of the edge server and the cloud server. Step 2: The service placement agent outputs a service placement decision based on the system status The task scheduling agent outputs task scheduling decisions based on the system status Step 3: Solve the optimal user and sensor transmit power decision based on the Dinkelbach algorithm Step 4: Make placement decisions based on services and task scheduling decisions Using KKT conditions to solve the optimal edge server computing resource allocation decision and transmission rate allocation decisions Step 5: The user offloads the task according to the task scheduling decision, the sensor uploads the service data according to the service placement decision, the edge server and the cloud server receive the sensor data, process the user task, and return the task result to the user.

Images