Distributed task unloading and service caching joint optimization method and device

Abstract

The invention provides a distributed task unloading and service caching joint optimization method and device. The method comprises the steps of obtaining a computing task of a user associated with each distributed node in a current time slot; determining task processing time delay based on a processing mode of the distributed node on the computing task of the user; constructing an optimization problem which comprises a target function and a plurality of constraint conditions, wherein the target function aims at minimizing the average value of task processing time delays of all distributed nodes completing calculation tasks of all users; each distributed node is regarded as an intelligent agent with independent learning and decision-making functions; and solving the optimization problem by adopting a multi-agent-based deep reinforcement learning algorithm to obtain a task unloading strategy and a service caching strategy which are globally optimal in the current time slot. According to the method, the algorithm convergence speed can be greatly increased, the problem of huge data volume during joint decision making can be solved, and smaller task execution time delay can be obtained.

Description

Technical Field

[0001] This application relates to the field of mobile communication technology, and in particular to a method and apparatus for joint optimization of distributed task offloading and service caching. Background Technology

[0002] In existing solutions to the joint optimization problem of task offloading and service caching in distributed systems, the joint optimization problem is usually decomposed into a task offloading optimization problem and a service caching optimization problem. The two independent sub-problems are solved by heuristic alternating optimization or hierarchical optimization, and the sub-strategies are solved separately.

[0003] Since the task unloading process and service caching process are interconnected, their decision-making processes influence each other. Decomposing the problem into subproblems fails to adequately consider the coupling between them, making it impossible to reach the globally optimal solution. Summary of the Invention

[0004] In view of this, this application provides a distributed task offloading and service caching joint optimization method and apparatus to solve the above-mentioned technical problems.

[0005] In a first aspect, embodiments of this application provide a distributed task offloading and service caching joint optimization method, applied to a distributed system with a cloud server and multiple distributed nodes, comprising: Get the computing tasks associated with each user on each distributed node within the current time slot; The task processing latency is determined based on how distributed nodes handle user computing tasks. The optimization problem is constructed, including: an objective function that aims to minimize the average task processing latency of all distributed nodes completing all users' computing tasks, and multiple constraints; Each distributed node is treated as an intelligent agent with independent learning and decision-making capabilities; the optimization problem is solved using a multi-agent deep reinforcement learning algorithm to obtain the globally optimal task offloading strategy and service caching strategy within the current time slot.

[0006] Secondly, embodiments of this application provide a distributed task offloading and service caching joint optimization apparatus, applied to a distributed system having a cloud server and multiple distributed nodes, comprising: The acquisition unit is used to acquire the computing tasks of multiple users associated with each distributed node in the current time slot; The determination unit is used to determine the processing method of user computing tasks based on distributed nodes and to determine the task processing latency; The building blocks are used to construct optimization problems, including: an objective function that aims to minimize the average task processing latency of all distributed nodes completing all users' computing tasks, and multiple constraints; The solution unit treats each distributed node as an agent with independent learning and decision-making capabilities; it uses a multi-agent deep reinforcement learning algorithm to solve the optimization problem and obtain the globally optimal task offloading strategy and service caching strategy within the current time slot.

[0007] Thirdly, embodiments of this application provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method of embodiments of this application.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the methods of embodiments of this application.

[0009] Fifthly, embodiments of this application provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the method of embodiments of this application.

[0010] This application can significantly accelerate the convergence speed of the algorithm and solve the problem of large data volume in joint decision-making, and can achieve smaller task execution latency. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0012] Figure 1 A flowchart of the distributed task offloading and service caching joint optimization method provided in the embodiments of this application; Figure 2 A schematic diagram of a distributed system provided in an embodiment of this application; Figure 3 Functional structure diagram of the distributed task offloading and service caching joint optimization device provided in the embodiments of this application; Figure 4 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0014] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0015] First, a brief introduction to the design concept of the embodiments of this application will be given.

[0016] A distributed system comprises multiple distributed nodes with certain computing and storage resources. By caching corresponding service environments, such as operating systems and databases, computational tasks can be offloaded to multiple nodes for execution. This parallel processing capability enables distributed systems to handle large amounts of data and complex computational tasks. However, due to storage space limitations, each distributed node can only select a portion of the system's service environment for storage. Therefore, the execution latency of the computational task depends on both the effectiveness of the task offloading strategy and the quality of the service environment caching strategy.

[0017] In existing solutions to the joint optimization problem of task offloading and service caching in distributed systems, the joint optimization problem is usually decomposed into a task offloading optimization problem and a service caching optimization problem, and the sub-problems are solved by heuristic alternating optimization or hierarchical optimization. An iterative cache update algorithm based on Gibbs sampling is proposed to formulate a service caching strategy. At the same time, a heuristic load scheduling algorithm is proposed to formulate a load scheduling strategy. The two algorithms are used to iteratively solve the joint optimization problem of task offloading and service caching.

[0018] In the context of connected vehicles, tasks are divided into several subtasks, and task nodes are constructed based on the unloading priority of each subtask. For each vehicle, an unloading decision maximizing the unloading utility is formulated according to the unloading priority order corresponding to the task node and the task execution constraints. Furthermore, a service caching strategy is developed for the server executing the task based on service popularity. A two-stage heuristic algorithm ensures the execution of unloading decisions for each subtask.

[0019] Since the task unloading process and service caching process are interconnected, their decision-making processes influence each other. Decomposing the problem into subproblems fails to adequately consider the coupling between them, making it impossible to reach the globally optimal solution.

[0020] To address the aforementioned technical challenges, this application proposes a multi-agent deep reinforcement learning (MDRL) approach, employing a dual-actor-shared critic network architecture to solve a joint optimization strategy for task offloading and service caching, thereby minimizing the average task execution latency of the system. This method models the task offloading and service caching process in a distributed scenario as a Markov game and solves it using the MDRL algorithm. To address the coupling issue in the task offloading and service caching processes, a dual-actor network is designed to enable independent outputs of two actions within the same agent, while a shared critic network facilitates the collaborative optimization of the two strategies.

[0021] In the joint optimization problem of task offloading and service caching in distributed scenarios, the system model provides a variety of computational tasks, and the distributed nodes need to process a massive amount of data when making decisions on task offloading and service caching, and the system is constantly changing. Traditional heuristic algorithms struggle to converge when solving such problems and cannot find the optimal joint strategy to maximize the gains. The deep reinforcement learning algorithm used in this application can learn task features through continuous interaction with complex environments to achieve optimal decisions, making it suitable for solving optimization problems in uncertain environments; the powerful learning ability of deep neural networks can greatly accelerate the algorithm's convergence speed and solve the problem of massive data volume in joint decision-making.

[0022] Secondly, existing solutions typically decompose the joint optimization problem into two interrelated subproblems and solve the sub-policies separately. However, this independent policy-solving approach ignores the coupling relationship between the two policy optimization processes and is inefficient in complex scenarios, making it difficult to obtain the optimal joint policy. This application designs two policy networks within the same intelligent entity to handle task offloading and service caching decisions separately; simultaneously, a shared evaluation network ensures the collaborative optimization of the two policies to achieve the lowest average system task execution latency.

[0023] After introducing the application scenarios and design concepts of the embodiments of this application, the technical solutions provided by the embodiments of this application will be described below.

[0024] like Figure 1 As shown, this application provides a distributed task offloading and service caching joint optimization method, applied to a distributed system with one cloud server and multiple distributed nodes, including: Step 101: Obtain the computation tasks of multiple users associated with each distributed node in the current time slot; Step 102: Determine the task processing latency based on the way distributed nodes handle user computing tasks; Step 103: Construct the optimization problem, including: an objective function that aims to minimize the average task processing latency of all distributed nodes completing all users' computing tasks, and multiple constraints; Step 104: Treat each distributed node as an intelligent agent with independent learning and decision-making functions; use a multi-agent deep reinforcement learning algorithm to solve the optimization problem and obtain the globally optimal task offloading strategy and service caching strategy in the current time slot.

[0025] Then, each distributed node is controlled to perform corresponding task unloading and service environment update actions based on the globally optimal task unloading strategy and service caching strategy.

[0026] Specifically, distributed systems share storage and computing resources to collaboratively offload computing tasks issued by end users across multiple distributed nodes to meet users' low-latency service requirements. However, in this distributed collaborative service scenario, task offloading and service caching are coupled. Even if a node has sufficient computing and data storage resources, it cannot execute the corresponding computing task if it does not cache the relevant services. In other words, the task offloading strategy and the service environment caching strategy both affect the efficiency of task execution.

[0027] like Figure 2 As shown, the distributed system includes a cloud server. and Distributed nodes Indicates the first Distributed nodes (edge ​​devices) with computing power of The maximum storage capacity for the service environment is Maximum data storage capacity is The maximum number of parallel computing tasks is . express A collection of collaborating nodes. A service area of ​​a distributed node. There are M terminal users in the domain. Provided to individual end users Various task computing services. Indicates the first One user. Indicates the first Various computational tasks. Among them Indicates the size of the uploaded data. This indicates the number of CPU cycles required for the calculation. This indicates the services required to perform the task, and the storage space required to cache the environment for these services is... .

[0028] Divide continuous time into A separate time slot, Indicates the first There are 1 time slot, and the size of each time slot is fixed at 1. The user's location is fixed within each time slot.

[0029] No. individual users Sending task request frequency Represented as:

[0030] in, for Ranking of request frequency among all computing tasks in the system. As the skewness factor, when The larger the value, the more concentrated the demand.

[0031] Communication between distributed nodes and between nodes and the cloud is achieved using wired transmission. individual users exist Associated nodes Wireless communication is used between them, and the data transmission rate is... Represented as:

[0032] in, express The channel bandwidth; Indicates in and The number of associated end users; express and The channel gain between them is a constant. express Transmission power, Indicates noise power.

[0033] express exist The set of computational tasks currently being executed within the system. Maintain a queue of tasks waiting to be executed. This includes tasks that need to be executed locally and those that need to be offloaded from the collaboration edge.

[0034] When distributed nodes have remaining computing resources, then Team leader mission Allocate computing resources, assuming the waiting time for computing resource allocation is... The allocation of computing resources must meet the following constraints:

[0035] Distributed nodes need to perform task offloading and matching based on resource usage status and service environment storage status, that is, offloading computing tasks to local nodes, collaborative nodes or cloud servers that meet the task execution conditions for execution.

[0036] This embodiment addresses the joint optimization problem of task offloading and service caching in distributed scenarios. Due to limitations in their own computing resources and content storage, end users choose to send computing tasks to a distributed system for collaborative execution to achieve lower task execution latency.

[0037] In some embodiments, the objective function is:

[0038] in, The number of distributed nodes. The number of users associated with each distributed node. For the nth distributed node, For the m-th user, Indicates the current time slot Inner and nth distributed node Associated user set; For the nth distributed node The task processing latency of the associated m-th user It is a function.

[0039] In some embodiments, when the distributed node associated with the m-th user When the service environment required to execute a user's computing task is stored, the computing task will be executed directly on the local machine. Then the task processing delay for the m-th user for:

[0040] in, For the time slot size, To calculate the task's waiting time for execution; Let m be the size of the computation task for the m-th user. For data transmission rate; Time is needed to wait for computing resources to be allocated; The number of CPU cycles required to compute the computation task of the m-th user; For distributed nodes The maximum computing power.

[0041] In some embodiments, when the distributed node associated with the m-th user When the computing resources are insufficient or the service environment required to execute a user's computing task is not cached, the computing task associated with the m-th user will be offloaded to a collaborative node with execution capabilities. Execution at the location; Then the task processing delay for the m-th user for:

[0042] in, For the time slot size, To calculate the task's waiting time for execution; Let m be the size of the computation task for the m-th user. This refers to the transmission rate between distributed nodes. Time is needed to wait for computing resources to be allocated; The number of CPU cycles required to compute the computation task of the m-th user; For distributed nodes The maximum computing power.

[0043] In some embodiments, when the computing resources of all distributed nodes are scarce or the service environment required to execute the user's computing task is not cached, the computing task associated with the m-th user is offloaded to the cloud server for execution. Then the task processing delay for the m-th user for:

[0044] in, For the time slot size, To calculate the task waiting time, This refers to the communication time between distributed nodes and the cloud server.

[0045] In some embodiments,

[0046] in, It is a binary variable: express In the current time slot Execution cached Required service environment ; This indicates that it is not cached; When performing service environment caching, the storage capacity limit of the node service environment must be met:

[0047] in, Maximum service environment storage capacity; Maintain a task request queue The received computing task unloading requests are sequentially placed into The back of the line. At the beginning of each time slot from The head of the line took out a mission request. And based on the current system resource usage status and service environment cache status, Choose to uninstall. It is a binary variable. Indicates in Start time Tasks that need to be scheduled Uninstall to Execution at the location; This indicates that the task will not be uninstalled. (Due to the task execution location...) If the service environment required to execute the task is unique and needs to be cached in advance, then the task unloading strategy is... Constraints must be met:

[0048] in, It is a binary variable: Represents distributed nodes The task needs to be completed. Unload to collaborative distributed nodes Execution at the location; This means do not uninstall. Represents a distributed set of nodes. This refers to a cloud server.

[0049] In some embodiments, a multi-agent deep reinforcement learning algorithm is used to solve the optimization problem to obtain a globally optimal task unloading and service caching strategy; including: Initialize the Actor-Offload network parameters, Actor-Cache network parameters, and Shared-Critic network parameters for each agent; At each time step, perform the following steps until the maximum time step is reached: By inputting the local observations of each agent at the current time step into the Actor-Offload network and the Actor-Cache network, the probability distributions of task offloading actions and service cache update actions are obtained. Task unloading actions are obtained by probability sampling from the probability distribution of task unloading actions, and service cache update actions are obtained by probability sampling from the probability distribution of service cache update actions. Perform task unloading and service cache update actions to obtain the reward function and local observations for the next time step; Calculate the average discounted reward at the current time step based on the execution trajectories of all agents; Based on the execution trajectories of all agents, calculate the action advantage function value of each agent at the current time step, and put the advantage function value and the agent's execution trajectory into the experience buffer pool; The data in the experience buffer pool is randomly sorted, and a batch of data of size is selected from it. The dataset; For each agent, update the parameters of the Shared-Critic network using gradient descent; for each agent, update the parameters of the Actor-Offload network and the Actor-Cache network using gradient ascent.

[0050] In some embodiments, the reward function includes: the reciprocal of the execution latency of the scheduled task at the current time step; the sum of the reciprocal of the execution latency of the service environment moved into the cache at the current time step from the next time step to the time before being moved out of the cache again; and the sum of the reciprocal of the execution latency of the service environment moved out of the cache at the current time step from the next time step to the time before being moved into the cache again.

[0051] Specifically, in the Markov game model, each distributed node can be regarded as an agent with independent learning and decision-making functions. The time step, state space, action space and reward function of each agent in the Markov game model are defined in turn.

[0052] Time step: The end time of each time slot is considered a time step. At the end of each time step, all nodes simultaneously make decisions regarding task unloading and service caching. A time step is a collection of discrete moments, denoted as... Initial moment Each agent randomly selects a portion of the service environment and caches it locally.

[0053] State Space: Distributed nodes need to execute appropriate offloading strategies for user-sent task offloading requests at each time step based on the system's computing resource usage status and service ring cache status. Simultaneously, they need to update the local service environment cache status to handle computing task offloading requests within a certain time period. Both actions need to be performed by the same agent, and the information considered in action selection differs. Therefore, [the following is missing from the original text: "will..."] exist The joint state space is defined as follows: Among them, the uninstallation status Recorded as:

[0054] express The status information considered when making a task unloading decision includes... , and They represent exist The service environment cache status, remaining data storage capacity, and remaining number of parallel tasks are as follows. , and Representing cooperative intelligent agents exist The service environment cache state set, the remaining data storage capacity set, and the remaining task parallel count set are as follows: express exist Status information of computational tasks requiring scheduling; cache status express exist The task request was received in the previous time slot. A set of frequencies. When When a task uninstallation action or a service cache update action is required, the status information of the other action is set to zero.

[0055] Action Space: Since each agent can perform different scheduling actions in response to different service requests, this proposal provides a space to accommodate different types of decision-making needs. exist The task unloading requirement and the service environment cache update requirement are designed with action spaces, represented as follows: .in express exist The action space for task unloading is below. express exist The action space for updating the service environment cache.

[0056] Reward Function: The reward function quantifies the agent's behavior in the environment and provides feedback signals. The agent updates its policy function based on these reward signals by interacting with the environment to maximize the expected cumulative reward. Definition exist The reward function below It consists of three parts: (1) the task to be scheduled at the current time step. (2) The execution delay reciprocal; (3) The service environment that is moved into the cache at the current time step. The sum of the reciprocals of the task execution latency from the next time step to the time when the cache is requested again; (3) the service environment that is removed from the cache in this time step. The sum of the reciprocals of the task execution latency from the next time step until the request is made again before being moved into the cache. The first two parts are given a positive reward, and the third part is given a negative reward.

[0057] In the Markov game model, it can be observed that each agent needs two different policies for task unloading actions and service environment update actions. The approach of using a single policy network maintained by agents in general multi-agent deep learning algorithm frameworks cannot meet these requirements.

[0058] This application designs a MA-DASC network based on multi-agent deep reinforcement learning. In this network, each Actor network of the agents learns a policy independently and generates actions by focusing on state information of different dimensions, thereby achieving separation of the policy learning process. Simultaneously, the network achieves collaborative optimization of multi-policy networks in multi-agent scenarios through global value function estimation via a shared Critic network.

[0059] Each agent in the MA-DASC network Maintain a parameter as State value function The network input is An agent at the current time step Local observation below Global state information obtained by splicing Output global state value It also participates in the parameter updates of the policy network.

[0060] The value network loss function is expressed as:

[0061] in, express An agent at the current time step The average discount reward obtained is calculated using the following formula:

[0062] in, is the reward discount factor, and is a constant.

[0063] The Actor-Offload network and Actor-Cache network maintain parameters as follows: and policy function and .Will exist Local observation information below Input the Actor-Offload or Actor-Cache network and output the current time step. Probability distribution of all executable task unloading actions and service cache update actions and .

[0064] Based on the same inventive concept, embodiments of this application provide a distributed task offloading and service caching joint optimization apparatus, applied to a distributed system with one cloud server and multiple distributed nodes, see below. Figure 3 As shown, the distributed task offloading and service caching joint optimization apparatus 200 provided in this application embodiment includes at least: The acquisition unit 201 is used to acquire the computing tasks of multiple users associated with each distributed node in the current time slot; The determining unit 202 is used to determine the task processing delay based on the processing method of the user's computing tasks by the distributed nodes; Construction unit 203 is used to construct the optimization problem, including: an objective function aimed at minimizing the average task processing latency of all distributed nodes completing all users' computing tasks, and multiple constraints; Solving unit 204 is used to treat each distributed node as an intelligent agent with independent learning and decision-making functions; it uses a multi-agent deep reinforcement learning algorithm to solve the optimization problem and obtain the globally optimal task offloading strategy and service caching strategy in the current time slot.

[0065] It should be noted that the principle of the distributed task offloading and service cache joint optimization device 200 provided in this application embodiment to solve the technical problem is similar to the method provided in this application embodiment. Therefore, the implementation of the distributed task offloading and service cache joint optimization device 200 provided in this application embodiment can refer to the implementation of the method provided in this application embodiment, and the repeated parts will not be described again.

[0066] Based on the same inventive concept, embodiments of this application also provide an electronic device, such as... Figure 4 As shown, it includes a memory and a processor. The memory stores an executable program, and the processor executes the executable program to implement the steps of the distributed task offloading and service caching joint optimization method provided in the above embodiments.

[0067] The aforementioned processor can be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0068] Since the electronic device described in this application embodiment is an electronic device equipped with a memory that implements the distributed task offloading and service cache joint optimization method disclosed in this application embodiment, those skilled in the art can understand the structure and variations of the electronic device described in this application embodiment based on the distributed task offloading and service cache joint optimization method described in this application embodiment, and therefore will not be described again here.

[0069] This application also provides a computer-readable storage medium storing a computer program thereon, which, when run by a processor, implements the steps of the distributed task offloading and service caching joint optimization method provided in the above embodiments.

[0070] The storage medium in this embodiment may be included in an electronic device; or it may exist independently and not be assembled into an electronic device. The storage medium carries one or more computer programs, which, when executed, implement the steps of the data flow behavior identification method based on multi-source logs provided in the above embodiment.

[0071] It should be understood that the various solutions in this embodiment have the same technical effects as those in the above method embodiments, and will not be repeated here.

[0072] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. Optionally, specific examples in this embodiment can refer to the examples described in any embodiment of this application, which will not be repeated here. Obviously, those skilled in the art should understand that the various modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular hardware and software combination.

[0073] This application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the distributed task offloading and service caching joint optimization method provided in the above embodiments.

[0074] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions targeted in the blocks may occur in a different order than those targeted in the drawings. For example, two consecutively represented blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0075] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. Multitasking and parallel processing may be advantageous in certain environments. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this application. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

Claims

1. A distributed task offloading and service caching joint optimization method, applied to a distributed system with one cloud server and multiple distributed nodes, characterized in that, include: Get the computing tasks associated with each user on each distributed node within the current time slot; The task processing latency is determined based on how distributed nodes handle user computing tasks. The optimization problem is constructed, including: an objective function that aims to minimize the average task processing latency of all distributed nodes completing all users' computing tasks, and multiple constraints; Each distributed node is treated as an intelligent agent with independent learning and decision-making capabilities; the optimization problem is solved using a multi-agent deep reinforcement learning algorithm to obtain the globally optimal task offloading strategy and service caching strategy within the current time slot.

2. The method according to claim 1, characterized in that, The objective function is: in, The number of distributed nodes. The number of users associated with each distributed node. For the nth distributed node, For the m-th user, Indicates the current time slot Inner and nth distributed node Associated user set; For the nth distributed node The task processing latency of the associated m-th user It is a function.

3. The method according to claim 2, characterized in that, When the distributed node associated with the m-th user When the service environment required to execute a user's computing task is stored, the computing task will be executed directly on the local machine. Then the task processing delay for the m-th user for: in, For the time slot size, To calculate the task's waiting time for execution; Let m be the size of the computation task for the m-th user. For data transmission rate; Time is needed to wait for computing resources to be allocated; The number of CPU cycles required to compute the computation task of the m-th user; For distributed nodes The maximum computing power.

4. The method according to claim 2, characterized in that, When the distributed node associated with the m-th user When the computing resources are insufficient or the service environment required to execute a user's computing task is not cached, the computing task associated with the m-th user will be offloaded to a collaborative node with execution capabilities. Execution at the location; Then the task processing delay for the m-th user for: in, For the time slot size, To calculate the task's waiting time for execution; Let m be the size of the computation task for the m-th user. This refers to the transmission rate between distributed nodes. Time is needed to wait for computing resources to be allocated; The number of CPU cycles required to compute the computation task of the m-th user; For distributed nodes The maximum computing power.

5. The method according to claim 2, characterized in that, When all distributed nodes are short of computing resources or do not have cached the service environment required to execute user computing tasks, the computing task associated with the m-th user will be offloaded to the cloud server for execution. Then the task processing delay for the m-th user for: in, For the time slot size, To calculate the task waiting time, This refers to the communication time between distributed nodes and the cloud server.

6. The method according to claim 2, characterized in that, The constraints include: in, It is a binary variable: express In the current time slot Execution cached Required service environment ; This indicates that it is not cached; Indicates the first Various computational tasks, among which Indicates the size of the uploaded data. This indicates the number of CPU cycles required for the calculation. This indicates the service environment required to execute the task, and the storage space required to cache this service environment is... ; in, Maximum service environment storage capacity; in, It is a binary variable: Represents distributed nodes The task needs to be completed. Unload to collaborative distributed nodes Execution at the location; This means do not uninstall. Represents a distributed set of nodes. This refers to a cloud server.

7. The method according to claim 1, characterized in that, The optimization problem is solved using a multi-agent deep reinforcement learning algorithm to obtain the globally optimal task unloading strategy and service caching strategy; including: Initialize the Actor-Offload network parameters, Actor-Cache network parameters, and Shared-Critic network parameters for each agent; At each time step, perform the following steps until the maximum time step is reached: By inputting the local observations of each agent at the current time step into the Actor-Offload network and the Actor-Cache network, the probability distributions of task offloading actions and service cache update actions are obtained. Task unloading actions are obtained by probability sampling from the probability distribution of task unloading actions, and service cache update actions are obtained by probability sampling from the probability distribution of service cache update actions. Perform task unloading and service cache update actions to obtain the reward function and local observations for the next time step; Calculate the average discounted reward at the current time step based on the execution trajectories of all agents; Based on the execution trajectories of all agents, calculate the action advantage function value of each agent at the current time step, and put the advantage function value and the agent's execution trajectory into the experience buffer pool; The data in the experience buffer pool is randomly sorted, and a batch of data of size is selected from it. The dataset; For each agent, update the parameters of the Shared-Critic network using gradient descent; for each agent, update the parameters of the Actor-Offload network and the Actor-Cache network using gradient ascent.

8. The method according to claim 7, characterized in that, The reward function includes: the reciprocal of the execution latency of the scheduled task at the current time step; the sum of the reciprocal of the execution latency of the service environment moved into the cache at the current time step from the next time step to the time before being moved out of the cache again; and the sum of the reciprocal of the execution latency of the service environment moved out of the cache at the current time step from the next time step to the time before being moved into the cache again.

9. A distributed task offloading and service caching joint optimization device, applied to a distributed system with one cloud server and multiple distributed nodes, characterized in that, include: The acquisition unit is used to acquire the computing tasks of multiple users associated with each distributed node in the current time slot; The determination unit is used to determine the processing method of user computing tasks based on distributed nodes and to determine the task processing latency; The building blocks are used to construct optimization problems, including: an objective function that aims to minimize the average task processing latency of all distributed nodes completing all users' computing tasks, and multiple constraints; The solution unit treats each distributed node as an agent with independent learning and decision-making capabilities; it uses a multi-agent deep reinforcement learning algorithm to solve the optimization problem and obtain the globally optimal task offloading strategy and service caching strategy within the current time slot.

10. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as claimed in any one of claims 1-8.

Images