Edge cloud network adaptive multi-time scale model resource scheduling system and method
By using an edge cloud network adaptive multi-timescale model resource scheduling system, and leveraging a hierarchical agent structure and a multi-agent reinforcement learning framework, model deployment and task offloading decisions are decoupled. This solves the problem of poor adaptability to dynamic environmental changes in edge cloud networks, achieves a balance between cost and latency, reduces operating costs, and improves system adaptability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XI AN JIAOTONG UNIV
- Filing Date
- 2026-03-13
- Publication Date
- 2026-06-05
AI Technical Summary
Existing model resource scheduling schemes are difficult to adapt to dynamic environmental changes in edge cloud networks, do not fully consider the time-scale coupling of model deployment and task offloading decisions, and have complex policy coupling when reinforcement learning is introduced.
An edge cloud network adaptive multi-timescale model resource scheduling system is adopted. A hierarchical agent structure is constructed through a hierarchical partially observable Markov decision process (H-POMDP). A multi-agent reinforcement learning framework with centralized training and distributed execution is used to decouple model deployment and task offloading decisions, and optimization decisions are made at both large and small timescales.
While ensuring low latency for tasks, it reduced operating costs, improved the system's adaptability and stability, achieved a balance between cost and latency, reduced operating costs by approximately 55%, and had an online inference time of only 15-17 milliseconds.
Smart Images
Figure CN122152532A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of interdisciplinary technology of network systems and artificial intelligence, and specifically relates to an adaptive multi-timescale model resource scheduling system and method for edge cloud networks. Background Technology
[0002] With the rapid development of large language models (LLMs), deploying them in edge cloud networks to provide low-latency, highly available intelligent services has become an important trend. However, LLM models have a huge number of parameters and are computationally intensive, placing extremely high demands on the storage and computing resources of edge nodes. In edge cloud networks, model resource scheduling involves two key decision levels: first, when to update the model deployment rules, i.e., changing the rules on which edge nodes cache which LLM models; and second, when to update the task offloading rules, i.e., changing the rules on which nodes with the corresponding deployed models should route real-time inference tasks for execution. These two types of decisions have significant differences and strong coupling in terms of time scale: model deployment rules are high-overhead, long-term decisions involving model downloads, API loading, etc., and should not be changed frequently; while task offloading rules require sub-second response times to adapt to dynamically changing loads and network conditions.
[0003] Currently, two main model resource scheduling schemes are employed: static multi-timescale scheduling and dynamic threshold multi-timescale scheduling. Static multi-timescale scheduling sets different fixed update intervals for different decision levels (e.g., minute-level updates for model deployment, second-level updates for task unloading). This reduces costs to some extent, but fixed intervals cannot adapt to dynamic environmental changes, and a suboptimal trade-off between cost and latency still exists during load fluctuations. Dynamic threshold multi-timescale scheduling sets task update thresholds for the current edge side. When the task load exceeds the threshold, an update action is triggered, and the algorithm updates the current scheduling strategy. This also reduces costs to some extent, but fixed thresholds also cannot adapt to dynamic environmental changes, leading to extremely high latency during periods of calm traffic, harming user experience and service provider interests. Furthermore, most existing methods handle model deployment or task unloading in isolation, failing to fully consider the coupling between the two on different time scales and lacking systematic modeling of operator economic benefits. Although some research has attempted to introduce reinforcement learning methods for resource scheduling, it faces problems such as complex policy coupling, partially observable environment, and lack of explicit modeling of model loading latency in multi-timescale decision-making scenarios. In summary, current model resource scheduling schemes are not well adapted to the dynamic changes in the edge cloud network environment; the deployment of multiple isolated models and the decision-making of task offloading do not fully consider the strong coupling between the two time scales, and the policy coupling of reinforcement learning in multi-time scale decision-making scenarios becomes complicated. Summary of the Invention
[0004] This invention provides an adaptive multi-timescale model resource scheduling system and method for edge cloud networks. The purpose is to solve the problems in current model resource scheduling schemes, such as difficulty in adapting to the dynamic changes in the edge cloud network environment; multiple isolated processing of model deployment and task offloading decisions, which do not fully consider the strong coupling between the two time scales; and the complex policy coupling caused by introducing reinforcement learning in multi-timescale decision-making scenarios.
[0005] To achieve the above objectives, the present invention adopts the following technical solution: This invention provides an adaptive multi-timescale model resource scheduling system for edge cloud networks, comprising: Network monitoring module: Collects status information from edge servers and the cloud, and transmits the status information to the model resource scheduling base layer module, the model resource scheduling service layer module, and the H-POMDP construction module respectively; H-POMDP building module: It formalizes the multi-timescale model resource scheduling problem into a hierarchical partially observable Markov decision process, and constructs a hierarchical agent structure containing a basic layer agent for model resource scheduling and a service layer agent for model resource scheduling. The basic layer agent and the service layer agent are respectively responsible for the decision of scheduling update timing and scheduling rules. The core controller module defines each edge server as an independent intelligent agent, adopts a training paradigm of centralized training and distributed execution, utilizes the computing power of the edge side to make decentralized decisions, and drives the intelligent agents of the basic layer of model resource scheduling and the intelligent agents of the service layer of model resource scheduling to learn the optimal strategy and generate scheduling update actions. Model resource scheduling base layer module: Receives scheduling update actions, makes decisions on model deployment and update strategies on a large time scale, follows the storage capacity constraints of edge nodes during execution, and models the impact of model download and API loading latency on task availability; Model resource scheduling service layer module: Receives scheduling update actions, makes decisions on real-time task offloading strategies on a small time scale, and follows task uniqueness, model availability and link bandwidth constraints during execution.
[0006] In some implementations, the network monitoring module collects the task request status of the task requests received by each edge server and the required LLM model type, the resource status of each edge server, the current available storage space, computing resources and communication bandwidth, and the model deployment status of each node, including the LLM models currently cached and their deployment time and availability status. The network monitoring module also transmits the collected status information to the core controller module.
[0007] Furthermore, in the H-POMDP building module, the binary decision signal issued by the model resource scheduling base layer agent in a time slot represents whether a model deployment rule update is needed; the binary decision signal issued by the model resource scheduling service layer agent in a micro time slot represents whether a task unloading rule update is needed; the historical information sequence at the macro time scale covers multiple micro decision cycles, and the policy function of the model resource scheduling base layer agent determines whether to trigger a deployment rule update based on the macro historical decisions; the historical sequence at the micro time scale, i.e., the policy function of the model resource scheduling service layer agent, determines whether to update the unloading rule based on the current micro history and model availability status.
[0008] Furthermore, the decision-making process of the H-POMDP building blocks is defined by the following joint decision-making formula: .
[0009] in, For the model resource scheduling base layer intelligent agent n In the time slot t The binary decision signal emitted Intelligent agents serving the model resource scheduling layer n In the time slot t The binary decision signal emitted This is the base layer strategy function for model resource scheduling. This is the strategy function for the model resource scheduling service layer. This is a sequence of historical information for the foundational layer of model resource scheduling. This is a sequence of historical information for the model resource scheduling service layer. For the model k In the current time slot t Availability indicator.
[0010] Furthermore, the model resource scheduling base layer agent operates on minute-level time slots, determining the optimal time to update deployment rules based on long-term network trends, model access frequency, and the time interval since the last update; the model resource scheduling service layer agent operates on second-level time slots, deciding whether to update unloading rules based on current refined network observations and the availability status of upper-layer models.
[0011] In some implementations, the specific state design of the model resource scheduling base layer is as follows: ; in, For the model resource scheduling base layer agent in t The status information of the time slot, for The dimension vector represents a vector of historical request patterns. It is a direct prediction of the distribution of future requests and is obtained by statistical analysis of the request history over a period of time using the moving average method. for 0-1 vector representation model Has it already been deployed on the current edge server? superior; The number of suspended tasks; For resource utilization rate; This represents the time elapsed since the last deployment update.
[0012] In some implementations, the state of the model resource scheduling base layer agent in the model resource scheduling base layer module is composed of historical request patterns, current model deployment status, system load status, and time interval information. The specific state design of the model resource scheduling service layer is as follows: ; in, Representation model resource scheduling service layer intelligent agent n In the time slot t Status information, In the current time slot Task set arriving at edge nodes Required storage resources In the current time slot Task set arriving at edge nodes Required computing resources; In the current time slot The number of tasks that timed out on edge nodes. For the current time slot The resulting deployment costs.
[0013] Furthermore, the historical request pattern is the future model demand distribution prediction result obtained by statistical analysis of task requests from multiple past cycles using the moving average method; the system load status includes the number of suspended tasks and node resource utilization; the time interval information is the time elapsed since the last model deployment update.
[0014] In some implementations, the state of the model resource scheduling service layer agent in the model resource scheduling service layer module includes task resource requirements, deployment cost information, and task timeout status. Among them, task resource requirements are the total computing and storage resources required for the current task; deployment cost information is the model deployment cost incurred within the current macro cycle; and task timeout status is the number of tasks on the current node whose processing latency exceeds the user's tolerance threshold.
[0015] This invention also provides an adaptive multi-timescale model resource scheduling method for edge cloud networks, which is implemented based on the aforementioned adaptive multi-timescale model resource scheduling system for edge cloud networks, and includes the following steps: S1. Collect status information from edge servers and the cloud, and transmit the status information to the model resource scheduling base layer, model resource scheduling service layer and H-POMDP construction module respectively; S2. The multi-timescale model resource scheduling problem is formalized into a hierarchical partially observable Markov decision process. A hierarchical agent structure is constructed, which includes a basic layer agent for model resource scheduling and a service layer agent for model resource scheduling. The basic layer agent for model resource scheduling and the service layer agent for model resource scheduling are respectively responsible for the decision on scheduling update timing and scheduling rules. S3. Define each edge server as an independent intelligent agent, adopt a training paradigm of centralized training and distributed execution, utilize the computing power of the edge side to make decentralized decisions, drive the intelligent agents of the basic layer of model resource scheduling and the intelligent agents of the service layer of model resource scheduling to learn the optimal strategy and generate scheduling update actions. S4, the model resource scheduling base layer module receives scheduling update actions, makes decisions on model deployment and update strategies on a large time scale, follows the storage capacity constraints of edge nodes during execution, and models the impact of model download and API loading latency on task availability; S5, the model resource scheduling service layer module receives scheduling update actions, makes decisions on real-time task unloading strategies on a small time scale, and follows task uniqueness, model availability and link bandwidth constraints during execution.
[0016] Compared with existing technologies, the resource scheduling system and method for adaptive multi-timescale models in edge cloud networks of the present invention have the following beneficial effects: This invention presents an adaptive multi-timescale model resource scheduling system for edge cloud networks. It proposes a two-layer asynchronous timescale scheduling framework that decouples high-overhead model deployment decisions from low-latency task offloading decisions, allowing the two layers to update adaptively at different frequencies. This mechanism balances cost and latency, overcoming the performance bottlenecks of traditional static or dynamic threshold multi-timescale scheduling. The invention explicitly models model loading latency and deployment cost in the optimization objective, enabling reinforcement learning agents to directly learn the economic losses incurred by executing scheduling actions, thus learning decision strategies that control costs without compromising user experience. Experiments show that compared to static synchronous update methods, this method can reduce operating costs by approximately 55% under the same task latency. This invention designs a joint learning mechanism based on Multi-Agent Proximal Policy Optimization (MAPPO). The basic and service layer agents in model resource scheduling achieve coordinated optimization through reward collaboration and state sharing, eliminating the need for precise environmental modeling and exhibiting good deployability. Experimental verification on a real edge cloud platform based on a Raspberry Pi cluster and on various traffic datasets shows that the present invention exhibits excellent adaptability and stability under various request modes, with an online inference time of only about 15-17 milliseconds, which is much smaller than the scheduling interval, and has real-time deployment capability. Attached Figure Description
[0017] The accompanying drawings are provided to further understand the invention and constitute a part of this invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0018] Figure 1 This is a schematic diagram of the overall architecture of an adaptive multi-timescale model resource scheduling system for edge cloud networks according to the present invention; Figure 2 This is a schematic diagram illustrating the workflow of an adaptive multi-timescale model resource scheduling method for edge cloud networks according to the present invention. Figure 3 This is a schematic diagram of a centralized training and distributed execution paradigm in an edge cloud network adaptive multi-timescale model resource scheduling system according to the present invention. Detailed Implementation
[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0020] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0021] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0022] It should be noted that the apparatus and methods disclosed in the embodiments herein can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments herein. In this regard, each block in a flowchart or block diagram may represent a module, program, or part of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system to perform the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.
[0023] In addition, the functional modules in the various embodiments of this article can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0024] How to provide an adaptive multi-timescale model resource scheduling system and method for edge cloud networks based on reinforcement learning, which can decouple model deployment and task unloading into a two-layer asynchronous timescale decision problem, and adaptively learn the scheduling update timing of each layer through a multi-agent reinforcement learning framework, so as to minimize the long-term operation cost of service providers while ensuring low task latency.
[0025] Based on this, the present invention provides an adaptive multi-timescale model resource scheduling system for edge cloud networks, comprising: Network monitoring module: Collects status information from edge servers and the cloud, and transmits the status information to the model resource scheduling base layer module, the model resource scheduling service layer module, and the H-POMDP construction module respectively; H-POMDP building module: It formalizes the multi-timescale model resource scheduling problem into a hierarchical partially observable Markov decision process, and constructs a hierarchical agent structure containing a basic layer agent for model resource scheduling and a service layer agent for model resource scheduling. The basic layer agent and the service layer agent are respectively responsible for the decision of scheduling update timing and scheduling rules. The core controller module defines each edge server as an independent intelligent agent, adopts a training paradigm of centralized training and distributed execution, utilizes the computing power of the edge side to make decentralized decisions, and drives the intelligent agents of the basic layer of model resource scheduling and the intelligent agents of the service layer of model resource scheduling to learn the optimal strategy and generate scheduling update actions. Model resource scheduling base layer module: Receives scheduling update actions, makes decisions on model deployment and update strategies on a large time scale, follows the storage capacity constraints of edge nodes during execution, and models the impact of model download and API loading latency on task availability; Model resource scheduling service layer module: Receives scheduling update actions, makes decisions on real-time task offloading strategies on a small time scale, and follows task uniqueness, model availability and link bandwidth constraints during execution.
[0026] This invention decomposes the differences and coupling relationships between model deployment decisions (the basic model resource scheduling layer) and task unloading decisions (the service model resource scheduling layer) on a time scale, decoupling them and modeling them as a hierarchical multi-objective optimization problem. A joint learning mechanism based on Multi-Agent Proximal Policy Optimization (MAPPO) is designed. The agents in the basic model resource scheduling layer make decisions on model rule updates in minute-level time slots, explicitly modeling model loading latency and deployment costs. The agents in the service model resource scheduling layer respond to real-time task unloading rule updates in second-level time slots. The two layers achieve coordinated optimization through reward collaboration and state sharing, thereby autonomously learning an adaptive scheduling strategy oriented towards maximizing long-term profits without requiring precise environmental modeling.
[0027] Specifically, in the edge cloud network adaptive multi-timescale model resource scheduling system of the present invention: The network monitoring module is used to collect the resource status, task request status, model deployment status, etc. of edge servers and the cloud, and transmit them to the model resource scheduling base layer, model resource scheduling service layer and H-POMDP construction module; The model resource scheduling base layer module acts as the long-term decision-maker of the system, responsible for deciding on model deployment and update strategies on a larger time scale (minutes / hours). It receives scheduling decision vectors from the base layer core controller and decides whether to call the preset model deployment algorithm to update the deployment rules at the beginning of the current time slot. During execution, it strictly follows the storage capacity constraints of edge nodes and explicitly models the impact of model download and API loading latency on task availability. The model resource scheduling service layer module acts as the short-term decision-maker of the system, responsible for deciding the real-time task offloading strategy on a small time scale (second level); it receives the scheduling decision vector from the service layer core controller and decides whether to call the preset task offloading algorithm to update the offloading rules in the current time slot; during execution, it strictly follows the constraints of task uniqueness, model availability, and link bandwidth to ensure that the task is completed within the latency budget. The H-POMDP building module, whose input comes from the network monitoring module, formalizes the multi-timescale model resource scheduling problem into a hierarchical partially observable Markov decision process, and constructs a hierarchical intelligent agent structure, including a basic layer intelligent agent for model resource scheduling and a service layer intelligent agent for model resource scheduling, which are responsible for making decisions on scheduling update timing and scheduling rules, respectively. The core controller module defines each edge server as an independent intelligent agent. This architecture utilizes the computing power of the edge side for decentralized decision-making, while adopting a training paradigm of centralized training and distributed execution. It is used to drive the agents at each layer to learn the optimal policy and generate scheduling update actions. In certain operating conditions, the network monitoring module of this invention specifically: collects various status information from edge servers and the cloud, including: task request status: task requests received by each edge server and the required LLM model type; resource status: the current available storage space, computing resources and communication bandwidth of each edge server; model deployment status: the LLM models currently cached by each node and their deployment time and availability status; and transmits the collected status information to the H-POMDP construction module and the core controller module as input for decision-making.
[0028] Specifically, the H-POMDP building module of this invention formalizes the multi-timescale model resource scheduling problem into a hierarchical partially observable Markov decision process. Let... For the model resource scheduling base layer intelligent agent n In the time slot t The binary decision signal issued indicates whether a model deployment strategy update is needed; let... Intelligent agents serving the model resource scheduling layer n In micro time slots The binary decision signal issued indicates whether the task offloading strategy needs to be updated. This represents a historical information sequence on a macroscopic timescale. Covering multiple micro-decision cycles, the policy function of the basic layer agent Based on macro history Whether the decision triggers a deployment rule update; the historical sequence at the micro-timescale, i.e. The policy function of the service layer agent Based on current microhistory and model availability status The decision-making process of this H-POMDP building block can be defined by the following joint decision formula: ; in, For the model resource scheduling base layer intelligent agent n In the time slot t The binary decision signal emitted Intelligent agents serving the model resource scheduling layer n In the time slot t The binary decision signal emitted This is the base layer strategy function for model resource scheduling. This is the strategy function for the model resource scheduling service layer. This is a sequence of historical information for the foundational layer of model resource scheduling. This is a sequence of historical information for the model resource scheduling service layer. For the model k In the current time slot t Availability indicator.
[0029] Model resource scheduling base layer agent: The base layer operates on larger time slots (minute-level) and is responsible for deciding when to update model deployment rules. It determines the best time to update the deployment strategy based on long-term network trends, model access frequency, and the time interval since the last update, thereby avoiding the huge loading costs caused by frequent decision-making.
[0030] Model resource scheduling service layer agent: The service layer operates at smaller time slots (e.g., seconds), responsible for deciding when to update task offloading rules. Based on current refined network observations and the availability status of upper-layer models, it decides whether to update the current task offloading rules, which determine which tasks should be offloaded to which edge nodes.
[0031] The core controller module operates by driving the learning and optimization of the entire system through reinforcement learning. Specifically, it employs a multi-agent Actor-Critic reinforcement learning framework at the edge center to provide decision-making for the H-POMDP problem, with each edge server constituting an independent agent. Each agent comprises a distributed Actor network and a centralized Critic network; the Actor network relies on its local observations... or Generate scheduling update actions or This enables efficient, low-communication-overhead decentralized decision-making. To overcome the non-stationarity of decentralized decision-making environments, the Critic network utilizes global information during the training phase to approximate the value function, thereby more accurately evaluating the long-term value of joint actions and guiding the policy updates of individual Actor networks. Its policy gradient update follows the formula: ; in, For intelligent agents n The advantage function, For intelligent agents n The policy function is calculated using generalized advantage estimation. After training, only a lightweight Actor network is used for online inference to ensure decision-making efficiency. The policy function generated by the cooperation between the core controller module and the H-POMDP building module is passed to the model resource scheduling base layer and service layer, allowing them to complete scheduling and enter the next time slot state. ; In some operating conditions, the specific state design of the model resource scheduling base layer of this invention is as follows: the state of the agent in this layer is composed of four core pieces of information, used to comprehensively perceive long-term system trends. These include: historical request patterns, which predict future model demand distribution through statistical analysis of task requests over multiple past cycles; current model deployment status, i.e., the caching status of various types of models on edge nodes; system load status, covering the number of suspended tasks and node resource utilization; and time interval information, i.e., the time elapsed since the last model deployment update. These state information collectively provide a basis for the agent to decide whether to perform high-overhead model deployment rule updates. The specific state design of the model resource scheduling base layer is as follows: ; in, For the model resource scheduling base layer agent in t The status information of the time slot, for The dimension vector represents a vector of historical request patterns. It is a direct prediction of the distribution of future requests and is obtained by statistical analysis of the request history over a past period (e.g., H-frames) using the moving average method. for 0-1 vector representation model Has it already been deployed on the current edge server? superior. The number of suspended tasks; For resource utilization rate; This represents the time elapsed since the last deployment update.
[0032] Furthermore, the specific state design of the model resource scheduling service layer of this invention is as follows: the state of the agent in this layer focuses on real-time task processing requirements and consists of the following key information: task resource requirements, i.e., the total computing and storage resources required for the current task; deployment cost information, i.e., the model deployment cost incurred in the current macro cycle, used to perceive the impact of upper-layer decisions on lower layers; and task timeout status, i.e., the number of tasks on the current node whose processing latency exceeds the user's tolerance threshold. These real-time states enable the agent to quickly decide whether to update the task offloading rules, so as to maximize the low-latency completion of tasks while controlling offloading costs. The specific state design of the model resource scheduling service layer is as follows: ; in, Representation model resource scheduling service layer intelligent agent n In the time slot t Status information, In the current time slot Task set arriving at edge nodes Required storage resources In the current time slot Task set arriving at edge nodes Required computing resources; In the current time slot The number of tasks that timed out on edge nodes. For the current time slot The resulting deployment cost and status are derived from the concatenation of the variables mentioned above.
[0033] This invention also provides an adaptive multi-timescale model resource scheduling method for edge cloud networks, which is implemented based on the aforementioned adaptive multi-timescale model resource scheduling system for edge cloud networks, and includes the following steps: S1. Collect status information from edge servers and the cloud, and transmit the status information to the model resource scheduling base layer, model resource scheduling service layer and H-POMDP construction module respectively; S2. The multi-timescale model resource scheduling problem is formalized into a hierarchical partially observable Markov decision process. A hierarchical agent structure is constructed, which includes a basic layer agent for model resource scheduling and a service layer agent for model resource scheduling. The basic layer agent for model resource scheduling and the service layer agent for model resource scheduling are respectively responsible for the decision on scheduling update timing and scheduling rules. S3. Define each edge server as an independent intelligent agent, adopt a training paradigm of centralized training and distributed execution, utilize the computing power of the edge side to make decentralized decisions, drive the intelligent agents of the basic layer of model resource scheduling and the intelligent agents of the service layer of model resource scheduling to learn the optimal strategy and generate scheduling update actions. S4, the model resource scheduling base layer module receives scheduling update actions, makes decisions on model deployment and update strategies on a large time scale, follows the storage capacity constraints of edge nodes during execution, and models the impact of model download and API loading latency on task availability; S5, the model resource scheduling service layer module receives scheduling update actions, makes decisions on real-time task unloading strategies on a small time scale, and follows task uniqueness, model availability and link bandwidth constraints during execution.
[0034] In some embodiments, Reference Figure 1 The system in this embodiment consists of a network monitoring module, a model resource scheduling base layer, a model resource scheduling service layer, an H-POMDP construction module, a core controller module, and a routing and storage management module. The network monitoring module collects the real-time status of the edge server and the cloud; the H-POMDP construction module processes the status into a hierarchical partially observable sequence; the core controller module generates scheduling update decisions based on multi-agent reinforcement learning; and the model resource scheduling base layer and service layer execute model deployment and task offloading strategies, respectively.
[0035] The system model of this invention operates on discrete time slots and includes a set of cloud servers and edge servers. Define the model type set. Each model consists of static parameters. The representations are model size, fixed deployment cost, model download latency, and API loading latency, respectively. Task set. Each task is represented as a quintuple. , respectively, represent the required model type, maximum tolerable latency, task completion revenue, required computing power resources, and required memory resources.
[0036] The optimization objective of this invention is defined as maximizing the long-run average profit of the service provider, i.e., the difference between total revenue and total cost. The revenue model is based on user satisfaction and generates revenue only when the task is completed within the delay budget; costs include model deployment costs and task unloading costs.
[0037] refer to Figure 2 This demonstrates the workflow of a dual-time-scale scheduling framework. In any time slot... First, determine whether it is within the model resource scheduling base layer time slot. t The initial time. If so, then proceed to the basic layer scheduling decision stage: Each edge server agent decides whether to update the model deployment rules based on historical states and lower-layer latency feedback. If not updated, the deployment rules of the previous frame are continued; if updated, a preset deployment algorithm (such as a greedy strategy) is executed to update the deployment rules and update the model availability status. Then the service layer scheduling decision phase begins. If the current time is not within a time slot... t At the start, the process directly enters the service layer scheduling and decision-making phase. Each edge server agent in the service layer decides whether to update the task unloading rules based on the current upper-layer state and historical states. If not updated, the original unloading rules are maintained; if updated, a preset unloading algorithm is executed to update the unloading rules, and tasks arriving at the current moment are unloaded and executed. After completion, the instantaneous rewards of each layer are calculated to guide the agents in updating.
[0038] refer to Figure 3 This demonstrates the Actor-Critic algorithm architecture of centralized training and distributed execution in the core controller module. Each edge server agent contains an independent Actor network that generates scheduled update actions based on local observations; the centralized Critic network uses global information to evaluate the value of joint actions during the training phase, guiding the policy updates of each Actor network. After training, only the lightweight Actor network is used for online inference, achieving decentralized and efficient decision-making.
[0039] In several verification experiments, the performance of this invention was compared with different baseline algorithms (static multi-timescale update and dynamic threshold multi-timescale update) under various request modes. Experimental results show that this invention performs excellently under multiple request modes. Compared to the static multi-timescale update method, the operating cost is reduced by approximately 55%; compared to the load threshold method, the average task processing latency is reduced by approximately 30 seconds. The online inference time is only about 15-17 milliseconds, demonstrating the real-time performance of this invention.
[0040] In summary, the adaptive multi-timescale model resource scheduling system and method for edge cloud networks of this invention addresses the problem of strong coupling between model deployment and task unloading in edge cloud networks on a time scale, and the inability of traditional static scheduling to balance cost and user experience. While ensuring low task latency, it reduces the operating costs of service providers and is suitable for edge cloud network scenarios that support large language model services, demonstrating a certain degree of reliability and applicability.
[0041] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Anyone skilled in the art can readily implement the present invention according to the description and above. Any modifications, alterations, or equivalent variations made using the technical content disclosed above are equivalent embodiments of the present invention. Furthermore, any modifications, alterations, or variations made to the above embodiments based on the essential technology of the present invention are still within the protection scope of the present invention.
Claims
1. An adaptive multi-timescale model resource scheduling system for edge cloud networks, characterized in that, include: Network monitoring module: Collects status information from edge servers and the cloud, and transmits the status information to the model resource scheduling base layer module, the model resource scheduling service layer module, and the H-POMDP construction module respectively; H-POMDP building module: It formalizes the multi-timescale model resource scheduling problem into a hierarchical partially observable Markov decision process, and constructs a hierarchical agent structure containing a basic layer agent for model resource scheduling and a service layer agent for model resource scheduling. The basic layer agent and the service layer agent are respectively responsible for the decision of scheduling update timing and scheduling rules. The core controller module defines each edge server as an independent intelligent agent, adopts a training paradigm of centralized training and distributed execution, utilizes the computing power of the edge side to make decentralized decisions, and drives the intelligent agents of the basic layer of model resource scheduling and the intelligent agents of the service layer of model resource scheduling to learn the optimal strategy and generate scheduling update actions. Model resource scheduling base layer module: Receives scheduling update actions, makes decisions on model deployment and update strategies on a large time scale, follows the storage capacity constraints of edge nodes during execution, and models the impact of model download and API loading latency on task availability; Model resource scheduling service layer module: Receives scheduling update actions, makes decisions on real-time task offloading strategies on a small time scale, and follows task uniqueness, model availability, and link bandwidth constraints during execution.
2. The edge cloud network adaptive multi-timescale model resource scheduling system according to claim 1, characterized in that, The network monitoring module collects the task request status of each edge server and the required LLM model type, the resource status of each edge server and the current available storage space, computing resources and communication bandwidth, and the model deployment status of each node and the current cached LLM model and its deployment time and availability status. The network monitoring module also transmits the collected status information to the core controller module.
3. The edge cloud network adaptive multi-timescale model resource scheduling system according to claim 2, characterized in that, In the H-POMDP building module, the binary decision signal emitted by the model resource scheduling base layer agent in a time slot represents whether a model deployment rule update is needed; the binary decision signal emitted by the model resource scheduling service layer agent in a micro time slot represents whether a task unloading rule update is needed; the historical information sequence at the macro time scale covers multiple micro decision cycles, and the policy function of the model resource scheduling base layer agent determines whether to trigger a deployment rule update based on the macro historical decisions; the historical sequence at the micro time scale, i.e., the policy function of the model resource scheduling service layer agent, determines whether to update the unloading rule based on the current micro history and model availability status.
4. The edge cloud network adaptive multi-timescale model resource scheduling system according to claim 3, characterized in that, The decision-making process of the H-POMDP building block is defined by the following joint decision-making formula: ； in, For the model resource scheduling base layer intelligent agent n In the time slot t The binary decision signal emitted Intelligent agents serving the model resource scheduling layer n In the time slot t The binary decision signal emitted This is the base layer strategy function for model resource scheduling. This is the strategy function for the model resource scheduling service layer. This is a sequence of historical information for the foundational layer of model resource scheduling. This is a sequence of historical information for the model resource scheduling service layer. For the model k In the current time slot t Availability indicator.
5. The edge cloud network adaptive multi-timescale model resource scheduling system according to claim 4, characterized in that, The model resource scheduling base layer agent operates on minute-level time slots, determining the optimal time to update deployment rules based on long-term network trends, model access frequency, and the time interval since the last update. The model resource scheduling service layer agent operates on second-level time slots, deciding whether to update unloading rules based on current refined network observations and the availability status of upper-layer models.
6. The edge cloud network adaptive multi-timescale model resource scheduling system according to claim 1, characterized in that, The specific state design of the model resource scheduling base layer is as follows: ； in, For the model resource scheduling base layer intelligent agent n exist t The status information of the time slot, for The dimension vector represents a vector of historical request patterns. It is a direct prediction of the distribution of future requests and is obtained by statistical analysis of the request history over a period of time using the moving average method. for 0-1 vector representation model Has it already been deployed on the current edge server? superior; The number of suspended tasks; For resource utilization rate; This represents the time elapsed since the last deployment update.
7. The edge cloud network adaptive multi-timescale model resource scheduling system according to claim 1, characterized in that, In the model resource scheduling base layer module, the state of the model resource scheduling base layer agent is composed of historical request patterns, current model deployment status, system load status, and time interval information. The specific state design of the model resource scheduling service layer is as follows: ； in, Representation model resource scheduling service layer intelligent agent n In the time slot t Status information, In the current time slot Task set arriving at edge nodes Required storage resources In the current time slot Task set arriving at edge nodes Required computing resources; In the current time slot The number of tasks that timed out on edge nodes. For the current time slot The resulting deployment costs.
8. The edge cloud network adaptive multi-timescale model resource scheduling system according to claim 7, characterized in that, The historical request pattern is the result of predicting the distribution of future model demand after statistical analysis of task requests from multiple past cycles using the moving average method. The system load status includes the number of suspended tasks and node resource utilization. The time interval information is the time elapsed since the last model deployment and update.
9. The edge cloud network adaptive multi-timescale model resource scheduling system according to claim 1, characterized in that, In the model resource scheduling service layer module, the state of the model resource scheduling service layer agent includes task resource requirements, deployment cost information, and task timeout status; Among them, task resource requirements are the total computing and storage resources required for the current task; deployment cost information is the model deployment cost incurred within the current macro cycle; and task timeout status is the number of tasks on the current node whose processing latency exceeds the user's tolerance threshold.
10. An adaptive multi-timescale model resource scheduling method for edge cloud networks, implemented based on the adaptive multi-timescale model resource scheduling system for edge cloud networks as described in any one of claims 1-9, characterized in that... Includes the following steps: S1. Collect status information from edge servers and the cloud, and transmit the status information to the model resource scheduling base layer, model resource scheduling service layer and H-POMDP construction module respectively; S2. The multi-timescale model resource scheduling problem is formalized into a hierarchical partially observable Markov decision process. A hierarchical agent structure is constructed, which includes a basic layer agent for model resource scheduling and a service layer agent for model resource scheduling. The basic layer agent for model resource scheduling and the service layer agent for model resource scheduling are respectively responsible for the decision on scheduling update timing and scheduling rules. S3. Define each edge server as an independent intelligent agent, adopt a training paradigm of centralized training and distributed execution, utilize the computing power of the edge side to make decentralized decisions, drive the intelligent agents of the basic layer of model resource scheduling and the intelligent agents of the service layer of model resource scheduling to learn the optimal strategy and generate scheduling update actions. S4, the model resource scheduling base layer module receives scheduling update actions, makes decisions on model deployment and update strategies on a large time scale, follows the storage capacity constraints of edge nodes during execution, and models the impact of model download and API loading latency on task availability; S5, the model resource scheduling service layer module receives scheduling update actions, makes decisions on real-time task unloading strategies on a small time scale, and follows task uniqueness, model availability and link bandwidth constraints during execution.