Edge network cache unloading routing cooperation system

Abstract

The invention discloses an edge network cache unloading routing collaboration system, which is characterized in that three steps of cache placement, task unloading and routing request are uniformly and collaboratively optimized, and routing updating, cache placement and request processing are respectively processed by adopting three different time granularities. A comprehensive resource model is constructed by considering computing resources, storage resources and communication resources, a polynomial time algorithm with precision and resistance guarantee and a reinforcement learning optimization strategy are established, and a distributed solution scheme is designed. According to the method, the scheduling efficiency is gradually improved through continuous learning and adjustment, the system can achieve near-real-time optimal scheduling in a complex and changeable environment, and the overall service quality and the resource utilization efficiency are greatly improved.

Description

Technical Field

[0001] This research belongs to the field of wireless communications and distributed computing in computer networks, specifically involving an edge network cache offloading routing collaborative system. Background Art

[0002] In the network services sector, changes are marked by a focus on service differentiation, increased resource consumption, and increased latency sensitivity, particularly with the growing use of AI applications. This shift has led to stricter requirements for storage, computing, and communication within the network services framework. Edge networks, being closer to end users, facilitate refined service customization, efficient resource organization, and sophisticated request scheduling, better meeting these demands than centralized service networks. However, edge networks present several thorny challenges:

[0003] First, most solutions focus only on cache placement and task offloading, ignoring the selection of request transmission paths in edge networks. Different transmission paths impose additional network load on the server and interact with cache placement and task offloading strategies.

[0004] Second, existing solutions usually assume that the edge network framework performs unaffected routing between edge servers, or only provide caching and offloading strategies for a single request, ignoring the dynamic adaptation and adjustment of caching, offloading, and request delivery paths during the collaborative scheduling process;

[0005] 3. With the popularization of AI applications, the amount of data that needs to be transmitted and processed in edge networks has increased significantly, which puts higher requirements on collaborative scheduling. Summary of the Invention

[0006] The present invention is to meet the following requirements:

[0007] Meeting synergy requirements: With the increasing popularity of AI applications, many applications need to transmit large amounts of data in edge networks (such as those used in video applications), and may even involve intermediate data processed by AI models, such as KV caches in large language models (LLMs). Existing solutions for edge network caching and offloading usually assume that the edge network framework performs unaffected routing between edge servers. Some existing solutions provide offloading strategies for services, or focus only on caching strategies. Some well-designed solutions provide caching and offloading strategies for individual requests. However, these solutions ignore the dynamic adaptation and adjustment of caching, offloading, and request delivery paths during collaborative scheduling. It is crucial for this solution to consider the synergy of caching, offloading, and routing.

[0008] Meeting the requirements for a network-storage-computing model: The need for a stateful edge framework. In the context of edge computing, requests, servers, and networks are all stateful. In particular, in AI-driven edge services, the computational requirements of different AI tasks vary significantly. The transmission of image data can create bottlenecks for routing and network resources, while the storage of KVCache in LLMs poses a challenge to server storage capabilities. Some solutions assume that all caches in the edge network consume equal server storage space or computing power. However, this assumption ignores the variations in task complexity and cache size in edge networks, making the model less effective for current edge computing. In this system, this solution modifies this assumption and takes into account the cache, computing, and storage resources of each server, as well as the resources consumed by each task. This approach significantly enhances the applicability of this system.

[0009] Considering inter-frame requirements: EdgeNetwork is sensitive to the cost of service cache migration in many scenarios. Some papers assume that the scheduling and cache transfer processes do not incur any additional resource consumption. However, this assumption ignores the impact of inter-frame cache migration. This solution employs a multi-frame optimization that considers cache migration, further improving overall model integrity and scheduling efficiency.

[0010] Meeting real-time solution requirements: In the actual online application scenario, the cache placement of scheduling frames occurs without knowing the number, source, and destination of different requests in advance. Some solutions make unrealistic assumptions that the number, source, and destination of all scheduling frame requests are known and fixed for the cache. This idealized strategy lacks the necessary data support in actual applications and may lead to a loss of scheduling performance. In this solution, this limitation is addressed by adopting an algorithm that predicts the demand for the next frame, thereby achieving efficient planning. By achieving sublinear regret, CARDS provides expected guarantees in practical applications.

[0011] The need for multiple time granularities and real-time performance: In edge computing environments, request offloading and processing, cache placement, and routing configuration can occur at different frequencies, ranging from low to high time sensitivity. Current research often considers caching and request processing at a unified frequency for optimization. In particular, DRL solutions may amplify the overhead of request scheduling, resulting in processing delays, or sacrifice the accuracy of the framework for real-time solutions. In terms of routing, existing solutions typically adopt the Link State Routing Protocol (LSRP) or the Distance Vector Routing Protocol (DVRP).

[0012] Meeting deployment and implementation requirements: A distributed implementation is required. Despite the significant computational and communication costs inherent in scheduling, distributed scheduling computation strategies are generally feasible in most cases. Some solutions assume a single server is responsible for orchestrating all edge servers, resulting in significant computational resource consumption during the planning process. In this system, this limitation is addressed by introducing a distributed scheduling algorithm with coarse-grained caching. This approach reduces the required computational resources, enabling CARDS to achieve scheduling results within an acceptable timeframe.

[0013] The need for adaptive deployment: Network systems exhibit a high degree of variability, especially in today's constantly evolving service landscape. Edge system deployment should adapt to the network environment. Some solutions assume that framework parameters remain static and unchanging. This assumption fails to account for the dynamic nature of edge network environments, leading to suboptimal scheduling results. In this system, this solution addresses this limitation by introducing an adaptive deployment method that adjusts CARDS parameters based on the network environment. This approach ensures adaptability to changing network conditions, thereby improving scheduling efficiency.

[0014] The purpose of the present invention can be achieved through the following technical solutions:

[0015] The present invention discloses an edge network cache offloading routing collaborative system, comprising:

[0016] Unify cache placement, task offloading, and request routing for collaborative optimization.

[0017] Three different time granularities are used to handle routing updates, cache placement, and request processing respectively;

[0018] Build a comprehensive resource model considering computing resources, storage resources, and communication resources;

[0019] Establish polynomial-time algorithms with accuracy and adversarial guarantees, as well as reinforcement learning optimization strategies, and design distributed solution solutions.

[0020] Preferably, the three different time granularities are specifically: dividing time into three granularities of interval, frame and slot, corresponding to routing update, cache placement and request processing respectively;

[0021] Interval (maximum granularity): Update routing information of all edge servers;

[0022] Frame: performs coarse-grained colocation of caches;

[0023] Slots (smallest granularity): Provide fine-grained optimal offloading and routing for requests.

[0024] Preferably, the collaborative optimization is specifically as follows: before each frame starts, based on historical request data and current routing conditions, coarse-grained cache placement is performed, and within each slot, fine-grained optimal offloading and routing solutions are provided for specific requests.

[0025] Preferably, the polynomial time algorithm is designed to achieve a solution in polynomial time while ensuring accuracy;

[0026] Reinforcement learning optimization uses deep reinforcement learning technology to continuously optimize the decision-making process and adapt to the dynamically changing network environment;

[0027] Distributed implementation adopts a distributed deployment method to reduce the pressure of single-point computing and improve system scalability while ensuring the collaborative optimization effect.

[0028] Preferably, the comprehensive resource model includes a time granularity model, a cache placement model, a task offloading and computation model, a routing and communication model, and a cache migration model;

[0029] In an edge network, when a user requests to submit a request to the nearest server n, the server determines the service type l requested by the user. Such a request is called an ln request. Multiple servers can process l requests, and there are multiple feasible paths between two specific servers. Therefore, different ln requests may be transmitted to different servers m through different paths in the same frame.

[0030] Preferably, in the time granularity model: in the CARDS framework, time is divided into three descending granularities: interval, frame and slot, and I∈I is used to represent each interval, f∈F is used to represent each frame, and s∈S is used to represent each slot. In each interval, the routing information of all edge servers is updated. For network nodes that are not edge servers within the network, their routes are not considered, which greatly reduces the maintenance cost of routing information in CARDS. Before the start of each frame, coarse-grained co-location of caches is performed, and the goal is to optimize cache placement by considering previous request data and offloading routing scenarios. In each slot, the goal is to provide fine-grained optimal offloading and routing for these requests, ensuring that the maximum number of requests can be satisfied within the constraints of computing, storage and communication. One update of routing information corresponds to multiple cache placement instances, and each cache placement instance corresponds to multiple rounds of user request processing. This means that one interval corresponds to multiple frames, and one frame corresponds to multiple slots;

[0031] In the cache placement model, the server's cache is primarily constrained by storage capacity and the diversity of services provided, thus requiring the following two constraints. First, the server's storage resources are limited. The sum of the cache space occupied by all l services placed on edge server m should not exceed the cache storage space. Second, the server's allocated resources for the cache are limited. The cost of placing all services l in each frame should be less than the maximum cost of cache placement, denoted as B.

[0032] In the task offloading and computational model, the focus during task offloading is on the server's computing resources and capacity to ensure that each task is assigned to the appropriate server. The following constraints apply. First, the probability standardization must be ensured. For any type of request ln, the sum of the probabilities m satisfied across all edge servers should not exceed 1. Second, the system imposes constraints on server adaptability. For any request l and edge server m, only edge server m capable of handling request l can process the request. Users can flexibly control the connectivity of request links.

[0033] In the routing and communication model: For the routing process, the main considerations include the integrity of task execution and the communication and routing resources of the server, aiming to equip each task with the most favorable routing trajectory. First, for the physical communication resource limit of each link, the sum of the communication resources consumed by all requests should not exceed the resource capacity of the link. Second, for the physical communication resource limit of each server, the sum of the network resource consumption caused by all requests passing through the server should not exceed the total network resource capacity of the server. Third, for the wireless communication resource limit of each server, the sum of the wireless communication resources consumed by all requests passing through the server should not exceed the total wireless communication resource capacity of the server.

[0034] In the cache migration model: By migrating cache between frames, the cache cost can be converted into migration cost. If the required cache already exists in a nearby server, migration can be chosen instead of re-caching, which may reduce the cache cost.

[0035] Preferably, as time goes by in the online adversarial guarantee, the cumulative loss of the framework will increase sub-linearly, and the formula is:

[0036]

[0037] Among them, R T Represents regret. Represents the benefit of cache placement when request information is known. Indicates the benefit of cache placement when unknown request information is available.

[0038] Preferably, the polynomial time algorithm includes a single-frame buffer placement algorithm and a multi-frame buffer placement algorithm;

[0039] Single-frame cache placement algorithm: The cache process is considered as k rounds. In each round, the best cache is selected and added to the currently determined cache set. This system introduces pruning logic and only searches until the target value stops increasing or other constraints are saturated. The total time complexity of the algorithm is as follows:

[0040] O(min(args)×l 2 ×n 2 ×(O(LP)))

[0041] Here, min(args) represents the size of the feasible parameter space, l represents the number of service types, n represents the number of servers, and O(LP) represents the time complexity of linear programming.

[0042] Multi-frame Cache Placement Algorithm: A dual-optimization greedy algorithm for multi-frame coarse-grained collaborative caching is proposed, which is inspired by the insights gained from the single-frame cache optimization algorithm. Each greedy algorithm needs to solve the LP formula for all selected frames at once. If a joint scheme for multi-frame scheduling is chosen, the number of binary variables is proportional to the number of frames, which makes the time complexity unacceptable. Therefore, this system proposes the following two-stage optimization algorithm:

[0043] Preferably, the reinforcement learning optimization strategy is specifically: Request scheduling algorithm: Deep reinforcement learning (DRL) is used to solve NP-hard combinatorial optimization problems (COPs), but it does not provide a systematic approach to performance enhancement or optimality guarantees. In other cases, constraint programming (CP) is a useful tool for solving COPs, but choosing the right branch may incur a large computational cost. A hybrid approach is introduced that combines deep reinforcement learning DRL and constraint programming CP to solve combinatorial optimization problems COPs formulated within a dynamic programming DP framework. Deep reinforcement learning DRL technology is used to solve problems that essentially manifest as constraint programming CP challenges. The optimal hyperparameters (e.g., learning-rate) and reinforcement learning models (e.g., DQN, PPO) are determined through k-fold cross validation, and the RL network model is further fine-tuned to cope with fine-grained offloading and routing.

[0044] Preferably, the distributed solution: This system employs a distributed deployment method suitable for coarse-grained collaborative caching. If each server planned its own cache placement, the collaborative system would inevitably lose accuracy and fall into a local optimum. However, considering the benefits of searching all caches, the process can be performed simultaneously with the placement of each cache. The following deployment method is provided. In each round, the algorithm executes in parallel on each server, and the results are summarized at the end of the round. More precisely, the caches planned for search are ranked in order according to the conditions n, l, and v, and then fairly distributed among all servers for processing.

[0045] The beneficial effects of the present invention are as follows:

[0046] This paper decomposes the edge network into three time granularities, from fine to coarse, and maps these granularities to computing, storage, and communication resources, as well as offloading, caching, and routing. It uses a polynomial-time solution with guaranteed lower bounds for submodular functions, and dynamic programming as a bridge between constraint programming and deep reinforcement learning. It provides a framework that considers inter-frame interactions and an adaptive deployment strategy, which increases the number of requests processed per unit time by several times and significantly shortens scheduling decision-making time.

[0047] The present invention gradually improves scheduling efficiency through continuous learning and adjustment. This method enables the system to achieve near-real-time optimal scheduling in complex and changing environments, greatly improving the overall service quality and resource utilization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The present invention will be further described below with reference to the accompanying drawings;

[0049] Figure 1 This is a system architecture diagram of the present invention;

[0050] Figure 2 Flowchart for the implementation of the system of the present invention; DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0052] This system considers a typical edge network environment, with |L| service types distributed across |N| servers. Each server deploys multiple services simultaneously, but the number of services on a server is limited by system resources, such as communication bandwidth, computing power, and cache costs. A user's request for service l is processed by the optimal server n selected by the submission framework. This system calculates the cache locations across all servers and determines the optimal destination and delivery path for the request.

[0053] Reference Figures 1 to 2 The present invention discloses an edge network cache offloading routing collaborative system, comprising:

[0054] Unify cache placement, task offloading, and request routing for collaborative optimization.

[0055] Three different time granularities are used to process routing updates, cache placement, and request processing respectively. Specifically, the three different time granularities are: time is divided into interval, frame, and slot granularities, corresponding to routing updates, cache placement, and request processing respectively;

[0056] Interval (maximum granularity): Update routing information of all edge servers;

[0057] Frame: performs coarse-grained colocation of caches;

[0058] Slot (minimum granularity): provides fine-grained optimal offloading and routing for requests;

[0059] Build a comprehensive resource model considering computing resources, storage resources, and communication resources;

[0060] Establish polynomial-time algorithms with accuracy and adversarial guarantees, as well as reinforcement learning optimization strategies, and design distributed solution solutions.

[0061] The collaborative optimization is specifically as follows: before the start of each frame, based on historical request data and current routing conditions, coarse-grained cache placement is performed, and within each slot, fine-grained optimal offloading and routing solutions are provided for specific requests.

[0062] The polynomial time algorithm is designed to achieve the solution in polynomial time while ensuring accuracy;

[0063] Reinforcement learning optimization uses deep reinforcement learning technology to continuously optimize the decision-making process and adapt to the dynamically changing network environment;

[0064] Distributed implementation adopts a distributed deployment method to reduce the pressure of single-point computing and improve system scalability while ensuring the collaborative optimization effect.

[0065] In an edge network, when a user requests to submit a request to the nearest server n, the server determines the service type l requested by the user. Such a request is called an ln request. Multiple servers can process l requests, and there are multiple feasible paths between two specific servers. Therefore, different ln requests may be transmitted to different servers m through different paths in the same frame. At the same time, this system will update the cache in the edge network to enhance its future service capabilities. CARDS aims to enable the edge framework to process as many requests as possible under the constraints of computing power, communication and storage resources.

[0066] Typically, in the routing update step, each edge server updates its routing information. In the cache placement step, at the beginning of each frame, the system automatically locates a cache based on past request patterns, taking into account routing paths to optimize cache performance. In the task offloading step, a server or cache allocates all pending requests at the start of each slot. In the request routing step, after specifying a routing endpoint at the beginning of each slot, a routing path is assigned to each request.

[0067] The comprehensive resource model includes a time granularity model, a cache placement model, a task offloading and computation model, a routing and communication model, and a cache migration model;

[0068] In the comprehensive resource model, an edge network is considered, which provides a set of L services and consists of a set of N edge servers. Each edge server n has storage resources Rn, computing resources Wn and communication resources Kn. A ln request represents a request for service l on edge server n∈N. γl, ωl, and κl are used to represent the storage, computing, and communication resources consumed by service l, respectively. For routing paths, a set of O complete paths can be selected between two edge servers. Usually, for set O, only the first few shorter paths are considered instead of all feasible paths. The subscript o∈O is introduced to indicate that the number of paths selected by the request from ln to m is o. The known variable dnmop is introduced to indicate whether the path numbered o from n∈N to m∈N passes through p∈N. If |O|=1, the model degenerates into fixed routing. When |O| adopts the maximum number of feasible paths, the model will reflect the cache placement considering any path. Due to different time granularities, the request processing process is divided into three steps: routing update, cache placement, and task scheduling. In each step, the goal is to maximize the number of requests that the network can handle while satisfying the network resource constraints, computing power resource constraints, and storage resource constraints in the edge network.

[0069] In the time granularity model: In the CARDS framework, time is divided into three descending granularities: interval, frame and slot. I∈I is used to represent each interval, f∈F is used to represent each frame, and s∈S is used to represent each slot. In each interval, the routing information of all edge servers is updated. For network nodes that are not edge servers within the network, their routes are not considered, which greatly reduces the maintenance cost of routing information in CARDS. Before the start of each frame, coarse-grained co-location of caches is performed, and the goal is to optimize cache placement by considering previous request data and offloading routing scenarios. In each slot, the goal is to provide fine-grained optimal offloading and routing for these requests, ensuring that the maximum number of requests can be satisfied under the constraints of computing, storage and communication. One update of routing information corresponds to multiple cache placement instances, and each cache placement instance corresponds to multiple rounds of user request processing. This means that one interval corresponds to multiple frames, and one frame corresponds to multiple slots;

[0070] In the cache placement model, the server's cache is primarily constrained by storage capacity and the diversity of services provided, thus requiring the following two constraints. First, the server's storage resources are limited. The sum of the cache space occupied by all l services placed on edge server m should not exceed the cache storage space. Second, the server's allocated resources for the cache are limited. The cost of placing all services l in each frame should be less than the maximum cost of cache placement, denoted as B.

[0071] In the task offloading and computational model, the focus during task offloading is on the server's computing resources and capacity to ensure that each task is assigned to the appropriate server. The following constraints apply. First, the probability standardization must be ensured. For any type of request ln, the sum of the probabilities m satisfied across all edge servers should not exceed 1. Second, the system imposes constraints on server adaptability. For any request l and edge server m, only edge server m capable of handling request l can process the request. Users can flexibly control the connectivity of request links.

[0072] In the routing and communication model: For the routing process, the main considerations include the integrity of task execution and the communication and routing resources of the server, aiming to equip each task with the most favorable routing trajectory. First, for the physical communication resource limit of each link, the sum of the communication resources consumed by all requests should not exceed the resource capacity of the link. Second, for the physical communication resource limit of each server, the sum of the network resource consumption caused by all requests passing through the server should not exceed the total network resource capacity of the server. Third, for the wireless communication resource limit of each server, the sum of the wireless communication resources consumed by all requests passing through the server should not exceed the total wireless communication resource capacity of the server.

[0073] In the cache migration model: By migrating cache between frames, the cache cost can be converted into migration cost. If the required cache already exists in a nearby server, migration can be chosen instead of re-caching, which may reduce the cache cost.

[0074] The current problem remains the offline version. To use the framework mentioned above, we need to obtain the number of all types of ln requests in this frame. Using RT to represent regret, we can now obtain a mathematical expression for regret. In general systems, RT changes linearly over time, and the function f(t) is expressed in a sublinear form. However, as time increases in online services within the online adversarial guarantee, the cumulative loss of the framework will increase sublinearly, as shown by the formula:

[0075]

[0076] Among them, R T Represents regret. Represents the benefit of cache placement when request information is known. Indicates the benefit of cache placement when unknown request information is available.

[0077] To predict the number of ln requests in a frame, a common approach is to take the average of all past frames. In practice, network requests often exhibit both peaks and regular patterns. Therefore, the expected number of requests is inaccurate, and cache placement based on inaccurate predictions is also inappropriate. Cache placement based on this prediction method has a linear lower bound, given the uneven distribution of network requests. In practical applications, a prediction method should be designed that predicts the number of requests for the next frame close to the actual value, using the network status of the previous frames as the basis for the prediction. Cache placement for that frame is performed based on the prediction, thus achieving an online version of sublinear regret. Notably, sublinear regret is an optimization of cache placement. The exact number of requests for a slot is known during the detailed offloading and routing steps, so there is no prediction regret.

[0078] The goal of the prediction algorithm proposed in this paper is to make the predicted number of network requests for the next frame as close as possible to the actual value, while ensuring sublinear regret for the target value. The input includes the predicted number of network requests for all past frames, the actual number of network requests for all past frames, and other network information, including server computing power, storage capacity, and network traffic. The output is the predicted number of network requests for the next frame. This system compares the performance of HedgeSelector with several other network traffic prediction models and time series models, such as LSTM and ARIMA.

[0079] The polynomial time algorithm includes a single-frame buffer placement algorithm and a multi-frame buffer placement algorithm;

[0080] Single-frame cache placement algorithm: The cache process is considered as k rounds. In each round, the best cache is selected and added to the currently determined cache set. This system introduces pruning logic and only searches until the target value stops increasing or other constraints are saturated. The total time complexity of the algorithm is as follows:

[0081] O(min(args)×l 2 ×n 2 ×(O(LP)))

[0082] Here, min(args) represents the size of the feasible parameter space, l represents the number of service types, n represents the number of servers, and O(LP) represents the time complexity of linear programming.

[0083] Multi-frame Cache Placement Algorithm: A dual-optimization greedy algorithm for coarse-grained multi-frame collaborative caching is proposed, inspired by insights gained from single-frame cache optimization algorithms. Each greedy algorithm requires solving the LP formula for all selected frames. If a joint multi-frame scheduling scheme is chosen, the number of binary variables is proportional to the number of frames, making the time complexity unacceptable. Therefore, the following two-stage optimization algorithm is proposed.

[0084] When using an approximate algorithm to solve multi-frame optimization, there is no need to consider the impact of a single cache on all frames, but only the impact of the cache on the frame it is in. This is because, except for the cache migration cost, all other resources are completely reset in subsequent frames, and this optimization can significantly reduce the computational load.

[0085] Second, regarding the search order for cache placement, a naive consideration is to combine Algorithm 1 with the first optimization strategy, starting the search from the first frame and then proceeding frame by frame. This approach effectively ensures that cache placement is favorable in every frame. However, it does not fully address the impact of inter-frame cache migration and only considers that subsequent frames borrow cache from the previous frame.

[0086] Subsequently, a flooding approach integrated with the first optimization strategy can be considered, where the number of caches searched in each round is limited to l×n (equivalent to the number of caches in a single frame), and the optimal cache placement is determined from them. In each round, the caches searched must satisfy the following two conditions:

[0087] The location of the cache has not yet been determined.

[0088] One or more other caches for the same frame have already been identified, or this cache has been identified in a neighboring frame, or there are no newly identified caches.

[0089] This strategy better considers the impact of inter-frame on cache migration, but it cannot fully ensure the advantage of cache placement in later frames. Based on the above discussion, a second optimization strategy is proposed, which continuously integrates Algorithm 1 with the flooding method, while taking into account the interdependencies in cache migration and the advantages of each cache within its respective frame.

[0090] The reinforcement learning optimization strategy is specifically as follows: Request Scheduling Algorithm: Deep reinforcement learning (DRL) is used to solve NP-hard combinatorial optimization problems (COPs), however, it does not provide a systematic approach to performance enhancement or optimality guarantees. In other cases, constraint programming (CP) is a useful tool for solving COPs, but choosing the right branch may incur significant computational costs. A hybrid approach is introduced that combines deep reinforcement learning (DRL) and constraint programming (CP) to solve combinatorial optimization problems (COPs) formulated within a dynamic programming (DP) framework. Deep reinforcement learning (DRL) techniques are employed to solve problems that essentially manifest as constraint programming (CP) challenges. The optimal hyperparameters (e.g., learning-rate) and reinforcement learning models (e.g., DQN, PPO) are determined through k-fold cross-validation, and the RL network model is further fine-tuned to cope with fine-grained offloading and routing.

[0091] Distributed Solution: This system employs a distributed deployment method suitable for coarse-grained collaborative caching. If each server planned its own cache placement, the collaborative system would inevitably lose accuracy and become trapped in a local optimum. However, considering the benefits of searching all caches, the process can be performed simultaneously with the placement of each cache. Therefore, the following deployment method is proposed. In each round, the algorithm is executed in parallel on each server, and the results are summarized at the end of the round. More precisely, the caches planned for search are ranked in order according to the criteria n, l, and v, and then fairly distributed among all servers for processing.

[0092] This system also addresses an extended application scenario: within a small edge network (typically characterized by a limited number of servers), the request path is determined solely by a fixed routing table, which does not change over time or with network congestion. In this case, cache placement lags behind path determination, and this system treats this situation as a special case of a fixed path for applications in small, stable network environments. More specifically, for each Ln request to a specific server m in the model, there is only one routing path.

[0093] In practical applications, sometimes the delivery path of a request is determined by periodic routing table updates, which does not change in real time with network conditions. This phenomenon is mainly observed in extended edge networks with a large number of servers and is called the periodic update routing special case. The coarse-grained and fine-grained models of the periodic update routing special case are the same as those of the fixed routing special case. The difference is that at the beginning of each frame, a DNMP solution is required for this frame, and the DNMP value is used to influence the coarse-grained cache. In the DNMP solution, you should not only choose the fastest path as the DNMP. Instead, you should try to satisfy more requests based on the average number of previously served requests.

[0094] For different types of edge networks, such as compute-intensive, storage-intensive, and network-intensive, the greedy objective can be transformed from the incremental value of the objective function to the cost-effectiveness, with E representing the incremental value and S representing the storage space occupied by the current service cache. For example, E / S can be used as the index to evaluate the current round of cache xln selection.

[0095] The above formulas are all obtained by collecting a large amount of data and performing software simulation to obtain data, and a formula that is close to the actual value is selected. The coefficients in the formula are set by those skilled in the art according to actual conditions. The above are only preferred specific implementation methods of the present invention, but the protection scope of the present invention is not limited to this. Any technician familiar with this technical field, within the technical scope disclosed by the present invention, can make equivalent replacements or changes based on the technical solution and inventive concept of the present invention, which should be covered within the protection scope of the present invention.

Claims

1. An edge network cache offloading routing coordination system, characterized in that: include: Unify cache placement, task offloading, and request routing for collaborative optimization. Three different time granularities are used to handle routing updates, cache placement, and request processing respectively; Build a comprehensive resource model considering computing resources, storage resources, and communication resources; Establish polynomial-time algorithms with accuracy and adversarial guarantees, as well as reinforcement learning optimization strategies, and design distributed solution solutions.

2. The edge network cache offloading routing coordination system according to claim 1, characterized in that: The three different time granularities are specifically: dividing time into interval, frame and slot granularities, corresponding to routing update, cache placement and request processing respectively; Interval: Update routing information of all edge servers; Frame: performs coarse-grained colocation of caches; Slots: Provide fine-grained optimal offloading and routing for requests.

3. The edge network cache offloading routing coordination system according to claim 2, characterized in that: The collaborative optimization is specifically as follows: before the start of each frame, based on historical request data and current routing conditions, coarse-grained cache placement is performed, and within each slot, fine-grained optimal offloading and routing solutions are provided for specific requests.

4. The edge network cache offloading routing coordination system according to claim 1, characterized in that: The polynomial time algorithm is designed to achieve the solution in polynomial time while ensuring accuracy; Reinforcement learning optimization uses deep reinforcement learning technology to continuously optimize the decision-making process and adapt to the dynamically changing network environment; Distributed implementation adopts a distributed deployment method to reduce the pressure of single-point computing and improve system scalability while ensuring the collaborative optimization effect.

5. The edge network cache offloading routing coordination system according to claim 1, characterized in that: The comprehensive resource model includes a time granularity model, a cache placement model, a task offloading and computation model, a routing and communication model, and a cache migration model; In an edge network, when a user requests to submit a request to the nearest server n, the server determines the service type l requested by the user. Such a request is called an ln request. Multiple servers can process l requests, and there are multiple feasible paths between two specific servers. The same ln request may be transmitted to different servers m through different paths in the same frame.

6. The edge network cache offloading routing coordination system according to claim 5, characterized in that: In the time granularity model: In this framework, time is divided into three descending granularities: interval, frame and slot. I∈I is used to represent each interval, f∈F is used to represent each frame, and s∈S is used to represent each slot. In each interval, the routing information of all edge servers is updated. For network nodes that are not edge servers in the network, their routes are not considered. In each slot, fine-grained optimal offloading and routing are requested. In the cache placement model: the sum of the space occupied by the caches of all l services placed on edge server m should not exceed the cache storage space, and the consumption of placing all services l in each frame should be less than the maximum consumption of cache placement, denoted as B; In the task offloading and computation model, we ensure the standardization of probabilities. For any type of ln request, the sum of the probabilities m of being satisfied on all edge servers should not exceed 1. Secondly, this system constrains the adaptability of the servers. For any request l and edge server m, only edge server m that has the ability to handle request l can process the request. Users can flexibly control the connectivity of the request link. In the routing and communication model: for each link's physical communication resource limit, the sum of the communication resources consumed by all requests should not exceed the resource capacity of the link. For each server's physical communication resource limit, the sum of the network resource consumption caused by all requests passing through the server should not exceed the total network resource capacity of the server. For each server's wireless communication resource limit, the sum of the wireless communication resources consumed by all requests passing through the server should not exceed the total wireless communication resource capacity of the server. In the cache migration model: By migrating cache between frames, the cache cost is converted into migration cost. If the required cache already exists in a nearby server, migration can be chosen instead of re-caching.

7. The edge network cache offloading routing coordination system according to claim 1, characterized in that: As time goes by in the online adversarial guarantee, the cumulative loss of the framework will increase sub-linearly, and its formula is: Among them, R T Represents regret. Indicates the benefit of cache placement when the request information is known, Indicates the benefit of cache placement when unknown request information is available.

8. The edge network cache offloading routing coordination system according to claim 1, characterized in that: The polynomial time algorithm includes a single-frame buffer placement algorithm and a multi-frame buffer placement algorithm; Single-frame cache placement algorithm: The cache process is considered as k rounds. In each round, the best cache is selected and added to the currently determined cache set. This system introduces pruning logic and only searches until the target value stops increasing or other constraints are saturated. The total time complexity of the algorithm is as follows: O(min(args)×l 2 ×n 2 ×(O(LP))) Where min(args) represents the size of the feasible parameter space, l represents the number of service types, n represents the number of servers, and O(LP) represents the time complexity of linear programming. Multi-frame buffer placement algorithm: Each greedy algorithm needs to solve the LP formula for all selected frames at once. If a joint scheme of multi-frame scheduling is chosen, the number of binary variables is proportional to the number of frames. In this case, the time complexity is unacceptable.

9. The edge network cache offloading routing coordination system according to claim 1, characterized in that: The reinforcement learning optimization strategy is specifically as follows: deep reinforcement learning (DRL) and constraint programming (CP) are combined to solve the combinatorial optimization problem (COP) formulated within the dynamic programming (DP) framework. The optimal hyperparameters and reinforcement learning model are determined through k-fold cross-validation, and the DRL network model is fine-tuned to cope with fine-grained offloading and routing.

10. The edge network cache offloading routing coordination system according to claim 1, characterized in that: The distributed solution is as follows: each server plans its own cache placement. Considering the benefits of searching all caches, the process can be performed simultaneously with the placement of each cache. In each round, the algorithm is executed in parallel on each server and the results are summarized at the end of the round.

Images