Edge cooperative generative inference dual-time scale cache and offload optimization method

Abstract

The application discloses an edge cooperative generative inference double-time scale cache and unloading optimization method, which comprises the following steps: step S1, modeling the whole system architecture; step S2, representing a target function as maximizing system benefits under the condition of meeting the minimum service quality constraints of user inference tasks; step S3, determining that the optimization target variables of step S2 are model cache parameters of each edge node, inference unloading decisions of user tasks and edge computing resource allocation decisions, and dividing the total optimization problem into two stages for solving. The application aims to effectively reduce cloud backhaul costs, edge computing energy consumption and model switching overhead under the premise of meeting the maximum tolerable time delay and other service quality constraints of multiple user inference tasks, thereby improving the overall operation benefits and cooperative processing capacity of the system.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and specifically to a joint optimization method for model caching, task offloading, and computing resource allocation for multi-user concurrent inference scenarios. Background Technology

[0002] In recent years, with the widespread adoption of 5G / 6G communication, smart terminals, and the rapid development of large-scale model technology, generative AI inference services have experienced explosive growth in scenarios such as intelligent question answering, text generation, image understanding, and multimodal interaction. These inference tasks are generally characterized by computational intensity, latency sensitivity, and sudden request bursts. Mobile users often face problems such as insufficient computing power, excessive energy consumption, and difficulty in guaranteeing response latency when performing inference on their local terminals. To improve service responsiveness, cloud-edge-device collaborative computing reduces end-to-end latency and alleviates cloud-side backhaul pressure by offloading some computing power to the network edge, providing inference services closer to the user.

[0003] However, deploying generative AI inference services for multi-user concurrency at the edge still faces multiple challenges. First, large models are bulky and require significant storage, while edge servers have limited GPU memory capacity, typically only able to cache a small number of models. When the edge server doesn't cache the requested model, it needs to download it from the cloud or offload the task to the cloud for execution, introducing additional model loading latency and cloud-side costs billed by data volume, potentially leading to a decline in service quality. Second, under high concurrency with multiple users, the edge needs to handle a large number of inference requests simultaneously. The uploading of tasks and the downloading of results over wireless links are tightly coupled with computing resource scheduling. Under non-ideal communication conditions, improper allocation of communication and computing resources will further cause queuing congestion and response timeouts. Third, to improve edge service capabilities, multiple edge servers can form an edge server alliance via wired links to achieve collaborative inference and load balancing. However, collaborative execution involves migrating tasks between the main edge and collaborating edges. How to make efficient decisions between "local execution—collaborative edge execution—cloud backup execution," and how to coordinate this with resource allocation strategies, is a key issue for system optimization.

[0004] In existing technologies, research on cloud-edge-device collaborative reasoning typically focuses on system architecture modeling, inference performance and latency modeling, and joint optimization decision-making. For example, the paper "A Joint Optimization Method for Cloud-Edge-Device Model Inference under Air-Ground Collaboration" (Application Publication No.: CN121706997A) proposes to construct a cloud-edge-device collaborative reasoning system in an air-ground collaborative scenario. Through inference performance modeling and latency modeling, it integrates variables such as drone selection, language model selection, task offloading, and drone trajectory into a unified optimization framework to reduce system costs and improve service quality. Based on this technical approach, it can be seen that existing solutions often focus on joint optimization around specific forms of collaborative reasoning architectures (such as air-ground collaboration), with decision variables emphasizing dimensions such as node selection, model selection, and trajectory planning. However, for the coupling relationship of "model caching—task offloading—computing power allocation" under limited edge-side model caching conditions, especially the time scale mismatch between the cross-period update characteristics of caching and the slot-level rapid decision-making of offloading / resource allocation, there is still a lack of unified modeling and hierarchical joint optimization mechanisms for edge server alliance collaboration scenarios. Furthermore, under conditions of multi-user concurrent access, competition for communication and computing resources intensifies. If a single time scale is used to uniformly solve the aforementioned coupled decisions, it can easily lead to excessively high online computational complexity or frequent policy updates with jitter. Simultaneously, the combined impact of spectrum reuse mechanisms (such as non-orthogonal multiple access) and inference scheduling also needs to be further comprehensively considered in the system model. Therefore, there is an urgent need for a technical solution that can jointly optimize model caching, inference offloading, and resource allocation at dual time scales within a cloud-edge-device three-layer architecture, targeting scenarios with limited edge caching, high multi-user concurrency, and edge alliance collaboration. This solution aims to improve overall system efficiency and reduce cloud-side costs and energy consumption while meeting service quality constraints such as user-tolerable latency. Summary of the Invention

[0005] In cloud-edge-device collaborative generative AI inference scenarios, high-concurrency requests from multiple users often face the technical challenge of limited edge server caching capacity. Simultaneously, existing resource scheduling often lacks overall coordination, leading to unreasonable task offloading and resource allocation decisions. These deficiencies further exacerbate technical problems such as task response timeouts, increased system costs, and insufficient collaborative efficiency. To address these issues in existing technologies, this invention aims to provide a hierarchical model caching and inference scheduling joint optimization method for cloud-edge-device collaborative generative AI inference. This method aims to effectively reduce cloud backhaul costs, edge computing energy consumption, and model switching overhead while meeting service quality constraints such as the maximum tolerable latency for multi-user inference tasks, thereby improving the overall operational efficiency and collaborative processing capabilities of the system.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: An edge-cooperative generative inference dual-time-scale caching and offloading optimization method includes the following steps: Step S1) Obtain system status information: Collect real-time system status in the cloud-edge-device collaborative environment. The system status includes the attributes of generative artificial intelligence model inference tasks generated concurrently by mobile users, the cache and computing power resource status of each edge node in the edge server alliance, and the network communication link status between nodes; among them, the task attributes include the maximum tolerable latency. Step S2) Generate joint optimization decision: With the optimization objective of maximizing the overall system benefit under the service quality constraint of satisfying the maximum tolerable latency, hierarchical decision generation with two time scales is performed based on the system state information obtained in step S1): In the slow time scale, model cache update decision for edge nodes is generated based on the historical task request popularity; in the fast time scale, the system state is input into the graph structure reinforcement learning policy network, and the offloading decision for the inference task and the edge computing resource allocation decision are output. Step S3) Execute caching and inference scheduling: Based on the model cache update decision, unloading decision and edge computing resource allocation decision obtained in step S2), execute the corresponding model replacement in the edge server alliance, and control the inference task to be unloaded and computed in parallel between local edge servers, collaborative edge servers or cloud servers in the overall system architecture, and return the inference result to the corresponding mobile user who initiated the inference task in step S1).

[0007] Step S1 includes the following steps: Step 1-1: Based on the constructed overall system architecture for cloud-edge-device collaborative generative artificial intelligence, identify the user terminal that currently generates the inference request, as well as the interconnection topology of the edge server alliance nodes participating in the collaboration; Steps 1-2: Based on the communication model, inference computing model, and energy consumption model between the user terminal and the edge node, and between the edge node and the cloud, quantify and obtain the data transmission latency, computing latency, and energy consumption parameters in the current state; Steps 1-3: Construct a hierarchical joint optimization mechanism.

[0008] In step 1-1, the overall system architecture is a three-layer network architecture, which includes a cloud-side model server, an edge alliance consisting of multiple edge nodes, and user terminals that generate inference requests. The interconnection topology between the edge nodes is defined as follows: The first layer: cloud servers provide full model caching and fallback inference capabilities, and charge for tasks offloaded to the cloud based on data volume; the second layer: the edge server alliance realizes low-overhead collaborative inference through wired links. The main edge server executes directly locally when "the cache is hit and resources are sufficient", otherwise, according to the offloading decision generated in step S2), the task is offloaded to the collaborative edge server or cloud server; the third layer: mobile users submit task requests to the nearby main edge server through wireless links to trigger inference services.

[0009] The cloud server, edge server alliance, mobile user, and inference task attributes of the above system are represented as follows: 1-1-1) System Set, Time Range, and Three-Tier Architecture Modeling: Define the system as a three-tier architecture of Cloud-Edge-End, including cloud servers, multiple edge servers, and a set of mobile users; define the discrete time range of resource transactions / scheduling as... , Indicates the time slot index. Represents the total number of time slots; defines the mobile user set as... , Indicates user index, Indicates the number of users; defines the edge server set as... , Indicates the server index. This indicates the number of edge servers; edge servers collaborate via wired links and form an Edge Alliance; collaboration within the alliance incurs no additional costs, while collaboration outside the alliance (offloaded to cloud servers) is billed based on data volume; 1-1-2) Mobile user task attributes and output data volume model: in any time slot Mobile users Generate a generative artificial intelligence model for inference tasks. Its characteristics are represented as follows: ,in, Indicates user In the time slot The amount of input data generated for the task (unit: MB). Indicates task The maximum tolerable latency, This indicates the generative artificial intelligence model identifier for the task request; Represents a set of models. Indicates the number of models; The amount of output data (response) for the task. Determined by the model complexity, it can be expressed as: , Representation and Model The relevant fixed output overhead factor reflects the basic output quantity used to generate the response; The scaling factor that represents the output as the input increases reflects the output expansion rate; 1-1-3) indicates that the decision of offloading a user task between the primary edge server, the collaborative edge server, or the cloud server is a variable. That is, when This indicates that the task is executed on the main edge server associated with the user; when This indicates that the task is forwarded from the primary edge server to a collaborative edge server within the alliance for execution, and the collaborative edge server is denoted as... That is, a collection of edge servers Except for the main edge server Other edge servers besides; when The time indicates that the task is offloaded to the cloud server for execution; the edge server's caching decision for the generative AI model is a binary variable. That is, when Time indicates edge server In the time slot The model was cached. ,when Time indicates edge server Uncached model ; indicates that the resource allocation decision of the edge server for tasks is a continuous variable. ,in Represents edge server In the time slot For the task The amount of computing resources allocated (GHz).

[0010] For each edge server Configure computing and storage resources, with computing power being [missing information]. (Unit: GHz), GPU memory capacity is (Unit: GB) Used for caching generative artificial intelligence models ;Model The features are simplified to their size (Unit: GB) and the number of CPU cycles required to process each MB of data. (Unit: Cycles / MB); Multiple edge servers collaborate via wired links, forming an edge server alliance. Collaboration within the alliance incurs no additional costs, while offloading to the cloud incurs billing based on data volume; Cloud servers cache all models. It supports downloading uncached models from edge servers and handles inference requests that edge servers cannot complete.

[0011] In steps 1-2, the communication model at least characterizes data transmission latency and transmission energy consumption, and the computation model at least characterizes inference computation latency and computation energy consumption. The inference task includes the amount of input data, the amount of output data, and the maximum tolerable end-to-end latency parameter. Specifically, the communication model, computation model, and energy consumption model are as follows: 1-2-1) For the uplink communication process from the mobile user to the edge server, the NOMA-SIC mechanism is adopted: Let the user... Uplink power is and edge servers The channel gain is The noise power is Uplink bandwidth is ;Please press the user Arranged in descending order If the user Located in the The uplink communication rate is calculated based on the NOMA serial interference cancellation SIC mechanism. Uplink transmission delay is Energy consumption is ; For the downlink transmission process from the edge server to the mobile user: the downlink bandwidth is... Edge server Downlink power is The downlink communication rate was calculated based on the NOMA-SIC mechanism. Downlink latency is Energy consumption is ; For the main edge server With collaborative edge servers The wired collaborative transmission process between edge servers: The wired transmission rate between edge servers is... The task migration latency is then... Energy consumption is ,in, Indicates the main edge server To collaborative edge server Transmit power for wired data transmission, return power , Similarly; For the wired backhaul process from the edge server to the cloud: the wired backhaul rate from the edge server to the cloud is... Upload latency Energy consumption is ,in, Represents edge server For wired data transmission to the cloud, using a "two-hop downlink backhaul strategy," the downlink latency and energy consumption for transmitting the cloud-side results back to the user are as follows: ,in, This indicates the downlink backhaul power of the cloud; the "two-hop downlink backhaul strategy" refers to the inference result being first backhauled from the execution node to the main edge server, and then sent to the user by the main edge server via the wireless downlink; 1-2-2) When the edge server is in a time slot For users Allocate computing resources At that time, the edge reasoning latency is ,in Representation Model Number of CPU cycles required to process each MB of data. Represents edge server In the time slot Assigned to task The computing resources required (GHz); when the task is executed on a cloud server, the inference latency is... ,in, This represents the equivalent computing resources (GHz or equivalent processing speed) provided by the cloud for inference tasks, used to characterize cloud-based inference capabilities; when tasks are offloaded to collaborative edge servers... During execution, task completion latency includes migration latency and collaborative reasoning latency. ,in Indicates collaborative edge server Inference latency for task execution and These represent migration and backhaul latency, respectively. 1-2-3) Edge Server The computational energy consumption for performing the inference task is The computing energy consumption of a cloud server performing inference tasks is... ,in and For edge servers And the computing energy consumption coefficient of cloud servers.

[0012] In steps 1-3, the constructed hierarchical joint optimization mechanism includes a slow-time-scale model heat evaluation and greedy caching decision mechanism, and a fast-time-scale graph structure reinforcement learning-based inference unloading and resource allocation mechanism. The graph structure reinforcement learning mechanism uses edge topology and state aggregation to obtain collaborative context and output hybrid actions. The hierarchical joint optimization mechanism operates based on model request popularity, cache capacity constraints, task latency constraints, and edge computing resource status, specifically as follows: 1-3-1) Model popularity assessment and slow time-scale cache update: In each cache update cycle, the system is based on edge servers. Observed historical task requests, statistics for each model On the edge server The popularity of requests on the platform, combined with model size. Edge server cache capacity Perform cache candidate sorting; update the model caching decision on the edge server accordingly. This allows for prioritizing the caching of more popular and cost-effective models while meeting cache capacity constraints. 1-3-2) For any edge server The candidate model set is sorted in descending order according to the comprehensive index of "popularity-size", and a greedy method is used to select models to enter the cache one by one until the cache capacity constraint is met. Get the updated caching strategy ,in Represents edge server In the time slot caching model , Indicates uncached model ; 1-3-3) In each time slot The system is based on the current cache state. Link transmission model and computation model to generate offloading decisions With resource allocation decisions , where: when Time indicates task By the main edge server Execute locally; when The time indicates that the task is handled by the main edge server. Offload to Collaborative Edge Server CES Execute; when This indicates that the task is unloaded to the cloud server for execution; and is achieved through allocation. Make the inference delay satisfy the maximum tolerable delay. Service quality constraints; 1-3-4) Cooperative scheduling is carried out based on graph structure reinforcement learning. By aggregating states and outputting hybrid actions, the topology of the edge server alliance is used as the graph structure input to process the edge server state and cache state. The task state is aggregated into a neighborhood to obtain a state representation that reflects the collaborative context. Based on this, a reinforcement learning strategy is used to output hybrid actions, where the discrete part corresponds to the unloading decision. The continuous part corresponds to resource allocation The strategy is iteratively updated based on the system benefit feedback after execution, so that the fast time-scale scheduling strategy can be adaptively adjusted according to the dynamic changes in requests and resources. 1-3-5) Caching decisions in slow timescales Updates are triggered according to the cache update cycle, and frequent cache jitter is constrained by the cost of model switching. In the fast time scale, online decisions are made on unloading and resource allocation during the period when the cache policy is fixed. When the cache policy is updated, the fast time scale scheduling strategy makes new decisions based on the updated cache state to achieve joint optimization of cache-unloading-resource allocation in two time scales.

[0013] In step S2), the model cache update decision is represented as a cache decision set. The unloading decision is represented as an unloading decision set. Edge computing resource allocation decisions are represented as resource allocation decision sets. Step S2) specifically includes the following steps: Step 2-1: Obtain user satisfaction and revenue; Step 2-2: Obtain system cost items; Steps 2-3: Obtain system benefits and propose long-term optimization goals; In step 2-1, for any user In the time slot The total task completion delay is denoted as Then, the time delay satisfaction is calculated as follows: and calculate user The benefits are: ,in Pay coefficient for users; calculate edge server In the time slot The income is: ,in, Indicates in time slot Access to edge server A subset of mobile users; In step 2-2, the cloud inference cost is calculated as follows: ,in The cloud unit data cost factor ($ / MB) is used to calculate energy consumption costs. ,in The energy price coefficient ($ / J) For the task The total energy consumption generated at the edge layer; the cost of switching the computational model is: ,in Represents edge server In the time slot Should the new model be cached? , For the model The comprehensive unit cost coefficient (unit: $ / GB). Model size (GB); In steps 2-3, the edge alliance is calculated in the time slot. The overall benefits are: The long-term optimization objective of the system is expressed as maximizing the overall alliance benefit across all time slots: .

[0014] Step S3 determines the variables to be optimized for the optimization objective described in Step S2, and decomposes the overall optimization problem into two sub-problems: slow-time-scale cache optimization and fast-time-scale online scheduling. Step S3 includes the following sub-steps: Step 3-1: Identify the variables to be optimized; Step 3-2: Determine the constraints of the optimization objective; Step 3-3: Decompose the overall optimization problem into two phases of sub-problems; Steps 3-4: Solve the slow-time-scale caching subproblem and the fast-time-scale scheduling subproblem respectively; In step 3-1, to maximize the long-term benefits of the system, the following decision variables are optimized: unloading the decision set. Model cache decision set Resource allocation decision set ; and with To optimize the objective; In step 3-2, the constraints should include at least: 1) Cache capacity constraints: 2) Computational resource constraints: and edge servers The total amount of allocated resources shall not exceed its computing power. 3) Service quality constraints: ;4) Unloading feasibility constraints: When selecting or When this is not met, the corresponding execution node's cache status and resource availability requirements should be satisfied; if not, selection is allowed. 5) Variable value constraints: Unloading to the cloud as a fallback option; , ; In step 3-3, the overall optimization problem is divided into two stages: the first stage is the slow-time-scale model cache optimization stage, which updates the cache based on historical request popularity and cache capacity constraints within the cache update cycle. The second stage is a joint scheduling stage for fast-timescale inference offloading and resource allocation. Under the condition of a fixed caching strategy, online generation is performed for each time slot. and ; In steps 3-4, the first stage uses a greedy caching strategy based on model heat assessment to solve the problem. and through To suppress frequent switching, the second stage employs an online scheduling strategy based on graph structure reinforcement learning to solve the problem. and The strategy is iteratively updated based on the system benefits defined in step S2 to obtain a stable scheduling strategy under dynamic requests from multiple users.

[0015] In steps 3-4, the specific steps for solving the slow-time-scale model caching subproblem and the fast-time-scale inference offloading and resource allocation subproblem are as follows: 3-4-1) Slow time scale: Greedy model cache solution based on request popularity; 1) Popularity statistics: When the cache update cycle arrives, for each edge server... Statistical analysis of models within historical time slots The frequency of requests determines the model's request popularity. 2) Candidate ranking: combined with model size Edge server cache capacity The candidate models are sorted to obtain a cache priority sequence; 3) Greedy loading: Select models to be written to the cache in sequence according to the priority order, until the cache capacity constraint is met. This allows us to obtain the caching decision within the cache update cycle. ; 4) Switching Suppression: If this cache update involves loading a new model, calculate the model switching cost. Furthermore, frequent loading is suppressed in subsequent cache updates to reduce the impact of cache jitter on system efficiency; 3-4-2) Fast timescale: Online offloading and resource allocation under a fixed caching strategy: in each time slot Internally, based on a fixed caching strategy Build a system containing tasks Cache status The system status, including link communication parameters and computing resource availability information, is input into the policy network to output offloading decisions. With resource allocation decisions ;When the output or If the model availability or resource availability requirements of the corresponding execution node are not met, the unloading decision will be adjusted to... As a safety net; then, the benefits of a single time-slot system are calculated based on communication, computing, and energy consumption models. and with the stated The policy network parameters are updated as learning feedback, and the process is repeated in the next time slot. 3-4-3) Strategy Iteration Updates and Simultaneous Execution of Dual Time Scales: 1) Online iteration: Based on the benefit feedback of each time slot within a fast time scale. The graph structure reinforcement learning strategy is iteratively updated to improve long-term effectiveness under multi-user dynamic requests; 2) Synchronization mechanism: Within each cache update cycle, a 3-4-1 cache update decision is first executed. The caching strategy is fixed during this period to execute 3-4-2 and 3-4-3; when the caching strategy is updated, the online scheduling strategy continues to be generated based on the new caching state. and This enables the coordinated operation of slow-timescale cache updates and fast-timescale online scheduling.

[0016] Compared with the prior art, the present invention has the following technical effects: 1) Existing technologies often design model caching, task offloading, and resource allocation separately or optimize them independently within a single edge node. This makes it difficult to depict the strong coupling relationship between the three in multi-user concurrent AI inference scenarios, resulting in low cache hit rate, high cloud offloading ratio, and large end-to-end latency fluctuations. This invention is aimed at cloud-edge-device collaborative inference systems, jointly modeling communication, computing, and energy costs, and achieving joint optimization of caching, offloading, and resource allocation under the collaborative conditions of edge server alliances, thereby improving the accessibility and overall efficiency of inference services.

[0017] 2) Existing technologies typically use a single time scale for global optimization, which can easily lead to problems such as excessively high online solution complexity or frequent cache updates causing policy jitter. This invention adopts a dual-time-scale hierarchical mechanism, which uses model caching as a slow time-scale decision and unloading and resource allocation as fast time-scale online decisions. It also introduces model switching costs to suppress frequent model changes, enabling the system to maintain stable operation and achieve better long-term benefits under dynamic request and heterogeneous resource conditions.

[0018] 3) Existing technologies rarely utilize the topological relationships and neighborhood state interaction information between edge nodes in edge collaborative inference, resulting in insufficient adaptability of collaborative scheduling to load changes, cache state changes, and link state changes. This invention introduces a graph-based reinforcement learning strategy into fast timescale scheduling, utilizing alliance topology and state aggregation to construct a collaborative context, and outputting offloading and resource allocation decisions. This enables dynamic load sharing and adaptive path selection for concurrent requests from multiple users, thereby improving the system's scheduling efficiency and collaborative gains. Attached Figure Description

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a schematic diagram of the system model in this invention; Figure 2 This is a training convergence curve diagram of the present invention; Figure 3 This is a line graph showing the change in the total utility of the edge alliance as a function of the number of edge servers in this invention. Figure 4 This is a line graph showing the change in system utility with the number of users in this invention. Figure 5 This is a bar chart comparing the utilization rate of cache resources with the cache capacity of the edge server in this invention. Figure 6 This is a bar chart comparing the utilization rate of computing resources with the changes in computing resources of the edge server in this invention; Figure 7 This is a line graph showing the comparison of resource utilization rate as a function of the number of users in this invention. Figure 8 This is a heatmap showing how system utility changes as a function of "cache capacity × computing resources" in this invention. Figure 9 This is a performance distribution diagram of the algorithm of the present invention; Figure 10 This is a performance comparison chart of the algorithm of this invention under different parameter settings; Figure 11 This is the overall flowchart of the present invention. Detailed Implementation

[0020] A hierarchical model caching and inference scheduling joint optimization method for cloud-edge-device collaborative generative artificial intelligence inference includes the following steps: Step S1: Model the entire system architecture; Step S2: This means that the objective function is to maximize the system efficiency while satisfying the minimum service quality constraints of the user's inference task; Step S3: Determine the optimization objective variables of Step S2 as the model cache parameters of each edge node, the inference offloading decision of user tasks, and the edge computing resource allocation decision. Divide the overall optimization problem into two stages for solution. The first stage is the slow time-scale model cache optimization stage, and the second stage is the fast time-scale joint scheduling stage of inference offloading and resource allocation.

[0021] Step S1 includes the following steps: Step 1-1: Describe the overall architecture for cloud-edge-device collaborative generative AI inference. The architecture includes at least a cloud-side model server, an edge alliance consisting of multiple edge nodes, and a user terminal that generates inference requests. Define the interconnection topology between the edge nodes. Steps 1-2: Describe the communication model, inference computing model, and energy consumption model between the user terminal and the edge node, and between the edge node and the cloud. The communication model should at least represent the data transmission latency and transmission energy consumption, and the computing model should at least represent the inference computing latency and computing energy consumption. The inference task should at least include the amount of input data, the amount of output data, and the maximum tolerable end-to-end latency parameter. Steps 1-3: Describe the hierarchical joint optimization mechanism, including a slow-timescale model heat evaluation and greedy caching decision mechanism, and a fast-timescale graph structure reinforcement learning-based inference unloading and resource allocation mechanism. The graph structure reinforcement learning mechanism uses edge topology and state aggregation to obtain collaborative context and output hybrid actions. Steps 1-4: Analyze the characteristics of the proposed joint optimization mechanism. The characteristics include at least the dual time-scale decoupling characteristic, the feasibility guarantee under cache capacity and resource constraints, the adaptability of the scheduling strategy to edge topology changes, and the characteristics of algorithm complexity and convergence stability.

[0022] In step 1-1, the attributes of the cloud server, edge server alliance, mobile user, and inference task in the system are represented as follows: Step 1-1-1) System Set, Time Range, and Three-Tier Architecture Modeling: Define the system as a three-tier architecture of Cloud-Edge-End, including cloud servers, multiple edge servers, and a set of mobile users. Define the discrete time range of resource transactions / scheduling as follows: , Indicates the time slot index. Represents the total number of time slots; defines the mobile user set as... , Indicates user index, Indicates the number of users; defines the edge server set as... , Indicates the server index. This indicates the number of edge servers; edge servers collaborate via wired links and form an Edge Alliance. Collaboration within the alliance incurs no additional costs, while collaboration outside the alliance (offloaded to cloud servers) is billed based on data volume.

[0023] Step 1-1-2) Mobile user task attributes and output data volume model: in any time slot Mobile users Generate a generative artificial intelligence model for inference tasks. Its characteristics are represented as follows: ,in, Indicates user In the time slot The amount of input data generated for the task (unit: MB). Indicates task The maximum tolerable latency, This indicates the generative artificial intelligence model identifier for the task request; Represents a set of models. Indicates the number of models.

[0024] The amount of output data (response) for the task. Determined by the model complexity, it can be expressed as: , Representation and Model The relevant fixed output overhead factor reflects the basic output quantity used to generate the response; It represents the proportionality coefficient of output growth as input increases, reflecting the output expansion rate.

[0025] Step 1-1-3) represents the decision of whether the user task is offloaded between the primary edge server, the collaborative edge server, or the cloud server, as a variable. That is, when Time indicates in time slot user The task is handled by its corresponding main edge server. Send to Collaborative Edge Server (CES) Processing; when The time indicates that it is from the main edge server Local processing; when This indicates that the task is offloaded to the cloud for processing; it also indicates that the edge server's caching decision for the generative AI model is a binary variable. That is, when Time indicates edge server In the time slot The model was cached. ,when Time indicates edge server Uncached model ; indicates that the resource allocation decision of the edge server for tasks is a continuous variable. ,in Represents edge server In the time slot For the task The amount of computing resources allocated (GHz).

[0026] For each edge server Configure computing and storage resources, with computing power being [missing information]. (Unit: GHz), GPU memory capacity is (Unit: GB) Used for caching generative artificial intelligence models ;Model The features are simplified to their size (Unit: GB) and the number of CPU cycles required to process each MB of data. Multiple edge servers collaborate via wired links, forming an edge server alliance. Collaboration within the alliance incurs no additional costs, while offloading to the cloud incurs billing based on data volume. Cloud servers cache all models. It supports downloading uncached models from edge servers and handles inference requests that edge servers cannot complete.

[0027] A three-layer network architecture is defined. In the first layer, the cloud server provides full model caching and fallback inference capabilities, and charges for tasks offloaded to the cloud based on data volume. In the second layer, the edge server consortium achieves low-overhead collaborative inference through wired links. The main edge server executes the task locally when the cache is hit and resources are sufficient; otherwise, the task is offloaded to the collaborative edge server (CES) or the cloud server according to the offloading decision. In the third layer, mobile users submit task requests to the nearest main edge server via wireless links to trigger the inference service. This architecture effectively improves the accessibility and collaborative efficiency of the inference service under conditions of limited cache capacity and heterogeneous computing power.

[0028] In steps 1-2, the communication, computing, and energy consumption models between mobile users and edge servers, between edge servers, and between edge servers and cloud servers are as follows: Step 1-2-1) For the uplink communication process from the mobile user to the edge server, the NOMA-SIC mechanism is adopted: Let the user... Uplink power is and edge servers The channel gain is The noise power is Uplink bandwidth is Press the user Arranged in descending order If the user Located in the For each bit, its uplink communication rate is calculated based on the NOMA serial interference cancellation (SIC) mechanism. Uplink transmission delay is Energy consumption is .

[0029] For the downlink transmission process from the edge server to the mobile user: the downlink bandwidth is... Edge server Downlink power is The downlink communication rate was calculated based on the NOMA-SIC mechanism. Downlink latency is Energy consumption is .

[0030] For the main edge server With collaborative edge servers The wired collaborative transmission process between edge servers: The wired transmission rate between edge servers is... The task migration latency is then... Energy consumption is , return , Similarly.

[0031] For the wired backhaul process from the edge server to the cloud: the wired backhaul rate from the edge server to the cloud is... Upload latency Energy consumption is Using a "two-hop downlink backhaul strategy", the downlink latency and energy consumption for transmitting cloud-side results back to the user are: ,in, This indicates the downlink backhaul power of the cloud; the "two-hop downlink backhaul strategy" means that the inference result is first backhauled from the execution node to the main edge server, and then sent to the user by the main edge server via the wireless downlink.

[0032] Step 1-2-2) When the edge server is in the time slot For users Allocate computing resources At that time, the edge reasoning latency is ,in Representation Model Number of CPU cycles required to process each MB of data. Represents edge server In the time slot Assigned to task The computing resources required (GHz). When the task is executed on a cloud server, the inference latency is... ,in, This represents the equivalent computing resources (GHz or equivalent processing speed) provided by the cloud for the inference task, used to characterize cloud-based inference capabilities. When the task is offloaded to a collaborative edge server... During execution, task completion latency includes migration latency and collaborative reasoning latency. ,in Indicates collaborative edge server Inference latency for task execution and These represent migration and backhaul delays, respectively.

[0033] 1-2-3) Edge Server The computational energy consumption for performing the inference task is The computing energy consumption of a cloud server performing inference tasks is... ,in and For edge servers And the computing energy consumption coefficient of cloud servers.

[0034] In steps 1-3, the hierarchical joint optimization mechanism needs to operate based on model request popularity, cache capacity constraints, task latency constraints, and edge computing resource status, as calculated below: Step 1-3-1) Model popularity assessment and slow timescale cache update: Within each cache update cycle, the system is based on edge servers. Observed historical task requests, statistics for each model On the edge server The popularity of requests on the platform, combined with model size. Edge server cache capacity Perform cache candidate sorting; update the model caching decision on the edge server accordingly. This allows for prioritizing the caching of models with higher popularity and better cost-effectiveness while meeting cache capacity constraints.

[0035] Step 1-3-2) For any edge server The candidate model set is sorted in descending order according to the comprehensive index of "popularity-size", and a greedy method is used to select models to enter the cache one by one until the cache capacity constraint is met. Get the updated caching strategy ,in Represents edge server In the time slot caching model , Indicates uncached model .

[0036] Step 1-3-3) In each time slot The system is based on the current cache state. Link transmission model and computation model to generate offloading decisions With resource allocation decisions , where: when Time indicates task By the main edge server Execute locally; when The time indicates that the task is handled by the main edge server. Offload to Collaborative Edge Server CES Execute; when This indicates that the task is unloaded to the cloud server for execution; and is achieved through allocation. Make the inference delay satisfy the maximum tolerable delay. Service quality constraints, etc.

[0037] Steps 1-3-4) Conduct collaborative scheduling based on graph structure reinforcement learning. By aggregating states and outputting hybrid actions, the topology of the edge server alliance is used as the graph structure input to process the edge server state and cache state. The task state is aggregated into a neighborhood to obtain a state representation that reflects the collaborative context. Based on this, a reinforcement learning strategy is used to output hybrid actions, where the discrete part corresponds to the unloading decision. The continuous part corresponds to resource allocation The strategy is iteratively updated based on the system benefit feedback after execution, so that the fast time-scale scheduling strategy can be adaptively adjusted according to the dynamic changes in requests and resources.

[0038] Steps 1-3-5) Caching Decisions in a Slow Time Scale Updates are triggered according to the cache update cycle, and frequent cache jitter is constrained by the cost of model switching. In the fast time scale, online decisions are made on unloading and resource allocation during the period when the cache policy is fixed. When the cache policy is updated, the fast time scale scheduling strategy makes new decisions based on the updated cache state to achieve joint optimization of cache-unloading-resource allocation in two time scales.

[0039] In steps 1-4, the hierarchical joint optimization mechanism proposed in this invention should meet the following basic properties: Step 1-4-1) Feasibility (Constraint Satisfaction): In any time slot The mechanism generates unloading decisions. Cache decision With resource allocation decisions At the same time, system physical and service quality constraints should be met, including edge server cache capacity constraints. Non-negativity and upper limit constraints of computational resources (and subject to edge server computing power) (constraints), and the maximum tolerable latency constraint for the task. This ensures that the output decision is an executable solution even under conditions of limited cache capacity and heterogeneous computing power.

[0040] Step 1-4-2) Dual-Time-Scale Stability (Cache-Scheduling Decoupling): The mechanism adopts a hierarchical structure of slow-time-scale cache update decisions and fast-time-scale execution of unloading and resource allocation decisions, wherein the cache decision... It remains unchanged within the preset cache update cycle, and only changes within a fast timescale. and Perform online adjustments; and introduce model switching costs. Suppressing strategy jitter caused by frequent mold changes improves system stability and the sustainability of long-term benefits.

[0041] Step 1-4-3) Cooperative Adaptability (Topology Awareness and State Aggregation): The mechanism utilizes wired links for coordination within the edge server consortium, and the scheduling strategy is generated... and By combining alliance topology relationships with neighborhood status aggregation information, the system can adaptively select the main edge, collaborative edge, or cloud execution path when the load, cache hit status, or link conditions between edge servers change, and achieve dynamic scheduling and load sharing for concurrent requests from multiple users.

[0042] Step 1-4-4) Computational Efficiency and Deployability: The mechanism decomposes the complex joint optimization problem into two sub-problems: "slow-timescale cache decision" and "fast-timescale scheduling decision." Cache updates employ a greedy selection based on heat assessment to reduce solution overhead, while fast-timescale scheduling uses a reinforcement learning policy network for forward inference to output decisions, enabling rapid generation in each time slot. and This allows the system to maintain acceptable online computational complexity as its scale increases, and supports distributed deployment and real-time operation at the edge.

[0043] Step S2 includes the following steps: Step 2-1: Calculate user satisfaction and revenue; Step 2-2: Calculate the system cost items; Steps 2-3: Calculate system benefits and provide long-term optimization objectives.

[0044] In step 2-1, for any user In the time slot The total task completion delay is denoted as Then, the time delay satisfaction is calculated as follows: and calculate user The benefits are: ,in The user pay coefficient is calculated. Further, the edge server is used for calculation. In the time slot The income is: .

[0045] In step 2-2, the cloud inference cost is calculated as follows: ,in This represents the unit data cost factor in the cloud ($ / MB). Energy costs are calculated as follows: ,in The energy price coefficient ($ / J) For the task The total energy consumption generated at the edge layer. The cost of switching the computational model is: ,in Represents edge server In the time slot Should the new model be cached? , For the model The comprehensive unit cost coefficient (unit: $ / GB). Model size (GB).

[0046] In steps 2-3, the edge alliance is calculated in the time slot. The overall benefits are: The long-term optimization objective of the system is expressed as maximizing the overall alliance benefit across all time slots: .

[0047] In step S3, the variables to be optimized for the optimization objective described in step S2 are determined, and the overall optimization problem is decomposed into two sub-problems: slow-time-scale cache optimization and fast-time-scale online scheduling; including the following sub-steps: Step 3-1: Identify the variables to be optimized; Step 3-2: Determine the constraints of the optimization objective; Step 3-3: Decompose the overall optimization problem into two phases of sub-problems; Steps 3-4: Solve the slow-time-scale caching subproblem and the fast-time-scale scheduling subproblem respectively.

[0048] In step 3-1, to maximize the long-term benefits of the system, the following decision variables are optimized: unloading the decision set. Model cache decision set Resource allocation decision set And with To optimize the objective.

[0049] In step 3-2, the constraints should include at least: 1) Cache capacity constraints: 2) Computational resource constraints: and edge servers The total amount of allocated resources shall not exceed its computing power. 3) Service quality constraints: ;4) Unloading feasibility constraints: When selecting or When this is not met, the corresponding execution node's cache status and resource availability requirements should be satisfied; if not, selection is allowed. 5) Variable value constraints: Unloading to the cloud as a fallback option; , .

[0050] In step 3-3, the overall optimization problem is divided into two stages: the first stage is the slow-time-scale model cache optimization stage, which updates the cache based on historical request popularity and cache capacity constraints within the cache update cycle. The second stage is a joint scheduling stage for fast-timescale inference offloading and resource allocation. Under the condition of a fixed caching strategy, online generation is performed for each time slot. and .

[0051] In steps 3-4, the first stage employs a greedy caching strategy based on model heat assessment to solve the problem. and through To suppress frequent switching, the second stage employs an online scheduling strategy based on graph structure reinforcement learning to solve the problem. and Based on the system benefits defined in step S2, the strategy is iteratively updated to obtain a stable scheduling strategy under dynamic requests from multiple users. The specific steps are as follows: 3-4-1) Slow Time Scale: Greedy model cache solution based on request popularity. 1) Popularity Statistics: When the cache update cycle arrives, for each edge server... Statistical analysis of models within historical time slots 1) Model request popularity: 1) Request frequency; 2) Candidate ranking: 2) Combining model size. Edge server cache capacity 3) Greedy loading: Select models sequentially according to the priority sequence and write them to the cache until the cache capacity constraint is met. This allows us to obtain the caching decision within the cache update cycle. 4) Switching suppression: If this cache update involves loading a new model, calculate the model switching cost. Furthermore, frequent loading is suppressed in subsequent cache updates to reduce the impact of cache jitter on system efficiency.

[0052] 3-4-2) Fast timescale: Online offloading and resource allocation under a fixed caching strategy. In each time slot... Internally, based on a fixed caching strategy The online scheduling process is executed cyclically according to the following procedure: Step A (Input Status): Input the system status of the current time slot, which includes at least: user tasks. Cache status Link communication parameters and computing resource availability information.

[0053] Step B (Output Decision): Input the system state into the policy network, and output the offloading decision and resource allocation decision, wherein the offloading decision is... Resource allocation decision-making When output or However, if the required model is not cached at the corresponding execution node or the resources are unavailable, the offloading decision for this task will be adjusted to... As a fallback, to meet the feasibility constraints of unloading.

[0054] Step C (Calculation of Benefits): Calculate the total task latency based on the communication, computing, and energy consumption model. Total energy consumption of the edge layer Furthermore, the benefits of a single time slot system were calculated. .

[0055] Step D (Update Parameters): Based on the stated As a learning feedback signal, the policy network parameters are updated, thereby improving the decision-making quality in subsequent time slots.

[0056] Step E (Entering the next time slot): Let Then proceed to the next time slot and repeat steps A to D until the online scheduling of all time slots is completed.

[0057] 3-4-3) Strategy iteration and updates are executed synchronously across two time scales. 1) Online iteration: Within the fast time scale, updates are based on the benefit feedback of each time slot. 1) Iteratively update the graph structure reinforcement learning strategy to improve long-term effectiveness under multi-user dynamic requests; 2) Synchronization mechanism: In each cache update cycle, first execute the 3-4-1 cache update decision. The caching strategy is fixed during this period to execute 3-4-2 and 3-4-3; when the caching strategy is updated, the online scheduling strategy continues to be generated based on the new caching state. and This enables the coordinated operation of slow-timescale cache updates and fast-timescale online scheduling.

[0058] Example: Figure 1 A system model diagram is provided, which describes a three-layer collaborative inference architecture of cloud, edge, and device: mobile users generate generative AI inference tasks in discrete time slots and offload them to the main edge server via wireless links; the main edge server completes the inference locally and returns the results when there is a cache hit and sufficient resources, and offloads the task to a collaborative edge server or cloud server to complete the inference and return the results; the cloud server caches the full model and provides fallback inference and model download support, and the edge servers form an alliance through wired links to achieve low-overhead collaboration.

[0059] Figure 2 The algorithm training convergence graph is presented. Both the HMGRL proposed in this invention and the comparison method show the convergence trend of reinforcement learning during the training process. However, HMGRL is better overall in terms of convergence speed, fluctuation range and final reward, indicating that its hierarchical cache-schedule structure can learn a stable and effective joint decision-making strategy faster under multi-user concurrency and multi-edge collaboration conditions.

[0060] Table 1. Comparison of total alliance utility under different numbers of edge servers (rounded to four decimal places):

[0061] Table 1 and Figure 3 The paper presents a comparison of system utility as the number of edge servers changes. When the number of edge servers increases from 2 to 6, the overall consortium utility of each algorithm increases with the number of servers, but the rate of increase varies significantly. The HMGRL algorithm maintains the highest utility and shows the most significant improvement across the entire range, indicating that its "heat-driven caching + topology collaborative scheduling" can more fully utilize the cache pool expansion and collaborative path richness brought about by the increase in the number of nodes. In contrast, methods lacking effective collaborative utilization or with weak caching strategies have limited gains, demonstrating the scalability advantage of this invention in multi-node expansion scenarios.

[0062] Table 2 Comparison of total alliance utility under different numbers of users (rounded to four decimal places):

[0063] Table 2 and Figure 4 The results demonstrate the system utility as the number of users changes. As the number of concurrent users increases from 5 to 25, the utility of each algorithm increases during low-to-medium load periods, while performance bottlenecks appear to varying degrees during high load periods. The proposed HMGRL algorithm remains optimal under all load conditions, achieving more significant benefits in the medium load range. Although its utility slightly declines under high load due to resource contention, it still significantly outperforms the comparative methods, indicating that the proposed algorithm has stronger robustness and scalability under dynamic loads.

[0064] Figure 5 The results show the changes in cache resource utilization as the edge server cache capacity changes. Under different average cache capacity levels, the cache resource utilization of the five algorithms shows a decreasing trend with increasing capacity, reflecting the principle that "the number of high-frequency models is limited and capacity expansion has marginal diminishing returns." In various capacity scenarios, the HMGRL algorithm consistently maintains the highest cache utilization, indicating that it can accurately select high-value models based on request popularity and maintain high effective utilization even after expansion. Random or weakly sensitive caching strategies are more prone to resource waste under large capacities, further verifying the effectiveness of the popularity-aware caching mechanism.

[0065] Figure 6 This study demonstrates the trend of computing resource utilization as edge server computing resources change. As the available computing power of edge servers increases, the overall computing resource utilization of each algorithm decreases, consistent with the relative utilization decline caused by "increased capacity while relatively stable demand." The HMGRL algorithm exhibits higher resource utilization across all computing power levels, indicating its ability to achieve more efficient task allocation and computing power utilization through model cache hits and collaborative scheduling. In contrast, comparative methods, due to insufficient collaboration and caching capabilities, are prone to insufficient computing power utilization or uneven workload distribution, resulting in lower overall utilization.

[0066] Table 3 Computational Resource Utilization vs. Number of Users (rounded to four decimal places):

[0067] Table 3 and Figure 7 The results show the changes in computing resource utilization as the number of users increases. As the number of users increases from 5 to 25, the CPU utilization of each algorithm generally increases, reflecting that increased load leads to more intensive edge-side scheduling. The HMGRL algorithm maintains the highest utilization across the entire range, indicating that it can more accurately identify server idle levels and cache hit probability under high load, thus achieving near-optimal offloading and resource allocation. Methods lacking coordination or with weak caching strategies are more prone to waiting and migration overhead when the user base increases, resulting in significantly lower utilization efficiency.

[0068] Figure 8 The study presents a comparison of system utility as a function of "cache capacity × computing resources". System utility exhibits a monotonically increasing trend with both average cache capacity and average computing resources, demonstrating a clear coupling characteristic: the benefits of increasing cache capacity are more pronounced when computing power is high, while the utility improvement from simply expanding the cache is slower when computing power is limited. This phenomenon indicates that the system's inference capability is jointly determined by "cache hit rate and edge computing power"; when both are sufficient, the HMGRL invention can fully leverage the advantages of collaborative caching and dynamic resource allocation, achieving a higher level of system utility.

[0069] Figure 9 The algorithm performance distribution diagram is presented, and the system utility box plot reflects the performance distribution and stability of different algorithms under multiple scenario conditions. The box positions of the HMGRL algorithm in this invention are higher and more concentrated, indicating that it not only has better average utility, but also less fluctuation and stronger robustness. Although some comparative methods may achieve better results in individual scenarios, their overall dispersion is large and their stability is insufficient, further demonstrating the generalization ability and consistency advantage of this invention in multi-user, multi-server heterogeneous environments.

[0070] Figure 10The performance comparison results of the algorithm of this invention under different parameter settings are presented. (Different model unit cost coefficients are also discussed.) Under certain conditions, system utility varies. The overall trend is downward, and different inference parameters... The corresponding curves maintain a stable relative ranking: the larger the inference parameters (the larger the output and the higher the inference complexity), the lower the system utility and the more significant the decrease. This result indicates that the increased cost of model switching and cache update will inhibit frequent caching and replacement at the edge, thus affecting the hit rate and latency overhead; it also shows that the present invention can maintain a consistent performance trend and good robustness under cost disturbances and changes in inference complexity, verifying the rationality of the cache-computation collaborative structure.

Claims

1. An edge co-synergistic generative inference dual-time scale cache and offload optimization method, characterized in that, Includes the following steps: Step S1) Obtain system status information: Collect real-time system status in the cloud-edge-device collaborative environment. The system status includes the attributes of generative artificial intelligence model inference tasks generated concurrently by mobile users, the cache and computing power resource status of each edge node in the edge server alliance, and the network communication link status between nodes; among them, the task attributes include the maximum tolerable latency. Step S2) Generate joint optimization decision: With the optimization objective of maximizing the overall system benefit under the service quality constraint of satisfying the maximum tolerable latency, hierarchical decision generation with two time scales is performed based on the system state information obtained in step S1): In the slow time scale, model cache update decision for edge nodes is generated based on the historical task request popularity; in the fast time scale, the system state is input into the graph structure reinforcement learning policy network, and the offloading decision for the inference task and the edge computing resource allocation decision are output. Step S3) Execute caching and inference scheduling: Based on the model cache update decision, unloading decision and edge computing resource allocation decision obtained in step S2), execute the corresponding model replacement in the edge server alliance, and control the inference task to be unloaded and computed in parallel between local edge servers, collaborative edge servers or cloud servers in the overall system architecture, and return the inference result to the corresponding mobile user who initiated the inference task in step S1).

2. The method of claim 1, wherein, Step S1 includes the following steps: Step 1-1: Based on the constructed overall system architecture for cloud-edge-device collaborative generative artificial intelligence, identify the user terminal that currently generates the inference request, as well as the interconnection topology of the edge server alliance nodes participating in the collaboration; Steps 1-2: Based on the communication model, inference computing model, and energy consumption model between the user terminal and the edge node, and between the edge node and the cloud, quantify and obtain the data transmission latency, computing latency, and energy consumption parameters in the current state; Steps 1-3: Construct a hierarchical joint optimization mechanism.

3. The method according to claim 2, characterized in that, In step 1-1, the overall system architecture is a three-layer network architecture, which includes a cloud-side model server, an edge alliance consisting of multiple edge nodes, and user terminals that generate inference requests. The interconnection topology between the edge nodes is defined as follows: First layer: Cloud servers provide full model caching and fallback inference capabilities, and charge for tasks offloaded to the cloud based on data volume; The second layer: The edge server alliance realizes low-overhead collaborative inference through wired links. The main edge server executes directly locally when the cache is hit and resources are sufficient. Otherwise, the task is unloaded to the collaborative edge server or cloud server according to the unloading decision generated in step S2). Layer 3: Mobile users submit task requests to nearby main edge servers via wireless links to trigger inference services.

4. The method according to claim 3, characterized in that, The cloud server, edge server alliance, mobile user, and inference task attributes of the system are represented as follows: 1-1-1) System Set, Time Range, and Three-Tier Architecture Modeling: Define the system as a three-tier architecture of cloud-edge-device, including cloud servers, multiple edge servers, and a set of mobile users; define the discrete time range of resource transactions / scheduling as... , Indicates the time slot index. Represents the total number of time slots; defines the mobile user set as... , Indicates user index, Indicates the number of users; defines the edge server set as... , Indicates the server index. This indicates the number of edge servers; edge servers collaborate via wired links and form an edge server alliance; collaboration within the alliance incurs no additional costs, while offloading to cloud servers outside the alliance is billed based on data volume. 1-1-2) Mobile user task attributes and output data volume model: in any time slot Mobile users Generate a generative artificial intelligence model for inference tasks. Its characteristics are represented as follows: ,in, Indicates user In the time slot The amount of input data generated for the task. Indicates task The maximum tolerable latency, This indicates the generative artificial intelligence model identifier for the task request; Represents a set of models. Indicates the number of models; Task output data volume Determined by the model complexity, it can be expressed as: , Representation and Model The relevant fixed output overhead coefficient reflects the basic output quantity used to generate the response; The scaling factor that represents the output as the input increases reflects the output expansion rate; 1-1-3) indicates that the decision of user task offloading between the primary edge server, collaborative edge server, or cloud server is a variable. That is, when When indicates that the task is executed on the main edge server associated with the user; when This indicates that the task is forwarded from the primary edge server to a collaborative edge server within the alliance for execution, and the collaborative edge server is denoted as... That is, a collection of edge servers Except for the main edge server Other edge servers besides; when The time indicates that the task is offloaded to the cloud server for execution; the edge server's caching decision for the generative AI model is a binary variable. That is, when Time indicates edge server In the time slot The model was cached. ,when Time indicates edge server Uncached model ; indicates that the resource allocation decision of the edge server for tasks is a continuous variable. ,in Represents edge server In the time slot For the task The amount of computing resources allocated.

5. The method of claim 4, wherein, For each edge server Configure computing and storage resources, with computing power being [missing information]. GPU memory capacity is Used for caching generative artificial intelligence models ;Model The features are simplified to their size And the number of CPU cycles required to process each MB of data. Multiple edge servers collaborate via wired links, forming an edge server alliance. Collaboration within the alliance incurs no additional costs, while collaboration outside the alliance is billed based on data volume. Cloud servers cache all models. It supports downloading uncached models from edge servers and handles inference requests that edge servers cannot complete.

6. The method according to claim 2, characterized in that, In steps 1-2, the communication model at least characterizes data transmission latency and transmission energy consumption, and the computation model at least characterizes inference computation latency and computation energy consumption. The inference task includes the amount of input data, the amount of output data, and the maximum tolerable end-to-end latency parameter. Specifically, the communication model, computation model, and energy consumption model are as follows: 1-2-1) For the uplink communication process from the mobile user to the edge server, the NOMA-SIC mechanism is adopted: Let the user... Uplink power is and edge servers The channel gain is The noise power is Uplink bandwidth is ;Please press the user Arranged in descending order If the user Located in the The uplink communication rate is calculated based on the NOMA serial interference cancellation SIC mechanism. ; The uplink transmission delay is , and the energy consumption is ; For the downlink transmission process from the edge server to the mobile user: the downlink bandwidth is... Edge server Downlink power is The downlink communication rate was calculated based on the NOMA-SIC mechanism. Downlink latency is Energy consumption is ; For the main edge server With collaborative edge servers The wired collaborative transmission process between edge servers: The wired transmission rate between edge servers is... The task migration latency is then... Energy consumption is ,in, Indicates the main edge server To collaborative edge server Transmit power for wired data transmission, return power , Similarly; For the wired backhaul process from the edge server to the cloud: the wired backhaul rate from the edge server to the cloud is... Upload latency Energy consumption is ,in, Represents edge server For wired data transmission to the cloud, using a "two-hop downlink backhaul strategy," the downlink latency and energy consumption for transmitting the cloud-side results back to the user are as follows: ,in, This indicates the downlink backhaul power of the cloud; the "two-hop downlink backhaul strategy" refers to the inference result being first backhauled from the execution node to the main edge server, and then sent to the user by the main edge server via the wireless downlink; 1-2-2) When the edge server is in a time slot For users Allocate computing resources At that time, the edge reasoning latency is ,in Representation Model Number of CPU cycles required to process each MB of data. Represents edge server In the time slot Assigned to task The amount of computing resources required; when the task is executed on the cloud server, the inference latency is... ,in, This represents the equivalent computing resources provided by the cloud for inference tasks, used to characterize cloud-based inference capabilities; when the task is offloaded to a collaborative edge server... During execution, task completion latency includes migration latency and collaborative reasoning latency. ,in Indicates collaborative edge server Inference latency for task execution and These represent migration and backhaul latency, respectively. 1-2-3) Edge Server The computational energy consumption for performing the inference task is The computing energy consumption of a cloud server performing inference tasks is... ,in and For edge servers And the computing energy consumption coefficient of cloud servers.

7. The method of claim 2, wherein, In steps 1-3, the constructed hierarchical joint optimization mechanism includes a slow-time-scale model heat evaluation and greedy caching decision mechanism, and a fast-time-scale graph structure reinforcement learning-based inference unloading and resource allocation mechanism. The graph structure reinforcement learning mechanism uses edge topology and state aggregation to obtain collaborative context and output hybrid actions. The hierarchical joint optimization mechanism operates based on model request popularity, cache capacity constraints, task latency constraints, and edge computing resource status, specifically as follows: 1-3-1) Model popularity assessment and slow time-scale cache update: In each cache update cycle, the system is based on edge servers. Observed historical task requests, statistics for each model On the edge server The popularity of requests on the platform, combined with model size. Edge server cache capacity Perform cache candidate sorting; update the model caching decision on the edge server accordingly. This allows for prioritizing the caching of more popular and cost-effective models while meeting cache capacity constraints. 1-3-2) For any edge server The candidate model set is sorted in descending order according to the "popularity-size" comprehensive index, and a greedy method is used to select models to enter the cache one by one until the cache capacity constraint is met. Get the updated caching strategy ,in Represents edge server In the time slot caching model , Indicates uncached model ; 1-3-3) In each time slot The system is based on the current cache state. Link transmission model and computation model to generate offloading decisions With resource allocation decisions , where: when Time indicates task By the main edge server Execute locally; when The time indicates that the task is handled by the main edge server. Offload to Collaborative Edge Server CES Execute; when This indicates that the task is unloaded to the cloud server for execution; and is achieved through allocation. Make the inference delay satisfy the maximum tolerable delay. Service quality constraints; 1-3-4) Cooperative scheduling is carried out based on graph structure reinforcement learning. By aggregating states and outputting hybrid actions, the topology of the edge server alliance is used as the graph structure input to process the edge server state and cache state. The task state is aggregated into a neighborhood to obtain a state representation that reflects the collaborative context. Based on this, a reinforcement learning strategy is used to output hybrid actions, where the discrete part corresponds to the unloading decision. The continuous part corresponds to resource allocation The strategy is iteratively updated based on the system benefit feedback after execution, so that the fast time-scale scheduling strategy can be adaptively adjusted according to the dynamic changes in requests and resources. 1-3-5) Caching decisions in slow timescales Updates are triggered according to the cache update cycle, and frequent cache jitter is constrained by the cost of model switching. In the fast time scale, online decisions are made on unloading and resource allocation during the period when the cache policy is fixed. When the cache policy is updated, the fast time scale scheduling strategy makes new decisions based on the updated cache state to achieve joint optimization of cache-unloading-resource allocation in two time scales.

8. The method according to one of claims 1 to 7, characterized in that In step S2), the model cache update decision is represented as a cache decision set. The unloading decision is represented as an unloading decision set. Edge computing resource allocation decisions are represented as resource allocation decision sets. Step S2) specifically includes the following steps: Step 2-1: Obtain user satisfaction and revenue; Step 2-2: Obtain system cost items; Steps 2-3: Obtain system benefits and propose long-term optimization goals; In step 2-1, for any user In the time slot The total task completion delay is denoted as Then, the time delay satisfaction is calculated as follows: and calculate user The benefits are: ,in Pay coefficient for users; calculate edge server In the time slot The income is: ,in, Indicates in time slot Access to edge server A subset of mobile users; In step 2-2, the cloud inference cost is calculated as follows: ,in The unit data cost coefficient in the cloud; the energy consumption cost is calculated as follows: ,in For energy price coefficient, For the task The total energy consumption generated at the edge layer; the cost of switching the computational model is: ,in Represents edge server In the time slot Should the new model be cached? , For the model The comprehensive unit cost coefficient, For model size; In steps 2-3, the edge alliance is calculated in the time slot. The overall benefits are: The long-term optimization objective of the system is expressed as maximizing the overall alliance benefit across all time slots: .

9. The method of claim 8, wherein, Step S3 determines the variables to be optimized for the optimization objective described in Step S2, and decomposes the overall optimization problem into two sub-problems: slow-time-scale cache optimization and fast-time-scale online scheduling. Step S3 includes the following sub-steps: Step 3-1: Identify the variables to be optimized; Step 3-2: Determine the constraints of the optimization objective; Step 3-3: Decompose the overall optimization problem into two phases of sub-problems; Steps 3-4: Solve the slow-time-scale caching subproblem and the fast-time-scale scheduling subproblem respectively; In step 3-1, to maximize the long-term benefit of the system, the following decision variables are optimized: the set of offloading decisions ; the set of model caching decisions ; Resource allocation decision set ; and as an optimization objective; In step 3-2, the constraints set include at least the following: 1) Cache capacity constraint: ; 2) Computational resource constraints: and edge servers The total amount of allocated resources shall not exceed its computing power. ; 3) Quality of service constraints: ; 4) unloading feasibility constraint: when selecting or , the corresponding execution node cache state and resource availability requirements should be met; when not met, allow to select unloading to the cloud as a bottom line; 5) variable value constraints: , ; In step 3-3, the overall optimization problem is divided into two stages: the first stage is the slow-time-scale model cache optimization stage, which updates the cache based on historical request popularity and cache capacity constraints within the cache update cycle. The second stage is a joint scheduling stage for fast-timescale inference offloading and resource allocation. Under the condition of a fixed caching strategy, online generation is performed for each time slot. and ; In steps 3-4, the first stage uses a greedy caching strategy based on model heat assessment to solve the problem. and through To suppress frequent switching, the second stage employs an online scheduling strategy based on graph structure reinforcement learning to solve the problem. and The strategy is iteratively updated based on the system benefits defined in step S2 to obtain a stable scheduling strategy under dynamic requests from multiple users.

10. The method of claim 9, wherein, In steps 3-4, the specific steps for solving the slow-time-scale model caching subproblem and the fast-time-scale inference offloading and resource allocation subproblem are as follows: 3-4-1) Slow time scale: Greedy model cache solution based on request popularity; 1) Hotness statistics: At the arrival of the cache update period, count the frequency of each model requested in the history time slot, get the model request hotness. Statistics of each model in the history time slot The frequency of each model requested in the history time slot is counted, and the model request hotness is obtained. 2) Candidate ranking: combined model size with edge server cache capacity ranking the candidate models to get a cache priority sequence; 3) Greedy loading: Select models to be written to the cache in sequence according to the priority order, until the cache capacity constraint is met. This allows us to obtain the caching decision within the cache update cycle. ; 4) Switching suppression: if the current cache update involves new model loading, calculate the model switching cost and suppress frequent loading in the following cache updates to reduce the impact of cache thrashing on system efficiency; 3-4-2) Fast timescale: Online offloading and resource allocation under a fixed caching strategy: in each time slot Internally, based on a fixed caching strategy Build a system containing tasks Cache status The system status, including link communication parameters and computing resource availability information, is input into the policy network to output offloading decisions. With resource allocation decisions ;When the output or If the model availability or resource availability requirements of the corresponding execution node are not met, the unloading decision will be adjusted to... As a safety net; then, the benefits of a single time-slot system are calculated based on communication, computing, and energy consumption models. and with the stated The policy network parameters are updated as learning feedback, and the process is repeated in the next time slot. 3-4-3) Strategy Iteration Updates and Simultaneous Execution of Dual Time Scales: 1) Online iteration: Based on the benefit feedback of each time slot within a fast time scale. The graph structure reinforcement learning strategy is iteratively updated to improve long-term effectiveness under multi-user dynamic requests; 2) Synchronization mechanism: Within each cache update cycle, a 3-4-1 cache update decision is first executed. The caching strategy is fixed during this period to execute 3-4-2 and 3-4-3; when the caching strategy is updated, the online scheduling strategy continues to be generated based on the new caching state. and This enables the coordinated operation of slow-timescale cache updates and fast-timescale online scheduling.

Images