Image-guided cloud revenue threshold routing with stateful demand estimation method and system

Abstract

The application discloses an image-guided cloud benefit threshold routing and stateful demand estimation method and system, which analyzes request metadata, obtains state hit flags and uncache context length from a state cache management module, estimates three execution demands of edges, clouds and splitting, superimposes queuing delays, and takes the difference between edge and cloud demands as cloud benefits; according to the joint gating of privacy permission, cloud call budget not exhausted, and cloud benefits not lower than the quantile threshold determined by the budget, and when cloud is allowed, the net benefit criterion selects splitting or whole cloud unloading according to the calculation saving minus state migration cost; the landing request is predicted to be completed by the availability of the work node, and when the SLO trigger value is exceeded and the rejection quota is not exhausted, the request is rejected and the work node time and cloud budget are not occupied. The application can distinguish between requests with cached prefixes and requests that need to be recalculated completely, avoid budget waste or lock, and protect the effective throughput of SLO under overload.

Description

Technical Field

[0001] This invention belongs to the field of large model inference technology under the cloud-edge system, specifically involving a cloud revenue threshold routing and stateful demand estimation method and system guided by profile. Background Technology

[0002] With the widespread application of large language models in long-context tasks, cloud-edge collaborative inference architecture has become an important deployment method that balances computing power and cost. The cloud has stronger computing power, but is constrained by cloud call budgets, privacy policies, bandwidth, and service deployment; edge nodes have a limited number of working nodes, but can handle requests with privacy-sensitive or low-latency requirements.

[0003] In existing technologies, data center-side prefix / context caching reuse schemes (such as patent US20250291722A1) rely solely on prefix reuse frequency for caching decisions, failing to consider cloud call budget and the impact of state transitions across "cloud-edge" links on routing. They also do not address cloud revenue thresholds estimated from offline profiles or SLO-oriented admission controls. Cloud-edge collaborative neural network / large model offloading and splitting schemes (such as patent CN113169990A) are oriented towards stateless DNN computation graph segmentation. The segmentation units are nodes of a fixed computation model, without modeling reusable prefix / session states that exist across boundaries with requests. They do not consider cloud call budget caps, lack routing criteria based on "budget-determined thresholds," and lack SLO admission controls. Another approach (such as patent CN119201476B) focuses on splitting pre-filling / decoding into different execution instances based on prompt tree local perception. It does not involve cloud revenue thresholds estimated by the profile segment, cloud call budget upper limit constraints, joint gating of "privacy / budget / threshold", nor does it involve SLO admission based on rejection quotas.

[0004] The aforementioned existing technologies have the following drawbacks: First, the routing perspective is stateless, treating long-context requests as independently routable inputs, failing to distinguish between "long requests with cached prefixes" and "long requests requiring complete context recalculation," resulting in the waste of scarce cloud call budget on requests with low marginal value; Second, budget and benefit are disconnected, traditional budgeting strategies only "proportionally limit the number of cloud visits," failing to use "cloud benefits" as a criterion for each type of request, causing budget to be wasted or locked by conservative thresholds; Third, benefit estimation ignores reusable state, failing to include "the reduction of uncached context due to cache hits" in demand estimation; Fourth, overload is ignored, if all requests are accepted indiscriminately during overload, queuing delays cause the SLO effective throughput to collapse.

[0005] In conclusion, a combined approach is needed to overcome the aforementioned shortcomings. Summary of the Invention

[0006] To address the aforementioned issues, this invention discloses a cloud revenue threshold routing method and system guided by user profiles, and establishes a joint scheme of "user profile-guided threshold routing + cloud call budget + state-related demand estimation" in a cloud-edge environment. This scheme optimizes the three components from their independent local aspects to form a closed-loop global decision-making mechanism, achieving low-tail latency, high budget utilization, and efficient SLO throughput.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] The cloud revenue threshold routing and stateful demand estimation method guided by user profiles includes the following steps:

[0009] S1 Access and Resolution: Receive long context requests and parse out the load source, task type, session identifier, prefix identifier, new prompt length, history length, output length, privacy level, and arrival time;

[0010] S2 Uncached Context Access: Obtain the hit flag and uncached context length from the state cache management module; the more hits, the smaller the portion to be pre-filled.

[0011] S3 Demand Estimation: Estimate the demand for edge, cloud, and split solutions respectively, and add the queuing delay of each current working node to obtain the estimated completion delay including queuing delay;

[0012] S4 Cloud Revenue Calculation: Calculate the difference between edge demand and cloud demand to obtain the cloud revenue for the request;

[0013] S5 Privacy Gating: If the requested privacy level does not allow cloud access, proceed directly to the edge and execute S6; otherwise, proceed to cloud budget / threshold gating S7.

[0014] S6 Edge Execution: The request is executed on the edge node without consuming cloud call budget;

[0015] S7 Budget and Threshold Gating: Cloud migration is only considered when "the proportion of cloud calls used is lower than the cloud call budget and the cloud revenue is not lower than the threshold of this group"; otherwise, S6 is executed at the edge.

[0016] S8 split benefit determination: Evaluate the split execution among the candidates that are allowed to be moved to the cloud. Only select to split S9 if the "computation savings - state transition costs" is a net benefit and is better than the whole cloud; otherwise, select to unload S10 to the whole cloud.

[0017] S9 Split Execution: Pre-filling is completed in the cloud and decoding is completed at the edge; it is only selected when the net split profit is positive.

[0018] S10 Cloud Unloading Execution: Unloads the entire request to the cloud for execution;

[0019] S11 Admission Decision: For requests that are about to be sent to the edge / cloud, predict their completion latency using the current availability of the worker node; if the SLO trigger value is exceeded and the rejection quota has not been exhausted, then S12 is rejected without consuming worker node time or cloud budget.

[0020] S12 Request Rejection: Does not consume worker node time or cloud call budget;

[0021] S13 dispatch and recording: dispatch execution and record metrics such as cloud call percentage, latency, SLO effective throughput and rejection rate.

[0022] As a supplement to the present invention, the cloud revenue threshold is calculated by grouping offline profile segments according to "task type × load source" and using the quantile determined by the cloud call budget.

[0023] Cloud revenue :

[0024] ;

[0025] Cloud revenue threshold (Image segment calculation, by group) ):

[0026] ;

[0027] in, , For edge computing and cloud computing needs, As the source of load, For the image segment request set, For privacy level, The quantile operator represents the percentage of cloud revenue that can be extracted from the set of requests in this group that are allowed to access the cloud. Quantiles This sets the upper limit for the proportion of cloud usage budget.

[0028] As a supplement to the present invention, in step S7, when the image threshold is too conservative and the cloud call budget is not fully utilized, budget filling correction is performed to lower the threshold to fully utilize the budget.

[0029] As a supplement to the present invention, in step S2, the uncached context length is the portion of the total request context that must be re-filled at the server endpoint, which is obtained by subtracting the prefix length of the cached data and the session length from the total length.

[0030] As a supplement to the present invention, in step S3, cloud requirements, edge requirements, and splitting requirements are:

[0031] ;

[0032] ;

[0033] ;

[0034] In the split formula The explicit term "state transition cost" is the key to whether the split is worthwhile.

[0035] , , These are respectively cloud / edge / split requirements. For device-to-cloud round-trip latency, For input transmission time, Input transmission time at the edge; The input token quantity and the amount of tokens to be migrated are given. For device-to-edge round-trip latency, , Edge-to-cloud round-trip latency / migration transmission time; , For cloud-prefilled / decoding coefficients, , Edge prefilling / decoding coefficients; Total context length, ; The length of the reusable prefix; For the available history length, For the new prompt length, The length of the uncached context is calculated and provided by the state cache management module based on the hit flag. For the output length, The number of bytes occupied by the reusable state per token.

[0036] Estimated completion delay including queuing delay :

[0037] ;

[0038] in, For the arrival time, For resources Previous working node The next available time, For resources The set of working nodes For the selected working node, r {e,c,s} correspond to edge, cloud, and splitting requirements, respectively. This corresponds to the service requirements at the edge, in the cloud, or in a split.

[0039] As a supplement to the present invention, there are three conditions for the conjunction of steps S5 and S7:

[0040] (1) Online routing determination:

[0041] ;

[0042] in, Privacy level; For service requests The percentage of cloud calls already consumed. The upper limit for the proportion of cloud usage budget. Cloud revenue thresholds grouped by task type × load source For the request The threshold of the group to which it belongs.

[0043] (2) Admission criteria:

[0044] ;

[0045] in To predict completion delays, This is the SLO trigger protection factor. For the SLO objective, This represents the current number of rejections. To refuse the quota;

[0046] (3) Constraints and evaluation indicators:

[0047] ;

[0048] ;

[0049] in, For the total number requested, For statistical time windows, For the strategy of request The selected action For SLO to achieve effective throughput, For rejection rate, The upper limit for the proportion of cloud usage budget. For the request The service completion was delayed.

[0050] As a supplement to the present invention, in step S9, the state transition cost is explicitly included in the demand estimation of the split execution, and the state transition cost is calculated based on the uncached state volume.

[0051] As a supplement to the present invention, in step S11, the rejected request in the SLO admission determination does not occupy cloud or edge worker node time, nor does it occupy cloud call budget.

[0052] The scenario and participants of this invention: This invention is aimed at "cloud-edge collaborative long context large model inference service system". The execution entity is the long context large model online decision control unit deployed in the cloud-edge system; the participants include user-side devices (initiating long context requests), edge nodes (with a limited number of working nodes) and cloud backend (with faster computing power, but constrained by cloud call budget, privacy policy, bandwidth and service deployment).

[0053] How the title object described in this invention enters the system: Long context requests initiated by user devices carry reusable states—including shared prefixes (repeated system prompts, search document prefixes, shared task templates) and multi-round session history. The request is parsed by the gateway to extract metadata—load source, task type, session identifier, prefix identifier, new prompt length, history length, output length, privacy level, and arrival time—and then enters the online decision control unit; the control unit obtains the "uncached context length" (calculated from the hit flag) corresponding to the request from the state cache management module, and then estimates cloud benefits and makes routing and admission decisions based on this.

[0054] The cloud benefits described in this invention are the difference in estimated demand between two solutions for the same request: "execution at the edge" and "execution in the cloud". A positive value indicates that the cloud is faster and worthwhile.

[0055] The cloud revenue threshold described in this invention is a quantile determined by the cloud call budget for a certain "task type × load source" group, used to determine the minimum cloud revenue at which requests for that group are "worth migrating to the cloud".

[0056] The reusable state described in this invention is an intermediate representation that can be reused across requests, divided into "prefix state" (shared prompt / retrieval document prefix) and "session state" (multi-turn session / multi-turn task accumulated history).

[0057] The SLO effective throughput and rejection rate described in this invention are: the throughput of requests completed within the SLO time limit without being rejected; and the proportion of rejected requests to the total number of requests. These two factors are used to jointly evaluate admission control.

[0058] This invention also introduces a profile-guided cloud revenue threshold routing and stateful demand estimation system, comprising:

[0059] The request feature encoder maps arriving long context requests into a unified tuple; the tuple includes the load source, task type, session identifier, prefix identifier, new prompt length, history length, output length, privacy level, and arrival time.

[0060] The Uncached Context Access Unit is used to receive the status hit flag and the length of the uncached context returned by the status cache management module, and incorporate the length into the edge / cloud / split demand estimation, so that the cloud revenue estimation can distinguish between prefix cached contexts and contexts that need to be recalculated.

[0061] The demand estimator is used to estimate the demand for three scenarios: edge execution, cloud execution, and split execution, and adds the queuing delay of the worker nodes to output the estimated completion delay including the queuing delay.

[0062] The cloud revenue profile storage unit stores the "cloud revenue distribution and threshold" from the profile segment. It stores the cloud revenue threshold determined by the cloud call budget in groups according to "task type × load source" for online routing gating query. At the same time, it records the budget usage to support budget filling and correction.

[0063] The routing / gating layer determines whether to migrate to the cloud based on the joint conditions of "privacy allowed → cloud budget not exhausted → cloud revenue ≥ threshold", and evaluates split execution when cloud migration is allowed; outputs one of four actions: "edge execution, cloud offloading, split execution, or rejection";

[0064] The SLO admission controller, after making a routing decision, predicts the request completion latency using the current worker node availability; if the SLO trigger value is exceeded and the rejection quota is not exhausted, the request is rejected without consuming cloud / edge worker node time; it outputs the effective SLO throughput and rejection rate.

[0065] The execution scheduler dispatches requests to edge worker nodes or cloud nodes for execution according to the selected action, and maintains the next available time for each worker node for demand estimation and admission determination; at the same time, it records indicators such as cloud call ratio, latency and throughput.

[0066] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0067] The present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the program to implement the steps of the above-described method.

[0068] The beneficial effects of this invention are as follows:

[0069] (1) Significantly eliminate the catastrophe of tail latency under budget constraints: By using cloud revenue threshold routing guided by profile, limited cloud calls are spent on requests with the highest marginal value; under the load constructed from public long context / multi-turn dialogue datasets, P95 latency is reduced by about 97%~98% compared to the naive budget baseline, and by about 14% compared to the ablation of “profile-guided but stateless”.

[0070] (2) Revenue estimation distinguishes between cached and recalculated requests: Cloud revenue is calculated based on the “uncached context length”, enabling the route to identify cheap requests with “prefix cached” and expensive requests that “require full recalculation”, thus avoiding wasting budget on low-revenue requests.

[0071] (3) Budget waste prevention and lock-in: The threshold is determined by the budget itself (quantile), and with the budget filling correction, the two failure modes of "conservative threshold lock-in budget" or "using the full budget but low returns" are avoided.

[0072] (4) Explicit state transition pricing to avoid incorrect splitting: Explicitly include split demand in pricing. This ensures that splitting is only selected when the "computation savings - migration cost" is a net benefit, thus avoiding the wrong selection of splitting when state transitions are expensive.

[0073] (5) Non-gaze, no semantic labeling required, easy to deploy: Requesting metadata only requires the source / task / session / prefix / approximate token length known to the gateway. It does not depend on future information, does not require semantic labels or model quality prediction, and can be run by simple online rules.

[0074] (6) Overload protection for effective SLO throughput: Admission control predicts completion delay based on worker node availability, and rejects requests when the threshold is exceeded and the rejection quota is not exhausted; under approximately 4x request overload, the effective throughput of 30-second SLO increases from approximately 9.5×10² to approximately 1.6×10². 4 Requests / hour, rejection rate of approximately 10%, and costs are disclosed truthfully based on a combined assessment of effective throughput and rejection rate. Attached Figure Description

[0075] Figure 1 This is a block diagram of the long-context large-model online decision control system of the present invention.

[0076] Figure 2 This is a flowchart illustrating the stateful online routing and admission decision-making process guided by the profiling of this invention. Detailed Implementation

[0077] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0078] Example 1 (System)

[0079] like Figure 1 As shown, this invention provides a profile-guided cloud revenue threshold routing and stateful demand estimation system, which includes:

[0080] The request feature encoder maps arriving long context requests into a unified tuple; the tuple includes the load source, task type, session identifier, prefix identifier, new prompt length, history length, output length, privacy level, and arrival time.

[0081] The Uncached Context Access Unit is used to receive the status hit flag and the length of the uncached context returned by the edge status caching module, and incorporate the length into the edge / cloud / split demand estimation, so that the cloud revenue estimation can distinguish between prefix cached contexts and contexts that need to be recalculated.

[0082] The demand estimator is used to estimate the demand for three scenarios: edge execution, cloud execution, and split execution, and adds the queuing delay of the worker nodes. It outputs the estimated completion delay including the queuing delay and calculates the cloud revenue based on the difference between edge demand and cloud demand.

[0083] The cloud revenue profile storage unit stores the "cloud revenue distribution and threshold" from the profile segment. It stores the cloud revenue threshold determined by the cloud call budget in groups according to "task type × load source" for online routing gating query. At the same time, it records the budget usage to support budget filling and correction.

[0084] The routing / gating layer receives the cloud revenue output by the demand estimator and the cloud revenue threshold output by the cloud revenue profile storage unit. It determines whether to migrate to the cloud based on the joint condition of "privacy allowed → cloud budget not exhausted → cloud revenue ≥ threshold". When migration to the cloud is allowed, it evaluates split execution and outputs one of four actions: "edge execution, cloud offloading, split execution, or rejection".

[0085] The SLO admission controller, after making a routing decision, predicts the request completion latency using the current worker node availability; if the SLO trigger value is exceeded and the rejection quota is not exhausted, the request is rejected without consuming cloud / edge worker node time; it outputs the effective SLO throughput and rejection rate.

[0086] The execution scheduler receives the actions output by the routing / gating layer and the allowance results output by the SLO admission controller. It dispatches the request to the edge worker node or cloud node for execution according to the selected action, and maintains the next available time for each worker node for demand estimation and admission determination. At the same time, it records indicators such as cloud call ratio, latency and throughput.

[0087] Example 2 (Main path: prefix cached → edge execution)

[0088] like Figure 2 As shown, this embodiment presents a path executed by the edge in this method.

[0089] A long context request for "multi-document retrieval question and answer" has a document prefix that was previously written to the edge state cache by a request in the same session, therefore the prefix state is hit. The length of the uncached context is significantly less than the total length, the cost of edge pre-filling is greatly reduced, the cloud benefit is negative (the cloud is not faster), and the privacy level allows edge execution; the control unit determines that the request is executed at the edge, and this embodiment reflects "cache hit discount → low cloud benefit → edge execution".

[0090] Example 3 (Branch: Context not cached, high cloud benefits → Cloud uninstallation)

[0091] like Figure 2 As shown in the figure, this embodiment provides a path for the cloud-based uninstallation execution in this method.

[0092] A certain "code completion" request is indicated by the shared system as reusable, but the long code context has not yet been cached and needs to be completely recalculated. Cloud benefits are high, and the current cloud call usage percentage has not yet reached the budget, and the privacy level allows cloud migration. The control unit determines that the code should be offloaded to the cloud. This example embodies the principle of "high cloud benefits + budget allowance → cloud migration".

[0093] Example 4 (Branch: Overload → Admission Denial)

[0094] like Figure 2 As shown, this embodiment provides a path for the method to be executed under overload conditions.

[0095] Under approximately 4x request overload, if the expected completion delay of a long context request exceeds the SLO trigger value and the current rejection count has not yet reached the rejection quota, the control unit determines to reject the request without consuming any worker node time or cloud budget; the remaining accepted requests can maintain effective SLO throughput; this embodiment reflects "admission rejection → maintaining effective throughput" and illustrates that the rejection rate and effective SLO throughput need to be evaluated together.

[0096] Technical effectiveness (load validation constructed based on publicly available benchmark datasets)

[0097] The stateful strategy guided by the profile reduces P95 latency by approximately 97%-98% compared to the naive budget baseline and by approximately 14% compared to "profile-guided but stateless" ablation.

[0098] Under approximately 4x overload, the effective throughput of a 30-second SLO increases from approximately 9.5 × 10² to approximately 1.6 × 10². 4 Requests per hour, SLO compliance rate significantly improved, rejection rate approximately 10%.

[0099] State transition pricing prevents split execution from being selected under the current transition cost (indicating that splitting is not inherently advantageous and transition costs must be explicitly priced).

[0100] Parameter examples

[0101] This embodiment provides examples of key technical parameters. Cloud call budget percentage limit. Set to 0.65; SLO target Use seconds as the reference, or 30 seconds; SLO trigger protection coefficient. Take 1.2; Reject quota The agreed-upon percentage of total requests, for example, approximately 10%; edge prefilling / decoding coefficients. , Calibrated by edge micro-reference, with a token prefilling time of 0.3ms / token and a decoding time of 25ms / token; cloud prefilling / decoding coefficients. , Cloud-based configuration and calibration are typically faster than edge computing; each token can reuse state bytes. Calibrated by edge measurement, using quantities ranging from hundreds of bytes to kilobytes per token; cloud revenue threshold. Depend on The quantile calculation is obtained and varies with task / source grouping; uncached context length. The value is calculated based on the hit status flag; the more hits, the smaller the value. (Total number of requests) For example, requests in the thousands; statistical time windows. Hourly level; SLO effective throughput Output metrics, unit: requests / hour, rejection rate As an output metric, approximately 10%.

[0102] The above values ​​are for illustrative purposes only; actual values ​​should be calibrated according to the measurement curves of the deployed devices. Cloud call budget, SLO target, rejection quota, etc., can be configured according to business needs.

[0103] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A cloud revenue threshold routing and stateful demand estimation method guided by user profiles, characterized by: Includes the following steps: S1 Access and Resolution: Receive long context requests and parse out the load source, task type, session identifier, prefix identifier, new prompt length, history length, output length, privacy level, and arrival time; S2 Uncached Context Access: Obtain the hit flag and uncached context length from the state cache management module; The more hits, the smaller the portion to be pre-filled; the uncached context length is the portion of the total request context that must be re-pre-filled at the server endpoint, obtained by subtracting the prefix length of the cache hits and the session length from the total length; S3 Demand Estimation: Estimate the demand for edge, cloud, and split solutions respectively, and add the queuing delay of each current working node to obtain the estimated completion delay including queuing delay; S4 Cloud Revenue Calculation: Calculate the difference between edge demand and cloud demand to obtain the cloud revenue for the request; S5 Privacy Gating: If the requested privacy level does not allow cloud access, proceed directly to the edge and execute S6; otherwise, proceed to cloud budget / threshold gating S7. S6 Edge Execution: The request is executed on the edge node without consuming cloud call budget; S7 Budget and Threshold Gating: Only when "the proportion of cloud calls used is lower than the cloud call budget, and the cloud revenue is not lower than the threshold of this group" will cloud migration be considered; otherwise, S6 will be executed at the edge. S8 split benefit determination: Evaluate the split execution among the candidates that are allowed to be moved to the cloud. Only select to split S9 if the "computation savings - state transition costs" is a net benefit and is better than the whole cloud; otherwise, select to unload S10 to the whole cloud. S9 Split Execution: Pre-filling is completed in the cloud and decoding is completed at the edge; it is only selected when the net split profit is positive. S10 Cloud Unloading Execution: Unloads the entire request to the cloud for execution; S11 Admission Decision: For requests that are about to be sent to the edge / cloud, predict their completion latency using the availability of the current worker node; If the SLO trigger value is exceeded and the rejection quota is not exhausted, then S12 will be rejected without consuming worker node time or cloud budget. S12 Request Rejection: Does not consume worker node time or cloud call budget; S13 dispatch and recording: dispatch execution and record metrics such as cloud call percentage, latency, SLO effective throughput and rejection rate.

2. The method according to claim 1, characterized in that: The cloud revenue threshold is calculated by grouping offline profile segments according to "task type × load source" and using the quantile determined by the cloud call budget; Cloud revenue : ； Cloud revenue threshold : ； in, , For edge computing and cloud computing needs, As the source of load, For the image segment request set, For privacy level, The quantile operator represents the percentage of cloud revenue that can be extracted from the set of requests in this group that are allowed to access the cloud. Quantiles This sets the upper limit for the proportion of cloud usage budget.

3. The method according to claim 1, characterized in that: In step S7, when the image threshold is too conservative and the cloud call budget is not fully utilized, budget filling correction is performed to lower the threshold to make full use of the budget.

4. The method according to claim 2, characterized in that: There are three conditions that satisfy the conjunction of steps S5 and S7: (1) Online routing determination: ； in, Privacy level; For service requests The percentage of cloud calls already consumed. The upper limit for the proportion of cloud usage budget. Cloud revenue thresholds grouped by task type × load source For the request The threshold of the group to which it belongs; (2) Admission criteria: ； in To predict completion delays, This is the SLO trigger protection factor. For the SLO objective, This represents the current number of rejections. To refuse the quota; (3) Constraints and evaluation indicators: ； ； in, For the total number requested, For statistical time windows, For the strategy of request The selected action For SLO to achieve effective throughput, For rejection rate, The upper limit for the proportion of cloud usage budget. For the request The service completion was delayed.

5. The method according to claim 1, characterized in that: In step S3, cloud requirements, edge requirements, and splitting requirements are discussed: ； ； ； In the split formula The term is explicitly defined as "state transition cost". , , These are respectively cloud / edge / split requirements. For device-to-cloud round-trip latency, For input transmission time, Input transmission time at the edge; The input token quantity and the amount of tokens to be migrated are given. For device-to-edge round-trip latency, , Edge-to-cloud round-trip latency / migration transmission time; , For cloud-prefilled / decoding coefficients, , Edge prefilling / decoding coefficients; Total context length, ; The length of the reusable prefix; For the available history length, For the new prompt length, The length of the uncached context is calculated and provided by the state cache management module based on the hit flag. For the output length, The number of bytes occupied by the reusable state per token. Estimated completion delay including queuing delay : ； in, For the arrival time, For resources Previous working node The next available time, For resources The set of working nodes For the selected work node, For the corresponding edge, cloud, or split service requirements.

6. The method according to claim 1, characterized in that: In step S9, the state transition cost is explicitly included in the demand estimation for split execution, and the state transition cost is calculated based on the uncached state volume.

7. The method according to claim 1, characterized in that: In step S11, during the SLO admission determination, rejected requests do not consume cloud or edge worker node time, nor do they consume cloud call budget.

8. A cloud revenue threshold routing and stateful demand estimation system guided by user profiles, used in the method described in any one of claims 1-7, characterized in that: include: The request feature encoder maps arriving long context requests into a unified tuple; the tuple includes the load source, task type, session identifier, prefix identifier, new prompt length, history length, output length, privacy level, and arrival time. The Uncached Context Access Unit is used to receive the status hit flag and the length of the uncached context returned by the status cache management module, and incorporate the length into the edge / cloud / split demand estimation, so that the cloud revenue estimation can distinguish between prefix cached contexts and contexts that need to be recalculated. The demand estimator is used to estimate the demand for three scenarios: edge execution, cloud execution, and split execution, and adds the queuing delay of the worker nodes to output the estimated completion delay including the queuing delay. The cloud revenue profile storage unit saves the "cloud revenue distribution and threshold" from the profile segment. It stores the cloud revenue threshold determined by the cloud call budget in groups according to "task type × load source" for online routing gating query. At the same time, it records the budget usage to support budget filling and correction. The routing / gating layer determines whether to migrate to the cloud based on the joint conditions of "privacy allowed → cloud budget not exhausted → cloud revenue ≥ threshold", and evaluates split execution when cloud migration is allowed; outputs one of four actions: "edge execution, cloud offloading, split execution, or rejection". The SLO admission controller predicts request completion latency based on the availability of the current worker node after making a routing decision. If the SLO trigger value is exceeded and the rejection quota has not been exhausted, the request is rejected without consuming time on the cloud / edge worker node. Output SLO effective throughput and rejection rate; The execution scheduler dispatches requests to edge worker nodes or cloud nodes for execution according to the selected action, and maintains the next available time for each worker node for demand estimation and admission determination; at the same time, it records the cloud call ratio, latency and throughput metrics.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by a processor, the program implements the steps of the method according to any one of claims 1-7.

10. An electronic device comprising a processor and a memory, characterized in that: The memory stores a computer program, and when the processor executes the program, it implements the steps of the method according to any one of claims 1-7.

Images