Method and system for scheduling large model inference tasks facing fuzzy deadline constraints

Abstract

This application discloses a scheduling method and system for large-model inference tasks with fuzzy deadline constraints. It utilizes a fuzzy deadline modeling mechanism based on trapezoidal membership functions, a world model based on a time-skip semi-Markov decision process, and a task scheduling method based on Gumbel noise-driven Monte Carlo tree search to transform fuzzy deadline constraints into continuous delay penalty signals in each scheduling decision. Combined with a prediction mechanism for task execution latency and cost, it addresses the state changes and resource heterogeneity of large-model inference tasks in dynamic environments, thereby optimizing inference task scheduling decisions and execution node selection, reducing average latency, execution cost, and deadline default rate. To improve the foresight and accuracy of scheduling decisions, a time-skip state modeling mechanism is introduced to predict system state evolution, and Gumbel noise is used to enhance the exploration capability during the search process, enabling effective evaluation and optimization of candidate scheduling strategies.

Description

Technical Field

[0001] This application relates to the fields of edge computing and task scheduling technology, specifically to a method and system for scheduling large model inference tasks with fuzzy deadline constraints. Background Technology

[0002] With the rapid development of large-scale deep learning models, large model inference tasks are widely used in scenarios such as industrial internet, intelligent manufacturing and edge intelligent services. These tasks are usually characterized by high computing resource requirements and strict response latency requirements, which puts forward higher requirements for task scheduling in cloud-edge collaborative environments.

[0003] Currently, in cloud-edge collaborative computing environments, inference tasks are typically generated by terminal devices and offloaded to edge servers or cloud servers for execution based on system resource status. However, due to the limited computing power of edge servers, dynamic changes in resource load, and uncertainties in the network environment, task execution latency is highly dynamic and unpredictable. Furthermore, large model inference processes usually involve multiple stages, such as pre-filling and decoding stages, with significant differences in computational overhead between different stages, making task scheduling more complex.

[0004] In practical applications, task deadlines are usually flexible and not strictly hard deadlines. Traditional scheduling methods typically use fixed deadline models, which are difficult to accurately characterize the task's tolerance for latency, and can easily lead to unreasonable scheduling decisions, thus affecting the overall system performance.

[0005] Therefore, there is an urgent need for a scheduling method for large model inference tasks in a cloud-edge collaborative environment that can ensure task completion latency while taking into account execution cost and system resource utilization efficiency. Summary of the Invention

[0006] Purpose of the invention: In order to overcome the above shortcomings, the purpose of this application is to provide a scheduling method for large model inference tasks oriented towards fuzzy deadline constraints.

[0007] To address the aforementioned technical problems, this application provides a scheduling method for large model inference tasks oriented towards fuzzy deadline constraints, including: S1: Model the large-scale inference task scheduling system in the cloud-edge collaborative computing environment and construct a fuzzy deadline constraint model. The modeling includes at least the following: task structure representation, cloud server and edge server modeling, resource heterogeneity modeling, task execution latency modeling, and task execution cost modeling. S2: For the fuzzy deadline constraint model, the membership function is used to map the task completion delay to continuous satisfaction and a delay penalty function is constructed based on the continuous satisfaction, thereby forming a reward function that includes task delay, resource cost and deadline default degree; S3: Based on the fuzzy deadline constraint model, construct a scheduling action space and construct a time-skip semi-Markov decision process model based on the scheduling action space and the system state, thereby constructing a world model that includes at least a state encoding model, a state prediction model, a time prediction model, a cost prediction model, and a policy network, and train the time-world model in the cloud. S4: Based on the world model, a scheduling decision search mechanism is constructed using the Gumbel noise-driven Monte Carlo tree search method, and candidate scheduling actions are evaluated and selected at each decision moment; S5: Input the current system state into the world model to make multi-step predictions of the future state, output a scheduling scheme, and allocate the inference task to the corresponding cloud server or edge server for execution according to the scheduling scheme, and update the system state according to the execution result.

[0008] As a preferred embodiment of this application, in S1, the method includes the following steps: S11: Obtain a large set of model inference tasks from existing data sources, where each inference task is represented as a task data structure consisting of task identifier, task arrival time, input token length, output token length, and deadline parameter; S12: Perform structured parsing on the reasoning task, extract task input and output feature information and task execution stage information, and abstract the reasoning task into a pre-filling stage and a decoding stage; S13: Construct computing and communication models for cloud servers and edge servers, and establish execution latency and resource consumption models for tasks on different computing nodes to obtain the execution latency and execution cost of tasks on each server, where different servers have heterogeneous computing capabilities and different unit token costs. S14: Based on the structured task data and system operation status information, construct task queue features and server resource status features to form a scheduling status representation, wherein the server resource status features include at least server identifier, computing power, current resource occupancy and queue length; S15: The scheduling state representation is used as model input for subsequent evaluation and selection of scheduling strategies.

[0009] As a preferred embodiment of this application, in S14, the method includes the following steps: S141: Based on the structured task data and system operation status information, extract task features and server resource features, wherein the task features include the task identifier, input token length, output token length, deadline parameter and corresponding inference stage information, and the system resource features include the identifier, computing power, cost, current resource occupancy and queue length of each server, so as to form the observation feature representation of each inference task at the current moment. S142: The observation feature representation is fused with the system resource feature representation to form a unified scheduling state representation used to describe the system state in the current cloud-edge collaborative environment.

[0010] As a preferred embodiment of this application, in S2, the method includes the following steps: S21: Set a lower deadline for each inference task to represent the preferred completion time and an upper deadline for each allowed completion time. S22: Obtain the actual completion time of the reasoning task and construct a satisfaction membership function based on the relationship between the actual completion time and the lower and upper limits of the deadline: ,in Record the delay time for task completion. This is the lower limit of the deadline. This is the upper limit of the deadline. A constant used to avoid a denominator of zero; S23: Construct a deadline penalty term based on the aforementioned satisfaction membership function: This is to increase the deadline penalty as the task completion time approaches or exceeds the deadline limit. S24: Calculate the total completion latency and execution cost of the inference task execution process, wherein the total completion latency includes at least queuing time, data transmission time and inference processing time, and the execution cost includes at least the resource occupation cost of the task on the target computing node. S25: The total completion delay, execution cost, and deadline penalty are processed to a unified dimension and combined according to preset weight coefficients to form a reward function: ,in, and These are the normalized task completion latency and execution cost, respectively. These are the weighting coefficients.

[0011] As a preferred embodiment of this application, in step S2, the method further includes the following steps: S26: Combine task information with computing node information to construct a scheduling action space for representing scheduling decisions to allocate inference tasks to different cloud servers or edge servers; S27: Based on the scheduling action space and system state, the scheduling process is modeled as a time-skipping semi-Markov decision process model consisting of state, option actions, state transitions and reward functions, wherein the option actions are time-extended actions with duration, used to describe the continuous scheduling behavior during task execution.

[0012] As a preferred embodiment of this application, in S3, the method includes the following steps: S31: Define the current system state at the current scheduling moment as a state variable in a semi-Markov decision process, wherein the system state includes at least the status of the task queue to be scheduled, the resource status of each cloud server and edge server, task deadline information, and task execution progress information. S32: Define the task scheduling behavior as a time-extended action and maintain the current action execution state during the duration of the time-extended action, and perform a state transition after the action ends; S33: Construct a state encoding model for feature compression and representation learning of the current system state, construct a state prediction model for predicting the potential state at the next moment after the action is executed, construct a time prediction model for predicting the duration of the time-extended action, construct a cost prediction model for predicting the execution cost corresponding to the candidate scheduling action, and construct a policy network for outputting the prior probability distribution of each candidate scheduling action according to the current system state. S34: Use historical scheduling samples to jointly train the state coding model, time prediction model, cost prediction model and policy network to obtain a world model that can represent the dynamic changes of the real scheduling environment.

[0013] As a preferred embodiment of this application, in step S4, the method includes the following steps: S41: Construct a Monte Carlo search tree with the current system state as the root node and simulate and evaluate candidate scheduling actions based on the world model; S42: The prior probability of the candidate scheduling action is randomly perturbed by introducing Gumbel noise to obtain the perturbed prior probability; wherein the Gumbel noise is generated in the following manner: ,in Let be a random variable that follows a standard uniform distribution. It is Gumbel noise; and satisfies the following relationship: ,in This represents the prior probability after adding Gumbel noise. The prior probability of the candidate scheduling action; S43: Construct an action selection scoring function based on the action value function and the perturbed prior probability, and select the optimal scheduling action according to the scoring function, wherein the scoring function satisfies the following relationship: ,in, Indicates the state Next action The estimated cumulative reward, These are the weighting coefficients. For scoring functions; S44: Expand the selected scheduling action and predict the next state, execution latency, and cost based on the world model; S45: Update the search path based on the prediction results, update the value estimate of each candidate scheduling action, and repeat the selection, expansion and backtracking process until the preset number of searches or termination conditions are reached, and output the optimal scheduling strategy.

[0014] In a preferred embodiment of this application, step S44 includes the following steps: S441: Expand the selected scheduling action and generate corresponding child nodes in the Monte Carlo search tree; S442: Predict the next state after the scheduling action is executed based on the world model; S443: Predict the execution latency and resource consumption corresponding to the scheduling action based on the world model; S444: Based on the predicted state, execution latency, and resource consumption, evaluate the value of the scheduling path corresponding to the scheduling action.

[0015] As a preferred embodiment of this application, in step S5, the method includes the following steps: S51: Determine the mapping relationship between the inference task and the target server based on the optimal scheduling action output by the Monte Carlo tree search process; S52: Generate a task scheduling scheme based on the mapping relationship to indicate that the inference task will be assigned to the corresponding cloud server or edge server for execution; S53: Output the task scheduling scheme as the current scheduling decision result and allocate the inference task to the corresponding target server for execution according to the task scheduling scheme, wherein the target server includes a cloud server or an edge server. S54: During task execution, the system state is updated and the scheduling process is repeated for newly arrived inference tasks based on the updated system state, so as to realize continuous scheduling of dynamic task flow.

[0016] This application also provides a large model inference task scheduling system oriented towards fuzzy deadline constraints using any of the methods described above, comprising: The constraint model building module is used to model the large model inference task scheduling system in the cloud-edge collaborative computing environment and build a fuzzy deadline constraint model. The modeling includes at least the task structured representation, cloud server and edge server modeling, resource heterogeneity modeling, task execution latency modeling, and task execution cost modeling. The function setting module is used to map the task completion delay to continuous satisfaction by using a membership function for the fuzzy deadline constraint model, and to construct a delay penalty function based on the continuous satisfaction, thereby forming a reward function that includes task delay, resource cost and deadline default degree. The world model construction module is used to construct a scheduling action space based on the fuzzy deadline constraint model and construct a time-skip semi-Markov decision process model based on the scheduling action space and the system state, thereby constructing a world model that includes at least a state encoding model, a state prediction model, a time prediction model, a cost prediction model, and a policy network, and training the time-world model in the cloud. The scheduling processing module is used to construct a scheduling decision search mechanism based on the world model using the Gumbel noise-driven Monte Carlo tree search method, and to evaluate and select candidate scheduling actions at each decision moment. The output and update module is used to input the current system state into the world model to make multi-step predictions of the future state, output a scheduling scheme and allocate the inference task to the corresponding cloud server or edge server for execution according to the scheduling scheme, and update the system state according to the execution result.

[0017] The technical solution described in this application has the following advantages over the prior art: 1. This application introduces a fuzzy deadline modeling method to transform the traditional hard deadline constraint into a continuous and measurable satisfaction function. It also combines task completion delay and execution cost for unified modeling, transforming the degree of deadline default into an optimizable continuous penalty signal. This can more accurately characterize the task's tolerance for delay, thereby achieving an effective trade-off between multiple objectives and improving the rationality and adaptability of scheduling strategies in complex application scenarios. 2. This application constructs a time-skipping semi-Markov decision process and world model to model scheduling actions as time-extended actions with duration, and jointly predicts the system state, execution delay and resource consumption after the scheduling action is executed. This avoids the redundant state unfolding problem caused by traditional fixed time step modeling, and realizes a forward-looking characterization of the dynamic evolution process of the system, thereby reducing the average cost and average delay by 14.58% and 74.65%, respectively.

[0018] 3. This application adopts a Monte Carlo tree search method based on Gumbel noise, which introduces a random perturbation mechanism while maintaining the search directionality, enhances the exploration ability in the search process, effectively alleviates the problem of premature convergence to local optima in the search process, thereby improving the stability and robustness of the scheduling strategy and realizing efficient scheduling of large model inference tasks in a dynamic cloud-edge collaborative environment. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1 This is a flowchart of the large model inference task scheduling method provided in the embodiments of this application.

[0021] Figure 2 This is a schematic diagram of the data distribution for inference tasks provided in an embodiment of this application.

[0022] Figure 3 This is a flowchart of the modeling method for a large-scale inference task scheduling system provided in the embodiments of this application.

[0023] Figure 4 This is an overall framework diagram of the scheduling method provided in the embodiments of this application; Figure 5 This is a flowchart of the scheduling state representation generation method provided in the embodiments of this application.

[0024] Figure 6 This is a flowchart of the reward function setting provided in the embodiments of this application.

[0025] Figure 7 This is a flowchart of the time-skipping semi-Markov decision process model construction method provided in the embodiments of this application.

[0026] Figure 8 This is a schematic diagram of the structure of the time-skipping semi-Markov decision process model provided in the embodiments of this application.

[0027] Figure 9 This is a flowchart of the world model construction method provided in the embodiments of this application.

[0028] Figure 10 This is a flowchart of the optimal scheduling strategy generation method provided in the embodiments of this application.

[0029] Figure 11This is a schematic diagram of the scheduling decision process of Monte Carlo tree search based on Gumbel noise driven by an embodiment of this application.

[0030] Figure 12 This is a flowchart of the value assessment method provided in the embodiments of this application.

[0031] Figure 13 This is a flowchart of a system status update method provided in an embodiment of this application.

[0032] Figure 14 These are schematic diagrams of ablation experiments for AC for each scheme provided in the embodiments of this application.

[0033] Figure 15 This is a schematic diagram of ablation experiments for each scheme provided in the embodiments of this application, considering the trade-off between AL and ST.

[0034] Figure 16 These are schematic diagrams of ablation experiments for DVRs provided in the embodiments of this application.

[0035] Figure 17 This is a schematic diagram of parameter sensitivity analysis for the number of simulations K provided in the embodiments of this application.

[0036] Figure 18 This is a comparative experimental diagram of each method provided in the embodiments of this application for AC and AL; where (a) is AC and (b) is AL.

[0037] Figure 19 These are comparative experiments for ST and DVR for each method provided in the embodiments of this application; where (a) is ST and (b) is DVR.

[0038] Figure 20 This is a module connection diagram of the large model inference task scheduling system provided in the embodiments of this application. Detailed Implementation

[0039] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0040] refer to Figure 1 As shown, in some embodiments, the large model inference task scheduling method oriented towards fuzzy deadline constraints includes the following steps: S1: Model the large-scale inference task scheduling system in the cloud-edge collaborative computing environment and construct a fuzzy deadline constraint model. The modeling includes at least the task structured representation, cloud server and edge server modeling, resource heterogeneity modeling, task execution latency modeling, and task execution cost modeling.

[0041] The large-model inference task scheduling system consists of a cloud layer, an edge layer, and a terminal layer from top to bottom, forming a typical cloud-edge-terminal collaborative computing architecture. The terminal layer consists of a group of terminal devices, which are mainly responsible for collecting industrial data and generating inference tasks. The generated inference tasks offload requests to the edge or cloud through the industrial access network. The edge layer consists of a group of edge servers, deployed close to the data source, which can provide low-latency inference computing services for terminal devices. The cloud layer consists of a group of cloud servers, which have stronger computing power and more abundant storage resources, and can provide computing support for complex inference tasks, large-scale concurrent requests, and high-load scenarios.

[0042] refer to Figures 2 to 4 As shown, specifically, in step S1, the method includes the following steps: S11: Reference Figure 2 As shown, a large model inference task set is obtained from an existing data source. Each inference task is represented by a task data structure consisting of a task identifier, task arrival time, input token length, output token length, and deadline parameter, which is used to characterize the computational requirements and deadline constraints of different inference tasks.

[0043] Step S11 further proposes preferred specific implementation methods, including task models; tasks It can be represented as a quintuple ,in Indicates the task identifier. Indicates the time stamp of the task arrival. Indicates the length of the token in the input request. Indicates the length of the token in the output response. The token length represents the deadline constraint corresponding to the task; the token length determines the computational cost of inference. Specifically, the length of the request token mainly affects the pre-filling stage of task inference, while the length of the response token mainly affects the decoding stage.

[0044] S12: Perform structured parsing on the reasoning task, extract task input and output feature information and task execution stage information, and abstract the reasoning task into a pre-filling stage and a decoding stage to form structured task data for scheduling decisions.

[0045] S13: Construct computing and communication models for cloud servers and edge servers, and establish execution latency and resource consumption models for tasks on different computing nodes to obtain the execution latency and execution cost of tasks on each server, where different servers have heterogeneous computing capabilities and different unit token costs.

[0046] Step S13 further proposes preferred specific implementation methods, including: Server Model: During task scheduling, the cloud-edge collaborative environment primarily provides inference computing services. The server set in the cloud-edge environment can be represented as follows: Since servers are heterogeneous in reality, a server can be represented as a quintuple. ,in Indicates the server identifier. and These represent the inference rates of the server during the pre-filling and decoding phases, respectively. and These represent the unit token cost for the server during the pre-filling and decoding phases, respectively; the edge server set contains [number of tokens]. An edge server, represented as Cloud server sets include One cloud server, represented as .

[0047] Computational and communication models: Assume binary variables Indicates task Was it assigned to a server? Its definition is as follows: Then the task On the server The inference processing time can be expressed as the sum of the pre-filling stage and the decoding stage time: ;set up and Representing tasks respectively If the start and end times are given, then the task's completion time is: Therefore, the task The total completion time can be defined as: ,in, It also includes the task's waiting time and execution time; therefore, the task's deadline constraint should be determined by comparing the total completion time. With tolerable cutoff threshold To determine.

[0048] Tolerable latency is a prerequisite for a Service Level Agreement (SLA). Its failure directly leads to compromise of the SLA and penalties on the service. To measure whether a task meets the SLA, the task is defined... The SLA default indicator variable is: The Deadline Default Rate (DVR) reflects the timeliness of a system, and its definition is as follows: Where W represents the task set; besides timeliness, execution cost is also a key optimization objective in cloud-edge collaborative inference scheduling, for tasks If it is assigned to a server Therefore, its execution cost is jointly determined by the inference costs of the pre-filling phase and the decoding phase; thus, the task The execution cost can be defined as: Accordingly, the total execution cost of the system within a scheduling cycle is expressed as: .

[0049] S14: Reference Figure 4 As shown, based on the structured task data and system operation status information, task queue features and server resource status features are constructed to form a scheduling status representation. The server resource status features include at least server identifier, computing power, current resource occupancy, and queue length.

[0050] refer to Figure 5 As shown, in S14, the method includes the following steps: S141: Based on the structured task data and system operation status information, extract task features and server resource features. The task features include the task identifier, input token length, output token length, deadline parameter, and corresponding inference stage information. The system resource features include the identifier, computing power, cost, current resource occupancy, and queue length of each server, to form an observation feature representation of each inference task at the current moment.

[0051] S142: The observation feature representation is fused with the system resource feature representation to form a unified scheduling state representation used to describe the system state in the current cloud-edge collaborative environment.

[0052] S15: The scheduling state representation is used as the model input for the evaluation and selection of subsequent scheduling strategies.

[0053] S2: For the fuzzy deadline constraint model, the membership function is used to map the task completion delay to continuous satisfaction, and a delay penalty function is constructed based on the continuous satisfaction, thereby forming a reward function that includes task delay, resource cost, and the degree of deadline default.

[0054] The fuzzy deadline refers to a situation where the deadline of a task includes hard deadlines, soft deadlines, and no deadlines, all of which are fuzzy. When the deadline of a task is a soft deadline, the deadline of the task is in the form of [a, b]. In general, the upper and lower limits of the deadline in this application are the attribute characteristics of the task itself.

[0055] refer to Figure 6 As shown, specifically, in step S2, the method includes the following steps: S21: Set a lower limit for the deadline to indicate the preferred completion time and an upper limit for the deadline to indicate the maximum tolerable completion time for each inference task.

[0056] S22: Obtain the actual completion time of the reasoning task and construct a satisfaction membership function based on the relationship between the actual completion time and the lower and upper limits of the deadline: ,in Record the delay time for task completion. This is the lower limit of the deadline. This is the upper limit of the deadline. A constant used to avoid a denominator of zero; when When, it indicates that the task deadline is a hard deadline. When the deadline is specified, it indicates that the deadline for the task is a soft deadline.

[0057] S23: Construct a deadline penalty term based on the aforementioned satisfaction membership function: The deadline penalty term is increased as the task completion time approaches or exceeds the deadline limit; this deadline penalty term is used to characterize the degree of breach of the task completion delay relative to the deadline constraint and participates in the calculation of the reward function.

[0058] S24: Calculate the total completion latency and execution cost during the execution of the inference task, wherein the total completion latency includes at least queuing time, data transmission time and inference processing time, and the execution cost includes at least the resource occupation cost of the task on the target computing node.

[0059] S25: The total completion delay, execution cost, and deadline penalty are processed to a unified dimension and combined according to preset weight coefficients to form a reward function: ,in, and These are the normalized task completion latency and execution cost, respectively. Since the weighting coefficients are fixed and the optimization objective during the scheduling process is clear, a fixed weighting coefficient is used.

[0060] refer to Figure 7 As shown, specifically, in step S2, the method further includes the following steps: S26: Combine task information with computing node information to construct a scheduling action space for representing scheduling decisions to allocate inference tasks to different cloud servers or edge servers; S27: Based on the scheduling action space and system state, construct a time-skipping semi-Markov decision process, thereby modeling the scheduling process as a time-skipping semi-Markov decision process model consisting of state, option actions, state transitions and reward functions, wherein the option actions are time-extended actions with duration, used to describe the continuous scheduling behavior during task execution.

[0061] Among them, the time-skipping semi-Markov decision process is a model that extends the traditional Markov decision process. It introduces the concept of time skipping, so that in the decision-making process, the state transition no longer takes place according to a fixed time step, but is triggered according to the occurrence of events or the progress of task execution. The core idea of ​​time-extended actions is that the execution of each action is no longer instantaneous, but spans a certain time range, and has a continuous impact on the system's state, execution latency, resource consumption and other factors during this period.

[0062] Specifically, in the time-skip semi-Markov decision process model: State S is the current system state, which includes at least task feature representation and system resource state feature representation. The task feature representation includes the length of the input token, the length of the output token, and the corresponding inference stage information of the task. The resource state feature representation includes the computing power of each server, the current resource occupancy, and the task queue status.

[0063] Action O is a scheduling decision action, which is represented by allocating inference tasks to target servers, including cloud servers or edge servers, and the action is an optional action with a duration.

[0064] The reward R is the feedback value obtained after performing the action in the current state. It is set based on the task completion delay, execution cost, and fuzzy deadline penalty. The task completion delay and execution cost are normalized and combined with the delay penalty value obtained from the fuzzy deadline membership function to form an immediate reward function. The reward function is used to guide scheduling decisions to reduce deadline defaults while reducing task completion delay and execution cost.

[0065] The fuzzy deadline penalty is obtained by inputting the task completion delay into a preset membership function to obtain a satisfaction value, and a delay penalty term is constructed based on the satisfaction value to reflect the degree to which the task meets the deadline constraint.

[0066] S3: Reference Figure 8 As shown, based on the fuzzy deadline constraint model, a scheduling action space is constructed, and a time-skipping semi-Markov decision process model is constructed based on the scheduling action space and the system state. In this way, a world model is constructed that includes at least a state encoding model, a state prediction model, a time prediction model, a cost prediction model, and a policy network, and the time-world model is trained in the cloud.

[0067] The world model is used to predict the system evolution during the execution of scheduling actions, including: taking the current system state and the option action as input, and predicting the system state at the next moment after the option action is executed. ,in, The state prediction network then predicts the execution duration corresponding to the chosen action. ,in, The network represents a latency prediction network; it then predicts the cost overhead during the execution of the chosen action. ,in, This represents a cost prediction network.

[0068] The world model is constructed based on a time-skipping semi-Markov decision process, representing scheduling actions as time-extended actions with durations. It triggers state transitions upon completion of the selected action, avoiding the gradual unfolding of intermediate states and thus improving the prediction and computational efficiency of the scheduling process. Based on the above prediction results, the world model performs a prospective evaluation of candidate scheduling strategies, and the policy network outputs the prior probabilities of actions. .

[0069] Specifically, the world model is trained within the MCTS framework to learn the state transitions and decision-making processes during task scheduling. The world model first simulates the state space of task scheduling and explores it multiple times using MCTS. MCTS gradually constructs a search tree representing the task scheduling process through continuous selection, expansion, simulation, and backpropagation. In each round of simulation, the world model selects an appropriate action based on the current state and obtains the next state through simulation, thereby gradually improving the model's predictive ability. The mean squared error (MSE) is used to calculate the difference between the actual and predicted states to measure the model's prediction error of task scheduling state transitions during simulation. The network parameters are shown in the table below. Table: Relevant Parameters of World Model Components refer to Figure 9 As shown, specifically, in step S3, the method includes the following steps: S31: Define the current system state at the current scheduling moment as a state variable in a semi-Markov decision process, wherein the system state includes at least the status of the task queue to be scheduled, the resource status of each cloud server and edge server, task deadline information, and task execution progress information. S32: Define the task scheduling behavior as a time-extended action and maintain the current action execution state during the duration of the time-extended action, and perform a state transition after the action ends; S33: Construct a state encoding model for feature compression and representation learning of the current system state, construct a state prediction model for predicting the potential state at the next moment after the action is executed, construct a time prediction model for predicting the duration of the time-extended action, construct a cost prediction model for predicting the execution cost corresponding to the candidate scheduling action, and construct a policy network for outputting the prior probability distribution of each candidate scheduling action according to the current system state. S34: Use historical scheduling samples to jointly train the state coding model, time prediction model, cost prediction model and policy network to obtain a world model that can represent the dynamic changes of the real scheduling environment.

[0070] S4: Reference Figure 11 As shown, based on the world model, a scheduling decision search mechanism is constructed using the Gumbel noise-driven Monte Carlo tree search method, and candidate scheduling actions are evaluated and selected at each decision moment.

[0071] The scheduling decision search mechanism is a Gumbel noise-driven Monte Carlo tree search method, which introduces random perturbations into the prior probability of actions to achieve the selection and expansion of candidate actions. The relevant parameters of Gumbel-MCTS are shown in the table below. Table: Relevant Parameters of Gumbel-MCTS refer to Figure 10 As shown, specifically, in step S4, the method includes the following steps: S41: Construct a Monte Carlo search tree with the current system state as the root node and simulate and evaluate candidate scheduling actions based on the world model.

[0072] S42: The prior probabilities of candidate scheduling actions are processed by introducing Gumbel noise to randomly perturb the prior probabilities of the candidate scheduling actions, resulting in perturbed prior probabilities; wherein the Gumbel noise is generated in the following manner: ,in Let be a random variable that follows a standard uniform distribution. It is Gumbel noise; it satisfies the following relationship: ,in This represents the prior probability after adding Gumbel noise. represents the prior probability of the candidate scheduling action.

[0073] S43: Construct an action selection scoring function based on the action value function and the perturbed prior probability, and select the optimal scheduling action according to the scoring function, wherein the scoring function satisfies the following relationship: ,in, Indicates the state Next action The estimated cumulative reward, These are the weighting coefficients. This is the scoring function.

[0074] S44: Expand the selected scheduling action and predict the next state, execution latency, and cost based on the world model.

[0075] refer to Figure 12 As shown, in step S44, the method includes the following steps: S441: Expand the selected scheduling action to generate a corresponding child node in the Monte Carlo search tree.

[0076] S442: Predict the next state after the scheduling action is executed based on the world model; during the search process, multiple steps are performed using the world model: ; S443: Predict the execution latency and resource consumption corresponding to the scheduling action based on the world model; S444: Based on the predicted state, execution latency, and resource consumption, evaluate the value of the scheduling path corresponding to the scheduling action.

[0077] S45: Update the search path based on the prediction results, update the value estimate of each candidate scheduling action, and repeat the selection, expansion and backtracking process until the preset number of searches or termination conditions are reached, and output the optimal scheduling strategy.

[0078] S5: Input the current system state into the world model to make multi-step predictions of the future state, output a scheduling scheme, and allocate the inference task to the corresponding cloud server or edge server for execution according to the scheduling scheme, and update the system state according to the execution result.

[0079] The system status update is as follows: , The system represents the state transition function of the real environment and includes the following rules: In this application, the system adopts a first-come, first-served (FCFS) task access mechanism. All arriving inference tasks enter the scheduling queue in the order of arrival, and the scheduler makes the scheduling decision. This application does not adopt a task preemption mechanism. After a task arrives at a service node, the system makes a scheduling decision. Once a task is scheduled to a computing node and starts execution, the task will continue to execute until it is completed or fails. This design conforms to the actual characteristics of edge computing scenarios and can reduce the system scheduling complexity. This application supports concurrent execution of multiple tasks. Multiple inference tasks in the system can exist simultaneously. The scheduler makes a joint decision based on the global state (including the states of all tasks and resources) at each decision moment, thereby realizing concurrent scheduling of multiple tasks. This application does not design a separate pipeline mechanism. Instead, it uses a world model to predict the execution time and cost of tasks, and the scheduling algorithm automatically forms a pipelined execution path across nodes.

[0080] In dynamic cloud-edge environments, network bandwidth and latency fluctuate. The world model takes the service node state as part of the environmental state as input and uses network state prediction to estimate execution latency and cost information to assist in determining scheduling decisions. Specifically, during the MCTS search process, the model predicts the execution overhead and state under different scheduling decisions. The scheduling strategy selects better nodes based on the prediction results, thereby avoiding high-latency or low-bandwidth paths in advance. Therefore, this application does not passively respond to network fluctuations, but rather makes forward-looking scheduling adjustments based on predictions.

[0081] To address the node failure issue, this application employs the following approach: when a node goes offline, it is removed from the set of available resources; the scheduling action space is dynamically updated accordingly, and subsequent scheduling decisions are made only on the remaining available nodes; and the scheduling of tasks that have not yet been executed will not be affected; this design ensures that the system will not crash or fail under dynamic node changes, but will maintain continuous schedulability.

[0082] The aforementioned mechanisms are conventional engineering implementation methods in edge computing scheduling systems and work in conjunction with the scheduling method proposed in this application. They do not constitute the innovation of this application, but provide necessary support for the implementation of this application.

[0083] refer to Figure 13 As shown, specifically, in step S5, the method includes the following steps: S51: Determine the mapping relationship between the inference task and the target server based on the optimal scheduling action output by the Monte Carlo tree search process; S52: Generate a task scheduling scheme based on the mapping relationship to indicate that the inference task will be assigned to the corresponding cloud server or edge server for execution; S53: Output the task scheduling scheme as the current scheduling decision result and allocate the inference task to the corresponding target server for execution according to the task scheduling scheme, wherein the target server includes a cloud server or an edge server. S54: During task execution, the system state is updated and the scheduling process is repeated for newly arrived inference tasks based on the updated system state, so as to realize continuous scheduling of dynamic task flow.

[0084] Therefore, in order to verify the effectiveness of the method of this application, the following ablation experiment was conducted, and the experimental results are shown below: In this experiment, the dataset comes from Burst GPT TRACE, a real-world tracing dataset based on an LLM service workload using ChatGPT (GPT 3.5) and GPT-4.

[0085] This application uses four key metrics to evaluate system performance: (1) Average Latency (AL) represents the average time taken from the start to the completion of a task, and is used to measure the overall response efficiency of the system.

[0086] (2) Average Cost (AC) represents the average amount of resources consumed or economic expenses incurred by each task during its execution.

[0087] (3) Deadline Violation Rate (DVR) represents the proportion of tasks that fail to be completed before the deadline among all tasks, serving as a measure of the degree to which the scheduling strategy satisfies the deadline constraint.

[0088] (4) Scheduling Time (ST) represents the computation time consumed by the scheduling algorithm for one decision.

[0089] Specifically, the following ablation experiments were conducted: To evaluate the contribution of each core component to the overall performance, this application tested the following five schemes within the same framework: a) the method of this application (Ours); b) ablation of time skipping SMDP (A-SMDP); c) ablation of Gumbel noise (A-GN); d) ablation of MCTS (A-MCTS); and e) ablation of the world model (A-WM). Then, ablation analysis was performed on the performance of each scheme based on the above four evaluation metrics.

[0090] refer to Figure 14 As shown, Figure 14 This represents the ablation experiments for each ablation protocol targeting the AC index; from Figure 14 As can be seen, Ours has the lowest average cost and the most concentrated distribution, with its box position generally lower than the other four ablation schemes, indicating that the proposed method not only effectively reduces costs but also has good stability. In contrast, the average costs of the other four ablation schemes all increased to varying degrees, indicating that each core module played a positive role in cost optimization. Looking at the result distribution, A-MCTS has the highest average cost, indicating that without a search planning mechanism, the scheduling strategy is more susceptible to local information, making it difficult to coordinate computational overhead, transmission costs, and resource consumption from a global perspective, thus leading to a significant increase in overall cost. A-WM also has a high average cost, indicating that the world model's ability to predict future state evolution and potential costs plays a crucial role in cost control. Once this module is removed, the strategy is prone to making more costly scheduling decisions due to a lack of foresight. Furthermore, although the average costs of A-SMDP and A-GN are also higher than Ours, the increase is relatively smaller. The results of A-SMDP show that the time-skip mechanism helps reduce frequent decisions and unnecessary state transitions, thereby improving cost performance. A-GN is closest to Ours, indicating that Gumbel noise primarily improves resource utilization efficiency indirectly by enhancing exploration capabilities and selection stability during the search process. Overall, the complete framework performs best in terms of cost control, and the modules exhibit good synergy.

[0091] To evaluate the overall performance of the complete method in terms of scheduling computational overhead and system operating efficiency, this application further conducted ablation experiments on the trade-off between AL and ST, and the results are referenced. Figure 15As shown, overall, Ours achieved the lowest AL while maintaining a low ST (Scheduled Time), indicating that the complete framework did not incur significant additional scheduling overhead due to the introduction of time-skip modeling, the world model, and Gumbel-MCTS search. Instead, it demonstrated a stronger advantage in decision quality, effectively reducing overall system latency. Looking at the ablation results, different components have different emphases on performance impact. A-MCTS had the lowest ST but the highest AL, indicating that simply reducing the computational cost of a single decision cannot guarantee system-level execution efficiency. Without a search planning mechanism, the policy is more prone to getting stuck in local decisions, leading to a significant increase in overall latency. A-WM also had a high AL, indicating that the world model plays an important role in predicting future state evolution and improving long-term decision quality. Furthermore, A-SMDP had a lower ST than Ours, but its AL was significantly higher, suggesting that while the time-skip mechanism introduces some scheduling computational overhead, it improves the overall grasp of long-sequence tasks, thus more effectively reducing system execution latency. The ST of A-GN is also lower than that of Ours, while AL, although lower than A-SMDP and A-WM, is still significantly higher than Ours. This indicates that the role of Gumbel noise is mainly to improve the exploration ability and selection stability during the search process, further enhancing the quality of scheduling results. Overall, although the complete method is not optimal in terms of scheduling time metrics, it achieves a more reasonable trade-off between scheduling time and average delay, demonstrating better overall efficiency.

[0092] refer to Figure 16The ablation experiments for each ablation scheme are shown. Overall, Ours has the lowest DVR and the smallest error bar range, indicating that the complete framework can not only effectively reduce the task default rate, but also has good stability. In contrast, the DVR of the other ablation schemes is significantly higher, indicating that each core component plays a positive role in scheduling efficiency under fuzzy deadline constraints. From the result ranking, A-MCTS has the highest DVR, indicating that without a search planning mechanism, the strategy is more susceptible to local state influences and it is difficult to coordinate the relationship between the current decision and subsequent resource consumption from a global perspective, thus making it easier for tasks to exceed the deadline. The default rate of A-WM is also relatively high, indicating that the world model's ability to predict future state evolution plays a crucial role in default control. Without this module, the scheduler struggles to predict potential congestion and latency accumulation, leading to increased default risk. Furthermore, the DVR of A-SMDP is significantly higher than Ours, suggesting that the time-skip mechanism helps organize scheduling decisions over longer time scales, reducing overdue events caused by frequent local responses. While the DVR of A-GN is also higher than Ours, it is closest to the complete method among all ablation schemes, indicating that Gumbel noise primarily enhances default control by improving exploration ability and selection stability during the search process. Overall, the complete framework performs best in both reducing DVR and maintaining outcome stability.

[0093] The following table summarizes the ablation experimental results for each protocol across four indicators: Table: Ablation experiments for each protocol targeting four indicators Among all ablation schemes, the complete method outperforms all others in the three core metrics of Cost, DVR, and Latency, demonstrating that the synergistic effect of each module significantly improves scheduling quality. Ours reduces AC to 0.715, DVR to 4.95%, and AL to 1.91s, showcasing strong overall optimization capabilities. Although the complete method's ST reaches 0.269s, higher than other ablation schemes, this additional computational overhead results in lower system execution latency, fewer deadline defaults, and better cost performance, indicating that the proposed method has higher practical value in terms of global performance.

[0094] In the MCTS-based scheduling framework, the number of simulations, K, determines the search intensity and computational cost of each decision stage. A smaller K leads to insufficient searching, thus reducing decision quality; a larger K, while improving search accuracy, significantly increases scheduling time. Therefore, K directly affects the trade-off between DVR and algorithm real-time performance. Parameter analysis of K can reveal the trend of performance changing with search intensity, determine whether there is diminishing marginal returns, and identify an optimal value that achieves a reasonable balance between scheduling quality and computational cost. This step can verify the stability and practicality of the algorithm while avoiding unnecessary computational overhead caused by over-searching. (Reference) Figure 17 The experiment illustrates the parameter sensitivity analysis of the number of simulations, K. The results show that as the number of simulations, K, increases, the deadline default rate decreases significantly, but the rate of decrease gradually diminishes, while the decision time increases approximately linearly. When K=10, the default rate is 8.49%, indicating insufficient search attempts, inadequate tree structure expansion, and low scheduling decision quality. Increasing K to 20 reduces the default rate to 6.02%, showing a significant performance improvement, indicating that increasing the number of simulations significantly improves search quality. When K continues to increase to 30 and 50, the default rates decrease to 4.91% and 4.18%, respectively, still showing improvement, but the rate of decrease becomes significantly smaller. When K further increases to 70 and 100, the default rates are 4.04% and 4.00%, respectively, almost stabilizing. This indicates that after K≥50, the system has entered a performance saturation range, and further increasing the number of searches yields extremely limited benefits. On the other hand, the decision time increases from 50.6ms to 260ms, showing an approximately linear upward trend. This aligns with the time complexity characteristics of MCTS, where computational overhead is proportional to the number of simulations. In summary, the experimental results demonstrate a clear diminishing marginal returns phenomenon. Within a small search range, increasing K significantly reduces the default rate. However, as the search scales up, performance improvements plateau, while computational costs continue to increase. Therefore, considering the overall trade-off between performance and computational overhead, K=50 can be seen as a relatively reasonable compromise.

[0095] To verify the effectiveness of the method, the following comparative experiment was conducted, and the results are shown below: To verify the effectiveness of the proposed algorithm, the following four mainstream algorithms were selected for comparative experiments: a) Ours, b) DRL

[72] , c) SMO

[80] , d) OptionZero

[81] , and e) random. Then, the performance of each algorithm was compared and analyzed based on the above four evaluation indicators.

[0096] refer to Figure 18As shown in (a), Ours has the lowest overall cost level and the most concentrated distribution, indicating that this method not only reduces the average cost but also exhibits less fluctuation across different operating groups, resulting in more stable cost control. In contrast, Random has a higher overall cost box, suggesting that random assignment is prone to dispatching tasks to inappropriate high-cost nodes, leading to increased costs. The cost distributions of OptionZero, SMO, and DRL fall between the two. OptionZero's cost level is lower than DRL and SMO, indicating that search-based decision-making still has an economic advantage, but its cost distribution is still significantly higher than Ours, reflecting further improvements in cost prediction and option-level planning. (Reference) Figure 18 As shown in (b), Ours has the lowest median and the shortest bin, indicating that the response time of most running tasks remains within a low range. Random has a significantly higher bin and upper limit, indicating that the random strategy is prone to queuing congestion or unreasonable cross-node scheduling, thus significantly increasing AL. In comparison, OptionZero and SMO have significantly better latency distributions than DRL and Random, with OptionZero having a slightly lower median than SMO, indicating that the search mechanism is more likely to find low-latency execution nodes under this task distribution. However, Ours is still significantly better than OptionZero, which shows that TS-SMDP-WM's prediction and planning of completion time can effectively reduce queuing time caused by erroneous dispatch, thereby reducing the overall AL.

[0097] refer to Figure 19 Figure (a) primarily reflects the differences in online decision-making overhead among different methods. Random has the lowest ST (Short Decision Time), with the box near the bottom, consistent with the characteristic that stochastic strategies require almost no computation. SMO also has a low ST, indicating that heuristic methods typically rely on simple rules or local information, resulting in low computational overhead. DRL's ST is higher than SMO and Random, mainly due to a single forward inference or state evaluation process. OptionZero and Ours have even higher STs, with Ours having the longest decision-making time. This is because the method requires multiple forward predictions using the world model and a Gumbel-MCTS search to evaluate candidate options at each decision. Nevertheless, Ours' ST remains within an acceptable range, yielding significant gains in default rate, latency, and cost, demonstrating a reasonable trade-off between performance improvement and decision-making overhead. The deadline default rate is one of the most critical quality indicators in the comparative experiments. (Reference) Figure 19As shown in (b), Ours has the lowest median default rate and a more concentrated distribution, indicating that the proposed method can more stably meet the task deadline requirements in most operating groups. Random has the highest default rate and greater fluctuations, indicating that random decision-making lacks basic time constraint awareness under deadline-constrained scenarios, which can easily lead to task timeouts. DRL's default rate is significantly higher than OptionZero and SMO, indicating that in this scenario, end-to-end decision-making relying solely on the policy network is more susceptible to the effects of training stability and state representation errors. SMO shows some improvement over DRL, indicating that high-quality heuristics have engineering usability under deadline constraints; while OptionZero's default rate is further lower than SMO, demonstrating the advantage of search-based planning for time-constrained objectives. Finally, Ours can still significantly reduce the default rate compared to OptionZero, indicating that TS-SMDP-WM has advantages in latency assessment and long-term planning, thus being more conducive to reducing the default risk in near-deadline scenarios.

[0098] The proposed method, as shown in the table below, achieves optimal performance across the three core metrics: AL, AC, and DVR. Table: Performance comparison experiment of each method (Mean±Std) Specifically, Ours achieves a significantly lower Altitude (AL) than other comparable methods. Simultaneously, its Deficit Rate (DVR) is significantly lower than OptionZero, SMO, DRL, and Random, indicating that Ours' method more effectively meets task deadline constraints and improves service stability. Regarding Accuracy (AC), Ours also performs worse than all comparable methods, demonstrating that it doesn't simply increase resource consumption to achieve a lower default rate, but rather achieves a comprehensive optimization of latency, cost, and default rate through more reasonable offloading decisions. Notably, among the baseline methods, OptionZero outperforms SMO and DRL overall in AL, AC, and DVR, demonstrating the effectiveness of search-based decision-making in complex scheduling scenarios. SMO, compared to DRL, is superior in latency, cost, and default rate, indicating that high-quality heuristics have strong engineering usability under this task distribution. Finally, in terms of Decision Cost (ST), Random and SMO have the lowest decision overhead, while Ours' ST is higher than the baseline but still within a tolerable range, indicating that Ours does indeed introduce additional online decision overhead. However, considering its significant advantages in AL, AC, and DVR, it can be seen that this additional overhead results in better system-level benefits, demonstrating better overall performance.

[0099] Therefore, for reference Figure 20As shown, in some embodiments of this application, this application also relates to a large model inference task scheduling system oriented towards fuzzy deadline constraints using the method described in any of the above embodiments, comprising: The constraint model construction module 201 is used to model the large model inference task scheduling system in the cloud-edge collaborative computing environment and construct a fuzzy deadline constraint model. The modeling includes at least the task structured representation, cloud server and edge server modeling, resource heterogeneity modeling, task execution latency modeling, and task execution cost modeling. The function setting module 202 is used to map the task completion delay to continuous satisfaction by using a membership function for the fuzzy deadline constraint model, and to construct a delay penalty function based on the continuous satisfaction, thereby forming a reward function that includes task delay, resource cost and deadline default degree. The world model construction module 203 is used to construct a scheduling action space based on the fuzzy deadline constraint model and construct a time-skip semi-Markov decision process model based on the scheduling action space and the system state, thereby constructing a world model that includes at least a state encoding model, a state prediction model, a time prediction model, a cost prediction model and a policy network, and training the time-world model in the cloud. The scheduling processing module 204 is used to construct a scheduling decision search mechanism based on the world model using the Gumbel noise-driven Monte Carlo tree search method, and to evaluate and select candidate scheduling actions at each decision moment. The output and update module 205 is used to input the current system state into the world model to make multi-step predictions of the future state, output a scheduling scheme and allocate the inference task to the corresponding cloud server or edge server for execution according to the scheduling scheme, and update the system state according to the execution result.

[0100] In some embodiments of this application, this application also relates to a computer-readable storage medium storing a computer program that, when loaded and executed by a processor, implements all or part of the steps of the large model inference task scheduling method for fuzzy deadline constraints according to any of the foregoing embodiments.

[0101] Specifically, the computer program includes instruction code for performing system modeling, fuzzy deadline constraint modeling, reward function construction, time-skip semi-Markov decision process modeling, world model construction and training, and Monte Carlo tree search scheduling decisions based on Gumbel perturbations. During execution, the instruction code invokes processor resources to generate task state representations, select scheduling actions, predict future states, estimate execution latency and cost, and output scheduling schemes. It can continuously trigger the scheduling decision process when tasks arrive dynamically, system states change, or node availability changes, thereby achieving dynamic online scheduling of large model inference tasks. The computer-readable storage medium can be any tangible storage medium capable of storing program code and being read by computing devices, including but not limited to read-only memory, random access memory, flash memory, solid-state drives, disk storage, optical disk storage, or other non-volatile or volatile storage media.

[0102] In some embodiments of this application, this application also relates to a computer device, the computer device including a processor, a memory and a communication interface, wherein the memory is used to store a computer program, and the processor is used to call and execute the computer program to implement the large model inference task scheduling method for fuzzy deadline constraints described in any of the above embodiments.

[0103] Specifically, when executing the computer program, the processor acquires large-scale model inference tasks and their corresponding input / output token information and deadline parameters, models cloud servers and edge servers, constructs task execution latency and execution cost models, and generates a reward function based on fuzzy deadline constraints. Further, the processor constructs a world model based on a time-skip semi-Markov decision process, predicts the system state, execution time, and execution cost after the scheduling action is executed, and searches and evaluates candidate scheduling actions using a Monte Carlo tree search method with Gumbel perturbation to output the optimal task scheduling scheme. Simultaneously, the processor distributes the scheduling scheme to the corresponding computing nodes for execution through the communication interface, and updates the system state and triggers the next round of scheduling decisions when a task is completed, a new task arrives, or the system state changes, thereby achieving efficient scheduling of large-scale model inference tasks in dynamic task flow scenarios. The computer device can be a cloud server, an edge server, or any node or control node in a distributed computing system composed of multiple nodes.

[0104] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0105] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A scheduling method for large-scale model inference tasks oriented towards fuzzy deadline constraints, characterized in that, Includes the following steps: S1: Model the large-scale inference task scheduling system in the cloud-edge collaborative computing environment and construct a fuzzy deadline constraint model. The modeling includes at least the following: task structure representation, cloud server and edge server modeling, resource heterogeneity modeling, task execution latency modeling, and task execution cost modeling. S2: For the fuzzy deadline constraint model, the membership function is used to map the task completion delay to continuous satisfaction and a delay penalty function is constructed based on the continuous satisfaction, thereby forming a reward function that includes task delay, resource cost and deadline default degree; S3: Based on the fuzzy deadline constraint model, construct a scheduling action space and construct a time-skip semi-Markov decision process model based on the scheduling action space and system state, thereby constructing a world model that includes at least a state encoding model, a state prediction model, a time prediction model, a cost prediction model, and a policy network. S4: Based on the world model, a scheduling decision search mechanism is constructed using the Gumbel noise-driven Monte Carlo tree search method, and candidate scheduling actions are evaluated and selected at each decision moment; S5: Input the current system state into the world model to make multi-step predictions of the future state, output a scheduling scheme, and allocate the inference task to the corresponding cloud server or edge server for execution according to the scheduling scheme, and update the system state according to the execution result.

2. The scheduling method for large model inference tasks oriented towards fuzzy deadline constraints according to claim 1, characterized in that, In S1, the method includes the following steps: S11: Obtain a large set of model inference tasks from existing data sources, where each inference task is represented as a task data structure consisting of task identifier, task arrival time, input token length, output token length, and deadline parameter; S12: Perform structured parsing on the reasoning task, extract task input and output feature information and task execution stage information, and abstract the reasoning task into a pre-filling stage and a decoding stage; S13: Construct computing and communication models for cloud servers and edge servers, and establish execution latency and resource consumption models for tasks on different computing nodes to obtain the execution latency and execution cost of tasks on each server, where different servers have heterogeneous computing capabilities and different unit token costs. S14: Based on the structured task data and system operation status information, construct task queue features and server resource status features to form a scheduling status representation, wherein the server resource status features include at least server identifier, computing power, current resource occupancy and queue length; S15: The scheduling state representation is used as model input for subsequent evaluation and selection of scheduling strategies.

3. The scheduling method for large model inference tasks oriented towards fuzzy deadline constraints according to claim 2, characterized in that, In S14, the method includes the following steps: S141: Based on the structured task data and system operation status information, extract task features and server resource features, wherein the task features include the task identifier, input token length, output token length, deadline parameter and corresponding inference stage information, and the system resource features include the identifier, computing power, cost, current resource occupancy and queue length of each server, so as to form the observation feature representation of each inference task at the current moment. S142: The observation feature representation is fused with the system resource feature representation to form a unified scheduling state representation used to describe the system state in the current cloud-edge collaborative environment.

4. A scheduling method for large model inference tasks oriented towards fuzzy deadline constraints, as described in claim 1 or 3, characterized in that, In S2, the method includes the following steps: S21: Set a lower deadline for each inference task to represent the preferred completion time and an upper deadline for each allowed completion time. S22: Obtain the actual completion time of the reasoning task and construct a satisfaction membership function based on the relationship between the actual completion time and the lower and upper limits of the deadline: ,in Record the delay time for task completion. This is the lower limit of the deadline. This is the upper limit of the deadline. A constant used to avoid a denominator of zero; S23: Construct a deadline penalty term based on the aforementioned satisfaction membership function: This is to increase the deadline penalty as the task completion time approaches or exceeds the deadline limit. S24: Calculate the total completion latency and execution cost of the inference task execution process, wherein the total completion latency includes at least queuing time, data transmission time and inference processing time, and the execution cost includes at least the resource occupation cost of the task on the target computing node. S25: The total completion delay, execution cost, and deadline penalty are processed to a unified dimension and combined according to preset weight coefficients to form a reward function: ,in, and These are the normalized task completion latency and execution cost, respectively. These are the weighting coefficients.

5. The scheduling method for large model inference tasks oriented towards fuzzy deadline constraints according to claim 4, characterized in that, In S2, the method further includes the following steps: S26: Combine task information with computing node information to construct a scheduling action space for representing scheduling decisions to allocate inference tasks to different cloud servers or edge servers; S27: Based on the scheduling action space and system state, the scheduling process is modeled as a time-skipping semi-Markov decision process model consisting of state, option actions, state transitions and reward functions, wherein the option actions are time-extended actions with duration, used to describe the continuous scheduling behavior during task execution.

6. The method for scheduling large model inference tasks oriented towards fuzzy deadline constraints according to claim 5, characterized in that, In S3, the method includes the following steps: S31: Define the current system state at the current scheduling moment as a state variable in a semi-Markov decision process, wherein the system state includes at least the status of the task queue to be scheduled, the resource status of each cloud server and edge server, task deadline information, and task execution progress information. S32: Define the task scheduling behavior as a time-extended action and maintain the current action execution state during the duration of the time-extended action, and perform a state transition after the action ends; S33: Construct a state encoding model for feature compression and representation learning of the current system state, construct a state prediction model for predicting the potential state at the next moment after the action is executed, construct a time prediction model for predicting the duration of the time-extended action, construct a cost prediction model for predicting the execution cost corresponding to the candidate scheduling action, and construct a policy network for outputting the prior probability distribution of each candidate scheduling action according to the current system state. S34: Use historical scheduling samples to jointly train the state coding model, time prediction model, cost prediction model and policy network to obtain a world model that can represent the dynamic changes of the real scheduling environment.

7. A scheduling method for large model inference tasks oriented towards fuzzy deadline constraints, as described in claim 1 or 6, characterized in that, In S4, the method includes the following steps: S41: Construct a Monte Carlo search tree with the current system state as the root node and simulate and evaluate candidate scheduling actions based on the world model; S42: The prior probability of the candidate scheduling action is randomly perturbed by introducing Gumbel noise to obtain the perturbed prior probability; wherein the Gumbel noise is generated in the following manner: ,in Let be a random variable that follows a standard uniform distribution. It is Gumbel noise; and satisfies the following relationship: ,in This represents the prior probability after adding Gumbel noise. The prior probability of the candidate scheduling action; S43: Construct an action selection scoring function based on the action value function and the perturbed prior probability, and select the optimal scheduling action according to the scoring function, wherein the scoring function satisfies the following relationship: ,in, Indicates the state Next action The estimated cumulative reward, These are the weighting coefficients. For scoring functions; S44: Expand the selected scheduling action and predict the next state, execution latency, and cost based on the world model; S45: Update the search path based on the prediction results, update the value estimate of each candidate scheduling action, and repeat the selection, expansion and backtracking process until the preset number of searches or termination conditions are reached, and output the optimal scheduling strategy.

8. The scheduling method for large model inference tasks oriented towards fuzzy deadline constraints according to claim 1, characterized in that, In step S44, the method includes the following steps: S441: Expand the selected scheduling action and generate corresponding child nodes in the Monte Carlo search tree; S442: Predict the next state after the scheduling action is executed based on the world model; S443: Predict the execution latency and resource consumption corresponding to the scheduling action based on the world model; S444: Based on the predicted state, execution latency, and resource consumption, evaluate the value of the scheduling path corresponding to the scheduling action.

9. A scheduling method for large model inference tasks oriented towards fuzzy deadline constraints according to claim 1, characterized in that, In S5, the method includes the following steps: S51: Determine the mapping relationship between the inference task and the target server based on the optimal scheduling action output by the Monte Carlo tree search process; S52: Generate a task scheduling scheme based on the mapping relationship to indicate that the inference task will be assigned to the corresponding cloud server or edge server for execution; S53: Output the task scheduling scheme as the current scheduling decision result and allocate the inference task to the corresponding target server for execution according to the task scheduling scheme, wherein the target server includes a cloud server or an edge server. S54: During task execution, the system state is updated and the scheduling process is repeated for newly arrived inference tasks based on the updated system state, so as to realize continuous scheduling of dynamic task flow.

10. A scheduling system for large model inference tasks oriented towards fuzzy deadline constraints, using the method described in any one of claims 1-9, characterized in that, include: The constraint model building module is used to model the large model inference task scheduling system in the cloud-edge collaborative computing environment and build a fuzzy deadline constraint model. The modeling includes at least the task structured representation, cloud server and edge server modeling, resource heterogeneity modeling, task execution latency modeling, and task execution cost modeling. The function setting module is used to map the task completion delay to continuous satisfaction by using a membership function for the fuzzy deadline constraint model, and to construct a delay penalty function based on the continuous satisfaction, thereby forming a reward function that includes task delay, resource cost and deadline default degree. The world model construction module is used to construct a scheduling action space based on the fuzzy deadline constraint model and construct a time-skip semi-Markov decision process model based on the scheduling action space and the system state, thereby constructing a world model that includes at least a state encoding model, a state prediction model, a time prediction model, a cost prediction model, and a policy network, and training the time-world model in the cloud. The scheduling processing module is used to construct a scheduling decision search mechanism based on the world model using the Gumbel noise-driven Monte Carlo tree search method, and to evaluate and select candidate scheduling actions at each decision moment. The output and update module is used to input the current system state into the world model to make multi-step predictions of the future state, output a scheduling scheme and allocate the inference task to the corresponding cloud server or edge server for execution according to the scheduling scheme, and update the system state according to the execution result.

Images