A method and system for automatically scheduling large model application services under limited computing power

Abstract

The application relates to the technical field of electric digital data processing, in particular to a large model application service automatic scheduling method and system under limited computing power. The method comprises the following steps: receiving a task request of a large model application service initiated by a subordinate platform; correcting the original estimated running time according to the task description information and historical task execution data of the subordinate platform to generate a corrected estimated running time; judging whether to execute the task initiated by the subordinate platform according to the corrected estimated running time and the current residual computing power; if the judgment is no, adding the task initiated by the subordinate platform to a waiting queue corresponding to a specified resource type, and based on the current state of the waiting queue, obtaining a predicted waiting duration of the task initiated by the subordinate platform, and returning response information containing the predicted waiting duration to the subordinate platform. The application improves the utilization efficiency of the computing power resources of the superior platform.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing technology, and in particular to an automatic scheduling method and system for large model application services under limited computing power. Background Technology

[0002] The demand for computing resources from lower-level platforms is increasing, but the computing resources deployed on these platforms are often limited. To optimize resource allocation, higher-level platforms provide large-model application services to lower-level platforms. Lower-level platforms can send task requests for these services to the higher-level platform to utilize its large models. However, higher-level platforms often interface with multiple lower-level platforms, and their computing resources are also limited. The higher-level platform needs to determine whether to execute tasks submitted by lower-level platforms based on pre-set response strategies. One processing flow is as follows: when a lower-level platform sends a task request for a large-model application service to the higher-level platform, the request includes the estimated runtime of the task. The higher-level platform then determines whether to execute the task based on this time and its remaining computing power. However, in some cases, the estimated runtime submitted by the lower-level platform is often based on its own experience or local environment, resulting in a significant deviation from the actual runtime in the higher-level platform's actual hardware environment. This deviation affects the higher-level platform's scheduling decisions for large models, potentially leading to incorrect allocation of computing resources and reducing the utilization efficiency of the higher-level platform's computing resources. Summary of the Invention

[0003] The purpose of this invention is to provide an automatic scheduling method and system for large model application services under limited computing power, so as to improve the utilization efficiency of computing resources of the upper-level platform.

[0004] According to a first aspect of the present invention, an automatic scheduling method for large model application services under limited computing power is provided, the method comprising the following steps:

[0005] S100, receive a task request for a large model application service initiated by a lower-level platform; the task request includes the original estimated runtime and task description information submitted by the lower-level platform; the task description information includes the task type.

[0006] S200, the original estimated running time is corrected based on the task description information and the historical task execution data of the lower-level platform to generate a corrected estimated running time; the historical task execution data includes the deviation between the historical estimated running time and the actual running time.

[0007] S300, based on the corrected estimated running time and the current remaining computing power, determines whether to execute the task initiated by the lower-level platform.

[0008] S400, if the determination is negative, the task initiated by the lower-level platform is added to the waiting queue corresponding to the specified resource type, and based on the current state of the waiting queue, the estimated waiting time of the task initiated by the lower-level platform is obtained, and the response information containing the estimated waiting time is returned to the lower-level platform; the specified resource type is the resource type corresponding to the task type included in the task description information.

[0009] According to a second aspect of the present invention, an automatic scheduling system for large model application services under limited computing power is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described automatic scheduling method for large model application services under limited computing power.

[0010] Compared with the prior art, the present invention has at least the following beneficial effects:

[0011] This invention corrects the original estimated runtime based on the task description information included in the task request of the large-scale application service initiated by the lower-level platform and the historical task execution data of the lower-level platform, generating a corrected estimated runtime. Thus, by introducing a correction mechanism based on historical task execution data, this invention reduces the error of the lower-level platform's original estimate, generating a corrected estimated runtime that more closely reflects the actual running conditions. This allows the scheduler of the upper-level platform to make execution decisions based on more reliable data, avoiding decision-making errors caused by inaccurate runtime information. Because the scheduling decision is based on a more accurate runtime prediction, the upper-level platform can allocate computing resources more precisely, reducing idle computing resources caused by early task completion and reducing computing resource scheduling conflicts caused by severe task timeouts. Therefore, with limited computing resources, it can accept and complete more computing tasks, improving the system's computing resource utilization efficiency.

[0012] Furthermore, if the present invention determines that it will not execute the task initiated by the lower-level platform, it will also return response information including the estimated waiting time to the lower-level platform. This feedback mechanism enables the lower-level platform to clearly understand the estimated waiting time of its task and make subsequent decisions based on this information, thereby significantly improving the collaboration efficiency between platforms. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1The flowchart illustrates the automatic scheduling method for large model application services under limited computing power provided in Embodiment 1 of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Example 1:

[0017] According to this embodiment, as Figure 1 As shown, an automatic scheduling method for large model application services under limited computing power is provided. The method includes the following steps:

[0018] S100, receive a task request for a large model application service initiated by a lower-level platform; the task request includes the original estimated runtime and task description information submitted by the lower-level platform; the task description information includes the task type.

[0019] In this embodiment, the large model application service is a service provided by the upper-level platform to the lower-level platform. The upper-level platform has built a large model, and the lower-level platform can submit a task request for the large model application service to the upper-level platform to use the large model built by the upper-level platform to perform the task. The original estimated running time is the task execution time estimated by the lower-level platform based on its own estimation, and the task type is the category of the task (such as data processing, model training, image recognition, etc.).

[0020] In this embodiment, the resource type (such as the GPU type used when a large model executes different types of tasks, where the GPU type can be understood as the GPU model) and computing power size corresponding to each task type are known. That is, for a certain task type, the resource type for processing the task of that task type is predetermined, and the computing power size allocated to the task when processing the task of that task type is also predetermined.

[0021] S200, the original estimated running time is corrected based on the task description information and the historical task execution data of the lower-level platform to generate a corrected estimated running time; the historical task execution data includes the deviation between the historical estimated running time and the actual running time.

[0022] In this embodiment, the corrected estimated running time is the value after correcting the original estimated time.

[0023] In this embodiment, historical data is used to compensate for deviations in the estimates made by lower-level platforms, aiming to obtain an estimated runtime closer to the actual value. As a specific implementation, correcting the original estimated runtime based on the task description information and historical task execution data from lower-level platforms includes:

[0024] S210, Obtain the platform profile of the lower-level platform based on the historical task execution data of the lower-level platform; the platform profile includes the average historical error rate and variance corresponding to different task types.

[0025] In this embodiment, the platform profile is a dataset reflecting the estimated capabilities of lower-level platforms, including the average historical error rate (the average level of estimated deviation) and variance (the degree of fluctuation in deviation) for different user types. In this embodiment, the average historical error rate and variance are obtained based on the error rate over a recent preset duration. The process of obtaining the error rate includes: for any task request, obtaining the difference between the actual running time and the original estimated time for that task request, and determining the ratio of the difference to the original estimated time as the error rate. The preset duration can be an empirical value, such as 5 days or 10 days. Therefore, the platform profile reflects the most recent estimated capabilities of the lower-level platforms.

[0026] S220, based on the task type included in the task description information, obtain the average historical error rate and the variance of the matching from the platform profile of the lower-level platform.

[0027] In this embodiment, the platform profile of the lower-level platform includes the average historical error rate and variance corresponding to different task types of the platform profile. Based on the task types included in the task description information, the average historical error rate and variance corresponding to the task types included in the task description information can be obtained from the platform profile, that is, the average historical error rate and variance of the matching.

[0028] S230, the original estimated running time is corrected based on the average historical error rate and variance of the matching to generate the corrected estimated running time.

[0029] As a specific implementation, correcting the original estimated running time based on the average historical error rate and the variance of the matching includes: if the variance of the matching is greater than a preset variance threshold, then the original estimated running time is corrected based on the average historical error rate and a preset first buffer coefficient; otherwise, the original estimated running time is corrected based on the average historical error rate and a preset second buffer coefficient; the first buffer coefficient is not equal to the second buffer coefficient.

[0030] In one specific implementation, the corrected estimated running time = original estimated time × (1 + average historical error rate) × buffer coefficient. Wherein, when the variance is greater than a preset variance threshold, the buffer coefficient is the first buffer coefficient; when the variance is less than or equal to the preset variance threshold, the buffer coefficient is the second buffer coefficient. Both the second and first buffer coefficients are greater than 1, and the second buffer coefficient is less than the first buffer coefficient. In one specific implementation, both the first and second buffer coefficients are empirical values, for example, the first buffer coefficient is 1.05 and the second buffer coefficient is 1.02.

[0031] Therefore, this embodiment corrects the original estimated running time in the task requests initiated by the lower-level platform, which can reduce the risk of system overload caused by the lower-level platform underestimating the task time, or the waste of computing power caused by the lower-level platform overestimating the task time.

[0032] S300, based on the corrected estimated running time and the current remaining computing power, determines whether to execute the task initiated by the lower-level platform.

[0033] As one specific implementation, S300 includes:

[0034] S310, obtain the computing power Q1 corresponding to the task type included in the task description information.

[0035] In this embodiment, for a given task type, the resource type used by the upper-level platform to process tasks of that task type is predetermined and fixed, as is the computing power used by the upper-level platform to process tasks of that task type. For example, a mapping table between task types, resource types, and computing power can be pre-established, and the resource type (i.e., the specified resource type) and computing power (i.e., Q1) corresponding to the task type included in the task description information can be obtained based on this table. This reduces scheduling complexity, facilitates management, and is suitable for scenarios with highly standardized task resource requirements, such as microservices and standardized jobs.

[0036] S320, Filter the current remaining computing power Q2 that corresponds to the specified resource type from the current remaining computing power.

[0037] In this embodiment, the current remaining computing power may include remaining computing power that is not of the specified resource type. Therefore, it is necessary to filter the current remaining computing power to obtain the current remaining computing power Q2 corresponding to the specified resource type.

[0038] S330, obtain the task load L1, L1=Q1×T, where T is the corrected estimated running time.

[0039] S340, if Q2≥Q1 and L1 <k×(L2×T0-∑ n i=1 (Ai ×T i )-∑ m j=1 (B j ×t j If the condition is met, the task initiated by the lower-level platform will be executed; otherwise, the task initiated by the lower-level platform will not be executed. Here, L2 is the total computing power corresponding to the specified resource type, T0 is the preset maximum task response time, k is the preset computing power buffer coefficient, and A... i T represents the computing power used by the i-th task in the waiting queue corresponding to a specified resource type. i This refers to the execution time of the i-th task in the waiting queue corresponding to a specified resource type, where i ranges from 1 to n, and n is the number of tasks in the waiting queue corresponding to the specified resource type; B j t represents the computing power used by the j-th task currently executing, corresponding to a specified resource type. j This represents the remaining runtime of the j-th task currently being executed for the specified resource type. The value of j ranges from 1 to m, where m is the number of tasks currently being executed for the specified resource type.

[0040] In this embodiment, the maximum task response time is the maximum time threshold from submission to execution of a promised or expected task, and this maximum task response time is a known preset value. The computing power buffer coefficient is a safety factor used to reserve a portion of the system capacity to cope with sudden high-priority tasks, prediction errors, or system fluctuations, thereby improving system stability; 0 <k<1。

[0041] As a specific implementation method, T i Specifically, t is the estimated runtime of the i-th task in the waiting queue corresponding to a specified resource type, after correction. j Specifically, it refers to the difference (t) between the estimated runtime of the j-th task currently being executed for a specified resource type and the actual execution time. j ≥0; if the difference is less than 0, then t is determined. j =0).

[0042] In this embodiment, Q2≥Q1 indicates that the upper-level platform currently has sufficient readily available computing power to start the task; otherwise, it indicates that the upper-level platform currently does not have sufficient readily available computing power to start the task. L1 represents the total amount of computing resources required for the task initiated by the lower-level platform from start to completion. L2×T0 represents the theoretical upper limit of the total computing resources that a specified resource type can provide within the maximum response time. n i=1 (A i ×T i ) represents the total amount of resources that all queued tasks corresponding to the specified resource type will consume during execution; ∑ mj=1 (B j ×t j This represents the total amount of resources required for tasks that have been started but not yet completed; L2×T0-∑ n i=1 (A i ×T i )-∑ m j=1 (B j ×t j This represents the theoretical resource margin that the system can still provide within the maximum response time after deducting the load of waiting queue tasks and currently executing tasks; k×(L2×T0-∑ n i=1 (A i ×T i )-∑ m j=1 (B j ×t j )) indicates the actual safe capacity allowed for tasks initiated by lower-level platforms.

[0043] It should be understood that determining not to execute a task initiated by a lower-level platform means that the task cannot be executed immediately. The task initiated by the lower-level platform needs to enter the waiting queue corresponding to the specified resource type and wait for processing.

[0044] Therefore, this embodiment satisfies the immediate computing power requirements while preventing long-running tasks from excessively occupying the capacity of future time windows, ensuring the overall responsiveness and throughput of the system, and achieving fine-grained scheduling under limited computing power.

[0045] S400, if the determination is negative, the task initiated by the lower-level platform is added to the waiting queue corresponding to the specified resource type, and based on the current state of the waiting queue, the estimated waiting time of the task initiated by the lower-level platform is obtained, and the response information containing the estimated waiting time is returned to the lower-level platform; the specified resource type is the resource type corresponding to the task type included in the task description information.

[0046] In this embodiment, if the determination is negative, it indicates that the lower-level platform failed to call the large model application service. If the determination is positive, the lower-level platform successfully called the large model application service, and the upper-level platform schedules the large model to execute the task initiated by the lower-level platform.

[0047] As a specific implementation method, the estimated waiting time for obtaining tasks initiated by lower-level platforms includes:

[0048] S410, identify all tasks in the waiting queue whose priority is higher than or equal to the priority of tasks initiated by the lower-level platform, and designate them as preceding tasks.

[0049] In this embodiment, the priorities corresponding to different task types are known, and the priority of a task can be determined based on the task type of the task in the waiting queue.

[0050] S420: Obtain the sum of the running time of all preceding tasks and the remaining running time of the currently executing task corresponding to the specified resource type to get the base waiting time.

[0051] In this embodiment, the base waiting time is ∑ n i=1 T i +∑ m j=1 t j .

[0052] S430, based on historical task arrival records, obtain the probability that a task initiated by a lower-level platform will not be interrupted within the basic waiting time.

[0053] As a specific implementation method, the probability that a task initiated by a lower-level platform will not be interrupted within the basic waiting time, based on historical task arrival records, includes:

[0054] S431, obtain the average number of tasks with a specified priority arriving within a unit of time based on historical task arrival records; the specified priority is a priority higher than that of tasks initiated by lower-level platforms.

[0055] In this embodiment, the historical task arrival records include the arrival time of tasks within a historical time period (e.g., the most recent 3 days or 7 days) and the task priority. Based on the historical arrival records, task arrival patterns can be obtained, and based on these patterns, task arrival predictions can be made. Optionally, the unit of time is a preset duration, such as 1 minute.

[0056] S432, obtain the probability P that no task of specified priority arrives within the basic waiting time; P=e -λ×t λ is the average number of tasks of a specified priority arriving per unit time, t is the base waiting time, and e is the base of the natural logarithm.

[0057] S440, the basic waiting time is determined as the expected waiting duration, and the probability is determined as the confidence level of the expected waiting duration; the confidence level of the expected waiting duration is used to construct response information.

[0058] In this embodiment, confidence level is a quantitative assessment of the credibility of the estimated waiting time. Therefore, based on the response information, the lower-level platform can determine the required waiting time and the corresponding probability.

[0059] In one specific implementation, in S400, if the determination is negative, the state changes of the waiting queue are continuously monitored while the task initiated by the lower-level platform is in the waiting queue; when a predetermined trigger event is detected, the estimated waiting time of the task initiated by the lower-level platform is re-acquired, and the re-acquired estimated waiting time is pushed to the lower-level platform; the predetermined trigger event includes: a higher-priority task is added to the waiting queue, and a preceding task is completed.

[0060] As a specific implementation, the method further includes: after the task initiated by the lower-level platform is completed, recording the actual running time of the task, and updating the platform profile of the lower-level platform based on the recorded actual running time. Thus, this embodiment can achieve the maintenance and updating of the platform profile of the lower-level platform.

[0061] This embodiment corrects the original estimated runtime based on the task description information included in the task request of the large model application service initiated by the lower-level platform and the historical task execution data of the lower-level platform, generating a corrected estimated runtime. Thus, by introducing a correction mechanism based on historical task execution data, this embodiment can reduce the error of the lower-level platform's original estimate and generate a corrected estimated runtime that is closer to the actual running conditions. This allows the scheduler of the upper-level platform to make decisions on whether to execute based on more reliable data, avoiding decision-making errors caused by inaccurate runtime information. Because the scheduling decision is based on a more accurate runtime prediction, the upper-level platform can allocate computing resources more precisely, reducing idle computing resources caused by early task completion and reducing computing resource scheduling conflicts caused by severe task timeouts. Therefore, with limited computing resources, it can accept and complete more computing tasks, improving the system's computing resource utilization efficiency.

[0062] Furthermore, in this embodiment, if it is determined that the task initiated by the lower-level platform will not be executed, response information including the estimated waiting time will be returned to the lower-level platform. This feedback mechanism enables the lower-level platform to clearly understand the estimated waiting time of its task and make subsequent decisions based on this information, thereby significantly improving the collaboration efficiency between platforms.

[0063] Example 2:

[0064] This embodiment provides an automatic scheduling system for large-scale model application services under limited computing power, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps:

[0065] S100, receive a task request for a large model application service initiated by a lower-level platform; the task request includes the original estimated runtime and task description information submitted by the lower-level platform; the task description information includes the task type.

[0066] S200, the original estimated running time is corrected based on the task description information and the historical task execution data of the lower-level platform to generate a corrected estimated running time; the historical task execution data includes the deviation between the historical estimated running time and the actual running time.

[0067] S300, based on the corrected estimated running time and the current remaining computing power, determines whether to execute the task initiated by the lower-level platform.

[0068] S400, if the determination is negative, the task initiated by the lower-level platform is added to the waiting queue corresponding to the specified resource type, and based on the current state of the waiting queue, the estimated waiting time of the task initiated by the lower-level platform is obtained, and the response information containing the estimated waiting time is returned to the lower-level platform; the specified resource type is the resource type corresponding to the task type included in the task description information.

[0069] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0070] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. It should also be understood that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.

Claims

1. A method for automatic scheduling of large-scale model application services under limited computing power, characterized in that, The method includes the following steps: S100, Receive a task request for a large model application service initiated by a lower-level platform; the task request includes the original estimated runtime and task description information submitted by the lower-level platform; the task description information includes the task type; the large model application service is a service provided by the upper-level platform to the lower-level platform; S200, the original estimated running time is corrected based on the task description information and historical task execution data of the lower-level platform to generate a corrected estimated running time; the historical task execution data includes the deviation between the historical estimated running time and the actual running time; the correction includes: S210, obtaining the platform profile of the lower-level platform based on the historical task execution data of the lower-level platform; the platform profile includes the average historical error rate and variance corresponding to different task types; S220, obtaining the matching average historical error rate and matching variance from the platform profile of the lower-level platform based on the task type included in the task description information; S230, correcting the original estimated running time based on the matching average historical error rate and variance to generate the corrected estimated running time. S300, based on the corrected estimated running time and the current remaining computing power, determine whether to execute the task initiated by the lower-level platform; S400, if the determination is negative, the task initiated by the lower-level platform is added to the waiting queue corresponding to the specified resource type, and based on the current state of the waiting queue, the estimated waiting time of the task initiated by the lower-level platform is obtained, and response information containing the estimated waiting time is returned to the lower-level platform; the specified resource type is the resource type corresponding to the task type included in the task description information; obtaining the estimated waiting time of the task initiated by the lower-level platform includes: S410, identifying all tasks in the waiting queue whose priority is higher than or equal to the priority of the task initiated by the lower-level platform as prerequisite tasks; S420, obtaining the sum of the running time of all prerequisite tasks and the remaining running time of the currently executing task corresponding to the specified resource type to obtain the basic waiting time; S430, obtaining the probability that the task initiated by the lower-level platform will not be interrupted within the basic waiting time based on historical task arrival records; S440, determining the basic waiting time as the estimated waiting time, and determining the probability as the confidence level of the estimated waiting time; the confidence level of the estimated waiting time is used to constitute the response information. S430 includes: S431, obtaining the average number of tasks of a specified priority arriving within a unit time based on historical task arrival records; the specified priority is a priority higher than that of tasks initiated by lower-level platforms; S432, obtaining the probability P that no task of the specified priority arrives within the basic waiting time; P=e -λ×t λ is the average number of tasks of a specified priority arriving per unit time, t is the base waiting time, and e is the base of the natural logarithm.

2. The automatic scheduling method for large model application services under limited computing power according to claim 1, characterized in that, Correcting the original estimated running time based on the average historical error rate and the variance of the matching includes: if the variance of the matching is greater than a preset variance threshold, then the original estimated running time is corrected based on the average historical error rate and a preset first buffer coefficient; otherwise, the original estimated running time is corrected based on the average historical error rate and a preset second buffer coefficient; the first buffer coefficient is not equal to the second buffer coefficient.

3. The automatic scheduling method for large model application services under limited computing power according to claim 1, characterized in that, In S400, if the determination is negative, the status changes of the waiting queue are continuously monitored while the task initiated by the lower-level platform is still in the waiting queue; when a predetermined trigger event is detected, the estimated waiting time of the task initiated by the lower-level platform is re-acquired and the re-acquired estimated waiting time is pushed to the lower-level platform. The predetermined trigger events include: a higher priority task being added to the waiting queue, and a preceding task completing execution.

4. The automatic scheduling method for large model application services under limited computing power according to claim 1, characterized in that, The method further includes: after the task initiated by the lower-level platform is completed, recording the actual running time of the task, and updating the platform profile of the lower-level platform based on the recorded actual running time.

5. The automatic scheduling method for large model application services under limited computing power according to claim 1, characterized in that, The S300 includes: S310, Obtain the computing power Q1 corresponding to the task type included in the task description information; S320, Filter the current remaining computing power Q2 that corresponds to the specified resource type from the current remaining computing power; S330, obtain the task load L1, L1=Q1×T, T is the corrected estimated running time; S340, if Q2≥Q1 and L1 <k×(L2×T0-∑ n i=1 (A i ×T i )-∑ m j=1 (B j ×t j If the condition is met, the task initiated by the lower-level platform will be executed; otherwise, the task initiated by the lower-level platform will not be executed. Here, L2 is the total computing power corresponding to the specified resource type, T0 is the preset maximum task response time, k is the preset computing power buffer coefficient, and A... i T represents the computing power used by the i-th task in the waiting queue corresponding to a specified resource type. i This refers to the execution time of the i-th task in the waiting queue corresponding to a specified resource type, where i ranges from 1 to n, and n is the number of tasks in the waiting queue corresponding to the specified resource type; B j t represents the computing power used by the j-th task currently executing, corresponding to a specified resource type. j This represents the remaining runtime of the j-th task currently being executed for the specified resource type. The value of j ranges from 1 to m, where m is the number of tasks currently being executed for the specified resource type.

6. An automatic scheduling system for large-scale model application services under limited computing power, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the automatic scheduling method for large model application services under limited computing power as described in any one of claims 1 to 5.

Images