Server-free computing resource scheduling method and system based on online algorithm
By recording job cold start status in real time through online algorithms and setting time windows to promptly open or close reserved instances, the cold start and high cost issues of DAG applications on serverless computing platforms are solved, achieving efficient and flexible resource scheduling.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-26
- Publication Date
- 2026-04-10
AI Technical Summary
When deploying DAG applications on serverless computing platforms, it is difficult to balance the performance impact of cold starts with the high cost of reserved instances. Especially under rapid changes or sudden traffic surges, existing technologies struggle to effectively schedule tasks in unknown situations.
A serverless computing resource scheduling method based on online algorithms is adopted. By comparing the observed value with the target K value, the cold start status of the job is recorded in real time. Time windows are set to open or close reserved instances in a timely manner to avoid cold starts and reduce costs.
It enables efficient and timely scheduling of serverless computing resources in the case of unknown jobs, avoids the impact of cold starts, reduces the cost of reserved instances, and improves the stability and flexibility of QoS.
Smart Images
Figure CN121833259A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of serverless computing resource scheduling technology, and particularly relates to a serverless computing resource scheduling method and system based on online algorithms. Background Technology
[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.
[0003] With the rapid development of the internet, software systems are becoming increasingly large-scale, leading to the rise of microservice architectures. Serverless computing, a product of cloud computing, is well-suited for deploying applications using microservice architectures due to its convenient deployment and maintenance, rapid elastic scaling, and precise billing models. Currently, cloud providers essentially offer computing resources in the form of instances, supporting two main instance modes: reserved and pay-as-you-go. In reserved mode, the cloud user manages the allocation and release of function instances. Once a reserved instance is configured, it will remain resident until the cloud user actively releases it. It's important to note that the cloud user must pay extra for reserved function instances. Pay-as-you-go mode means that Function Compute automatically scales up and down instances based on the number of function calls. Instance creation is automatically triggered by requests, and the cloud user only pays for the computing resources used. However, the first time a call is initiated, the instance needs to be cold-started.
[0004] Generally, service providers abstract applications using a microservices architecture into a set of functions orchestrated in a workflow and deployed on a serverless platform in the form of a directed acyclic graph (DAG) in logical order. To improve the competitiveness of DAG application services, service providers need to improve their Quality of Service (QoS) and reduce cloud costs. We use the end-to-end completion time of the job as the QoS.
[0005] Despite the numerous advantages of serverless platforms, the characteristics of DAG applications and the inherent properties of serverless computing still present many challenges when combined. Cold starts are a common overhead in serverless computing, especially in DAG applications. Since multiple functions need to process jobs in a logical order, the impact of cold starts is amplified, affecting QoS. Cloud providers have proposed reserving instances for each function to address this issue, but the cost of reserving instances is high. Reservation in the reservation mode uses a much larger billing granularity; in Alibaba Cloud Function Compute, this granularity reaches 10 seconds, which is 10,000 times smaller than the millisecond-level granularity of default computing resources. This means that using a reserved instance requires paying for at least 10 seconds of usage. This dictates that reservation mode cannot be used for the entire application, as this would result in exorbitant costs. Therefore, it is crucial to carefully manage the timing of opening and closing reserved instances and the number of instances that need to be changed, minimizing reservation costs while ensuring QoS. However, due to the complexity of job loads, the uncertainty of function execution, and the unknown size of job inputs, achieving this without knowing the future job load presents a significant challenge. Summary of the Invention
[0006] To address the technical problems mentioned above, this invention provides a serverless computing resource scheduling method and system based on online algorithms. The method makes scheduling decisions based on the comparison between observed values and the target K value, and opens new reserved instances in advance at appropriate time points. This is equivalent to scheduling serverless computing resources before the application performance may drop to the K value, avoiding cold start time for jobs during processing, and has high scheduling timeliness and reliability.
[0007] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of this invention provides a serverless computing resource scheduling method based on an online algorithm, comprising: Obtain cloud users' performance requirements for DAG applications and get target values for performance metrics; During the operation of the DAG application, record whether each job experiences a cold start time when passing through different functions, and mark the job as a qualified job or an unqualified job. For DAG applications, set time windows; at the end of each time window, obtain the usage status and concurrency of on-demand instances, and calculate the observations by combining the marking status of completed jobs. The observed value is compared with the performance target value. If the observed value is less than the performance target value, and a function uses an on-demand instance when processing the job, a new reserved instance is opened for the function at the end of the time window.
[0008] Furthermore, the observed values are: Where A represents the number of completed jobs that did not experience a cold start period, and S represents the total number of completed jobs. Indicates the number of on-demand instances used by a given function. This indicates the concurrency level of the function instances.
[0009] Furthermore, it also includes: if it is detected that some instances of a function have not been used during their lifetime, then these instances are closed.
[0010] Furthermore, the time window is the interval between making a serverless computing resource scheduling decision and scheduling resources based on the decision.
[0011] Furthermore, if a cold start period has occurred, the job will be marked as an unqualified job; otherwise, it will be a qualified job.
[0012] A second aspect of the present invention provides a serverless computing resource scheduling system based on an online algorithm, comprising: The K-value setting module is configured to: obtain the performance requirements of cloud users for DAG applications and obtain the target value of the performance index; The cold start job marking module is configured to: during the DAG application operation, record whether each job experiences a cold start time when passing through different functions, and mark the job as a qualified job or an unqualified job. The time window setting module is configured to set a time window for DAG applications. The observation calculation module is configured to: at the end of each time window, obtain the usage status and concurrency level of the on-demand instances, and calculate the observations by combining the marking of completed jobs; The serverless computing resource management module is configured to compare observed values with performance target values. If the observed value is less than the performance target value and a function uses on-demand instances when processing jobs, a new reserved instance will be opened for the function at the end of the time window.
[0013] Furthermore, the observed values are: Where A represents the number of completed jobs that did not experience a cold start period, and S represents the total number of completed jobs. Indicates the number of on-demand instances used by a given function. This indicates the concurrency level of the function instances.
[0014] Furthermore, the serverless computing resource management module is also configured to shut down certain instances of a function if it is detected that these instances have not been used within the specified time.
[0015] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the serverless computing resource scheduling method based on an online algorithm as described above.
[0016] A fourth aspect of the present invention provides a computer device including a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, wherein the processor executes the program to implement the steps of the serverless computing resource scheduling method based on an online algorithm as described above.
[0017] Compared with the prior art, the beneficial effects of the present invention are: This invention uses the comparison between observed values and target K values as the basis for scheduling decisions, and opens new reserved instances in advance at appropriate time points. This is equivalent to scheduling serverless computing resources before the application performance may drop to the K value, avoiding the job experiencing cold start time during processing, and has high scheduling timeliness and reliability.
[0018] This invention makes serverless computing resource scheduling decisions unaffected by job load, function transfer probability, and job size. Even when all future job conditions are unknown, it can quickly and accurately schedule serverless computing resources, effectively alleviating the performance degradation problem of DAG applications when facing complex workloads with rapid and disordered changes or sudden traffic. Attached Figure Description
[0019] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0020] Figure 1 This is a flowchart of a serverless computing resource scheduling method based on an online algorithm according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the serverless computing resource scheduling algorithm of Embodiment 1 of the present invention; Figure 3 This is an architecture diagram of a serverless computing resource scheduling system based on an online algorithm according to Embodiment 2 of the present invention; Figure 4 This is a schematic diagram of the structure of a computer device according to Embodiment 4 of the present invention. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.
[0022] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0023] Example 1 This embodiment provides a serverless computing resource scheduling method based on online algorithms.
[0024] To address the challenges described in the background section, an online scheduling algorithm is proposed to help application providers open and close reserved instances for DAG applications deployed on serverless platforms in real time, reducing cost waste while ensuring QoS. The proposed online scheduling algorithm can rapidly allocate serverless computing resources without knowing future jobs.
[0025] The following describes the problems encountered in ensuring QoS of DAG applications deployed on serverless computing platforms and the challenges in solving these problems.
[0026] Cold start is one of the most frequently studied overheads in serverless computing, occurring when a serverless function instance is first launched. On Function as a Service (FaaS) platforms, cold start involves starting a new container, configuring the runtime environment, and fetching and loading necessary libraries and dependencies. Cold start takes longer than the short execution of a function. The impact of cold start is particularly significant in Directed Acyclic Graph (DAG) applications, as a job typically involves the sequential processing of multiple functions, potentially leading to more function cold starts and thus a significant impact on QoS. To address this issue, a strategy of reserving instances for each function in the application can be adopted.
[0027] When avoiding cold starts by reserving instances, the cost of reserving instances must be fully considered. To keep instances active, cloud users need to pay additional fees for reserved function instances. Compared to the pay-as-you-go billing model of pay-as-you-go instances, continuously maintaining reserved instances for applications incurs higher reservation costs. Ideally, an appropriate number of reserved instances should be pre-activated before jobs arrive to avoid delays caused by cold starts, and instances should be promptly deactivated when no longer needed, thereby ensuring QoS and reducing the reservation costs. Therefore, mastering the dynamic activation and deactivation timing of reserved instances and determining the changing number of instances becomes a critical technical issue that urgently needs to be addressed.
[0028] The following challenges will be encountered in solving this problem: The complexity of job loads: In real-world applications, job load patterns are often unpredictable. It's impossible to know exactly when a job will arrive, making it extremely difficult to accurately determine when to launch reserved instances, especially in the face of rapidly changing or bursty traffic patterns. In such complex job load environments, making appropriate instance scheduling decisions becomes a significant challenge.
[0029] DAG Application Function Transition Probability: When a job arrives, not all functions in a DAG (Directed Acyclic Graph) application are necessarily required to participate in job processing. Due to conditional branches, service providers cannot determine which functions will be executed before the job actually arrives. This uncertainty in function transition probabilities may lead to the activation of too many reserved instances when some functions are considered more likely to process the job, resulting in over-provisioning of resources and additional costs; conversely, insufficient reserved instances may be activated when some functions are considered less likely to process the job, causing the job to experience cold start time and thus impacting QoS. Therefore, effectively and quickly allocating serverless computing resources when transition probabilities are unknown becomes another challenge that needs to be addressed.
[0030] Job Input Size: Since most DAG applications require processing user-provided input to generate the desired results, the execution time of many such applications is closely related to the size of the job input. When the size of the job input is unknown, service providers cannot accurately predict the execution time of each function. This uncertainty makes it difficult to determine when to open function reservation instances: opening too early incurs unnecessary reservation costs, while delaying opening may lead to cold starts, thus affecting QoS. Therefore, how to ensure QoS while reducing reservation costs when the job input size is unknown is a significant challenge.
[0031] This embodiment provides a serverless computing resource scheduling method based on an online algorithm. A K value is set according to the cloud user's performance requirements for the DAG application. During application execution, it records whether each job experiences a cold start time when passing through different functions in the DAG application. If a cold start time occurs, the job is marked as unqualified; otherwise, it is considered qualified. For the DAG application, a time window T is set. w Set and monitor the observed value K mAt the end of each time window, the serverless computing resource scheduling algorithm acquires observations and compares them with the K value. This allows it to quickly make scheduling decisions for serverless computing resources even when the workload, the internal function transition probability of the DAG application, and the job execution time are all unknown. This enables rapid allocation of serverless computing resources in the face of complex workloads with rapid and disordered changes or sudden traffic surges. The algorithm also adjusts the number of reserved instances based on the actual operation of the application, improving QoS while saving costs.
[0032] This embodiment provides a serverless computing resource scheduling method based on online algorithms, which uses the percentage of jobs that have experienced cold start time to the total number of jobs as an evaluation index. The target value (K value) of the evaluation index can be specified by the cloud user.
[0033] This embodiment provides a serverless computing resource scheduling method based on online algorithms, such as... Figure 1 As shown, it includes the following steps: Step 1: Based on the cloud user's performance requirements for DAG applications, the end-to-end completion time of the job is used as a performance indicator, and a target threshold for the job performance satisfaction rate is set, i.e., the performance indicator target value (K value). Step 2: During the DAG application's operation, record whether each job experiences a cold start time when passing through different functions. If it does, mark it as an unqualified job; otherwise, it is a qualified job. Step 3: Apply serverless computing resource scheduling algorithms to quickly make scheduling decisions for serverless computing resources, enabling rapid allocation of serverless computing resources in the face of complex workloads that are subject to rapid and disordered changes or sudden traffic surges.
[0034] Furthermore, serverless computing resource scheduling algorithms, such as Figure 2 As shown, it includes the following steps: Step 3-1: For DAG applications, set the time window. ; Step 3-2: Set observation values ; Step 3-3: At the end of each time window, obtain the observation value and compare it with the K value; Steps 3-4: If the observed values If the value is less than K and a function uses on-demand instances when processing a job, it means that the number of reserved instances that have been opened at this time is insufficient, which may have caused the job to experience a cold start time. In this case, a new reserved instance will be opened for the function at the end of the time window to avoid the job experiencing a cold start time in the future and affecting QoS. Steps 3-5: If it is detected that some instances of a function have not been used during their lifetime, these instances will be shut down to reduce reservation costs and thus reduce total costs.
[0035] Furthermore, time window This refers to how often a scheduling decision is made, and how serverless computing resources are rapidly allocated based on this decision in the face of complex workloads that are subject to rapid and disorderly changes or sudden traffic surges.
[0036] Furthermore, the observed values The calculation method is as follows: Where A represents the number of completed jobs that did not experience a cold start period, and S represents the total number of completed jobs. Indicates the number of on-demand instances used by a given function. This indicates the instance concurrency of the function, which means how many job requests a single function instance can handle simultaneously.
[0037] The reason for this design is that, in this embodiment, jobs are marked as qualified or unqualified upon completion based on whether they experienced a cold start period. However, at the end of a certain time window, some jobs may not have been completed but have already experienced the cold start period. These jobs are not marked but are already considered unqualified jobs. In other words, the actual performance of the application is lower than what is obtained in this embodiment at this time. The observed value reflects the worst-case scenario of the application's actual performance, and the observed value will never exceed the actual performance of the application. This embodiment uses the comparison between the observed value and the target K value as the basis for scheduling decisions. This is equivalent to scheduling serverless computing resources before the application performance may drop to the K value, which has high scheduling timeliness and reliability.
[0038] This embodiment provides a serverless computing resource scheduling method based on an online algorithm, which significantly mitigates the impact of cold start time on job execution time for DAG applications deployed on serverless computing. By setting observation values and using the comparison between the observation values and the target K value as the basis for scheduling decisions, new reserved instances are opened in advance at appropriate time points. This is equivalent to scheduling serverless computing resources before the application performance may degrade to the K value, avoiding cold start time for jobs during processing, and exhibiting high scheduling timeliness and reliability.
[0039] This embodiment provides a serverless computing resource scheduling method based on online algorithms. The determination of serverless computing resource scheduling decisions is not affected by job load, function transfer probability, and job size. Even when all future job conditions are unknown, serverless computing resources can be scheduled quickly and accurately, effectively alleviating the performance degradation problem of DAG applications when facing complex workloads with rapid and disordered changes or sudden traffic.
[0040] This embodiment provides a serverless computing resource scheduling method based on online algorithms, which effectively alleviates the problem of high cost losses in the process of improving performance. It closes idle reserved instances in a timely manner according to the specific situation within the time window, thereby reducing the reservation cost of instances.
[0041] Example 2 This embodiment provides a serverless computing resource scheduling system based on online algorithms, such as... Figure 3 As shown, it includes: K-value setting module, cold start job marking module, time window setting module, observation value calculation module, and serverless computing resource management module.
[0042] The K-value setting module is configured to set specific values for the performance indicators to be achieved based on the performance requirements of cloud users for DAG applications. The cold start job marking module is configured to: monitor the actual completion status of the job and mark it as a qualified job or an unqualified job based on whether the cold start time has elapsed. If the cold start time has elapsed, it is marked as an unqualified job; otherwise, it is marked as an unqualified job. The time window setting module is configured to: based on the cloud user’s needs for DAG application jobs, set an appropriate time window size according to experience, that is, how often to make a serverless computing resource scheduling decision and schedule resources according to the decision. The observation calculation module is configured to calculate observations based on the marking of completed jobs, the usage of instances, and the concurrency of instances. The serverless computing resource management module is configured to: primarily implement the serverless computing resource scheduling algorithm, the detailed implementation process of which can be found in step 3 of embodiment 1; and enable and disable reserved instances based on specific circumstances such as observation values, thereby saving costs while ensuring QoS.
[0043] This embodiment provides a serverless computing resource scheduling system based on an online algorithm, which can be easily deployed in a distributed manner.
[0044] This embodiment provides a serverless computing resource scheduling system based on online algorithms, which can provide seamless instance opening and closing functions without requiring any user intervention or auxiliary decision-making.
[0045] It should be noted that each module in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation processes are the same, so they will not be repeated here.
[0046] Example 3 This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a serverless computing resource scheduling method based on an online algorithm as described in Embodiment 1 above.
[0047] Example 4 This embodiment provides a computer device, such as... Figure 4 As shown, the system includes a computer-readable storage medium 1003, a processor 1001, a communication interface 1002, and a computer program stored on the computer-readable storage medium 1003 and executable on the processor 1001. The processor 1001, communication interface 1002, and computer-readable storage medium 1003 can be connected via a bus or other means. The communication interface 1002 is used to receive and send data. When the processor 1001 executes the program, it implements the steps of the serverless computing resource scheduling method based on an online algorithm as described in Embodiment 1 above.
[0048] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A serverless computing resource scheduling method based on online algorithms, characterized in that, include: Obtain cloud users' performance requirements for DAG applications and get target values for performance metrics; During the operation of the DAG application, record whether each job experiences a cold start time when passing through different functions, and mark the job as a qualified job or an unqualified job. For DAG applications, set time windows; at the end of each time window, obtain the usage status and concurrency of on-demand instances, and calculate the observations by combining the marking status of completed jobs. The observed value is compared with the performance target value. If the observed value is less than the performance target value, and a function uses an on-demand instance when processing the job, a new reserved instance is opened for the function at the end of the time window.
2. The serverless computing resource scheduling method based on online algorithms as described in claim 1, characterized in that, The observed values are: Where A represents the number of completed jobs that did not experience a cold start period, and S represents the total number of completed jobs. Indicates the number of on-demand instances used by a given function. This indicates the concurrency level of the function instances.
3. The serverless computing resource scheduling method based on online algorithms as described in claim 1, characterized in that, Also includes: If it is detected that some instances of a function have not been used within their lifetime, then these instances are closed.
4. The serverless computing resource scheduling method based on online algorithms as described in claim 1, characterized in that, The time window is the interval between making a serverless computing resource scheduling decision and scheduling resources based on the decision.
5. The serverless computing resource scheduling method based on online algorithms as described in claim 1, characterized in that, If a cold start period occurs, the job will be marked as an unqualified job; otherwise, it will be a qualified job.
6. A serverless computing resource scheduling system based on online algorithms, characterized in that, include: The K-value setting module is configured to: obtain the performance requirements of cloud users for DAG applications and obtain the target value of the performance index; The cold start job marking module is configured to: during the DAG application operation, record whether each job experiences a cold start time when passing through different functions, and mark the job as a qualified job or an unqualified job. The time window setting module is configured to set a time window for DAG applications. The observation calculation module is configured to: at the end of each time window, obtain the usage status and concurrency level of the on-demand instances, and calculate the observations by combining the marking of completed jobs; The serverless computing resource management module is configured to compare observed values with performance target values. If the observed value is less than the performance target value and a function uses on-demand instances when processing jobs, a new reserved instance will be opened for the function at the end of the time window.
7. The serverless computing resource scheduling system based on online algorithms as described in claim 6, characterized in that, The observed values are: Where A represents the number of completed jobs that did not experience a cold start period, and S represents the total number of completed jobs. Indicates the number of on-demand instances used by a given function. This indicates the concurrency level of the function instances.
8. The serverless computing resource scheduling system based on an online algorithm as described in claim 6, characterized in that, The serverless computing resource management module is also configured to shut down certain instances of a function if it is detected that these instances have not been used within the specified time.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the serverless computing resource scheduling method based on an online algorithm as described in any one of claims 1-5.
10. A computer device comprising a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the serverless computing resource scheduling method based on an online algorithm as described in any one of claims 1-5.