Distributed function scheduling optimization method and device under server-free architecture

Abstract

The embodiment of the invention provides a distributed function scheduling optimization method and device under a server-free architecture, and the method employs a dynamic priority scheduling strategy, dynamically adjusts the priority of a function task according to a predefined priority rule and real-time monitoring data, guarantees the priority processing of a key task, and improves the scheduling efficiency. And the system response speed and the resource utilization efficiency are improved. A distributed cache is adopted to store function tasks, execute environment snapshots and a dependency library, a preheating mechanism is supported, cold start delay is reduced, and function task execution efficiency is improved. Through a monitoring feedback mechanism, index data related to system operation are collected in real time, and according to the index data, a priority and a caching strategy are dynamically adjusted, so that adaptive optimization is realized, and the stability and robustness of the system are improved. And through dynamic priority scheduling and cache optimization, the system throughput and the resource utilization rate are improved.

Description

Technical Field

[0001] This application relates to the field of cloud computing technology, and particularly to a distributed function scheduling optimization method and device under a serverless architecture. Background Art

[0002] Cloud computing is a service model that provides computing resources over a network. Through cloud computing, computing resources such as servers, storage, databases, networks, software, etc. can be provided to clients with different requirements. The computing modes of cloud computing include server architectures and serverless architectures. Under the serverless architecture, users do not need to manage servers. By uploading code snippets to the cloud platform, the cloud platform is responsible for automatically allocating resources and executing the code to meet the specific needs of users.

[0003] In the cloud computing model based on the serverless architecture, the computing platform needs to perform function scheduling and optimization based on requests input by users. Since the computing platforms of cloud computing are widely used in various scenarios, there are numerous functions involved in the computing platform. Therefore, the computing platform needs to execute function scheduling according to the set scheduling strategy. For example, the computing platform can perform function scheduling based on a First In First Out (FIFO) queue. This method cannot distinguish the priorities of functions, resulting in critical tasks possibly being delayed. Especially in high-concurrency situations, it cannot meet the application requirements sensitive to latency.

[0004] Some cloud computing platforms also support setting static priorities for functions, that is, by pre-configuring the priorities of each function, the computing platform can perform function scheduling according to the configured priorities. However, when performing function scheduling based on static priorities, it is impossible to dynamically adjust according to real-time load and business requirements. When the load pattern changes, static priorities may lead to uneven resource allocation and affect the overall efficiency. Summary of the Invention

[0005] In view of this, embodiments of this application provide a distributed function scheduling optimization method and device under a serverless architecture, which can improve the efficiency and performance of the serverless computing platform, reduce cold start latency, and support priority scheduling in high-concurrency scenarios, enhancing the user experience and resource utilization rate to solve the problem of low function scheduling efficiency of cloud computing platforms.

[0006] According to one aspect of this application, a distributed function scheduling optimization method under a serverless architecture is provided. The method includes:

[0007] Obtain priority rules and real-time monitoring data;

[0008] Calculate a comprehensive priority according to the priority rule and the real-time monitoring data, where the comprehensive priority is the weighted sum result of the scores of multiple monitoring items in the real-time monitoring data;

[0009] Load the objective function into the multi-level task queue according to the comprehensive priority, where the objective function is a function determined according to the comprehensive priority and the queue length of the multi-level task queue;

[0010] Extract function tasks from the multi-level task queue, and perform startup function scheduling according to the cache status of the function tasks. The startup function includes a first function and a second function. The first function is a function task reused from the cache; the second function is a newly created function task;

[0011] Record the metric data after the execution of the startup function, and adjust the function scheduling policy according to the metric data and the real-time monitoring data. The metric data includes execution performance, resource usage data, and historical records; the function scheduling policy includes the priority rule, the calculation weight parameter of the comprehensive priority, and the queue management policy of the multi-level task queue.

[0012] Optionally, the method further includes:

[0013] Obtain historical function call data;

[0014] Extract the function call parameters within a preset period from the historical function call data. The function call parameters include function name, call time, and call frequency;

[0015] Predict the warm-up function based on the function call parameters, and load the warm-up function into the distributed cache;

[0016] Load the function-related information corresponding to the warm-up function into the distributed memory database. The function-related information includes at least one of the function code compilation result, the execution environment snapshot, the function input parameter mode, and the function resource requirement.

[0017] Optionally, predicting the warm-up function based on the function call parameters includes:

[0018] Obtain user behavior data, where the user behavior data includes user access path, search record, and execution history;

[0019] Construct a user behavior model based on the user behavior data, where the user behavior model is used to calculate the call probability of the function task;

[0020] Obtain system context information, and determine a warm-up strategy according to the system context information, where the warm-up strategy is dynamically adjusted according to the system context information and the deep Q-learning algorithm;

[0021] Generate the warm-up function list according to the warm-up strategy, where the warm-up function list includes a preset number of the warm-up functions screened according to the warm-up strategy; the warm-up function is a function task whose call probability is greater than or equal to a probability threshold and satisfies the warm-up strategy; the preset number is set according to the queue length of the multi-level task queue.

[0022] Optionally, the method further includes:

[0023] Collect update information, where the update information is the information generated after the function task is executed; the update information includes the updated code compilation result and the latest status information of the execution environment;

[0024] Obtain benchmark parameters of the cache update strategy, where the benchmark parameters include at least one of the version number of the function task, the execution timestamp, and the change impact degree;

[0025] Update the function task in the distributed cache based on the benchmark parameters and the update information.

[0026] Optionally, the method further includes:

[0027] Record the access information of the function task during the cache usage process, where the access information includes the access time and the access frequency;

[0028] Set a hierarchical elimination strategy for the multi-level cache according to the access information;

[0029] When the cache space of the current level of the multi-level cache is full, find the object to be replaced according to the hierarchical elimination strategy of the current level;

[0030] Remove the object to be replaced from the multi-level cache, and load the cache information corresponding to the new function resource into the multi-level cache.

[0031] Optionally, perform start function scheduling according to the cache status of the function task, including:

[0032] Periodically select functions to be executed from the multi-level task queue in the order of priority;

[0033] Find the relevant resources of the function to be executed in the distributed cache;

[0034] If the relevant resources of the function to be executed are found in the distributed cache, start the function to be executed using the execution environment in the cache to start the first function;

[0035] If the relevant resources of the to-be-executed function are not found in the distributed cache, allocate resources for the to-be-executed function and build an execution environment to start the second function.

[0036] Optionally, periodically select to-be-executed functions from the multi-level task queue in the order of priority, including:

[0037] Obtain task requirement information, where the task requirement information includes at least one of the business characteristics of the system, the function call frequency, and the desired response timeliness;

[0038] Set scheduling period parameters according to the task requirement information;

[0039] Periodically select to-be-executed functions from the multi-level task queue in the order of the comprehensive priority based on the scheduling period parameters.

[0040] Optionally, search for the relevant resources of the to-be-executed function in the distributed cache, including:

[0041] Send a query request to the distributed cache;

[0042] Record the waiting time after sending the query request;

[0043] If the waiting time is less than or equal to the time threshold, mark that the relevant resources of the to-be-executed function are found in the distributed cache;

[0044] If the waiting time is greater than the time threshold, mark that the relevant resources of the to-be-executed function are not found in the distributed cache.

[0045] Optionally, record the metric data after the execution of the start function and adjust the scheduling policy according to the metric data and the real-time monitoring data, including:

[0046] Periodically obtain multi-dimensional metric data according to the collection time interval parameter, where the metric data includes system resource utilization rate, function execution performance, and queue status;

[0047] Perform preprocessing on the metric data according to the preset processing items, where the preset processing items include one or more combinations of removing outliers, filling missing values, and format conversion;

[0048] Adjust the priority rule, the calculation weight parameter of the comprehensive priority, and the queue management policy of the multi-level task queue based on the preprocessed metric data.

[0049] According to another aspect of the present application, a distributed function scheduling optimization device under a serverless architecture is provided. The device includes:

[0050] An acquisition module, configured to acquire a priority rule and real-time monitoring data;

[0051] A dynamic priority management module, configured to calculate a comprehensive priority according to the priority rule and the real-time monitoring data, where the comprehensive priority is the weighted sum result of scores of multiple monitoring items in the real-time monitoring data;

[0052] A distributed cache module, configured to load a target function into a multi-level task queue according to the comprehensive priority, where the target function is a function determined according to the comprehensive priority and the queue length of the multi-level task queue;

[0053] A scheduling module, configured to extract function tasks from the multi-level task queue and perform start function scheduling according to the cache status of the function tasks. The start function includes a first function and a second function. The first function is a function task reused from the cache; the second function is a newly created function task;

[0054] A monitoring feedback module, configured to record metric data after the start function is executed, and adjust a function scheduling policy according to the metric data and the real-time monitoring data. The metric data includes execution performance, resource usage data, and historical records; the function scheduling policy includes the priority rule, the calculation weight parameter of the comprehensive priority, and the queue management policy of the multi-level task queue.

[0055] According to yet another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor. When the processor executes the program, the above-mentioned distributed function scheduling optimization method under a serverless architecture is implemented.

[0056] According to still another aspect of the present application, a storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the above-mentioned distributed function scheduling optimization method under a serverless architecture is implemented.

[0057] With the above technical solutions, the embodiments of the present application provide a distributed function scheduling optimization method and device under a serverless architecture. The method adopts a dynamic priority scheduling strategy, dynamically adjusts the function task priorities according to predefined priority rules and real-time monitoring data, ensures that critical tasks are processed first, and improves the system response speed and resource utilization efficiency. It uses a distributed cache to store function tasks, execution environment snapshots, and dependency libraries, and supports a warm-up mechanism to reduce cold start latency and improve the function task execution efficiency. Through a monitoring feedback mechanism, it collects metric data related to the system operation in real time, and dynamically adjusts the priorities and cache policies according to the metric data to achieve adaptive optimization and improve the system stability and robustness. Through dynamic priority scheduling and cache optimization, the system throughput and resource utilization rate are improved.

[0058] The above description is only an overview of the technical solutions of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features, and advantages of the present application more obvious and understandable, the specific embodiments of the present application are specifically given below. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0060] Figure 1 It is a schematic diagram of the cloud computing platform structure provided by the embodiments of the present application;

[0061] Figure 2 It is a schematic flowchart of a distributed function scheduling optimization method under a serverless architecture provided by the embodiments of the present application;

[0062] Figure 3 It is a schematic diagram of the distributed function scheduling optimization interaction process provided by the embodiments of the present application;

[0063] Figure 4 It is a schematic flowchart of the function scheduling process provided by the embodiments of the present application;

[0064] Figure 5 It is a schematic flowchart of the warm-up function scheduling process provided by the embodiments of the present application;

[0065] Figure 6 It is a schematic flowchart of the update information process in the distributed cache provided by the embodiments of the present application;

[0066] Figure 7 It is a schematic diagram of the distributed cache replacement object provided by the embodiments of the present application;

[0067] Figure 8Schematic diagram of a distributed function scheduling optimization device under a serverless architecture provided by an embodiment of the present application;

[0068] Figure 9 Schematic diagram of the hardware structure of the function scheduling optimization device provided by an embodiment of the present application;

[0069] Figure 10 Timing relationship diagram of function scheduling optimization provided by an embodiment of the present application. Detailed implementation manners

[0070] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other.

[0071] In the embodiments of the present application, the function does not refer to a function in the mathematical sense, but refers to a computing task or a set of computing tasks pre-constructed according to a specific cloud computing method. It is a task application deployed in a distributed service system and can be called by a client or the like with data processing and communication functions. Therefore, the function is also referred to as a function task.

[0072] Among them, cloud computing is a service model that provides computing resources through a network. Through cloud computing, computing resources such as servers, storage, databases, networks, software, etc. can be provided for clients with different requirements. For example, cloud computing can be used to build an e-commerce platform to meet the interaction needs of users and operators.

[0073] In some embodiments, the computing modes of cloud computing include a server architecture and a serverless architecture. Under the serverless architecture, users do not need to manage servers. By uploading code snippets to the cloud platform, the cloud platform is responsible for automatically allocating resources and executing the code to meet the specific needs of users. For example, e-commerce operators can use a serverless computing platform to process user requests such as product browsing, order processing, payment, etc.

[0074] In some application scenarios, in the cloud computing mode based on the serverless architecture, the computing platform needs to perform function scheduling and optimization based on requests input by users. For example, during a promotional event, the volume of order processing requests will surge, and the priority of payment requests is higher and needs to be processed first to ensure the completion of transactions. Since the computing platforms of cloud computing are widely used in various scenarios, there are many functions involved in the computing platform. Therefore, the computing platform needs to execute function scheduling according to the set scheduling strategy.

[0075] In some embodiments, the computing platform may perform function scheduling based on a First In First Out (FIFO) queue. When performing function scheduling based on the FIFO queue, it can be processed in the order in which function tasks enter the queue, that is, enqueue, adding a function task to the tail of the queue. Correspondingly, the elements that enter first will be taken out first, that is, dequeue, removing a function task from the head of the queue. This method cannot distinguish the priorities of functions, resulting in critical tasks possibly being delayed, especially in high-concurrency situations, and unable to meet the requirements of latency-sensitive applications.

[0076] In some embodiments, the cloud computing platform also supports setting the static priorities of functions, that is, by pre-configuring the priorities of each function, enabling the computing platform to perform function scheduling according to the configured priorities. However, when performing function scheduling based on static priorities, it cannot be dynamically adjusted according to real-time load and business requirements. When the load pattern changes, static priorities may lead to uneven resource allocation and affect the overall efficiency.

[0077] As Figure 1 shown, the cloud computing platform can respond to a user request and schedule the function task associated with the user request. During the scheduling process of the function task, the storage of the function task is involved. The storage of the function can include persistent storage and runtime caching. The storage location can be a two-layer architecture of a distributed cache and persistent storage, and the direct access to the persistent storage can be reduced through the cache, reducing latency. Among them, the persistent storage can be completed based on storage media such as a code repository and an image repository. The code repository can store the function source code, configuration files, and dependency manifests. The image repository can save the execution environment snapshot.

[0078] The runtime cache can use storage media such as a distributed memory database (Hazelcast) to cache the compilation results of function code, execution environment snapshots, dependency libraries, etc., for accelerating subsequent calls. In order to execute function tasks, the cloud computing platform can reduce the access pressure on the backend storage based on a specific caching strategy, improve the data access speed, and optimize the performance of the cloud computing platform.

[0079] In some embodiments, the cloud computing platform can use a basic caching mechanism to store function code or execution environment. For example, the function code or execution environment can be stored in the storage space shared by multiple application instances. However, this caching method lacks fine-grained control and optimization. For example, the cache may only store function code and does not include dependency libraries or pre-warmed execution environments, still resulting in cold start latency. And the cache replacement strategy may not be efficient enough to make the best use of cache resources.

[0080] After the cloud computing platform executes a function task, it can generate execution result information based on the execution result of the function task. The generated execution result information can be sent to the user client through the communication connection relationship with the user client to implement specific interaction functions.

[0081] Since the corresponding cloud computing task ends after the cloud computing platform sends the execution result information to the user client, the cloud computing platform lacks a monitoring and feedback mechanism. For example, the cloud computing system lacks real-time monitoring and feedback on function execution performance, resource utilization, and queue status, making it difficult for the system to dynamically adjust scheduling policies and caching policies according to the actual situation and unable to achieve adaptive optimization.

[0082] To solve problems such as low function scheduling efficiency, cold start latency, and inability to achieve adaptive optimization in the cloud computing platform, some embodiments of this application provide a distributed function scheduling optimization method under a serverless architecture. The method can run on the cloud computing platform and can be specifically executed by multiple functional modules deployed in the cloud computing platform. For example, the execution entity of the function is a computing node of the cloud computing platform, such as Figure 2 shown, the method includes:

[0083] S101. Obtain a priority rule and real-time monitoring data.

[0084] As Figure 3 shown, for function scheduling, the cloud computing platform can be configured with an acquisition module for obtaining a priority rule and implementation monitoring data. Among them, the priority rule is used to calculate the execution priority of the function task. The priority rule can perform priority calculation based on specific evaluation metrics. That is, the priority rule includes but is not limited to business criticality, service level agreement (SLA) requirements, deadline, call frequency, error rate, etc.

[0085] In some embodiments, the priority rule can be extended in multiple dimensions according to specific application requirements. For example, in addition to basic metrics such as business criticality, service level agreement (SLA) requirements, deadline, call frequency, average execution time, resource consumption, and error rate, indicators such as historical stability (HS), relative competitive intensity (RCI), and burstiness incidence (BI) can also be introduced.

[0086] Among them, historical stability is used to characterize the failure rate or abnormal fluctuation degree in the past execution of the function. The resource competition intensity is used to characterize the intensity of competition between the current system resources and other high-priority tasks. The burstiness factor is used to characterize the short-term surge degree of the function call frequency for quickly responding to bursty demands.

[0087] While obtaining the priority rules, the acquisition module can also perform real-time monitoring on the entire cloud computing platform to obtain real-time monitoring data. Among them, the real-time monitoring data can include function call patterns, call frequencies, average execution times, resource consumption, error rates, bursty traffic, etc.

[0088] S102. Calculate the comprehensive priority according to the priority rules and the real-time monitoring data.

[0089] After obtaining the priority rules and the real-time monitoring data, the cloud computing platform can calculate the comprehensive priority according to the priority rules and the real-time monitoring data. Among them, the comprehensive priority is the weighted summation result of the scores of multiple monitoring items in the real-time monitoring data.

[0090] To calculate the comprehensive priority, a dynamic priority queue management module can be deployed in the cloud computing platform, which is responsible for dynamically calculating and adjusting the priorities of function tasks according to predefined priority rules and real-time monitoring data. The dynamic priority queue management module can adopt an enhanced multi-level feedback queue structure and combine an adaptive weight adjustment mechanism and a prediction-based priority promotion strategy to achieve more intelligent and efficient function scheduling.

[0091] The dynamic priority queue management module can calculate the comprehensive priority based on a weighted adaptive priority algorithm. The weighted adaptive priority algorithm can dynamically adjust the weight parameters by combining the real-time system state and historical execution data on the basis of fixed weights. For example, when the system resources are tense, the resource consumption weight (RCW) is dynamically increased; in the case of bursty high concurrency scenarios, the call frequency weight (CFW) and the burstiness incidence weight (BIW) are increased.

[0092] In some embodiments, the comprehensive priority score (FPS) can be calculated according to the following formula:

[0093] FPS = (BCW * Business Criticality Score) + (SLAW * SLA Requirement Score) + (DW * Deadline Score) + (CFW * Call Frequency Score) + (AETW * Average Execution Time Score) + (RCW * Resource Consumption Score) + (ERW * Error Rate Score) + (HSW * Historical Stability Score) + (RCIW * Resource Competition Intensity Score) + (BIW * Burstiness Factor Score);

[0094] Among them, FPS represents the comprehensive priority score; BCW represents the business criticality weight; SLAW represents the service level agreement (SLA) requirement weight; DW represents the deadline weight; CFW represents the call frequency weight; AETW represents the average execution time weight; RCW represents the resource consumption weight; ERW represents the error rate weight; HSW represents the historical stability weight; RCIW represents the resource competition intensity weight; BIW represents the burstiness factor weight.

[0095] Since the comprehensive priority is the weighted sum result of the scores of multiple monitoring items in the real-time monitoring data. And the weights corresponding to multiple monitoring items can be predicted from historical data. In some embodiments, in order to calculate the comprehensive priority, the cloud computing platform can also use a regression model trained based on historical data to dynamically predict and optimize the weight parameters.

[0096] It can be seen that by calculating the comprehensive priority, the cloud computing platform can dynamically adjust the function priority according to predefined rules and real-time monitoring data, ensuring that critical tasks are processed first, and improving the system response speed and resource utilization efficiency. For example, during a promotional event, the volume of order processing requests will surge, and the payment request has a higher priority and needs to be processed first to ensure the completion of the transaction. Then, after calculating the comprehensive priority, the payment function can be set to a high priority to ensure that payment requests can be processed first under high concurrency, avoiding transaction failures caused by delays. Therefore, it can improve the user experience, increase the order conversion rate, and reduce the operating cost.

[0097] S103. Load the target function into the multi-level task queue according to the comprehensive priority.

[0098] After calculating the comprehensive priority, the cloud computing platform can load the target function into the multi-level task queue according to the comprehensive priority. That is, the dynamic priority queue management module continuously monitors the function call pattern and system resource status, dynamically adjusts the priority rule weights, calculates the comprehensive priority score (FPS), and then dynamically allocates the functions to the multi-level queues according to the priority score.

[0099] In some embodiments, the objective function is a function determined according to the comprehensive priority and the queue lengths of the multi-level task queues. For example, if the length of the multi-level task queue includes N tasks, the first N function tasks can be determined as the objective functions in the order of decreasing comprehensive priority.

[0100] In some embodiments, the cloud computing platform can also implement a multi-level queue dynamic adjustment and cross-queue migration mechanism. By monitoring metrics such as queue length, waiting time, and system resource load in real time, the scheduling time slice and capacity threshold of each priority queue are dynamically adjusted to achieve adaptive threshold adjustment.

[0101] In some embodiments, the cloud computing platform can also automatically merge low-priority queues or split high-priority queues according to the system load conditions. By dynamically splitting and merging the multi-level task queues, the flexibility of queue management can be improved.

[0102] In some embodiments, the cloud computing platform can also introduce a cooling time window through cross-queue migration and cooling strategies to limit the frequent cross-queue of functions within a short period of time and relieve the unnecessary overhead caused by frequent migration. When performing cross-queue migration, the migration conditions can also be optimized, such as adding multi-dimensional trigger conditions, including: the cumulative waiting time of the function exceeds a certain threshold; tasks in the high-priority queue require additional resources due to sudden traffic increase; the priority score of the function significantly increases or decreases in consecutive calculations, etc.

[0103] To meet the scheduling fairness and critical business guarantee mechanism, the cloud computing platform can have a critical business guarantee mode. In the critical business guarantee mode, by adding a Critical Business Queue (CBQ), an independent resource pool can be allocated for specific critical business functions to ensure that they can always be scheduled quickly.

[0104] The cloud computing platform can also provide a dynamic critical business switching mechanism, that is, based on real-time service traffic analysis, dynamically adjust which functions are marked as critical business. By introducing a Weighted Fair Queuing (WFQ) algorithm based on priority weights, while ensuring the fast response of high-priority tasks, the scheduling fairness is satisfied, and long-term starvation of low-priority tasks is avoided.

[0105] In some embodiments, the cloud computing platform can also implement real-time algorithm calibration and anomaly detection. Based on the priority calculation result verification mechanism, the cloud computing platform can use rule verification and machine learning models in parallel to verify the priority calculation results, avoiding abnormal scheduling behaviors caused by improper weight settings. To improve abnormal scheduling behaviors, the cloud computing platform can identify abnormal patterns, identify potential abnormal patterns, such as a sudden drop in call frequency, abnormal increase in resource consumption, etc., by analyzing function call data in real time, and trigger a re-evaluation.

[0106] S104. Extract function tasks from the multi-level task queue and execute startup function scheduling according to the cache status of the function tasks.

[0107] After loading the target function into the multi-level task queue, the cloud computing platform can also extract function tasks from the multi-level task queue and execute startup function scheduling according to the cache status of the function tasks. To implement startup function scheduling, a scheduling module can be deployed in the cloud computing platform. The scheduling module can preferentially select functions from the high-priority queue, check the cache status, and execute scheduling.

[0108] Among them, the startup function includes a first function and a second function. The first function is a function task reused from the cache; the second function is a newly created function task. Therefore, when the scheduling module selects a function to be executed from the priority queue, it can synchronously query the distributed cache module. The distributed cache module can adopt an optimized scheduling algorithm, an intelligent cache lookup strategy, and an asynchronous cache update mechanism to maximize the cache hit rate and scheduling efficiency.

[0109] In some embodiments, the distributed cache module is responsible for storing function code compilation results, execution environment snapshots, function input parameter patterns, resource requirements, and other metadata information, and adopts a high-performance distributed memory data grid, such as Hazelcast, Redis, etc. Through efficient memory storage capabilities and optimized data structures, function calls and executions are accelerated.

[0110] The distributed memory database (Hazelcast) can support cross-node memory sharing and low-latency access. For example, the cache content includes function code compilation results to avoid repeated compilation. The distributed memory database can store execution environment snapshots, that is, the runtime state of the container image, which can be used to achieve second-level startup. The distributed memory database is also used to store metadata, that is, function input parameter patterns, resource requirements, etc.

[0111] In some embodiments, it is also possible to form a dependency library by preloading third-party libraries required by the function. Moreover, when caching, the cache location may include a distributed memory database such as Hazelcast, as well as a local cache, that is, local cache copies of some high-frequency functions are retained on the computing nodes.

[0112] The distributed cache module can adopt a multi-level cache and a hierarchical elimination strategy to maximize the utilization of cache resources. Among them, the multi-level cache and the hierarchical elimination strategy can monitor the cache usage situation, and record the information of each cache item (function task) being accessed during the cache usage process, including the access time, access frequency, etc.

[0113] Based on the multi-level cache, the distributed cache module can divide the cache into multiple levels. For example, the distributed cache module can include cache levels such as L1 cache, L2 cache, and L3 cache. Among them, the L1 cache adopts the LRU strategy and has the characteristics of high priority and small capacity. The L2 cache adopts the LFU strategy and has the characteristics of medium priority and medium capacity. The L3 cache adopts a hybrid strategy based on frequency and time decay, Time-aware LFU (TLFU), and has the characteristics of low priority and large capacity.

[0114] In some embodiments, in order to implement the scheduling of function tasks, the scheduling module can perform cache queries through the distributed cache module. That is, as Figure 4 shown, performing the startup function scheduling according to the cache status of the function task further includes periodically selecting functions to be executed from the multi-level task queue in the order of priority, and searching for relevant resources of the function to be executed in the distributed cache. If the relevant resources of the function to be executed are found in the distributed cache, use the execution environment in the cache to start the function to be executed, so as to start the first function; if the relevant resources of the function to be executed are not found in the distributed cache, allocate resources for the function to be executed, and construct an execution environment to start the second function.

[0115] The scheduling module can periodically select functions to be executed from the priority queue in the order of priority and query the distributed cache. When performing the scheduling of the startup function, the scheduling module can first interact with the distributed cache module to query whether there are relevant resources of the function in the cache. If the relevant resources of the function to be executed are found in the cache, that is, the cache hits, then use the execution environment in the cache to directly start the function. By directly reusing the execution environment in the cache, the function can be started quickly. If the relevant resources of the function to be executed are not found in the cache, that is, the cache misses, then start the function creation process, including resource allocation, construction of the execution environment, etc. By creating a new execution environment and caching the relevant information after the function execution is completed.

[0116] In some embodiments, in order to periodically select functions to be executed from the multi-level task queue in the order of priority, the scheduling module may obtain task requirement information, where the task requirement information includes at least one of the business characteristics of the system, the function call frequency, and the desired response timeliness. Then, the scheduling cycle parameter is set according to the task requirement information, and functions to be executed are periodically selected from the multi-level task queue in the order of the comprehensive priority based on the scheduling cycle parameter.

[0117] The cloud computing platform may, based on the function selection and cache query algorithm, cause the scheduling module to initiate the scheduling process at a set time period, such as every 10 milliseconds. The set time period is a configurable parameter and the corresponding scheduling cycle parameter can be set according to the specific application scenario.

[0118] After the scheduling cycle is set, the scheduling can be triggered by a timer. Then, the scheduling module can first locate the queue with the highest priority. Then, function tasks to be executed are taken out in order from the head of the high-priority queue, and the functions in the high-priority queue are preferentially selected according to the first-in, first-out principle. For the taken-out functions, a query request is sent to the distributed cache module to check whether relevant resources corresponding to the function, as well as other metadata such as the function input parameter pattern and resource requirements, exist in the cache. The relevant resources corresponding to the function to be executed may be an execution environment snapshot, including container images and dependency libraries, etc.

[0119] If the cache hits, that is, the corresponding function execution environment and other resources are found in the distributed cache module. The scheduling module can directly obtain these resources from the cache, quickly start the function, and record the function execution-related information. For example, the execution time and resource usage, for subsequent use by the monitoring feedback module.

[0120] If the cache misses, the resource allocation module is called to create a new execution environment according to the function task creation process, including allocating computing resources and constructing the execution environment according to the function code, etc. After the function execution ends, according to the cache update policy, the updated relevant information, such as the execution environment status and code compilation result, is cached in the distributed cache module for subsequent reuse.

[0121] The scheduling time period parameter determines how often the scheduling module performs function selection and scheduling operations. For example, function selection and scheduling operations are performed every 10 milliseconds. It can be adjusted according to factors such as the business characteristics of the cloud computing platform, the function call frequency, and the desired response timeliness. If the business has extremely high requirements for real-time performance, the cycle can be appropriately shortened; if the system resources are limited and the function calls are relatively infrequent, the cycle can be appropriately extended. The value range is generally between a few milliseconds and dozens of milliseconds or even seconds, depending on the actual scenario.

[0122] Similarly, the queue priority parameter is used to define the priority value or level of a function corresponding to factors such as different business criticalities, service level agreement (SLA) requirements, and deadlines in the queue. For example, functions with high business criticality, strict SLA requirements, and tight deadlines are set to the highest level (such as level 1), and the levels decrease sequentially for other functions (such as level 2, level 3, etc.). These priority levels determine the position of the function in the corresponding multi-level queue and the order of obtaining scheduling.

[0123] In some embodiments, to search for relevant resources of the to-be-executed function in the distributed cache, the cloud computing platform can set a cache query timeout to improve query efficiency. That is, the scheduling module can send a query request to the distributed cache and record the waiting time after sending the query request. Then, compare the waiting time with a pre-set time threshold. If the waiting time is less than or equal to the time threshold, it is marked that the relevant resources of the to-be-executed function are found in the distributed cache, that is, cache hit; if the waiting time is greater than the time threshold, it is marked that the relevant resources of the to-be-executed function are not found in the distributed cache, that is, cache miss.

[0124] By setting cache-related parameters, that is, the cache query timeout, the scheduling module can set a maximum waiting time limit for the response after sending a query request to the distributed cache, that is, the time threshold, to alleviate the long-term blockage of the scheduling process caused by cache module failures or network problems. The time threshold can be reasonably set according to the network environment and cache module performance. For example, it is set to 100 milliseconds. If the cache response is not received after this time, it can be regarded as a query failure and processed as a cache miss.

[0125] For example, the scheduling module can obtain configuration information such as the scheduling time period parameter, queue priority parameter, and cache-related parameter by reading the configuration file or system initialization parameters, and complete the initialization settings of the scheduling module itself and related modules based on these parameters, such as the initialization of the multi-level queue structure and the connection configuration of the distributed cache module.

[0126] Then, start each round of scheduling loop at the time interval set by the scheduling time period parameter. In each round of loop, first traverse the priority queue, starting from the highest priority queue, and sequentially take out the to-be-executed function tasks. For each function task taken out, send a query request to the distributed cache module, and pass the function-related identification information, including unique identifiers such as function name and version number, for accurate cache query.

[0127] Check the priority queue according to the set time period, and retrieve function tasks starting from the high-priority queue. Then send a query request to the distributed cache module to check if the cache is hit. If it is hit, obtain the execution environment from the cache and start the function, and at the same time record execution-related information such as the execution time of the function for subsequent monitoring feedback. If it is not hit, call the resource allocation module to create a new execution environment, start the function, and cache the relevant information after the function execution ends.

[0128] In some embodiments, intelligent snapshot lifecycle management can also be adopted to dynamically adjust the snapshot lifecycle and eviction policy according to function call frequency, resource consumption, and version updates.

[0129] It should be noted that the computing nodes of the cloud computing platform can be triggered by the scheduling module to execute, that is, the scheduling module selects the function instances to be executed according to the dynamic priority queue and cache status. If the cache is hit, that is, the function code and execution environment exist in the distributed cache, the scheduling module will directly call the idle computing nodes and quickly start the function by reusing the execution environment snapshots (container images, dependency libraries) in the cache. If the cache is not hit, the computing nodes need to pull the function code from the persistent storage (such as the code repository), create a new execution environment (such as starting a container), execute the function after initialization, and cache the execution environment snapshots to the distributed storage. The resources (CPU, memory) of the computing nodes are dynamically allocated by the scheduling module to ensure the resource requirements of high-priority functions first.

[0130] S105. Record the metric data after the execution of the startup function, and adjust the function scheduling policy according to the metric data and the real-time monitoring data.

[0131] After the execution of the startup function scheduling, the cloud computing platform can also record the metric data after the execution of the startup function, and adjust the function scheduling policy according to the metric data and the real-time monitoring data. Among them, the metric data includes execution performance, resource usage data, and historical records; the function scheduling policy includes the priority rules, the calculation weight parameters of the comprehensive priority, and the queue management policy of the multi-level task queue.

[0132] Therefore, the cloud computing platform can also be deployed with a monitoring feedback module. The monitoring feedback module is responsible for real-time monitoring of key metrics such as the resource utilization rate, function execution performance, and queue status of the entire cloud computing platform, and feeding back the processed monitoring data to the dynamic priority queue management module and the distributed cache system to achieve adaptive scheduling and cache optimization.

[0133] The monitoring and feedback module can adopt multi-dimensional monitoring, intelligent anomaly detection, and predictive feedback mechanisms to improve the accuracy of monitoring and the timeliness of feedback. In some embodiments, the monitoring and feedback module can monitor indicators such as system resource utilization, function execution performance, and queue status in real time. Probe technology or built-in monitoring components of the system can be used to collect multi-dimensional data such as system resource utilization, function execution performance, and queue status in real time. Among them, the system resource utilization can include parameters such as CPU, memory, and network bandwidth; the function execution performance can include parameters such as execution time, response time, and error rate; the queue status can include parameters such as queue length and queuing waiting time. After obtaining the indicator data through collection, the indicator data can be transmitted to the dynamic priority queue management module so that the dynamic priority queue management module can adjust the function priority and cache policy according to the data changes.

[0134] In some embodiments, when the monitoring and feedback module records the indicator data after the start function is executed and adjusts the scheduling policy according to the indicator data and the real-time monitoring data, it can also periodically obtain multi-dimensional indicator data according to the collection time interval parameter. Among them, the indicator data includes system resource utilization, function execution performance, and queue status. Then, preprocess the indicator data according to the preset processing items, and the preset processing items include one or more combinations of removing outliers, filling missing values, and format conversion. Then, adjust the priority rule, the calculation weight parameter of the comprehensive priority, and the queue management strategy of the multi-level task queue based on the preprocessed indicator data.

[0135] After obtaining the indicator data through collection, the monitoring and feedback module can remove outliers and error data in the collected data according to the data preprocessing algorithm. For example, if the CPU usage rate collected due to probe failure and other reasons is significantly beyond the reasonable range, such data will be excluded. For the extremely short or extremely long abnormal execution time data in the function execution performance data caused by special network fluctuations and other reasons, it can be filtered by setting a threshold range. For missing values, if the corresponding data cannot be obtained at some data collection points due to temporary failures and other reasons, the valid data collected last time can be used for filling, or filled according to the average value, median, etc. of the same type of indicators to ensure the integrity of the data.

[0136] After the index data and real-time monitoring data are fed back to the dynamic priority queue management module, the dynamic priority queue management module can dynamically adjust the function priorities and caching policies to achieve adaptive scheduling optimization. To this end, after receiving the monitoring data, the dynamic priority queue analyzes it based on built-in algorithms and rules. For example, when it is found that the average execution time of a certain function continues to increase and its corresponding business criticality is relatively high, the priority of this function will be increased according to the predefined priority adjustment algorithm, taking into account factors such as the change range of the execution time and the weight of business importance. On the other hand, when it is detected that the call frequency of a certain type of function has dropped significantly, its priority will be correspondingly reduced.

[0137] For cache policy adjustment, if it is found based on the monitoring data that the cache hit rate is too low, it may decide to adjust the cache preheating mechanism, such as increasing the range of functions to be preheated, increasing the preheating frequency, etc., or changing the cache replacement policy, such as switching from LRU to LFU, which can be determined according to the cache resource utilization and the change of function access patterns.

[0138] For example, the monitoring feedback module in the cloud computing platform can collect various system index data at a certain time interval (such as every second) by setting the data collection time interval parameter. And preprocess the collected data, such as data cleaning, format conversion, etc. Specify what target format the original data is to be converted into, such as JSON, XML, etc., as well as specific format specification requirements, including the naming rules of data fields, the organization form of data structures, etc., to ensure the consistency of data understanding and processing among modules.

[0139] Then the processed data is transmitted to the dynamic priority queue management module. The dynamic priority queue management module can adjust the weight parameters of the priority. According to the adjustment information fed back by the dynamic priority queue management module, such as the priority of a certain function is increased and cache preheating is required, relevant modules such as the distributed cache module are notified to perform corresponding operations.

[0140] By applying the technical solutions of the above embodiments, a distributed function scheduling optimization method provided by an embodiment of the present application can effectively alleviate the problems of function cold start delay and low scheduling efficiency in a serverless architecture through multi-dimensional dynamic priority scheduling, deep caching of environment snapshots, and predictive adaptive optimization. Among them, multi-dimensional dynamic priority scheduling is different from the priority allocation of static rules. By introducing multi-dimensional dynamic priority calculation, real-time monitoring data such as call frequency, execution time, resource consumption, and context information is combined with predefined rules such as SLA, deadline, and business criticality to construct a dynamic priority vector, which more comprehensively and accurately reflects the actual importance of the function at a specific moment.

[0141] By constructing a priority vector model, the comprehensive priority is no longer a single value, but a vector containing multiple dimensions, such as business priority, call frequency, resource consumption ratio, execution time variance, context correlation degree, etc. Different weights can be assigned to each dimension and adjusted dynamically according to the monitoring data.

[0142] Through adaptive weight adjustment, a feedback loop is introduced. According to system performance metrics such as average response time, queue length, and resource utilization, the weights of each dimension are adjusted to make the priority calculation more in line with the actual operating conditions. And according to the multi-level priority queue group, a grouped and hierarchical queue structure is adopted, and each group of queues corresponds to different scheduling strategies, such as the delay-sensitive queue group, the throughput optimization queue group, and the burst traffic response queue group.

[0143] It is also possible to combine rules and machine learning algorithms to dynamically adjust the position of functions in each queue to achieve dynamic priority adjustment. For example, the weighted moving average method smooths the monitoring data, and decision trees / neural networks predict the future resource requirements and call patterns of functions.

[0144] By setting up a priority decay and promotion mechanism, priority decay is introduced to avoid starvation of low-priority functions, and priority promotion is introduced to avoid long waiting times for high-priority functions.

[0145] The method also supports deep caching of environment snapshots. By constructing a hierarchical caching system, a multi-level caching architecture is adopted, and caching strategies such as local caching, distributed memory caching, and persistent storage are used. In the distributed caching module, function code and complete execution environment snapshots are cached, including pre-configured container images, pre-loaded dependency libraries, and pre-warmed runtime contexts such as established database connections and cached data. The function startup time is significantly shortened through deep caching.

[0146] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, in order to fully illustrate the specific implementation process of this embodiment, some embodiments of this application also provide a distributed function scheduling optimization method under a serverless architecture, such as Figure 5 shown. Based on the above embodiments, this method can also introduce a cache preheating mechanism, that is, the method includes:

[0147] S201. Obtain historical function call data;

[0148] S202. Extract function call parameters within a preset period from the historical function call data, where the function call parameters include function name, call time, and call frequency;

[0149] S203. Predict preheated functions based on the function call parameters and load the preheated functions into the distributed cache;

[0150] S204. Load the function-related information corresponding to the warm-up function into the distributed memory database, where the function-related information includes at least one of the function code compilation result, execution environment snapshot, function input parameter mode, and function resource requirements.

[0151] The cloud computing platform can, according to the cache warm-up algorithm, pre-load the codes and execution environments of commonly used functions into the cache in accordance with the predicted function call patterns, historical data, user behaviors, and context information, so as to reduce the cold start latency.

[0152] When executing the warm-up algorithm, the cloud computing platform can first perform data collection and analysis, that is, collect historical function call data through the acquisition module, and obtain the detailed situation of function calls within a certain period from the system records, including information such as function name, call time, call frequency, etc.

[0153] Then, use data analysis methods such as time series analysis to analyze the call patterns, process the collected data, and identify which functions are likely to be frequently called in the near future. For example, by statistically analyzing the trend of function call frequency over time, if the call frequency of a certain function shows an obvious upward trend within a specific period and this trend continues until near the current time, it can be determined that the function is likely to be frequently called.

[0154] After obtaining the data and the analysis results, a list of warm-up functions that need to be warmed up can be screened out according to the analysis results. And load the warm-up functions into the cache, and for the screened warm-up functions, load the corresponding function code compilation results, execution environment snapshots, function input parameter modes, resource requirements, and other information into a distributed memory database such as Hazelcast, and use the high-efficiency memory storage ability of the distributed memory database to complete the storage operation, so as to prepare for the subsequent rapid function calls.

[0155] In some embodiments, when performing data collection and analysis, user behavior data such as user access paths, search records, purchase histories, etc. can also be combined to build a user behavior model, so as to predict the function calls that the user may trigger next through the user behavior model. And the system context information such as the current time, holidays, promotional activities, etc. can also be combined to predict the changes in function call patterns. Use Q-learning to dynamically adjust the warm-up strategy according to indicators such as the hit rate and performance improvement of warm-up.

[0156] Therefore, when predicting warm-up functions based on the function call parameters, the cloud computing platform can first obtain user behavior data, which includes user access paths, search records, and execution histories. Then, a user behavior model is constructed based on the user behavior data, and the user behavior model is used to calculate the call probability of the function task. Next, system context information is obtained, and a warm-up strategy is determined according to the system context information. The warm-up strategy is dynamically adjusted according to the system context information and the deep Q-learning algorithm. And a list of warm-up functions is generated according to the warm-up strategy. The list of warm-up functions includes a preset number of the warm-up functions screened according to the warm-up strategy. The warm-up function is a function task whose call probability is greater than or equal to a probability threshold and satisfies the warm-up strategy. The preset number is set according to the queue length of the multi-level task queue.

[0157] By constructing a user behavior model and a warm-up strategy determined according to system context information, predictive adaptive optimization can be achieved. That is, by introducing a prediction model based on machine learning, analyzing historical function call patterns, resource consumption, and business trends, predicting function call situations and resource requirements in a future period of time, and achieving proactive performance optimization.

[0158] The cloud computing platform can predict according to the call pattern and resource requirements, that is, use time series analysis, machine learning algorithms, and statistical models for prediction. Based on the predicted cache warm-up and resource pre-allocation, snapshots can be pre-loaded into the cache according to the prediction results, and computing resources can be pre-allocated. It is beneficial to achieve dynamic resource adjustment and elastic scaling, that is, according to the predicted resource requirements, dynamically adjust the resource allocation of computing nodes, and increase the number of computing nodes or resource quotas in advance during the high-load prediction period.

[0159] In some embodiments, in order to obtain more accurate model prediction results, the cloud computing platform can also perform model training and iterative optimization, that is, by continuously collecting system operation data, training and optimizing the prediction model to improve prediction accuracy.

[0160] By applying the technical solutions of the above embodiments, the method can set a cache warm-up mechanism, realize predictive analysis based on function call patterns, pre-load function resources that may be frequently called into the cache in advance, and further reduce the possibility of cold start. Through a proactive cache management strategy, the response speed and throughput of the system can be significantly improved in high-concurrency scenarios. And more intelligent warm-up can be combined with predictive adaptive optimization.

[0161] Furthermore, as a refinement and extension of the specific implementation manner of the above embodiments, in order to completely illustrate the specific implementation process of this embodiment, some embodiments of this application also provide a distributed function scheduling optimization method in a serverless architecture, such as Figure 6As shown, based on the above embodiments, a differential snapshot update strategy can also be introduced, that is, the method includes:

[0162] S301. Collect update information, where the update information is the information generated after the function task is executed; the update information includes the updated code compilation result and the latest status information of the execution environment;

[0163] S302. Obtain the benchmark parameters of the cache update strategy, where the benchmark parameters include at least one of the version number of the function task, the execution timestamp, and the degree of change impact;

[0164] S303. Update the function task in the distributed cache based on the benchmark parameters and the update information.

[0165] After the cloud computing platform executes the scheduling of the startup function, the function tasks can be sequentially executed according to the scheduling process. After the function is executed, information can be obtained, that is, when a certain function is executed, the relevant updated information generated by it is collected, such as the updated code compilation result, the latest status information of the execution environment, etc.

[0166] Then perform the update decision judgment, that is, make a judgment according to the established cache update strategy. For example, based on the version number comparison, if the version number of the newly executed function is higher than the corresponding version number stored in the cache, it is considered that the cache needs to be updated; or based on the timestamp update, if the new execution timestamp is updated and meets the preset update conditions, it is determined to update the cache.

[0167] In some embodiments, it is also possible to evaluate the degree of impact of code changes on the function execution result and performance based on the change impact assessment. When the change may lead to significant performance improvement or function change, the cache is updated.

[0168] Then execute the cache update operation according to the determined update decision, and update the obtained updated function-related information to the distributed cache according to the corresponding rules to ensure that the function information in the cache is always in the latest and available state, facilitating accurate reuse when called again later.

[0169] In some embodiments, for large functions or images, an incremental update strategy can be adopted, that is, only the changed parts are updated.

[0170] It can be seen that by introducing the differential snapshot update strategy, incremental update and differential storage can be achieved, and only the changed parts in the snapshot are updated, reducing the overhead and network transmission volume. For example, using layered images or differential backups.

[0171] In some embodiments, in order to make better use of the distributed cache space, the cloud computing platform can also maximize the use of limited cache resources by judging the replacement of the memory resources of function tasks. That is, as Figure 7 shown, the method further includes:

[0172] S401. Record the access information of the function task during cache usage, where the access information includes the access time and access frequency;

[0173] S402. Set a hierarchical elimination strategy for the multi-level cache according to the access information;

[0174] S403. When the current-level cache space of the multi-level cache is full, find the object to be replaced according to the hierarchical elimination strategy of the current level;

[0175] S404. Remove the object to be replaced from the multi-level cache, and load the cache information corresponding to the new function resource into the multi-level cache.

[0176] Through replacement judgment, when the cache space of a certain level is full and a new function resource needs to be loaded into the cache, the object to be replaced can be found according to the elimination strategy of the cache at that level. For some caches that have little impact on performance, a probabilistic elimination strategy is adopted. Among them, the parameters related to the cache replacement strategy can include the time precision parameter for recording the access information of each cache item, and the parameters related to reinforcement learning, such as the learning rate, discount factor, exploration rate, etc. The parameters related to the cache update strategy can include the specific rules for version number comparison, the time stamp precision and time interval threshold, and the change impact evaluation threshold, etc.

[0177] Then, perform a replacement operation on the found object to be replaced, that is, the found cache item to be replaced can be removed from the cache, and the cache information corresponding to the new function resource can be loaded into the cache to complete the cache replacement operation, so as to ensure that the cache always stores the most frequently used and most valuable function resources, and maximize the use of limited cache resources.

[0178] In some embodiments, the distributed cache module can adopt historical data windows of different capacities according to different analysis purposes, so as to obtain historical data of different volumes. The distributed cache module can also set cache space parameters, including the total cache space, and capacity quotas according to different cache levels.

[0179] For example, after the cloud computing platform is started and runs, according to historical data or pre-configured information, a cache warm-up process can be started to load function resources that may be frequently called into the distributed memory database and initialize the reinforcement learning model.

[0180] After the function execution is completed, update the relevant updated code compilation results, execution environment information, etc. to the distributed cache according to a certain cache update strategy. Then, according to the eviction strategies and probabilistic eviction mechanisms of each level of cache, evict the cache items that are no longer needed. The reinforcement learning model continuously adjusts the warm-up strategy according to the system performance metrics.

[0181] Further, as a specific implementation of the distributed function scheduling optimization method in the serverless architecture described in the above embodiments, some embodiments of the present application also provide a distributed function scheduling optimization device in the serverless architecture, as Figure 8 shown, the device includes:

[0182] An acquisition module, configured to acquire a priority rule and real-time monitoring data;

[0183] A dynamic priority management module, configured to calculate a comprehensive priority according to the priority rule and the real-time monitoring data, where the comprehensive priority is a weighted sum result of scores of multiple monitoring items in the real-time monitoring data;

[0184] A distributed cache module, configured to load a target function into a multi-level task queue according to the comprehensive priority, where the target function is a function determined according to the comprehensive priority and the queue length of the multi-level task queue;

[0185] A scheduling module, configured to extract function tasks from the multi-level task queue, and execute start function scheduling according to the cache status of the function tasks, where the start function includes a first function and a second function, the first function is a function task reused from the cache; the second function is a newly created function task;

[0186] A monitoring feedback module, configured to record the metric data after the start function is executed, and adjust the function scheduling strategy according to the metric data and the real-time monitoring data, where the metric data includes execution performance, resource usage data, and historical records; the function scheduling strategy includes the priority rule, the calculation weight parameter of the comprehensive priority, and the queue management strategy of the multi-level task queue.

[0187] As Figure 9 shown, the distributed function scheduling optimization device in the serverless architecture is a distributed system including multiple modules, and may include: a processor, a memory, an input / output interface, and a communication interface. Among them, the processor is responsible for executing the logic of each module, including priority calculation, cache query, execution environment management, resource allocation, monitoring data analysis, etc. It can be a general-purpose CPU, GPU, or other special processors.

[0188] The memory is used to store various data. The memory may include a main memory, a distributed cache, and a persistent storage. The main memory is used to store the running program code, function code, execution environment snapshots, metadata, etc. The distributed cache is used to build a distributed memory database (such as Hazelcast, RedisCluster, etc.) or a storage system compatible with memory semantics, and can store function code compilation results, execution environment snapshots, dependent libraries, input parameter patterns, etc. The persistent storage is used to store priority rule configurations, monitoring data history records, etc.

[0189] The input / output interface is used to interact with external devices or systems. The input / output interface may include a function call request receiving interface, a monitoring data collection interface, a management interface, and a log output interface. Among them, the function call request receiving interface receives function call requests from clients. The monitoring data collection interface is used to receive monitoring data from the monitoring system. The management interface allows administrators to configure priority rules, cache policies, etc. The log output interface is used to output system logs and monitoring data.

[0190] The communication interface is used for communication between modules and communication with external systems. The communication interface may include an inter-module communication interface, a communication interface with the distributed cache, and a communication interface with the monitoring system. The inter-module communication interface can use methods such as RPC (Remote Procedure Call) or message queues for inter-module communication. The communication interface with the distributed cache can use the client API provided by the distributed cache for communication. The communication interface with the monitoring system can use the API or protocol provided by the monitoring system for communication.

[0191] The distributed function scheduling optimization device under the entire serverless architecture can solve the latency problem caused by function cold start in the serverless architecture and the scheduling efficiency problem in high-concurrency scenarios. By dynamically adjusting function priorities and utilizing the distributed cache, functions with high priorities and cache hits are preferentially scheduled, thereby improving system throughput and resource utilization.

[0192] As Figure 10 shown, in specific applications, users can submit tasks through the client, that is, the client initiates a function call request. The scheduling module can receive the submitted task and query the distributed cache module according to the received task. The distributed cache module then performs cache query and processing. If the required function and its execution environment are found in the cache (including the warm-up resource pool), the scheduler directly obtains them from the cache and hands them over to the function execution engine for execution, thereby shortening the startup time. If the cache is not hit, the scheduler triggers the resource preparation module to create a new execution environment according to the creation process and then hands it over to the function execution engine for execution. After execution, the relevant information will be cached in the distributed cache for subsequent call reuse.

[0193] The device reuses the function execution engine to execute functions, that is, the function code is actually executed through the function execution engine. After the function execution is completed, the result is returned to the client.

[0194] The monitoring and feedback module in the device can monitor the system running status in real time, such as resource utilization rate, function execution performance, etc., and feed the monitoring data back to the dynamic priority queue management module for adjusting the function priority and cache policy.

[0195] The device also passes through the dynamic priority queue management module. The dynamic priority queue management module can dynamically adjust the priority of functions according to the monitoring data and predefined priority rules, and provide the priority information to the scheduler.

[0196] For example, the device can calculate and adjust the function priority according to the predefined rules and real-time monitoring data, and put the function into the corresponding multi-level queue. According to the predefined rules and the real-time monitoring data obtained from the monitoring and feedback module, dynamically calculate and adjust the function priority. Then, put the functions into different queues according to the priority. Thus, push the function tasks according to the queue priority, that is, the dynamic priority queue management module pushes the function tasks to be executed to the scheduling module according to the priority order of the queues.

[0197] After receiving the function task, the scheduling module first queries the distributed cache and judges whether the required function code and execution environment have been cached. Then, return the query result (hit or miss) to the scheduling module. So that the scheduling module can perform branch processing, that is, according to the cache hit situation, the process is divided into two branches. When the cache hits, reuse the execution environment in the cache to start the function. The scheduling module directly obtains the cached execution environment from the distributed cache and quickly starts the function execution, avoiding the cold start delay. When the cache misses, a new execution environment is created according to the creation process, including downloading function code, configuring the running environment, loading dependent libraries, etc.

[0198] After being scheduled by the scheduling module, the function can be executed in the function execution engine. After the function execution is completed, the function execution engine caches relevant information such as function code, execution environment snapshot, input parameter mode, etc. in the distributed cache for subsequent call reuse.

[0199] The device also uses the monitoring and feedback module to monitor the resource utilization rate and function execution performance metrics in real time, including resource utilization rates of CPU, memory, network, etc., and performance metrics such as function execution time, error rate, etc. The monitoring and feedback module feeds the collected monitoring data back to the dynamic priority queue management module for dynamically adjusting the function priority and the cache policy of the distributed cache, forming a closed-loop optimization process.

[0200] By deploying an acquisition module, a dynamic priority management module, a distributed cache module, a scheduling module, and a monitoring and feedback module in the device, the device can achieve extremely fast cold start. By preheating and caching function code, environment, and dependencies in a distributed memory database, functions with high priority and cache hits are preferentially executed during scheduling. This eliminates the process of repeated loading and initialization during cold start, enabling functions to start almost instantaneously, and reducing the cold start latency by more than 80%. For example, in an e-commerce promotion scenario, the product recommendation function can instantly respond to user requests.

[0201] Moreover, the device can achieve higher throughput. Through the dynamic priority queue management module, the execution order is reasonably arranged according to factors such as the importance of the function and resource requirements. High-priority functions are preferentially executed, avoiding long-term occupation of resources by low-priority functions. Combined with distributed caching, the resource waiting time during function startup and execution is reduced, and the system throughput is increased by 50%-200%. For example, a social platform can process a large amount of messages and user status updates faster.

[0202] The device can also provide better resource utilization. By real-time monitoring of function call patterns and resource requirements, the function priority is dynamically adjusted, and resources are accurately allocated to the functions that most need them. The cache system reuses function resources, reducing repeated allocation and initialization, increasing the utilization rate of resources such as CPU and memory by 30%-60%, and reducing cloud computing costs. For example, data-intensive applications can utilize resources more efficiently and avoid waste.

[0203] The device can also ensure the fast response of critical services. Through the dynamic priority mechanism, critical service functions (such as financial transactions, payment confirmations, etc.) are ensured to obtain high priority and preferentially obtain resources from the cache for execution. In a high-concurrency scenario, the response time of critical service functions can be controlled within milliseconds, ensuring the normal operation of the service and service quality. For example, an online payment system can also quickly complete payment confirmation during peak periods.

[0204] By applying the technical solutions of the above embodiments, a distributed function scheduling optimization device under a serverless architecture provided by an embodiment of the present application can adopt a dynamic priority scheduling strategy, dynamically adjust the function task priority according to predefined priority rules and real-time monitoring data, ensure that critical tasks are preferentially processed, and improve the system response speed and resource utilization efficiency. Use a distributed cache to store function tasks, execution environment snapshots, and dependency libraries, and support the preheating mechanism to reduce cold start latency and improve the execution efficiency of function tasks. Through the monitoring and feedback mechanism, collect index data related to system operation in real time, and dynamically adjust the priority and cache strategy according to the index data to achieve adaptive optimization and improve system stability and robustness. Through dynamic priority scheduling and cache optimization, improve system throughput and resource utilization.

[0205] It should be noted that for other corresponding descriptions of each functional unit involved in the distributed function scheduling optimization device under a serverless architecture provided in the embodiments of the present application, reference may be made to the corresponding descriptions in the distributed function scheduling optimization method under the serverless architecture provided in the above embodiments, which will not be elaborated herein.

[0206] The embodiments of the present application further provide a computer device, which may specifically be a personal computer, a server, a network device, etc. The computer device includes a bus, a processor, a memory, and a communication interface, and may further include an input / output interface and a display device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the method embodiments are implemented.

[0207] Those skilled in the art can understand that the structure of the above computer device is only a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components, or combine certain components, or have different component arrangements.

[0208] In one embodiment, a computer-readable storage medium is further provided. The computer-readable storage medium may be non-volatile or volatile, and has a computer program stored thereon. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0209] In one embodiment, a computer program product is further provided, including a computer program. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0210] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data that have been authorized by the user or fully authorized by all parties.

[0211] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it may include the processes of the embodiments of the above methods.

[0212] Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application may include at least one of non-volatile and volatile memories. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc.

[0213] Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0214] The databases involved in the embodiments provided in this application may include at least one of relational databases and non-relational databases. Non-relational databases may include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application may be general-purpose processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0215] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0216] The above-described embodiments merely represent several implementation manners of this application. Their descriptions are relatively specific and detailed, but should not be construed as limiting the patent scope of this application. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application should be subject to the appended claims.

Claims

1. A distributed function scheduling optimization method under a serverless architecture, characterized in that The method includes: Obtaining a priority rule and real-time monitoring data; Calculating a comprehensive priority according to the priority rule and the real-time monitoring data, where the comprehensive priority is the weighted sum result of scores of multiple monitoring items in the real-time monitoring data; Loading an objective function into a multi-level task queue according to the comprehensive priority, where the objective function is a function determined according to the comprehensive priority and the queue length of the multi-level task queue; Extracting function tasks from the multi-level task queue, and performing start function scheduling according to the cache status of the function tasks. The start function includes a first function and a second function. The first function is a function task reused from the cache; the second function is a newly created function task; Recording the metric data after the start function is executed, and adjusting the function scheduling policy according to the metric data and the real-time monitoring data. The metric data includes execution performance, resource usage data, and historical records; the function scheduling policy includes the priority rule, the calculation weight parameter of the comprehensive priority, and the queue management policy of the multi-level task queue.

2. The method according to claim 1, wherein The method further includes: Obtaining historical function call data; Extracting function call parameters within a preset period from the historical function call data, where the function call parameters include function name, call time, and call frequency; Predicting a warm-up function based on the function call parameters, and loading the warm-up function into a distributed cache; Loading function-related information corresponding to the warm-up function into a distributed memory database, where the function-related information includes at least one of function code compilation results, execution environment snapshots, function input parameter modes, and function resource requirements.

3. The method according to claim 2, wherein Predicting a warm-up function based on the function call parameters includes: Obtaining user behavior data, where the user behavior data includes user access paths, search records, and execution history; Constructing a user behavior model based on the user behavior data, where the user behavior model is used to calculate the call probability of the function task; Obtaining system context information, and determining a warm-up strategy according to the system context information, where the warm-up strategy is dynamically adjusted according to the system context information and a deep Q-learning algorithm; Generating a list of warm-up functions according to the warm-up strategy, where the list of warm-up functions includes a preset number of the warm-up functions screened according to the warm-up strategy; the warm-up function is a function task whose call probability is greater than or equal to a probability threshold and satisfies the warm-up strategy; the preset number is set according to the queue length of the multi-level task queue.

4. The method according to claim 1, wherein The method further includes: Collecting update information, where the update information is information generated after a function task is executed; the update information includes updated code compilation results and the latest status information of the execution environment; Obtaining benchmark parameters of a cache update strategy, where the benchmark parameters include at least one of the version number of the function task, the execution timestamp, and the degree of change impact; Updating the function task in the distributed cache based on the benchmark parameters and the update information.

5. The method according to claim 1, characterized in that The method further includes: Record the accessed information during the cache usage of the function task, where the accessed information includes the access time and access frequency; Set a hierarchical elimination strategy for the multi-level cache according to the accessed information; When the cache space of the current level in the multi-level cache is full, find the object to be replaced according to the hierarchical elimination strategy of the current level; Remove the object to be replaced from the multi-level cache, and load the cache information corresponding to the new function resource into the multi-level cache.

6. The method according to claim 1, wherein Execute startup function scheduling according to the cache status of the function task, including: Periodically select functions to be executed from the multi-level task queue in the order of priority; Find the relevant resources of the function to be executed in the distributed cache; If the relevant resources of the function to be executed are found in the distributed cache, start the function to be executed using the execution environment in the cache to start the first function; If the relevant resources of the function to be executed are not found in the distributed cache, allocate resources for the function to be executed and construct an execution environment to start the second function.

7. The method according to claim 6, characterized in that, Periodically select functions to be executed from the multi-level task queue in the order of priority, including: Obtain task requirement information, where the task requirement information includes at least one of the business characteristics of the system, the function call frequency, and the desired response timeliness; Set the scheduling period parameter according to the task requirement information; Periodically select functions to be executed from the multi-level task queue based on the scheduling period parameter in the order of the comprehensive priority.

8. The method according to claim 6, characterized in that Find the relevant resources of the function to be executed in the distributed cache, including: Send a query request to the distributed cache; Record the waiting time after sending the query request; If the waiting time is less than or equal to the time threshold, mark that the relevant resources of the function to be executed are found in the distributed cache; If the waiting time is greater than the time threshold, mark that the relevant resources of the function to be executed are not found in the distributed cache.

9. The method according to claim 1, wherein Record the metric data after the startup function is executed, and adjust the scheduling strategy according to the metric data and the real-time monitoring data, including: Periodically obtain multi-dimensional metric data according to the collection time interval parameter, where the metric data includes system resource utilization rate, function execution performance, and queue status; Perform preprocessing on the metric data according to the preset processing items, where the preset processing items include one or more combinations of removing outliers, filling missing values, and format conversion; Adjust the priority rule, the calculation weight parameter of the comprehensive priority, and the queue management strategy of the multi-level task queue based on the preprocessed metric data.

10. A distributed function scheduling optimization device under a serverless architecture, characterized in that, The device includes: An acquisition module for acquiring the priority rule and real-time monitoring data; A dynamic priority management module for calculating a comprehensive priority according to the priority rule and the real-time monitoring data, where the comprehensive priority is the weighted sum result of the scores of multiple monitoring items in the real-time monitoring data; A distributed cache module, configured to load a target function into a multi-level task queue according to the comprehensive priority, where the target function is a function determined based on the comprehensive priority and the queue length of the multi-level task queue; A scheduling module, configured to extract function tasks from the multi-level task queue and perform start function scheduling according to the cache status of the function tasks. The start functions include a first function and a second function. The first function is a function task reused from the cache; the second function is a newly created function task; A monitoring and feedback module, configured to record the metric data after the execution of the start function and adjust the function scheduling policy according to the metric data and the real-time monitoring data. The metric data includes execution performance, resource usage data, and historical records; the function scheduling policy includes the priority rule, the calculation weight parameter of the comprehensive priority, and the queue management policy of the multi-level task queue.

Images