Serverless function calculation method and system with instance-level capacity expansion configuration

Abstract

The invention provides a Serverless function calculation method and a Serverless function calculation system with instance level capacity expansion configuration. The method comprises the following steps: receiving a capacity expansion threshold value and a concurrency upper limit configuration parameter of a function instance; monitoring the number of concurrent requests of a single function instance in real time; when the number of concurrent requests reaches a capacity expansion threshold value, triggering an asynchronous capacity expansion operation; requests exceeding the concurrence upper limit are distributed in a balanced mode through multi-level request scheduling; executing batch capacity expansion and contraction operation based on the length of the message queue; and implementing asynchronous request processing. The system comprises a controller, a configuration module, a monitoring module, a capacity expansion module, a scheduling module, a batch capacity expansion and contraction module and an asynchronous request processing module. The method has the beneficial effects that through instance-level capacity expansion configuration and multi-level request scheduling, the resource utilization rate and the request processing efficiency of the Serverless function are remarkably improved, meanwhile, the cold start delay is reduced, and the flexibility and the expandability of the system are enhanced.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing technology, and in particular to the function as a service (FaaS) technology in serverless computing. Specifically, the present invention relates to a method and system for optimizing serverless function computing resource management and performance. Background Art

[0002] As an emerging model of cloud computing, serverless computing has important applications in the field of software development and deployment. It allows developers to focus on writing and running application code without having to manage the underlying infrastructure. In the serverless architecture, function as a service (FaaS) is a core component that enables developers to build and deploy applications in units of functions, thereby achieving higher flexibility and scalability. With the popularity of microservice architecture and event-driven programming, serverless computing is increasingly used in web applications, the Internet of Things, data processing and other fields.

[0003] At present, serverless function computing is mainly implemented by container-based function instance management and automatic scaling solutions. This solution usually includes technologies such as function code encapsulation, containerized deployment, request routing, and load balancing. When a function call request is received, the system will start a new container instance or use an existing instance to execute the function code. In order to cope with load changes, the system will automatically increase or decrease the number of function instances according to preset rules. In addition, some platforms also use preheating pool technology to pre-create a certain number of container instances to reduce cold start delays.

[0004] However, this solution has problems such as low resource utilization efficiency, high cold start latency, and coarse scaling granularity. In scenarios with large load fluctuations, fixed scaling rules often cannot respond in time, resulting in resource waste or request accumulation. At the same time, each function instance can usually only process one request. This one-to-one mapping relationship limits the system's concurrent processing capabilities. In addition, frequent container creation and destruction operations will also increase system overhead and affect overall performance.

[0005] To solve the above problems, some studies have proposed optimization methods based on instance reuse and request batch processing. These methods improve resource utilization by allowing a single instance to handle multiple requests, and reduce system overhead by batch processing requests. Although these methods have improved resource utilization efficiency and system performance to a certain extent, there are still problems such as insufficient flexibility in scaling, simple load balancing strategies, and limited asynchronous processing capabilities. Especially when faced with complex and changing load patterns, these methods find it difficult to achieve refined resource management and efficient request scheduling.

[0006] Therefore, there is an urgent need for a Serverless function computing method and system that can achieve instance-level refined expansion configuration, multi-level request scheduling, and efficient asynchronous processing. This method should be able to dynamically adjust the expansion and contraction strategy and concurrent processing capabilities according to the characteristics and load conditions of different functions, while providing a flexible load balancing mechanism and efficient asynchronous request processing capabilities. The development of such an advanced Serverless function computing technology has become a research hotspot in this field. It will help improve the performance, reliability, and resource utilization efficiency of Serverless computing, and provide better support for various application scenarios. Summary of the invention

[0007] The purpose of the present invention is to solve the problems of low resource utilization efficiency, high cold start delay, coarse scaling granularity, etc. in existing Serverless function computing. In addition, the present invention also aims to provide a flexible load balancing mechanism and efficient asynchronous request processing capabilities to cope with complex and changeable load patterns.

[0008] To achieve the above objectives, the present invention provides a Serverless function computing method and system with instance-level expansion configuration, characterized in that: fine-grained resource management and request processing are achieved by setting instance-level expansion thresholds and concurrency limits. In some embodiments, the method also includes multi-level request scheduling and batch expansion and contraction operations to improve the overall performance and resource utilization of the system.

[0009] Specifically, the method of the present invention includes receiving and storing configuration parameters of the expansion threshold and the concurrency upper limit of the function instance. These parameters can be set separately for each function instance to achieve refined resource control.

[0010] Furthermore, the method monitors the number of concurrent requests of a single function instance in real time, and when the number of requests reaches the expansion threshold, triggers the asynchronous expansion operation of the instance and creates a new function instance.

[0011] Preferably, the present invention adopts a multi-level request scheduling mechanism to evenly distribute requests that exceed the concurrency upper limit to other available instances, thereby ensuring load balancing and efficient operation of the system.

[0012] Optionally, the method also includes performing batch expansion and contraction operations based on changes in message queue lengths to improve resource management efficiency.

[0013] In one embodiment, the present invention implements asynchronous request processing, asynchronously writes received requests into a message queue, and a scheduler pulls batches of requests from the queue for processing, thereby improving the concurrent processing capability of the system.

[0014] In certain embodiments, the present invention further includes an adaptive parameter adjustment step to dynamically adjust the concurrency upper limit and the capacity expansion threshold according to application characteristics, historical load data, and real-time performance indicators.

[0015] In addition, the present invention can also achieve cold start optimization through technologies such as preheating pool maintenance and function code caching, thereby further reducing the delay of function calls.

[0016] In other embodiments, the system of the present invention supports single-node and multi-node cluster deployment, and has cross-region load balancing capabilities to adapt to application scenarios of different scales and complexities.

[0017] In a preferred embodiment, the system of the present invention includes a controller, a configuration module, a monitoring module, a capacity expansion module, a scheduling module, a batch capacity expansion module and an asynchronous request processing module. These modules work together to achieve efficient Serverless function computing.

[0018] In another preferred embodiment, the method of the present invention combines asynchronous expansion operations, multi-level request scheduling and batch expansion and contraction to achieve efficient utilization and flexible management of resources while ensuring system performance.

[0019] By adopting the above scheme, the present invention has the following beneficial effects: (1) Through instance-level expansion configuration, more refined resource management is achieved and resource utilization is improved; (2) The multi-level request scheduling mechanism is adopted to significantly improve the system's load balancing capability and request processing efficiency; (3) Through asynchronous expansion operations and batch expansion and contraction, resource management overhead is reduced and system response speed is improved; (4) Implementing asynchronous request processing enhances the system's concurrent processing capabilities and adapts to high-load scenarios; (5) Through adaptive parameter adjustment and cold start optimization, the system performance and user experience are further improved.

[0020] In summary, the present invention provides an efficient, flexible and scalable Serverless function computing method and system, which significantly improves resource utilization efficiency, system performance and user experience, and provides important support for the further development and application of Serverless computing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 This is the basic architecture diagram of the Serverless function computing system. It shows the core components of the system, including the relationship between the controller, configuration module, monitoring module, expansion module, scheduling module, batch expansion module, and asynchronous request processing module.

[0023] Figure 2 It is an instance-level expansion configuration parameter diagram. It describes key parameters such as expansion threshold, concurrency limit, minimum number of instances, maximum number of instances, and cool-down period, as well as how they affect the dynamic adjustment process of the system.

[0024] Figure 3 This is a flowchart of asynchronous capacity expansion operations. It shows the interaction between the capacity expansion decision engine, resource pre-allocation module, request routing module, resource readiness signal module, and preheating mechanism.

[0025] Figure 4 It is a diagram of a multi-level request scheduling mechanism. It describes how request classifiers, priority queue managers, load balancers, resource monitors, and dynamic scheduling policy generators work together to achieve efficient request scheduling.

[0026] Figure 5 This is a flow chart of batch scaling operations. It shows the relationship and workflow between the request queue monitor, batch operation decision engine, resource pool manager, batch operation executor, and dynamic threshold adjuster.

[0027] Figure 6 It is a diagram of the asynchronous request processing mechanism. It describes how request receivers, message queues, request distributors, parallel processor pools, result collectors, and asynchronous response senders work together to achieve efficient asynchronous request processing.

[0028] Figure 7 It is a diagram of the adaptive parameter adjustment mechanism. It shows the closed-loop adjustment process between the performance monitor, load signature analyzer, parameter impact model, optimization decision engine, parameter adjustment executor, and feedback evaluator.

[0029] Figure 8 It is a cold start optimization and fault tolerance processing mechanism diagram. It describes how the warm-up pool manager, code cache optimizer, error detector, automatic retry module, failover controller, and degraded service manager work together to improve system performance and reliability. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] In the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as a limitation on the present invention.

[0032] Example 1: Basic system architecture.

[0033] Reference Figure 1 This embodiment provides a basic architecture of a Serverless function computing system with instance-level expansion configuration. The system mainly includes the following core components: a controller 101, a configuration module 102, a monitoring module 103, an expansion module 104, a scheduling module 105, a batch expansion module 106, and an asynchronous request processing module 107.

[0034] As the center of the system, controller 101 is responsible for coordinating the work of each module and managing the overall system status. It receives external requests and decides how to handle them based on the current system status and configuration. Controller 101 implements a novel decision-making mechanism that can maximize resource utilization while ensuring system stability.

[0035] Configuration module 102 is used to receive and manage configuration parameters for the capacity expansion threshold and concurrency limit of function instances. These parameters can be set individually for each function instance, achieving unprecedented refined resource control. Configuration module 102 also implements a dynamic configuration update mechanism that allows these parameters to be adjusted in real time while the system is running without restarting the system.

[0036] Monitoring module 103 is responsible for real-time monitoring of the number of concurrent requests for a single function instance. It uses an innovative lightweight monitoring algorithm that can provide millisecond-level monitoring accuracy while minimizing system overhead. Monitoring module 103 not only monitors the number of requests, but also collects multi-dimensional performance indicators including CPU usage, memory usage, network I / O, etc., to provide comprehensive data support for subsequent decision-making.

[0037] When the number of concurrent requests reaches the expansion threshold, the expansion module 104 triggers the asynchronous expansion operation of the instance and creates a new function instance. The module implements an intelligent prediction algorithm that can start the expansion operation in advance based on the historical load pattern and the current system status, effectively reducing the expansion delay. The expansion module 104 also supports multiple expansion strategies, including fixed number expansion, percentage expansion, and adaptive expansion, which can be flexibly selected according to different application scenarios.

[0038] The scheduling module 105 distributes the requests exceeding the concurrency limit to other available instances through multi-level request scheduling. It implements an innovative adaptive scheduling algorithm that can dynamically adjust the scheduling strategy according to factors such as the real-time load status of the instance and network latency. The scheduling module 105 supports multiple load balancing algorithms, including but not limited to polling, weighted polling, least number of connections, and consistent hashing, and can automatically select the optimal algorithm according to application characteristics.

[0039] The batch expansion and contraction module 106 triggers batch expansion or contraction operations according to changes in the message queue length. This module introduces a new batch operation mechanism, which significantly reduces resource management overhead while ensuring system responsiveness by setting a dynamically adjusted batch operation trigger threshold.

[0040] The asynchronous request processing module 107 is responsible for asynchronously writing the received requests into the message queue, and the scheduler pulls the requests from the queue in batches for processing. This module implements an efficient asynchronous processing mechanism that can significantly improve the concurrent processing capability of the system while ensuring the order of request processing.

[0041] These modules work together to form an efficient, flexible and scalable Serverless function computing system. The system workflow is as follows: (1) The configuration module 102 first loads and initializes the system configuration, including the expansion threshold and concurrency upper limit of each function instance; (2) The monitoring module 103 continuously monitors the number of concurrent requests and performance indicators of each function instance; (3) When the controller 101 receives a new function call request, it first consults the scheduling module 105; (4) The scheduling module 105 decides how to allocate requests based on the current system load and configuration parameters; (5) If the number of concurrent requests of all available instances reaches the concurrent upper limit, the scheduling module 105 will notify the expansion module 104; (6) The expansion module 104 triggers an asynchronous expansion operation to create a new function instance according to a preset expansion strategy; (7) At the same time, the asynchronous request processing module 107 writes the excess request into the message queue, waiting for processing; (8) The batch expansion and contraction module 106 continuously monitors the message queue length and triggers batch expansion and contraction operations when necessary; (9) After the new function instance is created, the scheduling module 105 starts to allocate the requests in the queue to the new instance for processing.

[0042] The system architecture of this embodiment has the following innovations and advantages: It realizes unprecedented instance-level refined resource control, greatly improving resource utilization. It introduces intelligent prediction and adaptive scheduling algorithms, significantly reducing cold start delays and load imbalance problems. It innovatively combines asynchronous processing and batch operations, improving concurrent processing capabilities while reducing system overhead. It designs a flexible modular architecture, making the system scalable and adaptable.

[0043] Example 2: Instance-level capacity expansion configuration.

[0044] Reference Figure 2 This embodiment describes in detail the implementation method of the instance-level expansion configuration in the present invention. This novel configuration method allows for refined resource management of each function instance, significantly improving the resource utilization and performance of the system.

[0045] like Figure 2 As shown in the figure, instance-level expansion configuration mainly includes the following key parameters: Scaling Threshold (ST): 201; Concurrency Limit (CL): 202; Minimum number of instances (MI): 203; Maximum number of instances (MAX): 204; Cooldown Period (CP): 205.

[0046] The expansion threshold (ST) 201 defines the number of concurrent requests that trigger the creation of a new instance. When the number of concurrent requests for an instance reaches the ST, the system starts the process of creating a new instance. The setting of ST requires a trade-off between response time and resource utilization. A lower ST may provide faster response time, but may cause resource waste; a higher ST can improve resource utilization, but may increase the waiting time of requests.

[0047] The concurrency limit (CL) 202 specifies the maximum number of requests that a single instance can process simultaneously. When the number of concurrent requests to an instance reaches CL, new requests will be routed to other instances or put into a waiting queue. The setting of CL needs to take into account the complexity of the function, resource requirements, and expected performance level.

[0048] The minimum number of instances (MI) 203 and the maximum number of instances (MAX) 204 define the minimum number of instances that a function should maintain at any time and the maximum number of instances that it is allowed to scale to, respectively. These two parameters help control resource usage and costs while ensuring basic service availability.

[0049] The cooldown period (CP) 205 is the minimum time interval between two expansion operations, which is used to prevent the system from over-expanding. This parameter helps the system remain stable when the load fluctuates.

[0050] This embodiment introduces an innovative dynamic parameter adjustment mechanism. The system automatically adjusts these parameters based on real-time monitored performance indicators and historical load data. The adjustment process uses a newly developed machine learning algorithm that can predict short-term load changes and optimize parameter settings accordingly.

[0051] The mathematical model of parameter adjustment is as follows: ST_new = ST_current * (1 + α * (AVG_RT - TARGET_RT) / TARGET_RT) Among them, ST_new is the adjusted expansion threshold, ST_current is the current expansion threshold, AVG_RT is the average response time in the past period of time, TARGET_RT is the target response time, and α is the adjustment coefficient (0 < α < 1).

[0052] Similarly, the adjustment formula for CL is: CL_new = CL_current * (1 + β * (AVG_CPU - TARGET_CPU) / TARGET_CPU) Where CL_new is the adjusted concurrency limit, CL_current is the current concurrency limit, AVG_CPU is the average CPU utilization, TARGET_CPU is the target CPU utilization, and β is the adjustment coefficient (0 < β < 1).

[0053] The workflow for instance-level capacity expansion configuration is as follows: (1) When the system is initialized, the preset configuration parameters are loaded for each function; (2) The monitoring module continuously monitors the number of concurrent requests and performance indicators of each instance; (3) When the number of concurrent requests of a certain instance reaches ST, the expansion operation is triggered; (4) The expansion module creates new instances while ensuring that the total number of instances does not exceed MAX; (5) If the number of concurrent requests for an instance reaches CL, new requests will be routed to other instances; (6) The system evaluates performance indicators regularly (e.g., every 5 minutes) and adjusts parameters as needed; (7) After each expansion operation, the system will comply with CP and no new expansion will be performed during this period.

[0054] The innovation of this embodiment is that it realizes unprecedented fine-grained resource control, and each function instance can have a unique configuration. It introduces a dynamic parameter adjustment mechanism, which enables the system to adaptively respond to load changes. It optimizes parameter settings through intelligent algorithms to maximize resource utilization while ensuring performance.

[0055] This dynamically adjusted instance-level capacity expansion configuration can significantly improve resource utilization while significantly reducing average response time, which has obvious advantages over traditional fixed configuration methods.

[0056] Embodiment 3: Asynchronous expansion operation.

[0057] This embodiment describes in detail the asynchronous expansion operation in the present invention, which is an innovative resource management method that can significantly improve the system's response speed and resource utilization efficiency. Figure 3 ,The core components and processes of asynchronous expansion operations are shown below.

[0058] The core idea of ​​asynchronous expansion operation is to decouple the expansion decision from the actual resource allocation, allowing the system to prepare new computing resources in parallel while processing the current request. This approach greatly reduces the waiting time in traditional synchronous expansion methods and improves the overall throughput of the system.

[0059] When the monitoring module detects that the number of concurrent requests of the instance reaches the expansion threshold, the controller (301) will immediately trigger the asynchronous expansion operation. The expansion decision engine (302) first evaluates the current system status and the predicted load trend, and then selects the most suitable expansion strategy. This embodiment supports three main expansion strategies: fixed number expansion, percentage expansion and adaptive expansion.

[0060] The fixed number expansion strategy adds a predetermined number of new instances each time, which is suitable for scenarios with relatively stable load changes. The percentage expansion strategy adds new instances based on a certain percentage of the current number of instances, which is suitable for handling sudden but predictable loads. The adaptive expansion strategy is a major innovation of this invention. It uses machine learning algorithms to analyze historical load patterns and current system status to dynamically determine the optimal expansion quantity.

[0061] Once the expansion strategy is determined, the resource pre-allocation module (303) will start the resource preparation process. This process includes allocating computing resources, preparing the operating environment, preloading function codes, etc. At the same time, the request routing module (304) continues to allocate newly arrived requests to existing instances to ensure the continuous operation of the system.

[0062] The resource ready signal module (305) is responsible for monitoring the readiness status of the new instance. When the new instance is ready, it sends a ready signal to the controller. After receiving the signal, the controller immediately updates the available instance pool, allowing the request routing module to start allocating requests to the new instance.

[0063] This embodiment also introduces an innovative preheating mechanism (306). Before the new instance is fully ready, the preheating mechanism sends some simulated requests to it to help the new instance quickly reach the optimal performance state. This method effectively reduces the cold start problem and further improves the response speed of the system.

[0064] Asynchronous capacity expansion operations show significant advantages in high-load scenarios. Compared with traditional synchronous capacity expansion methods, it can respond to load changes more quickly, effectively reduce average response time, and improve system throughput. Especially when dealing with burst traffic, asynchronous capacity expansion operations can respond to load increases more quickly and effectively prevent system overload.

[0065] Embodiment 4: Multi-level request scheduling.

[0066] This embodiment describes in detail the multi-level request scheduling mechanism in the present invention, which is an innovative load balancing and request processing method that can significantly improve the resource utilization and request processing efficiency of the system. Figure 4 ,The core components and processes of the multi-level request scheduling mechanism are shown below.

[0067] The core idea of ​​the multi-level request scheduling mechanism is to adopt a multi-level, differentiated scheduling strategy based on the different characteristics of the request and the current state of the system. This approach can more accurately match requests with resources and improve overall system performance.

[0068] like Figure 4 As shown in the figure, the multi-level request scheduling mechanism mainly includes the following components: Request classifier (401): responsible for preliminary classification of incoming requests; Priority Queue Manager (402): manages request queues of different priorities; Load Balancer (403): implements multiple load balancing algorithms; Resource Monitor (404): monitors system resource status in real time; Dynamic scheduling strategy generator (405): dynamically generates scheduling strategies according to system status.

[0069] When a request arrives at the system, it is first classified by the request classifier (401). The basis for classification includes, but is not limited to: the urgency of the request, resource requirements, expected execution time, etc. This step uses a newly developed fast classification algorithm that can complete the initial analysis of the request in milliseconds.

[0070] The classified requests are sent to the priority queue manager (402). An innovative multi-level feedback queue mechanism is implemented here, which not only considers the initial priority of the request, but also dynamically adjusts the priority according to the waiting time of the request to prevent low-priority requests from being starved for a long time.

[0071] The load balancer (403) is one of the core components of this mechanism. It implements multiple load balancing algorithms, including but not limited to: polling, weighted polling, least number of connections, consistent hashing, etc. The innovation of the present invention is that the load balancer can dynamically select the optimal load balancing algorithm according to the real-time system status and request characteristics.

[0072] The resource monitor (404) continuously monitors the resource usage of each instance in the system, including CPU usage, memory usage, network I / O, etc. These data provide an important basis for the generation of dynamic scheduling strategies.

[0073] The dynamic scheduling strategy generator (405) is another innovation of this embodiment. It uses a machine learning algorithm to generate the optimal scheduling strategy in real time based on historical data and current system status. This method can adapt to different load modes and system states and always maintain efficient resource utilization.

[0074] The workflow of multi-level request scheduling is as follows: (1) After the request arrives at the system, it is initially classified by the request classifier; (2) The classified requests enter the corresponding priority queue; (3) The load balancer selects requests from the queue and assigns them to the appropriate instance based on the currently optimal load balancing algorithm. (4) The resource monitor continuously monitors the system status and feeds the data back to the dynamic scheduling strategy generator; (5) The dynamic scheduling strategy generator regularly updates the scheduling strategy based on feedback data.

[0075] This embodiment also introduces an innovative adaptive scheduling mechanism. The system will dynamically adjust the processing weights and scheduling strategies of queues at all levels according to the actual operation situation. For example, when it is detected that the average response time of high-priority requests exceeds the threshold, the system will automatically increase the processing weight of the high-priority queue.

[0076] Compared with the traditional single scheduling strategy, the multi-level request scheduling mechanism can more effectively reduce the average response time of the system and improve the system throughput. The advantages of this mechanism are more obvious when processing mixed loads (including both computing-intensive and I / O-intensive requests).

[0077] It should be noted that the effect of the multi-level request scheduling mechanism is closely related to other components of the system (such as asynchronous expansion operations and instance-level expansion configuration). Through the collaborative work of these components, the system can achieve more efficient and flexible resource management and request processing.

[0078] Embodiment 5: Batch expansion and contraction operations.

[0079] This embodiment describes in detail the batch expansion and contraction operation in the present invention, which is an innovative resource management method that can significantly improve the resource utilization efficiency and response speed of the system. Figure 5 The core components and processes of batch scaling operations are shown below.

[0080] The core idea of ​​batch scaling is to reduce resource management overhead and improve system stability and efficiency by merging multiple scaling requests. This method is particularly suitable for handling high-frequency load fluctuations and large-scale concurrent requests.

[0081] like Figure 5 As shown in the figure, batch scaling operations mainly include the following components: Request queue monitor (501): continuously monitors the changes in the length of the message queue; Batch operation decision engine (502): determines whether to trigger a batch operation according to a preset strategy; Resource Pool Manager (503): responsible for actual resource allocation and recycling; Batch operation executor (504): executes batch expansion or reduction operations; Dynamic Threshold Adjuster (505): Dynamically adjusts the trigger threshold of batch operations according to system performance.

[0082] The request queue monitor (501) is the starting point of the entire mechanism. It uses an innovative lightweight monitoring algorithm that can track changes in the length of the message queue in real time without affecting system performance. This algorithm uses sampling technology and prediction models to detect significant changes in queue length within milliseconds.

[0083] The batch operation decision engine (502) is one of the core innovations of this embodiment. It decides whether to trigger a batch operation based on multiple factors such as the changing trend of the queue length, the current system load, historical data, etc. The decision process uses a newly developed multi-factor decision model, which can maximize resource utilization efficiency while ensuring system stability.

[0084] The core formula of the decision model is as follows: BatchOperationScore = α * QueueLengthChange + β * SystemLoad + γ *HistoricalPattern Among them, α, β, and γ are weight coefficients, QueueLengthChange is the rate of change of queue length, SystemLoad is the current system load, and HistoricalPattern is the load prediction value based on historical data. When BatchOperationScore exceeds the preset threshold, the batch operation is triggered.

[0085] The resource pool manager (503) is responsible for maintaining a dynamic resource pool. It adopts an innovative pre-allocation strategy to prepare a certain amount of resources in advance when the load is low to cope with possible sudden loads. This strategy significantly reduces the delay of resource allocation and improves the response speed of the system.

[0086] The batch operation executor (504) is the component that actually performs the expansion or reduction operation. It uses parallel processing technology and can handle the creation or destruction of multiple instances at the same time. This method greatly improves the efficiency of batch operations and reduces the time overhead of resource management.

[0087] The dynamic threshold adjuster (505) is another innovation. It dynamically adjusts the trigger threshold of batch operations by analyzing system performance indicators and historical operation effects. This adaptive mechanism enables the system to maintain optimal performance under different load modes.

[0088] The workflow of batch scaling operations is as follows: (1) The request queue monitor continuously monitors changes in the message queue length; (2) When significant changes are detected, the batch operation decision engine evaluates whether a batch operation needs to be triggered; (3) If it is decided to perform a batch operation, the resource pool manager will prepare the required resources; (4) The batch operation executor processes the creation or destruction of multiple instances in parallel; (5) After the operation is completed, the dynamic threshold adjuster updates the trigger threshold according to the operation effect.

[0089] Compared with the traditional method of scaling up and down one by one, batch scaling can effectively reduce resource management overhead and shorten the average response time of the system. Especially when dealing with burst traffic, this method can adjust resource configuration more quickly to meet demand, showing obvious advantages.

[0090] Example 6: Asynchronous request processing.

[0091] This embodiment describes in detail the asynchronous request processing mechanism of the present invention, which is an innovative request processing method that can significantly improve the concurrent processing capability and resource utilization of the system. Figure 6 ,The core components and processes of the asynchronous request processing mechanism are shown below.

[0092] The core idea of ​​asynchronous request processing is to decouple the reception, processing and response of requests, and improve the throughput and response speed of the system through message queues and parallel processing. This method is particularly suitable for handling a large number of concurrent requests and long-running tasks.

[0093] like Figure 6 As shown in the figure, the asynchronous request processing mechanism mainly includes the following components: Request receiver (601): responsible for receiving and initially processing incoming requests; Message Queue (602): stores pending requests; Request Dispatcher (603): Gets requests from the queue and distributes them to the processor; Parallel processor pool (604): contains multiple processor instances for processing requests in parallel; Result Collector (605): collects and manages processing results; Asynchronous response sender (606): sends the processing result to the client asynchronously.

[0094] The request receiver (601) uses an innovative fast parsing algorithm that can complete the initial parsing and classification of requests within milliseconds. This algorithm uses optimized data structures and parallel processing technology to greatly reduce the initial processing time of requests.

[0095] The message queue (602) is one of the core components of this mechanism. It uses a newly developed high-performance distributed queue system that supports multi-level priorities and dynamic partitioning. This design not only improves the processing efficiency of the queue, but also enhances the scalability and fault tolerance of the system.

[0096] The request dispatcher (603) implements an intelligent dispatch algorithm. The algorithm considers multiple factors such as request priority, estimated execution time, current system load, etc., and dynamically determines the request dispatch strategy. This method can better balance the system load and improve the overall processing efficiency.

[0097] The parallel processor pool (604) is another innovation of this embodiment. It adopts a dynamically scalable processor pool design, which can automatically adjust the number of processors according to the current load. Each processor is an independent execution unit and can process multiple requests in parallel.

[0098] The result collector (605) uses an innovative asynchronous result management mechanism, which can efficiently collect and organize results from different processors and support early return of some results, thereby reducing the waiting time of the client.

[0099] The asynchronous response sender (606) uses batch sending and compression technology to efficiently return the processing results to the client. It also implements an intelligent retry mechanism to improve reliability under unstable network conditions.

[0100] The workflow of asynchronous request processing is as follows: (1) The request receiver receives and quickly parses the incoming request; (2) The parsed request is written to the message queue; (3) The request distributor obtains requests from the queue and distributes the requests to processors in the parallel processor pool according to the intelligent distribution algorithm; (4) The processor processes the requests in parallel and sends the results to the result collector; (5) The result collector organizes and manages the processing results; (6) The asynchronous response sender returns the result to the client.

[0101] This embodiment also introduces an innovative adaptive batch processing mechanism. The system dynamically adjusts the size and frequency of batch processing according to the current load and request characteristics to achieve the best balance between low latency and high throughput.

[0102] Compared with the traditional synchronous processing method, the asynchronous request processing mechanism can significantly improve the system's concurrent processing capabilities and reduce the average response time. Especially when processing long-running tasks, this method can better utilize system resources and provide a better user experience.

[0103] Embodiment 7: Adaptive parameter adjustment.

[0104] This embodiment describes in detail the adaptive parameter adjustment mechanism in the present invention, which is an innovative system optimization method that can significantly improve the adaptability and performance of the system. Figure 7 ,The core components and processes of the adaptive parameter adjustment ,mechanism are shown as follows.

[0105] The core idea of ​​adaptive parameter adjustment is to dynamically adjust key system parameters by real-time monitoring of system performance and load characteristics, so that the system always maintains the best operating state. This method is particularly suitable for dealing with changing workloads and complex operating environments.

[0106] like Figure 7 As shown in Figure 1, the adaptive parameter adjustment mechanism mainly includes the following components: Performance Monitor (701): collects various system performance indicators in real time; Load characteristic analyzer (702): analyzing characteristics and patterns of workload; Parameter Impact Model (703): Establish a relationship model between parameter changes and system performance; Optimization decision engine (704): generating a parameter adjustment strategy according to the analysis results; Parameter adjustment executor (705): performs parameter adjustment operations; Feedback Evaluator (706): Evaluates the effect of parameter adjustments and provides feedback.

[0107] The performance monitor (701) uses an innovative distributed monitoring architecture that can collect comprehensive performance indicators in real time with extremely low system overhead. These indicators include but are not limited to: CPU utilization, memory usage, network I / O, request response time, throughput, etc.

[0108] The load characteristic analyzer (702) implements a novel multi-dimensional load analysis algorithm. The algorithm can not only identify the basic characteristics of the load (such as request frequency and resource consumption), but also capture complex load patterns (such as periodic fluctuations and burst traffic). This in-depth analysis provides an accurate basis for subsequent parameter adjustments.

[0109] The parameter impact model (703) is one of the core innovations of this embodiment. It uses machine learning technology, especially reinforcement learning and time series analysis, to establish a dynamic parameter-performance relationship model. This model can accurately predict the impact of parameter changes on system performance and provide support for optimization decisions.

[0110] The optimization decision engine (704) uses a multi-objective optimization algorithm. It can find the best balance between multiple performance goals that may conflict (such as high throughput, low latency, and resource utilization). The decision-making process takes into account short-term performance improvement and long-term system stability, avoiding system instability caused by over-optimization.

[0111] The parameter adjustment actuator (705) implements a progressive parameter adjustment strategy. It reduces the impact of parameter changes on the system by making small and fast adjustments, while ensuring the timeliness of the adjustment.

[0112] The feedback evaluator (706) uses an innovative real-time evaluation mechanism. It can quickly detect the effect of parameter adjustment and trigger a rollback operation when necessary, ensuring that the system is always in a controllable state.

[0113] The workflow of adaptive parameter adjustment is as follows: (1) The performance monitor continuously collects system performance indicators; (2) The load characteristic analyzer analyzes workload characteristics in real time; (3) The parameter impact model predicts the potential impact of parameter adjustments based on historical data and current status; (4) The optimization decision engine takes all factors into consideration and generates a parameter adjustment strategy; (5) The parameter adjustment executor performs parameter adjustment according to the strategy; (6) The feedback evaluator monitors the effect of adjustments and provides feedback to optimize future decisions.

[0114] This embodiment also introduces an innovative multi-level parameter adjustment mechanism. System parameters are divided into fast-adjusting parameters (such as concurrency limits) and slow-adjusting parameters (such as resource allocation strategies). This hierarchical design enables the system to be optimized at different time scales, which can quickly respond to short-term fluctuations and adapt to long-term trend changes.

[0115] Compared with fixed parameter configuration, the adaptive parameter adjustment mechanism can more effectively reduce the average response time of the system, improve resource utilization, and enhance the stability of the system when facing sudden traffic and complex loads.

[0116] Embodiment 8: Cold start optimization and fault tolerance processing.

[0117] This embodiment describes in detail the cold start optimization and fault tolerance processing mechanism in the present invention, which is an innovative method for improving system performance and ensuring reliability, and can significantly improve user experience and system stability. Figure 8 ,The core components and processes of cold start optimization and fault tolerance ,processing mechanism are shown as follows.

[0118] The core idea of ​​cold start optimization is to reduce the function initialization time through preheating and caching technology, while fault tolerance aims to improve the reliability of the system through multi-level error detection and recovery mechanisms. These two mechanisms work together to improve the overall performance and stability of the system.

[0119] like Figure 8 As shown in the figure, the cold start optimization and fault tolerance processing mechanism mainly includes the following components: Preheating Pool Manager (801): maintains and manages the preheating instance pool; Code Cache Optimizer (802): optimizes the loading and caching strategy of function codes; Error detector (803): monitors various errors and anomalies in the system in real time; Automatic retry module (804): intelligently retry failed operations; Failover Controller (805): performs failover when an instance or node fails; Degraded Service Manager (806): Provides degraded service in extreme cases.

[0120] The preheat pool manager (801) implements an innovative dynamic preheat strategy. It not only maintains a basic preheat instance pool, but also dynamically adjusts the number and type of preheat instances based on historical call patterns and predicted load changes. This approach greatly reduces the probability of cold starts and avoids waste of resources.

[0121] The code cache optimizer (802) uses a novel multi-layer cache mechanism. It establishes a multi-level cache in memory, SSD, and distributed cache systems, and uses a machine learning algorithm to predict the most likely functions to be called and loads them into a faster cache layer in advance. This approach significantly reduces the loading time of functions.

[0122] The error detector (803) is an important innovation of this embodiment. It uses an anomaly detection algorithm based on deep learning, which can identify various abnormal patterns in the system in real time, including performance anomalies, resource leakage, network failures, etc. This high-precision error detection provides a reliable foundation for subsequent fault-tolerant processing.

[0123] The automatic retry module (804) implements an intelligent retry strategy. Instead of simply retrying for a fixed number of times, it dynamically adjusts the number and interval of retries based on the error type, system load, and historical success rate. This method greatly improves the success rate of retries while avoiding the additional burden on the system caused by invalid retries.

[0124] The failover controller (805) uses a new fast failover mechanism. It maintains a real-time system topology map and can identify failed nodes and reroute requests within milliseconds. This mechanism greatly improves the system's ability to recover from local failures.

[0125] The Degraded Service Manager (806) implements a progressive service degradation strategy. When the system encounters a serious failure or extreme load, it can gradually reduce the service quality according to the predefined priority to ensure the availability of core functions. This approach provides the system with a last line of defense and guarantees basic service capabilities in the worst case.

[0126] The workflow of cold start optimization and fault tolerance is as follows: (1) The preheating pool manager continuously maintains a certain number of preheating instances to cope with sudden requests; (2) The code cache optimizer preloads frequently used function codes into a multi-level cache; (3) When a request arrives, the system prioritizes using preheated instances and cached code to minimize cold start delays; (4) The error detector monitors the system operation status in real time and detects abnormalities in a timely manner; (5) When an error is detected, the automatic retry module first attempts to solve the problem by retrying; (6) If the retry fails, the failover controller transfers the request to a healthy node or instance; (7) In extreme cases, the degraded service manager initiates service degradation to ensure the basic availability of the system.

[0127] The cold start optimization mechanism of this embodiment can effectively reduce the average cold start time, while the fault tolerance processing mechanism can improve the availability of the system. This mechanism shows obvious advantages especially when facing burst traffic and complex error scenarios.

[0128] For those skilled in the art, many improvements and modifications can be made without violating the principles of the present invention, all of which fall within the scope of protection of the present invention. For example, the present invention can be combined with virtualization, edge computing and other technologies to further expand the application scenarios and effects of intelligent scheduling.

Claims

1. A serverless function computing method with instance-level expansion configuration, characterized in that: The following steps are involved: (1) Receive and store the configuration parameters of the function instance's expansion threshold and concurrency limit; (2) Real-time monitoring of the number of concurrent requests for a single function instance; (3) When the number of concurrent requests reaches the expansion threshold, the asynchronous expansion operation of the instance is triggered and a new function instance is created; (4) Repeat steps (2) and (3) until the number of instances meets the concurrency requirement or reaches the concurrency limit; (5) Through a multi-level request scheduling mechanism, requests that exceed the concurrency limit are evenly distributed to other available instances; (6) Perform batch expansion and contraction operations based on changes in the message queue length; (7) Implement asynchronous request processing, asynchronously write the received requests to the message queue, and have the scheduler pull requests in batches from the queue for processing.

2. The method according to claim 1, characterized in that The configuration parameters of the expansion threshold and concurrency upper limit are set separately for each function instance and support dynamic adjustment at runtime.

3. The method according to claim 1, characterized in that The asynchronous expansion operation creates a new function instance according to a preset expansion strategy, where the expansion strategy includes fixed quantity expansion, percentage expansion, or adaptive expansion.

4. The method according to claim 1, characterized in that: The balanced allocation method includes but is not limited to polling, weighted polling, random, minimum number of connections or consistent hashing.

5. The method according to claim 1, characterized in that It also includes an adaptive parameter adjustment step to dynamically adjust the concurrency limit and expansion threshold based on application characteristics, historical load data, and real-time performance indicators.

6. The method according to claim 1, characterized in that The batch expansion and contraction operation includes: Set the batch operation trigger threshold; When the number of pending capacity expansion or reduction requests reaches the trigger threshold, batch operations are performed; During batch operations, the system load is continuously monitored and the batch operation scale is dynamically adjusted when necessary.

7. The method according to claim 1, characterized in that It also includes cold start optimization steps to reduce cold start latency through techniques such as warm-up pool maintenance and function code caching.

8. A Serverless function computing system with instance-level expansion configuration, characterized in that: include: Controller, used to coordinate the work of various modules in the system; The configuration module is used to receive and manage the configuration parameters of the expansion threshold and concurrency limit of the function instance; The monitoring module is used to monitor the number of concurrent requests of a single function instance in real time; The expansion module is used to trigger the asynchronous expansion operation of the instance when the number of concurrent requests reaches the expansion threshold and create a new function instance; The scheduling module is used to evenly distribute requests that exceed the concurrency limit to other available instances through multi-level request scheduling; Batch expansion and contraction module, used to trigger batch expansion or contraction operations according to changes in the message queue length; The asynchronous request processing module is used to asynchronously write the received requests to the message queue, and the scheduling module pulls the requests in batches from the queue for processing.

9. The system according to claim 8, characterized in that It also includes an adaptive parameter adjustment module, which is used to dynamically adjust the concurrency limit and expansion threshold according to application characteristics, historical load data and real-time performance indicators.

10. The system according to claim 8, characterized in that Also includes: Fault tolerance module, used to handle expansion failure or instance crash, including but not limited to automatic retry, failover and downgrade processing; Cold start optimization module, which is used to reduce cold start latency through technologies such as warm-up pool maintenance and function code caching; The system supports single-node and multi-node cluster deployment, and has cross-region load balancing capabilities.

Images