A scheduling system for efficiently scheduling serverless function computation tasks in large-scale scenarios

Abstract

This invention discloses a scheduling system for efficiently scheduling serverless function computation tasks in large-scale scenarios. The system includes: a Storage module providing caching functionality for the apiserver component and storage functionality for the etcd component; a Client module providing a Kubebuilder-compatible SDK API interface; an Apiserver module ensuring that external services and tools continue to use their original operating methods; a WatchCache module ensuring that the event flow in the List-Watch mechanism is consistent with the native system; and a Persister module persisting data to a BadgerDB database, enabling data recovery from the persisted BadgerDB database. This invention improves computational efficiency and performance in large-scale serverless scenarios and reduces waiting time.

Description

Technical Field

[0001] This invention relates to the field of scheduling technology for serverless function computing tasks, and in particular to a scheduling system for efficiently scheduling serverless function computing tasks in large-scale scenarios. Background Technology

[0002] Serverless is a cloud computing model where developers don't need to manage server infrastructure. Unlike traditional computing models, Serverless allows developers to upload their application code to a cloud provider, which then automatically handles all server management, scaling, and maintenance tasks. Developers only need to focus on the code itself, without worrying about the underlying computing resources. Function Compute tasks are a core concept in the Serverless model. They typically execute in a stateless manner and can run automatically when triggered by events. Function Compute task execution is event-driven; external event sources generate events that trigger task execution. A function Compute task can be triggered when a user visits a webpage or when an internal system undergoes a state change. Each task executes independently and typically terminates immediately upon completion to avoid resource waste. Billing for function Compute tasks is based on actual resource usage and execution time, rather than charging a fixed amount of server resources.

[0003] In a complex network system, besides web backend services, there may be a large number of computational tasks that need to be executed, such as video encoding / decoding and database backup. These computational tasks are often triggered by a specific event, commonly a timer or a change in system state. Furthermore, these tasks are typically stateless, executed only once, and all dependent resources can be released after completion. Based on these characteristics, function computation is particularly well-suited for executing these computational tasks.

[0004] Currently, Kubernetes-based scheduling systems are a popular choice in the serverless field. Leveraging Kubernetes' controllers and Custom Resources (CRDs) mechanisms, automated application management and scaling can be achieved. In a Kubernetes-based scheduling system, the scheduler is a key component responsible for adjusting the number of Pods based on the service's request load. When the load increases, the scheduler requests Kubernetes to schedule more Pods to handle these requests; when the load decreases, it reduces the number of Pods. Furthermore, the scheduler is responsible for finding suitable nodes in the cluster to run new tasks, i.e., creating the corresponding containers (Pods). When a new Pod is created, the scheduler selects a node to run the Pod based on factors such as node resource usage, node affinity, and anti-affinity strategies. However, in large-scale scenarios, Kubernetes-based serverless scheduling systems may encounter bottlenecks.

[0005] Specifically, Kubernetes' architecture consists of multiple components, including etcd, apiserver, and controller-manager. This fragmented architecture requires network communication for interaction between components. Each operation (such as creating, updating, or deleting resources) necessitates communication between the apiserver and etcd, introducing additional network latency. This is particularly problematic during large-scale serverless tasks, where numerous resource requests need to be passed from the apiserver to etcd and serialized / deserialized using JSON. This serialization process can become a bottleneck in high-concurrency scenarios, leading to significant latency. For serverless frameworks, executing complex computational tasks often involves numerous Pod startup, configuration, and scheduling requests, all of which rely on the efficient response of etcd and the apiserver. In large-scale scenarios, the overhead of JSON serialization and deserialization amplifies with the number of tasks, impacting overall execution efficiency.

[0006] Furthermore, controllers in Kubernetes are event-driven, listening for resource changes (such as Pod creation and deletion) and executing corresponding operations based on these changes. In a Kubernetes-based serverless framework, the controller's role may include scheduling functions and scaling Pod resources. When network communication is delayed, the controller's response to resource changes will also be delayed, leading to controller execution delays. This delay not only affects the startup speed of individual serverless tasks but can also block the serial execution of controllers, especially when there are dependencies between multiple controllers. For example, if controller A in the controller-manager needs to wait for controller B to complete its operation, and controller B's execution is blocked due to network latency, then the processing of the entire serverless task will be postponed. In addition, controllers are distributed, and different controllers may need to coordinate the processing of the same type of resources. Delay in any controller can trigger a chain reaction, reducing the overall task scheduling and execution speed of the serverless framework. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing technologies by proposing a scheduling system for efficiently scheduling Serverless function computation tasks in large-scale scenarios.

[0008] The objective of this invention is achieved through the following technical solution: a scheduling system for efficiently scheduling Serverless function computation tasks in large-scale scenarios, comprising:

[0009] The Storage module provides the caching functionality of the original apiserver component and the storage functionality of the etcd component, and integrates Webhook and List-Watch mechanisms. The Storage module uses the caching functionality of the apiserver component and the storage functionality of the etcd component to store data objects in memory, where data objects are organized according to namespaces during storage.

[0010] The Client module provides a Kubebuilder-compatible SDK API interface for use by the upper-level controller to implement CRUD operations on data objects, and supports event push and Webhook mechanism logic.

[0011] The Apiserver module provides an HTTP API interface consistent with the native Kubernetes system, enabling seamless compatibility with Kubernetes tools such as client-go and kubectl, ensuring that external services and tools continue to interact with the system using their familiar operating methods.

[0012] The WatchCache module implements the same event push mechanism as the native Kubernetes system, ensuring consistency between the event flow in the List-Watch mechanism and the native system. Specifically, when a Kubernetes controller needs to understand the changes in a resource within the cluster, it first executes a List operation to obtain the current resource status information, and then uses a Watch operation to subscribe to future change events. This allows the controller to respond promptly to changes in the resource status of the cluster, achieving automated management and scaling.

[0013] The Persister module is used to persist data to the BadgerDB library, which is the underlying key-value database, to ensure that the system can recover data from the persisted BadgerDB library after a failure or crash.

[0014] Furthermore, the Storage module is divided into two layers: ① Lower layer Storage: used to directly store all data objects; ② Upper layer Storage: used to organize and classify the lower layer Storage through namespaces; wherein, the lower layer Storage stores data objects of a specified category, and the upper layer Storage stores the data objects stored in the lower layer Storage according to the namespace.

[0015] Furthermore, the List-Watch mechanism is a mechanism in Kubernetes used to monitor changes in the status of cluster resources in real time. The List-Watch mechanism includes List operations and Watch operations. The List operation is used to obtain all status information of all relevant resources in the cluster at once; the Watch operation is used to continuously monitor changes in resources and push update events.

[0016] Furthermore, the Apiserver module specifically includes:

[0017] The Apiserver module parses the external HTTP requests it receives according to the OpenAPI specification and converts the parsed results into operation requests to the Client module. The Apiserver module routes the operation requests to the corresponding processing functions, which interact with the Client module to perform specific data operations. After the Client module completes the data operations, the Apiserver module constructs an HTTP response based on the operation results. The format of the HTTP response strictly follows the OpenAPI specification and includes the returned status code and the structure of the response body. The response body includes the resource data corresponding to the successful request or the error information and detailed error description corresponding to the failed request.

[0018] Furthermore, the WatchCache module specifically includes:

[0019] The WatchCache module stores all captured events in a circular queue. Within the WatchCache module, all events are sorted by resource version number. When an event is pushed to the WatchCache module, the WatchCache module distributes the event to the corresponding Informer component based on the resource version number. The Informer component is used to listen for changes in resource objects in the cluster and notify the corresponding controller of these changes. The controller then performs corresponding scheduling or control operations based on the changes in the resource objects.

[0020] Furthermore, the Persister module specifically includes:

[0021] When a data write request is received, the Persister module first submits the data to the BadgerDB database for persistence processing, and the BadgerDB database quickly persists the data to disk;

[0022] If the data is successfully persisted to the BadgerDB database, the Persister module will write the data to the in-memory storage of the Storage module; if any error or exception occurs during the data persistence process, the Persister module will throw an exception, preventing the data from being written to the Storage module.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0024] (1) This invention optimizes the shortcomings of the existing Kubernetes architecture through the design of a single-component architecture, especially the network communication latency and JSON serialization / deserialization overhead, which can improve the computing efficiency and reduce the waiting time in large-scale serverless scenarios. The scheduling system described in this invention combines various functional modules in Kubernetes in its design, integrating the multi-component architecture of Kubernetes into a single-component Memlet system, which significantly improves the performance, efficiency and compatibility in large-scale serverless scenarios, while also taking into account compatibility with the original Kubernetes system.

[0025] (2) By reducing network communication to memory-level operations, especially in Webhook and event push mechanisms, this invention significantly reduces latency and improves system response speed. This enables the system to maintain agile and efficient operation under high concurrency, adapt to the needs of rapid startup and expansion in Serverless scenarios, and improve the performance of the system in large-scale Serverless scenarios.

[0026] (3) By providing API interfaces (including HTTP API and SDK API) consistent with the native Kubernetes system, this invention ensures seamless compatibility with tools such as kubectl and client-go, thus guaranteeing compatibility and enabling users to migrate directly to the Memlet system without modifying existing application code, thereby reducing the complexity of migration.

[0027] (4) By retaining Kubernetes’ List-Watch mechanism and event push logic in the Memlet system, this invention ensures the stability and consistency of resource management in large-scale clusters, enabling the controller to rely on the existing event handling mechanism to maintain the normal operation of the system in a dynamic environment.

[0028] (5) By combining persistent operations with efficient memory storage, this invention effectively reduces the burden of disk I / O while ensuring fast data persistence and high system availability. Since persistent scenarios involve more writing than reading, the design of the Persister module pays special attention to the efficiency of write operations, thereby improving the resource efficiency of the system in large-scale Serverless scenarios.

[0029] (6) Through interface consistency design, this invention enables external tools and services to continue to interact with the system in a familiar way without having to be aware of changes in the underlying storage, thereby avoiding modifications to the upper-level logic and making the system operation transparent. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating the architecture of the scheduling system (memlet system) for efficiently scheduling Serverless function computation tasks in large-scale scenarios according to the present invention.

[0031] Figure 2 This is a schematic diagram illustrating the migration process from a Kubernetes system to a memlet system according to the present invention;

[0032] Figure 3 This is a flowchart of the Persister module of the present invention. Detailed Implementation

[0033] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not intended to limit this application.

[0034] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0035] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "in response to determination," or "includes." Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process or method. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0036] The present invention will now be described in detail with reference to the accompanying drawings. Unless otherwise specified, the features of the following embodiments and implementations can be combined with each other.

[0037] The scheduling system for efficiently scheduling Serverless function computation tasks in large-scale scenarios, as described in this invention, is called the memlet system. It is implemented based on the Kubernetes architecture. The memlet system incorporates various functional modules from Kubernetes while maintaining compatibility with existing Kubernetes systems. For example... Figure 1 As shown, the memlet system specifically includes the Storage module, Client module, Apiserver module, WatchCache module, and Persister module.

[0038] In this embodiment, the Storage module provides the caching function of the original apiserver component and the storage function of the etcd component, and integrates the functions of Webhook and List-Watch mechanisms. The Storage module uses the caching function of the apiserver component and the storage function of the etcd component to store data objects in memory. The data objects are organized according to namespaces when stored, which can ensure efficient data management and fast access.

[0039] It should be understood that the Storage module is the core of the memlet system. It undertakes the caching function of the original apiserver component in the original Kubernetes system and the storage function of the etcd component. It also integrates the functions of the Webhook mechanism and the List-Watch mechanism.

[0040] It's important to note that the apiserver component is one of the core components of a Kubernetes cluster. It handles all external and internal API requests and routes them to the appropriate components for further processing. The apiserver component is the front-end interface for Kubernetes, providing CRUD (Create, Read, Update, Delete) operations and ensuring consistency and high availability for reading and writing cluster state. The apiserver component also handles authentication, authorization, admission control, and resource quotas. Through its caching capabilities, the apiserver component can quickly respond to client requests, reducing the frequency of interaction with backend databases (such as etcd) and thus improving overall performance.

[0041] etcd is the backend storage system for Kubernetes clusters, used to persistently store all cluster configuration information, state data, and objects. It's a distributed key-value store that supports high availability and consistency, ensuring that each node in the cluster accurately synchronizes its state. Through etcd, Kubernetes can reliably store all critical cluster data and provides fault tolerance mechanisms to handle node failures or network partitions.

[0042] Furthermore, the Storage module is divided into two layers: ① Lower-level Storage: This layer directly stores all data objects, ensuring efficient data object access and utilizing memory to accelerate data access. ② Upper-level Storage: This layer organizes and categorizes the lower-level Storage using namespaces, making data object management clearer and more convenient. The lower-level Storage stores data objects of a specified category, while the upper-level Storage stores the data objects stored in the lower-level Storage according to namespaces.

[0043] Furthermore, the Webhook mechanism is integrated into the Storage module to enhance scalability and flexibility. The Webhook mechanism allows callbacks to be triggered when specific events occur, such as object creation, update, or deletion operations. Through the Webhook mechanism, the memlet system can dynamically execute custom logic, such as security checks or auditing, thereby enhancing the scalability and customization capabilities of the memlet system.

[0044] Furthermore, the Storage module implements the List and Watch functionalities of the Kubernetes List-Watch mechanism. The List-Watch mechanism in Kubernetes is used for real-time monitoring of cluster resource status changes. It includes List and Watch operations. The List operation retrieves all status information for all relevant resources in the cluster at once, while the Watch operation continuously monitors resource changes and pushes update events, such as resource additions, updates, and deletions. Through the List-Watch mechanism, the Storage module effectively supports large-scale serverless scenarios, providing fast and reliable support for subsequent event pushing. Since data is stored in memory, access speed is extremely fast, which is crucial in scenarios requiring high performance and high scalability.

[0045] In this embodiment, the Client module provides an SDK API interface compatible with Kubebuilder for use by the upper-level controller to implement CRUD operations on data objects, and supports event push and Webhook mechanism logic.

[0046] It should be understood that the Kubebuilder-compatible SDK interface provided by the Client module ensures seamless integration between the memlet system and existing Kubernetes business logic. By implementing the corresponding factory methods, the Client module replaces the default Client implementation in the original dependency library, allowing applications to switch to the memlet system without modifying their code. While the Webhook mechanism is implemented via HTTP requests, in the memlet system, this Webhook mechanism is simplified to memory-level object passing and admission checks, significantly reducing latency.

[0047] In this embodiment, the Apiserver module plays a crucial bridging role in the memlet system. The Apiserver module provides an HTTP API interface consistent with the native Kubernetes system, achieving seamless compatibility with Kubernetes tools such as client-go and kubectl. This ensures that external services and tools can continue to interact with the system using their familiar operating methods without being aware of changes in the underlying storage.

[0048] It's important to understand that in the native Kubernetes system, the Apiserver module is the central hub of the entire cluster. It handles all API requests, manages all cluster resource objects, and persists these objects by interacting with the etcd component. The Apiserver module defines API interfaces using the OpenAPI specification, ensuring consistent communication between Kubernetes components and providing a rich set of HTTP API interfaces, enabling users and applications to easily interact with Kubernetes. In the memlet system, the Apiserver module inherits this design philosophy, emulating the native Kubernetes Apiserver module to implement HTTP API interfaces consistent with the original system.

[0049] It's worth noting that `client-go` is the official Go client library for Kubernetes. It allows developers to programmatically interact with Kubernetes clusters, perform CRUD operations, listen for events, and manage resource objects. `client-go` encapsulates complex API calls, enabling developers to easily build applications integrated with Kubernetes using Go. Through the consistent API provided by the `Apiserver` module, components in the memlet system can continue to interact with `client-go`, thus maintaining compatibility with existing Kubernetes applications.

[0050] Another Kubernetes tool, kubectl, is a command-line tool used to manage Kubernetes cluster resources. Users can use kubectl to send API requests to create, update, delete, and query Kubernetes resources. Because the memlet system's Apiserver module implements Kubernetes API interface compatibility, users can manage the memlet system through kubectl just like using Kubernetes, without needing to be aware of changes to the underlying storage structure.

[0051] Furthermore, the Apiserver module specifically includes: parsing received external HTTP requests (such as those from kubectl or client-go) according to the OpenAPI specification, and converting the parsing results into operation requests to the Client module; routing the operation requests to the corresponding processing functions, which interact with the Client module to perform specific data operations; after the Client module completes the data operations, the Apiserver module constructs an HTTP response based on the operation results, wherein the format of the HTTP response strictly follows the OpenAPI specification, and the HTTP response includes the returned status code, the structure of the response body, etc. The structure of the response body includes the resource data corresponding to the successful operation request or the error information and detailed error description corresponding to the failed operation request.

[0052] Specifically, when the Apiserver module receives an HTTP request, it first parses and validates the request according to the OpenAPI specification, converting the parsed result into an operation request for the Client module. This ensures that data storage and processing logic remains consistent with Kubernetes. The OpenAPI specification is the standard used in Kubernetes to define API interfaces, detailing information such as the path, parameters, and return value type of each API endpoint. Based on the OpenAPI specification, the Apiserver module can use this information to parse the request path, query parameters, and request body, and validate the request's validity. After parsing the request, the Apiserver module routes it to the corresponding processing function (i.e., the CRUD logic). These processing functions perform the actual operations through interaction with the memlet Client module. For example, when the request is to query a list of resources, the Apiserver module will convert the request into a query operation on the memlet Client to retrieve the corresponding resource data. The memlet Client module is responsible for interacting with the memlet system's in-memory database and performing specific data operations (i.e., CRUD operations). This design ensures that even if the underlying storage is switched from the etcd component in Kubernetes to the memlet system's in-memory database, external applications can still interact with the cluster through the same API interface. This change is transparent to external users and services; they do not need to be aware of the differences in the underlying architecture. Once the memlet Client module completes the specific data operation, the Apiserver module constructs an HTTP response based on the operation result. The HTTP response format strictly follows the Kubernetes OpenAPI specification, including the returned status code and the structure of the response body. For successful requests, the Apiserver module returns the corresponding resource data; for failed requests, it returns error information and a detailed error description. This design ensures that external services can process API responses as if they were in a native Kubernetes environment. Whether it's the kubectl command-line tool or client-go-based applications, they can continue to use their existing parsing and processing logic to seamlessly interact with the memlet system.

[0053] It's important to note that the OpenAPI specification in Kubernetes describes the structure and behavior of all APIs. The Apiserver module also implements this specification in the memlet system, ensuring full compatibility with the memlet system and Kubernetes. By using the OpenAPI specification, the Apiserver module can generate automated API documentation, enabling external services and tools to better understand and use the API interfaces. Furthermore, the OpenAPI specification supports client-side code generation, meaning developers can automatically generate memlet-compatible client code based on the OpenAPI specification, thereby accelerating application development and integration.

[0054] In addition, the Apiserver module extends the OpenAPI specification appropriately. Through these extensions, the Apiserver module not only provides functionality consistent with Kubernetes, but also fully leverages the advantages of the memlet system's in-memory database to achieve more efficient data access and processing.

[0055] In this embodiment, the WatchCache module plays a crucial role in the memlet system. The WatchCache module is used to implement the same event push mechanism as the native Kubernetes system to ensure that the event flow in the List-Watch mechanism is consistent with the native system. Specifically, when the Kubernetes controller (such as the Controller component) needs to know the changes of a certain resource in the cluster, it first executes the List operation to obtain the current resource status information, and then uses the Watch operation to subscribe to future change events so that the controller can respond to changes in the resource status in the cluster in a timely manner, thereby achieving automated management and scaling.

[0056] It's worth noting that the WatchCache module implements this List-Watch mechanism in the memlet system and optimizes event handling efficiency. Under this List-Watch mechanism, when an object in the Storage module changes (such as being added, updated, or deleted), a corresponding event is immediately generated. This event is then pushed to the WatchCache module for processing. First, a List operation is performed to obtain the current resource status information, and then a Watch operation is used to subscribe to future change events.

[0057] Furthermore, the WatchCache module specifically includes: The WatchCache module stores all captured events using a circular queue. This circular queue design features efficient memory usage, capable of storing a large number of events within a limited memory space, and supports fast enqueue and dequeue operations. This design not only improves system performance but also effectively handles a large number of events under high concurrency, ensuring the orderliness and efficiency of event processing. Within this WatchCache module, all events are sorted by their resource version number (ResourceVersion) to ensure that events with smaller resource version numbers are processed first. ResourceVersion is a key field in Kubernetes used to track changes to resource objects; its version number increments whenever a resource object changes. Therefore, by prioritizing events with smaller version numbers, the system can ensure the accuracy and consistency of resource states. Once an event is pushed to the WatchCache module, the WatchCache module distributes the event to the corresponding Informer component based on the ResourceVersion. The Informer component is an important concept in the Kubernetes client, and its role is to ensure that the controller responds to resource state changes in a timely manner to maintain the stability and correct operation of the cluster. The Informer component is responsible for listening to changes in resource objects in the cluster and notifying the corresponding controllers of these changes. These controllers will then perform corresponding scheduling or control operations based on the changes in resource objects.

[0058] It should be understood that the WatchCache module is a caching component of the Apiserver module in Kubernetes. It is mainly used to efficiently process and push change events of resource objects. By caching the most recently occurred resource change events, it reduces the access pressure of the Apiserver on the backend database (such as etcd), thereby improving system performance.

[0059] It's important to note that the WatchCache module's design strictly adheres to Kubernetes' event-driven mechanism, ensuring complete consistency with Kubernetes' List-Watch mechanism. This compatibility design allows upper-layer controllers and applications to continue operating as before without any modifications. For example, when an external Kubernetes client (such as kubectl or client-go) issues a List-Watch request to the memlet system, it first performs a List operation to retrieve all relevant resource status information in the cluster. Subsequently, it continuously performs Watch operations to monitor resource changes and push update events. Therefore, the WatchCache module can directly respond to these List-Watch requests, providing event streams and resource status information consistent with the native Kubernetes system.

[0060] In this embodiment, the Persister module is used to persist data to the BadgerDB library as the underlying key-value database (KV database) to ensure that the system can recover data from the persisted BadgerDB library after a failure or crash.

[0061] It should be understood that the Persister module plays a crucial role in ensuring persistence in the memlet system. Its design ensures that data can be recovered from the persistent file library after a failure or crash, guaranteeing data integrity and high system availability.

[0062] Furthermore, the Persister module specifically includes: such as Figure 3 As shown, when a data write request arrives, the Persister module first submits the data to the BadgerDB database for persistence. The BadgerDB database then quickly persists the data to disk. This efficient write operation not only improves the speed of data persistence but also ensures the system's stability when handling large-scale write requests. If the data is successfully persisted to the BadgerDB database, the Persister module writes the data to the in-memory storage of the Storage module. If any error or exception occurs during the data persistence process, the Persister module will throw an exception, preventing further data writing to the Storage module. This dual-protection mechanism ensures data reliability and prevents inconsistent or incomplete data from appearing in the system.

[0063] It's important to note that data is only truly stored in the Storage module after the persistence operation is successfully completed. If any errors or exceptions occur during persistence, the Persister module will throw an exception, preventing data from being written to the Storage module and thus preventing inconsistent or incomplete data from appearing in the system. This dual-protection mechanism ensures data reliability, and this design greatly improves the system's robustness, ensuring that the system maintains data integrity and consistency even in the event of unforeseen failures.

[0064] Furthermore, another design feature of the Persister module is that it primarily performs write operations, with very few read operations. This design choice significantly reduces the burden on disk I / O, focusing on efficient write operations and thus ensuring the speed and reliability of the data persistence process. In practical applications, since most data operations are write operations, this design of the Persister module can fully leverage the write efficiency advantages of the BadgerDB library, ensuring that the system can still respond quickly and complete data persistence under high concurrency.

[0065] For example, an e-commerce platform uses the Memlet system to build its real-time order processing system to handle the high-concurrency processing needs of millions of orders per day. Through Memlet's efficient event push mechanism and memory-level webhook verification, the platform can quickly process order creation, verification, and update operations in a large-scale serverless scenario, while maintaining seamless compatibility with the existing Kubernetes infrastructure, significantly improving the system's response speed and stability.

[0066] In summary, the key point of this invention lies in successfully achieving an efficient replacement of the native Kubernetes architecture in the memlet system through the optimization and integration of multiple modules. The migration process is as follows: Figure 2 As shown, while ensuring compatibility with existing Kubernetes systems, it significantly improves performance and response speed in large-scale serverless scenarios.

[0067] The memlet Client module provides an SDK API interface compatible with Kubebuilder, allowing upper-layer controllers to switch directly to the memlet system without code modifications, achieving seamless integration between the memlet system and the native Kubernetes system. This design ensures that the original controller logic continues to function normally when the underlying storage and event mechanisms are replaced. By implementing the corresponding factory methods, the memlet Client module can replace the default Client implementation in Kubernetes, maintaining compatibility with tools such as client-go and kubectl after switching to the memlet system. This compatibility design is particularly important in large-scale serverless application scenarios, ensuring that existing applications can quickly adapt to the new storage architecture and avoiding interruptions and complexities caused by the switch.

[0068] The Apiserver module implements an HTTP API interface consistent with Kubernetes within the memlet system. By strictly adhering to the OpenAPI specification, it ensures that external tools and services can continue to use the familiar HTTP API when interacting with the memlet system. External applications and operational tools (such as kubectl) do not need to be aware of changes in the underlying storage when interacting with the memlet system, thus maintaining operational consistency and continuity of the development experience. In serverless scenarios, the interface consistency of the Apiserver module enables rapid response and seamless scaling even in large-scale and high-concurrency environments.

[0069] The memlet Client module simplifies the Webhook mechanism, which previously relied on network communication, into memory-level object passing and admission checks. This optimization not only retains the functionality of the Webhook mechanism in Kubernetes but also improves overall system performance by reducing network latency. In large-scale serverless scenarios, this memory-level optimization is particularly critical, significantly reducing latency caused by network communication and enabling the system to maintain agile and efficient event processing capabilities even under high concurrency.

[0070] The WatchCache module implements the same List-Watch event push mechanism as Kubernetes in the memlet system, ensuring that the order and logic of event flow are completely consistent with Kubernetes, thus fully preserving the event push mechanism. This design allows upper-layer controllers to continue relying on existing event handling mechanisms, thereby maintaining stable performance and consistent behavior in large-scale clusters. In serverless application scenarios, the reliability and efficiency of the event push mechanism are crucial, as serverless applications typically involve a large number of dynamic resource management and real-time response requirements. The WatchCache module's design ensures that the system maintains consistency and stability even in highly dynamic environments.

[0071] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A scheduling system for efficiently scheduling serverless function computation tasks in large-scale scenarios, characterized in that, include: The Storage module provides the caching functionality of the original apiserver component and the storage functionality of the etcd component, and integrates Webhook and List-Watch mechanisms. The Storage module uses the caching functionality of the apiserver component and the storage functionality of the etcd component to store data objects in memory, where data objects are organized according to namespaces during storage. The Client module provides a Kubebuilder-compatible SDK API interface for use by the upper-level controller to implement CRUD operations on data objects, and supports event push and Webhook mechanism logic. The Apiserver module provides an HTTP API interface consistent with the native Kubernetes system, achieving seamless compatibility with Kubernetes tools client-go and kubectl, ensuring that external services and tools continue to interact with the system using their familiar operating methods; The WatchCache module is used to implement the same event push mechanism as the native Kubernetes system, to ensure that the event flow in the List-Watch mechanism is consistent with the native system. Specifically, when the Kubernetes controller needs to know the changes of a certain resource in the cluster, it first executes the List operation to obtain the current resource status information, and then uses the Watch operation to subscribe to future change events, so that the controller can respond to changes in the resource status in the cluster in a timely manner, and realize automated management and scaling. and The Persister module is used to persist data to the BadgerDB library, which is the underlying key-value database, to ensure that the system can recover data from the persisted BadgerDB library after a failure or crash.

2. The scheduling system for efficiently scheduling serverless function computation tasks in large-scale scenarios according to claim 1, characterized in that, The Storage module is divided into two layers: ① Lower layer Storage: used to directly store all data objects; ② Upper layer Storage: used to organize and classify the lower layer Storage by namespace. The lower layer Storage stores data objects of a specified category, while the upper layer Storage stores the data objects stored in the lower layer Storage according to the namespace.

3. The scheduling system for efficiently scheduling Serverless function computation tasks in large-scale scenarios according to claim 1, characterized in that, The List-Watch mechanism is a mechanism in Kubernetes used to monitor changes in the status of cluster resources in real time. The List-Watch mechanism includes List operations and Watch operations. The List operation is used to obtain all status information of all relevant resources in the cluster at once. The Watch operation is used to continuously monitor changes in resources and push update events.

4. The scheduling system for efficiently scheduling Serverless function computation tasks in large-scale scenarios according to claim 1, characterized in that, The Apiserver module specifically includes: The Apiserver module parses the external HTTP requests it receives according to the OpenAPI specification and converts the parsed results into operation requests to the Client module. The Apiserver module routes the operation requests to the corresponding processing functions, which interact with the Client module to perform specific data operations. After the Client module completes the data operations, the Apiserver module constructs an HTTP response based on the operation results. The format of the HTTP response strictly follows the OpenAPI specification and includes the returned status code and the structure of the response body. The response body includes the resource data corresponding to the successful request or the error information and detailed error description corresponding to the failed request.

5. The scheduling system for efficiently scheduling Serverless function computation tasks in large-scale scenarios according to claim 1, characterized in that, The WatchCache module specifically includes: The WatchCache module stores all captured events in a circular queue. Within the WatchCache module, all events are sorted by resource version number. When an event is pushed to the WatchCache module, the WatchCache module distributes the event to the corresponding Informer component based on the resource version number. The Informer component is used to listen for changes in resource objects in the cluster and notify the corresponding controller of these changes. The controller then performs corresponding scheduling or control operations based on the changes in the resource objects.

6. The scheduling system for efficiently scheduling Serverless function computation tasks in large-scale scenarios according to claim 1, characterized in that, The Persister module specifically includes: When a data write request is received, the Persister module first submits the data to the BadgerDB database for persistence processing, and the BadgerDB database quickly persists the data to disk; If the data is successfully persisted to the BadgerDB database, the Persister module will write the data to the in-memory storage of the Storage module; if any error or exception occurs during the data persistence process, the Persister module will throw an exception, preventing the data from being written to the Storage module.

Images