Scheduling system for efficiently scheduling Serverless function calculation task in large-scale scene

Abstract

The invention discloses a scheduling system for efficiently scheduling a Serverless function calculation task in a large-scale scene. The system comprises a Storage module, a task scheduling module, a task scheduling module and a task scheduling module, wherein the Storage module is used for providing a cache function of an apiserver component and a storage function of an etcd component; the Client module is used for providing an SDK (Software Development Kit) API (Application Program Interface) which is compatible with KubuBuilder; the Apiserver module is used for ensuring that external services and tools continue to use the original operation mode; the WatchCache module is used for ensuring that event circulation in the List-Watch mechanism is consistent with that in a native system; and the Persister module is used for persisting the data into the BadgerDB library, so that the data can be recovered from the persistent BadgerDB library. According to the method, the calculation efficiency and performance in a large-scale Serverless scene can be improved, and the waiting time is shortened.

Description

Technical Field

[0001] The present invention relates to the technical field of scheduling Serverless function computing tasks, and in particular to a scheduling system for efficiently scheduling Serverless function computing tasks in large-scale scenarios. Background Art

[0002] Serverless is a cloud computing model in which developers do not need to manage server infrastructure. Unlike traditional computing models, Serverless allows developers to upload application code to cloud providers, which automatically handle all server management, scaling, and maintenance tasks. Developers only need to focus on the code itself without having to consider the underlying computing resources. Among them, the function computing task is the core concept in the Serverless model. It is usually executed in a stateless manner and can run automatically when an event is triggered. The execution of the function computing task is event-driven, and external event sources generate events to trigger task execution. A function computing task can be triggered when a user visits a web page or when the state of an internal system changes. Each task is executed independently and usually terminates immediately after completion to avoid wasting resources. The billing method for function computing tasks is based on the actual resources used and execution time, rather than charging for fixed server resources.

[0003] In a complex network system, in addition to web backend services, there may be a large number of computing tasks that need to be performed, such as video encoding and decoding, database backup, etc. The triggering of such computing tasks often depends on an event, which may be a timer trigger or a change in system status. Moreover, such tasks are usually stateless and are executed once. After execution, the dependent resources can be released. Based on these characteristics, function computing scenarios are particularly suitable for executing these computing tasks.

[0004] Currently in the Serverless field, Kubernetes-based scheduling systems are a popular choice. With the help of Kubernetes' controller and custom resources (CRDs) mechanism, automated management and scaling of applications can be achieved. In a Kubernetes-based scheduling system, the scheduler is a key component responsible for adjusting the number of Pods based on the request load of the service. When the load increases, the scheduler will request Kubernetes to schedule more Pods to handle these requests; when the load decreases, it will reduce the number of Pods. In addition, the scheduler is also responsible for finding suitable nodes in the cluster to run new tasks, that is, creating corresponding containers (Pods). When a new Pod is created, the scheduler selects a node to run the Pod based on the node's resource usage, node affinity, anti-affinity and other policies. However, in large-scale scenarios, the Kubernetes-based Serverless scheduling system may encounter bottlenecks.

[0005] Specifically, the architecture of Kubernetes is divided into multiple components, including etcd, apiserver, controller-manager, etc. This split architecture requires network communication for interaction between components. Each operation (such as creating, updating, and deleting resources) needs to communicate with etcd through the apiserver, which brings additional network latency. Especially when performing large-scale Serverless tasks, a large number of resource requests need to be passed to etcd through the apiserver and serialized and deserialized in JSON format. This serialization process may become a bottleneck in high-concurrency scenarios, resulting in higher latency. For the Serverless framework, executing complex computing tasks often involves a large number of Pod startup, configuration, and scheduling requests, all of which rely on the efficient response of etcd and apiserver. In large-scale scenarios, the overhead of JSON serialization and deserialization will increase with the increase in the number of tasks, affecting the overall execution efficiency.

[0006] In addition, the controller in Kubernetes is event-driven. It listens to resource changes (such as Pod creation, deletion, etc.) and performs corresponding operations based on these changes. In the Serverless framework based on Kubernetes, the role of the controller may include scheduling functions, expanding Pod resources, etc. When there is a delay in network communication, the response to the resource change received by the controller will also be delayed accordingly, which will cause the controller execution delay. This delay will not only affect the startup speed of a single Serverless task, but also block the serial execution of the controller, especially when there are dependencies between multiple controllers. For example, if controller A in controller-manager needs to wait for controller B to complete its operation, and the execution of controller B is blocked due to network delay, then the processing of the entire Serverless task will be delayed. In addition, controllers are distributed, and different controllers may need to coordinate the processing of the same type of resources. The delay of any controller may trigger a chain reaction, resulting in a decrease in the overall task scheduling and execution speed of the Serverless framework. Summary of the invention

[0007] The purpose of the present invention is to address the deficiencies in the prior art and propose a scheduling system for efficiently scheduling Serverless function computing tasks in large-scale scenarios.

[0008] The objective of the present invention is to achieve the following technical solution: a scheduling system for efficiently scheduling Serverless function computing tasks in large-scale scenarios, comprising:

[0009] The Storage module is used to provide the cache function of the original apiserver component and the storage function of the etcd component, and integrates the functions of the Webhook mechanism and the List-Watch mechanism; the Storage module uses the cache function of the apiserver component and the storage function of the etcd component to store data objects in memory, where the data objects are organized according to the namespace when stored;

[0010] The Client module provides a Kubebuilder-compatible SDK API interface for use by upper-level controllers to implement the addition, deletion, modification, and query operations of data objects, and supports event push and Webhook mechanism logic;

[0011] Apiserver module, which provides an HTTP API interface consistent with the native Kubernetes system and achieves seamless compatibility with Kubernetes tools such as client-go and kubectl, to ensure that external services and tools continue to interact with the system using the original familiar operation methods;

[0012] The WatchCache module is used to implement the same event push mechanism as the Kubernetes native system to ensure that the event flow in the List-Watch mechanism is consistent with the native system. Specifically, when the Kubernetes controller needs to know the changes of a certain resource in the cluster, it first executes the List operation to obtain the current resource status information, and then uses the Watch operation to subscribe to future change events, so that the controller can respond to changes in the resource status in the cluster in a timely manner to achieve automated management and expansion; and

[0013] The Persister module is used to use the BadgerDB library as the underlying key-value database to persist data in the BadgerDB library to ensure that the system can recover data from the persistent BadgerDB library after a failure or crash.

[0014] Furthermore, the Storage module is divided into two layers, namely: ① Bottom-layer Storage: used to directly store all data objects; ② Upper-layer Storage: used to organize and classify the bottom-layer Storage through namespaces; wherein, the bottom-layer Storage stores data objects of specified categories, and the upper-layer Storage stores the data objects stored in the bottom-layer Storage according to the namespace.

[0015] Furthermore, the List-Watch mechanism is a mechanism in Kubernetes for real-time monitoring of cluster resource status changes. The List-Watch mechanism includes List operations and Watch operations, wherein the List operation is used to obtain all status information of all relevant resources in the cluster at one time; the Watch operation is used to continuously monitor resource changes and push update events;

[0016] Furthermore, the Apiserver module specifically includes:

[0017] The Apiserver module parses the external HTTP requests it receives according to the OpenAPI specification, and converts the parsing results into operation requests for the Client module; the Apiserver module routes the operation requests to the corresponding processing function, which interacts with the Client module to perform specific data operations; after the Client module completes the data operation, the Apiserver module builds an HTTP response based on the operation results, where the format of the HTTP response strictly follows the OpenAPI specification. The HTTP response includes the returned status code and the structure of the response body. The structure of the response body includes the resource data corresponding to the request for a successful operation or the error information and detailed error description corresponding to the request for a failed operation.

[0018] Furthermore, the WatchCache module specifically includes:

[0019] The WatchCache module stores all captured events through a circular queue; in the WatchCache module, all events are sorted according to the resource version number; when the event is pushed to the WatchCache module, the WatchCache module distributes the event to the corresponding Informer component according to the resource version number, where the Informer component is used to monitor changes in resource objects in the cluster and notify these changes to the corresponding controller, and the controller makes corresponding scheduling or control operations based on the changes in the resource objects.

[0020] Furthermore, the Persister module specifically includes:

[0021] When there is a data write request, the Persister module first submits the data to the BadgerDB library for persistence processing, and the BadgerDB library quickly persists the data to the disk;

[0022] If the data is successfully persisted to the BadgerDB database, the Persister module will write the data to the memory storage in the Storage module; if any error or exception occurs during the data persistence process, the Persister module will throw an exception to prevent the data from being written to the Storage module.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] (1) The present invention optimizes the deficiencies of the existing Kubernetes architecture, especially the network communication delay and JSON serialization / deserialization overhead, through the design of a single-component architecture, which can improve the computing efficiency and reduce the waiting time in large-scale Serverless scenarios; the scheduling system described in the present invention combines various functional modules in Kubernetes in design, integrates the multi-component architecture of Kubernetes into a single-component Memlet system, significantly improves the performance, efficiency and compatibility in large-scale Serverless scenarios, while taking into account the compatibility with the original Kubernetes system.

[0025] (2) The present invention significantly reduces latency and improves the response speed of the system by reducing network communications to memory-level operations, especially in the Webhook and event push mechanisms. This enables the system to maintain agile and efficient operations under high concurrency conditions, adapt to the needs of fast startup and expansion in Serverless scenarios, and improve the performance of the system in large-scale Serverless scenarios.

[0026] (3) The present invention ensures seamless compatibility with tools such as kubectl and client-go by providing an API interface (including HTTP API and SDK API) consistent with the Kubernetes native system, thereby ensuring compatibility and allowing users to directly migrate to the Memlet system without modifying the existing application code, thereby reducing the complexity of migration.

[0027] (4) The present invention ensures the stability and consistency of resource management in large-scale clusters by retaining the List-Watch mechanism and event push logic of Kubernetes in the Memlet system, so that the controller can rely on the existing event processing mechanism to maintain the normal operation of the system in a dynamic environment.

[0028] (5) The present invention effectively reduces the burden of disk I / O by combining persistence operations with efficient memory storage, while ensuring rapid data persistence and high availability of the system. Since there are more writes than reads in the persistence scenario, the design of the Persister module pays special attention to the efficiency of write operations, thereby improving the resource efficiency of the system in large-scale Serverless scenarios.

[0029] (6) The present invention uses interface consistency design to allow external tools and services to continue to interact with the system in a familiar manner without having to perceive changes in the underlying storage, thereby avoiding modifications to the upper-level logic and making system operations transparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is an architectural flow chart of a scheduling system (memlet system) for efficiently scheduling Serverless function computing tasks in large-scale scenarios according to the present invention;

[0031] Figure 2 This is a schematic diagram of the migration process from the Kubernetes system to the memlet system of the present invention;

[0032] Figure 3 This is a workflow diagram of the Persister module of the present invention. DETAILED DESCRIPTION

[0033] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. It should be understood that the above general description and the detailed description that follows are merely exemplary and explanatory and cannot limit the present application.

[0034] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0035] It should be understood that, although the term first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of..." or "when..." or "in response to determination". Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process and method including a series of elements include not only those elements, but also include other elements not clearly listed, or also include elements inherent to such process and method. In the absence of more restrictions, the elements limited by the statement "including one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements.

[0036] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0037] The scheduling system described in the present invention for efficiently scheduling Serverless function computing tasks in large-scale scenarios is called the memlet system, which is implemented based on the Kubernetes architecture. The memlet system is designed to combine various functional modules in Kubernetes while taking into account the compatibility with the original Kubernetes system. Figure 1 As shown, the memlet system specifically includes a Storage module, a Client module, an Apiserver module, a WatchCache module and a Persister module.

[0038] In this embodiment, the Storage module is used to provide the caching function of the original apiserver component and the storage function of the etcd component, and integrates the functions of the Webhook mechanism and the List-Watch mechanism; the Storage module uses the caching function of the apiserver component and the storage function of the etcd component to store data objects in memory, where data objects are organized according to namespaces when stored, which can ensure efficient data management and fast access.

[0039] It should be understood that the Storage module is the core of the memlet system. It assumes the caching function of the original apiserver component and the storage function of the etcd component in the original Kubernetes system, and also integrates the functions of the Webhook mechanism and List-Watch mechanism.

[0040] It should be noted that the apiserver component is one of the core components of the Kubernetes cluster, responsible for handling all external and internal API requests, and routing these requests to the corresponding components for subsequent processing. The apiserver component is the front-end interface of Kubernetes, providing CRUD (create, read, update, delete) operations, and ensuring consistency and high availability of read and write operations on cluster status. The apiserver component is also responsible for functions such as authentication, authorization, admission control, and resource quotas. Through the caching function of the apiserver component, the apiserver component can quickly respond to requests from the client, reduce the frequency of interaction with the backend database (such as the etcd component), and thus improve overall performance.

[0041] The etcd component is the backend storage system of the Kubernetes cluster, which is used to persistently store all configuration information, state data, and objects of the cluster. The etcd component is a distributed key-value storage system that supports high availability and consistency, ensuring that each node in the cluster can accurately synchronize its state. Through the etcd component, Kubernetes can reliably store all critical cluster data and provide a fault-tolerant mechanism to cope with situations such as node failures or network partitions.

[0042] Furthermore, the Storage module is divided into two layers: ① Bottom-layer Storage: used to directly store all data objects, which can ensure the high efficiency of data object access and use memory to speed up data access. ② Upper-layer Storage: used to organize and classify the bottom-layer Storage through namespace, making data object management clearer and more convenient. The bottom-layer Storage stores data objects of specified categories, while the upper-layer Storage stores the data objects stored in the bottom-layer Storage according to namespace.

[0043] Furthermore, the Webhook mechanism is integrated into the Storage module to achieve enhanced scalability and flexibility. The Webhook mechanism allows callbacks to be triggered when specific events occur, such as object creation, update, or deletion operations. Through the Webhook mechanism, the memlet system can dynamically execute custom logic, such as security checks or audits, thereby enhancing the scalability and customization capabilities of the memlet system.

[0044] Furthermore, the Storage module implements the List function and Watch function in the List-Watch mechanism in Kubernetes. The List-Watch mechanism is a mechanism in Kubernetes for real-time monitoring of cluster resource status changes. The List-Watch mechanism includes List operations and Watch operations. The List operation is used to obtain all status information of all relevant resources in the cluster at one time, and the Watch operation is used to continuously monitor resource changes and push update events, such as resource addition, update, and deletion events. Through the List-Watch mechanism, the Storage module can effectively support large-scale Serverless scenarios and provide fast and reliable support for subsequent event push. Since data is stored in memory, the access speed is very fast, which is crucial in scenarios that require high performance and high scalability.

[0045] In this embodiment, the Client module provides a Kubebuilder-compatible SDK API interface for use by the upper-level controller (Controller) to implement the addition, deletion, modification, and query operations of data objects, and supports event push and Webhook mechanism logic.

[0046] It should be understood that the Kubebuilder-compatible SDK interface provided by the Client module can ensure seamless integration of the memlet system with the existing business logic of Kubernetes; by implementing the corresponding factory method, the Client module replaces the default Client implementation in the original dependency library, allowing applications to switch to the memlet system without modifying the code. The Webhook mechanism is implemented through HTTP requests, and in the memlet system, this Webhook mechanism is simplified to memory-level object delivery and admission judgment, which greatly reduces latency.

[0047] In this embodiment, the Apiserver module plays a crucial bridging role in the memlet system. The Apiserver module is used to provide an HTTP API interface consistent with the Kubernetes native system, and achieve seamless compatibility with Kubernetes tools such as client-go and kubectl, to ensure that external services and tools continue to interact with the system using the original familiar operation methods without having to be aware of changes in the underlying storage.

[0048] It should be understood that in the Kubernetes native system, the Apiserver module is the hub of the entire cluster. It is responsible for processing all API requests, managing all resource objects in the cluster, and persisting these objects by interacting with the etcd component. The Apiserver module defines the API interface using the OpenAPI specification to ensure the consistency of communication between Kubernetes components, and provides a rich HTTP API interface so that users and applications can easily interact with Kubernetes. In the memlet system, the Apiserver module inherits this design concept and implements an HTTP API interface consistent with the original system by simulating the Apiserver module of the Kubernetes native system.

[0049] It should be noted that client-go is the official Go client library of Kubernetes, which allows developers to programmatically interact with Kubernetes clusters, perform CRUD operations, listen to events, manage resource objects, etc. client-go encapsulates complex API calls, allowing developers to easily build applications integrated with Kubernetes using the Go language. Through the consistent API provided by the Apiserver module, components in the memlet system can continue to interact with client-go, thereby maintaining compatibility with existing Kubernetes applications.

[0050] Another Kubernetes tool, kubectl, is a command-line tool for Kubernetes that is used to manage Kubernetes cluster resources. Users can send API requests through kubectl to create, update, delete, and query Kubernetes resources. Since the Apiserver module of the memlet system implements the compatibility of the Kubernetes API interface, users can manage the memlet system through kubectl just like using Kubernetes, without being aware of changes in the underlying storage structure.

[0051] Furthermore, the Apiserver module specifically includes: the Apiserver module parses the external HTTP requests it receives (such as requests from kubectl or client-go) according to the OpenAPI specification, and converts the parsing results into operation requests for the Client module; the Apiserver module routes the operation requests to the corresponding processing function, which interacts with the Client module to perform specific data operations; after the Client module completes the data operation, the Apiserver module builds an HTTP response according to the operation result, wherein the format of the HTTP response strictly follows the OpenAPI specification, and the HTTP response includes the returned status code, the structure of the response body, etc. The structure of the response body includes the resource data corresponding to the request for a successful operation or the error information and detailed error description corresponding to the request for a failed operation.

[0052] Specifically, when the Apiserver module receives an HTTP request, it first parses and verifies the request according to the OpenAPI specification, and converts the parsing result into an operation request for the Client module, which can ensure that the data storage and processing logic are consistent with Kubernetes. Among them, the OpenAPI specification is a standard for defining API interfaces in Kubernetes, which describes in detail the path, parameters, return value type and other information of each API endpoint. Based on the OpenAPI specification, the Apiserver module can use this information to parse the request path, query parameters and request body, and verify the legitimacy of the request. After parsing the request, the Apiserver module will route the request to the corresponding processing function (i.e., the logic of adding, deleting, modifying and checking), which performs actual operations through interaction with the memlet Client module. For example, when the request is to query a certain resource list, the Apiserver module will convert the request into a query operation on the memlet Client to obtain the corresponding resource data. The memlet Client module is responsible for interacting with the memlet system's in-memory database and performing specific data operations (i.e., add, delete, modify, and query). This design ensures that even if the underlying storage is switched from the etcd component in Kubernetes to the memlet system's in-memory database, external applications can still interact with the cluster through the same API interface; for external users and services, this change is transparent, and they do not need to perceive the differences in the underlying architecture. Once the memlet Client module completes the specific data operation, the Apiserver module will build an HTTP response based on the operation result; the format of the HTTP response strictly follows the Kubernetes OpenAPI specification, including the returned status code, the structure of the response body, etc.; for successful requests, the Apiserver module will return the corresponding resource data; for failed requests, it will return error information and detailed error descriptions. This design ensures that when external services process API responses, they can process them as in the native Kubernetes environment. Whether it is the kubectl command-line tool or the client-go-based application, they can continue to use the existing parsing and processing logic to interact seamlessly with the memlet system.

[0053] It should be noted that the OpenAPI specification is used in Kubernetes to describe the structure and behavior of all APIs. The Apiserver module also implements this specification in the memlet system to ensure full compatibility with the memlet system and Kubernetes system. By using the OpenAPI specification, the Apiserver module can generate automated API documentation to support external services and tools to better understand and use API interfaces. In addition, the OpenAPI specification also supports client code generation, which means that developers can automatically generate client code compatible with the memlet system based on the OpenAPI specification, thereby accelerating application development and integration.

[0054] In addition, the Apiserver module has been appropriately extended based on the implementation of the OpenAPI specification. Through these extensions, the Apiserver module can not only provide functions consistent with Kubernetes, but also make full use of the advantages of the memlet system memory database to achieve more efficient data access and processing.

[0055] In this embodiment, the WatchCache module plays a vital role in the memlet system. The WatchCache module is used to implement the same event push mechanism as the Kubernetes native system to ensure that the event flow in the List-Watch mechanism is consistent with the native system. Specifically, when the Kubernetes controller (such as the Controller component) needs to understand the changes in a certain resource in the cluster, it first executes the List operation to obtain the current resource status information, and then uses the Watch operation to subscribe to future change events, so that the controller can respond to changes in the resource status in the cluster in a timely manner to achieve automated management and expansion.

[0056] It should be noted that the WatchCache module implements this List-Watch mechanism in the memlet system and optimizes the efficiency of event processing. Under this List-Watch mechanism, when an object in the Storage module changes (such as adding, updating or deleting), a corresponding event will be generated immediately, and the event will be pushed to the WatchCache module for processing. First, the List operation is executed to obtain the current resource status information, and then the Watch operation is used to subscribe to future change events.

[0057] Furthermore, the WatchCache module specifically includes: the WatchCache module stores all captured events through a circular queue. The design of this circular queue has the characteristics of efficient memory usage, can store a large number of events in a limited memory space, and supports fast event enqueue and dequeue operations; this design not only improves the performance of the system, but also can effectively handle a large number of events under high concurrency, and can ensure the orderliness of events and the efficiency of processing. In the WatchCache module, all events are sorted according to the resource version number (ResourceVersion) to ensure that events with smaller resource version numbers can be processed first; ResourceVersion is a key field in Kubernetes for tracking changes in resource objects. Whenever a resource object changes, its version number will increase. Therefore, by giving priority to events with smaller version numbers, the system can ensure the accuracy and consistency of resource status. When the event is pushed to the WatchCache module, the WatchCache module distributes the event to the corresponding Informer component according to ResourceVersion. The Informer component is an important concept in the Kubernetes client. Its function is to ensure that the controller responds to resource status changes in a timely manner to maintain the stability and correct operation of the cluster. The Informer component is responsible for monitoring changes in resource objects in the cluster and notifying the corresponding controllers of these changes. These controllers will make corresponding scheduling or control operations based on the changes in resource objects.

[0058] It should be understood that the WatchCache module is a cache component of the Apiserver module in Kubernetes, which is mainly used to efficiently process and push change events of resource objects. It reduces the access pressure of Apiserver to the backend database (such as etcd) by caching the latest resource change events, thereby improving system performance.

[0059] It should be noted that the design of the WatchCache module strictly follows the event push mechanism of Kubernetes to ensure that it is completely consistent with the List-Watch mechanism of Kubernetes. This compatibility design allows the upper-level controllers and applications to continue to work in the original way without any modification. For example, when an external Kubernetes client (such as kubectl or client-go) sends a List-Watch request to the memlet system, it first performs a List operation to obtain all the status information of all relevant resources in the cluster at one time, and then executes the Watch operation to continuously monitor resource changes and push update events. Therefore, the WatchCache module can directly respond to these List-Watch requests and provide event streams and resource status information consistent with the Kubernetes native system.

[0060] In this embodiment, the Persister module is used to use the BadgerDB library as the underlying key-value database (KV database) to persist data in the BadgerDB library to ensure that the system can recover data from the persistent BadgerDB library after a failure or crash.

[0061] It should be understood that the Persister module plays a key role in ensuring persistence in the memlet system. Its design ensures that the system can recover data from the persistent file library after a failure or crash, ensuring data integrity and high availability of the system.

[0062] Furthermore, the Persister module specifically includes: Figure 3 As shown in the figure, when there is a data write request, the Persister module first submits the data to the BadgerDB library for persistence processing, and the BadgerDB library quickly persists the data to the disk; this efficient write operation not only improves the speed of data persistence, but also ensures the stability of the system when processing large-scale write requests. If the data is successfully persisted to the BadgerDB library, the Persister module will write the data to the memory storage in the Storage module; if any error or exception occurs during the data persistence process, the Persister module will throw an exception to prevent the data from continuing to be written to the Storage module. This dual protection mechanism ensures the reliability of the data and can prevent inconsistent or incomplete data from appearing in the system.

[0063] It should be noted that data will only be actually stored in the Storage module after the persistence operation is successfully completed; if any error or exception occurs during the persistence process, the Persister module will throw an exception to prevent data from being written to the Storage module, thereby preventing inconsistent or incomplete data from appearing in the system. This dual protection mechanism ensures data reliability, and this design greatly improves the robustness of the system, ensuring that the system can maintain data integrity and consistency even in the event of unforeseen failures.

[0064] In addition, another design feature of the Persister module is that it mainly performs write operations and rarely performs read operations. This design choice greatly reduces the burden of disk I / O and focuses on efficient write operations, thereby ensuring the speed and reliability of the data persistence process. In actual application scenarios, since most data operations are write operations, this design of the Persister module can give full play to the advantages of the BadgerDB library in write efficiency, ensuring that the system can still respond quickly and complete data persistence under high concurrency.

[0065] For example, an e-commerce platform uses the Memlet system to build its real-time order processing system to cope with the high-concurrency processing requirements of millions of orders per day. Through Memlet's efficient event push mechanism and memory-level Webhook verification, the platform can quickly process order creation, verification, update and other operations in large-scale Serverless scenarios, while maintaining seamless compatibility with the existing Kubernetes infrastructure, significantly improving the system's response speed and stability.

[0066] In summary, the key point of the present invention is to successfully achieve an efficient replacement of the Kubernetes native architecture in the memlet system by optimizing and integrating multiple modules. The migration process is as follows: Figure 2 As shown in the figure, while ensuring compatibility with the existing Kubernetes system, it significantly improves the performance and response speed in large-scale Serverless scenarios.

[0067] The memlet Client module provides an SDK API interface compatible with Kubebuilder, allowing the upper-level controller to switch directly to the memlet system without code modification, realizing seamless connection between the memlet system and the Kubernetes native system. This design ensures that the original controller logic can continue to run normally when the underlying storage and event mechanisms are replaced. By implementing the corresponding factory methods, the memlet Client module can replace the default Client implementation in Kubernetes, so that the system remains compatible with tools such as client-go and kubectl after switching to the memlet system. In large-scale application scenarios of Serverless, this compatibility design is particularly important, ensuring that existing applications can quickly adapt to the new storage architecture and avoiding interruptions and complexity caused by switching.

[0068] The Apiserver module implements an HTTP API interface consistent with Kubernetes in the memlet system. By strictly following the OpenAPI specification, it ensures that external tools and services can continue to use the familiar HTTP API when interacting with the memlet system. External applications and operation and maintenance tools (such as kubectl) do not have to perceive changes in the underlying storage when interacting with the memlet system, thereby maintaining the operational consistency of the system and the continuity of the development experience. In the Serverless scenario, the interface consistency of the Apiserver module enables fast response and seamless expansion in large-scale and high-concurrency environments.

[0069] The memlet Client module simplifies the Webhook mechanism that originally relied on network communication into memory-level object delivery and admission judgment. This optimization not only retains the functionality of the Webhook mechanism in Kubernetes, but also improves the overall performance of the system by reducing network latency. In large-scale Serverless scenarios, this memory-level optimization is particularly critical, as it can significantly reduce the latency caused by network communication, allowing the system to maintain agile and efficient event processing capabilities under high concurrency conditions.

[0070] The WatchCache module implements the same List-Watch event push mechanism as Kubernetes in the memlet system, ensuring that the order and logic of event flow are completely consistent with Kubernetes, and completely retaining the event push mechanism. This design allows the upper-level controller to continue to rely on the existing event processing mechanism, thereby maintaining stable performance and consistent behavior in large-scale clusters. In Serverless application scenarios, the reliability and efficiency of the event push mechanism are crucial, because Serverless applications usually involve a large amount of dynamic resource management and real-time response requirements. The design of the WatchCache module ensures that the system can maintain consistency and stability even in a highly dynamic environment.

[0071] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A scheduling system for efficiently scheduling Serverless function computing tasks in large-scale scenarios, characterized by: include: The Storage module is used to provide the cache function of the original apiserver component and the storage function of the etcd component, and integrates the functions of the Webhook mechanism and the List-Watch mechanism; the Storage module uses the cache function of the apiserver component and the storage function of the etcd component to store data objects in memory, where the data objects are organized according to the namespace when stored; The Client module provides a Kubebuilder-compatible SDK API interface for use by upper-level controllers to implement the addition, deletion, modification, and query operations of data objects, and supports event push and Webhook mechanism logic; Apiserver module, which provides an HTTP API interface consistent with the native Kubernetes system and achieves seamless compatibility with Kubernetes tools such as client-go and kubectl, to ensure that external services and tools continue to interact with the system using the original familiar operation methods; The WatchCache module is used to implement the same event push mechanism as the Kubernetes native system to ensure that the event flow in the List-Watch mechanism is consistent with the native system. Specifically, when the Kubernetes controller needs to understand the changes of a certain resource in the cluster, it first executes the List operation to obtain the current resource status information, and then uses the Watch operation to subscribe to future change events, so that the controller can respond to changes in the resource status in the cluster in a timely manner and realize automated management and expansion; and The Persister module is used to use the BadgerDB library as the underlying key-value database to persist data in the BadgerDB library to ensure that the system can recover data from the persistent BadgerDB library after a failure or crash.

2. According to claim 1, the scheduling system for efficiently scheduling Serverless function computing tasks in large-scale scenarios is characterized in that: The Storage module is divided into two layers, namely: ① Bottom-layer Storage: used to directly store all data objects; ② Upper-layer Storage: used to organize and classify the bottom-layer Storage through namespaces; wherein, the bottom-layer Storage stores data objects of specified categories, and the upper-layer Storage stores the data objects stored in the bottom-layer Storage according to the namespace.

3. According to claim 1, the scheduling system for efficiently scheduling Serverless function computing tasks in large-scale scenarios is characterized in that: The List-Watch mechanism is a mechanism in Kubernetes for real-time monitoring of cluster resource status changes. The List-Watch mechanism includes List operations and Watch operations. The List operation is used to obtain all status information of all relevant resources in the cluster at one time; the Watch operation is used to continuously monitor resource changes and push update events.

4. According to claim 1, the scheduling system for efficiently scheduling Serverless function computing tasks in large-scale scenarios is characterized in that: The Apiserver module specifically includes: The Apiserver module parses the external HTTP requests it receives according to the OpenAPI specification, and converts the parsing results into operation requests for the Client module; the Apiserver module routes the operation requests to the corresponding processing function, which interacts with the Client module to perform specific data operations; after the Client module completes the data operation, the Apiserver module builds an HTTP response based on the operation results, where the format of the HTTP response strictly follows the OpenAPI specification. The HTTP response includes the returned status code and the structure of the response body. The structure of the response body includes the resource data corresponding to the request for a successful operation or the error information and detailed error description corresponding to the request for a failed operation.

5. According to claim 1, the scheduling system for efficiently scheduling Serverless function computing tasks in large-scale scenarios is characterized in that: The WatchCache module specifically includes: The WatchCache module stores all captured events through a circular queue; in the WatchCache module, all events are sorted according to the resource version number; when the event is pushed to the WatchCache module, the WatchCache module distributes the event to the corresponding Informer component according to the resource version number, where the Informer component is used to monitor changes in resource objects in the cluster and notify these changes to the corresponding controller, and the controller makes corresponding scheduling or control operations based on the changes in the resource objects.

6. The scheduling system for efficiently scheduling Serverless function computing tasks in large-scale scenarios according to claim 1 is characterized in that: The Persister module specifically includes: When there is a data write request, the Persister module first submits the data to the BadgerDB library for persistence processing, and the BadgerDB library quickly persists the data to the disk; If the data is successfully persisted to the BadgerDB database, the Persister module will write the data to the memory storage in the Storage module; if any error or exception occurs during the data persistence process, the Persister module will throw an exception to prevent the data from being written to the Storage module.

Images