Method and apparatus for extending kubernetes scheduler on basis of webassembly

Abstract

A method for extending a Kubernetes scheduler on the basis of a WebAssembly, which relates to the technical field of Kubernetes cloud platforms. Scheduling logic is managed and configured, a WASM Runtime is loaded, a WASM Module instruction is generated after loading is completed, and then a WASM sandbox instance is generated, and finally, same is prepared to execute scheduling. Scheduling logic is implemented by means of the mutual cooperation between a Manager module, a VM module and an SI module, such that the problems of scheduling extension and supporting hot updates are solved, and a user is conveniently and securely supported to extend the scheduling logic. Further provided is an apparatus for extending a Kubernetes scheduler on the basis of a WebAssembly.

Description

A method and device for extending Kubernetes scheduler based on WebAssembly Technical Field

[0001] The present invention relates to the technical field of Kubernetes cloud platforms, and in particular to a method and device for extending a Kubernetes scheduler based on WebAssembly. Background Art

[0002] Kubernetes is a powerful container orchestration platform. Its scheduler is responsible for assigning pods to available nodes. The default scheduler, kube-scheduler, is a built-in component of Kubernetes. It uses a scheduling policy based on predicates and priorities to ensure that pods are scheduled on nodes that meet resource requirements, affinity constraints, and other requirements. Although kube-scheduler provides a set of default scheduling policies, these policies may not meet the needs of users with specialized requirements. For example, users may need to schedule pods based on customized business rules or optimize specific workload types. Therefore, Kubernetes v1.15 and later allow users to implement their own scheduling policies through the Scheduling Framework. However, users still face several pain points. 1) The barrier to entry for fully developing Scheduling Framework extensions is high, requiring a thorough understanding of the scheduler extension mechanism, including development and deployment. Furthermore, the process from development to debugging to production deployment is cumbersome and prone to errors. 2) In some scenarios, such as public cloud-hosted Kubernetes, deploying Scheduling Framework extensions is almost impossible. This patent attempts to provide a lighter and simpler way to extend scheduling, making it easier for users to extend Kubernetes' scheduling strategies, thereby solving the above problems.

[0003] For example, the patent invention with the existing application publication number CN116841718A provides a physical resource scheduling method and scheduler based on Kubernetes. The method obtains the scheduling information in the Kubernetes cluster, including the Pod information of the Pod to be scheduled and the load information of each node; then traverses each node, inputs the Pod information of the Pod to be scheduled and the load information of the current node into the scheduling evaluation model, and obtains the evaluation information of the Pod to be scheduled when it is scheduled to the current node output by the scheduling evaluation model; preferably, the Pod to be scheduled is scheduled based on the evaluation information corresponding to each node. This method introduces the load information of the current node, avoids the situation where the physical resources on the node are abnormal due to only considering hardware resources, ensures the safe operation of the Kubernetes cluster, and improves the stability of the Kubernetes cluster.

[0004] However, the above patent has the following problems: the scheduling logic including Pod information and load information of each node is controlled by scheduling Pod, but the problem of scheduling extension and hot update is not addressed. Secondly, although three methods, namely custom scheduler, scheduler extension program and scheduling framework, are provided, the implementation of the solutions is relatively cumbersome and hot update is not supported.

[0005] This invention solves the problems of scheduling extension and supporting hot updates. Secondly, in terms of scenarios, this patent is mainly aimed at Kubernetes-general scheduling scenarios such as public clouds and private clouds, allowing users to customize scheduling logic. Especially in public cloud scenarios, it is generally impossible for users to replace the kube-scheduler image. Finally, this patent is suitable for scenarios that extend the scheduling capabilities of Kubernetes. Based on the scheduling framework, the request for the scheduling extension point is transferred to the WASM sandbox instance for execution, which conveniently and safely supports the user to extend the scheduling logic. At the same time, WSSP supports hot-changing scheduling logic and supports different Pods to use different scheduling logic. Summary of the Invention

[0006] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of this application to avoid obscuring the purpose of this section, the abstract and the title of the invention, and such simplifications or omissions should not be used to limit the scope of the present invention.

[0007] To solve the above technical problems, the main purpose of the present invention is to provide a method for extending the Kubernetes scheduler based on WebAssembly, including:

[0008] S1. Create a sandbox scheduling plug-in based on kube-scheduler source code.

[0009] S2. Modify NewSchedulerCommand as the configuration file for kube-scheduler;

[0010] After S3 configuration is complete, the user develops the scheduling logic using a relevant language, compiles it into a wasm file, uploads it to the storage system, and writes and submits a WsspSchedulerConfiguration request change request.

[0011] S4. When the CR request is changed, the Pod annotation enhancement is started, and the wssp and schedulefragment are annotated. The annotation value is changed to the name of the CR request change in WsspSchedulerConfiguration.

[0012] S5. After the request change is completed, the scheduler is scheduled, in which the WSSP SI is called through extension points, and then the scheduling logic is called. After execution, the result is returned and output.

[0013] As a preferred solution of the method for extending the Kubernetes scheduler based on WebAssembly of the present invention, wherein:

[0014] The sandbox scheduling plug-in is denoted as wssp;

[0015] The wssp is based on the kube-scheduler source code and is input into the sandbox scheduling plug-in by modifying the NewSchedulerCommand;

[0016] WSSP itself is configured through the KubeSchedulerConfigurations and the schedulefragment parameter in args, and the schedulefragment value defaults to default:v1.0.0. The schedulefragment status is not required. The default scheduling logic default-v1.0.0.wasm is loaded into the WASM sandbox when the plug-in is initialized.

[0017] As a preferred solution of the method for extending the Kubernetes scheduler based on WebAssembly of the present invention, wherein:

[0018] The request to change CR is an output instruction issued by the scheduling logic, and the name of the request to change CR follows the format of {name}-{versionNo};

[0019] After the configuration is complete, the user develops the scheduling logic using the relevant language, compiles it into a wasm file, uploads it to the storage system, and writes and submits the WsspSchedulerConfiguration change request CR. When wssp watch receives the change request CR, it loads the .wasm file into the WASM sandbox.

[0020] As a preferred solution of the method for extending the Kubernetes scheduler based on WebAssembly of the present invention, wherein:

[0021] The Pod annotation is used for wssp / schedulefragment, and the logical scheduling is executed by changing the annotation value;

[0022] If the Pod has wssp / schedulefragment, it means that the Pod will use the scheduling logic in WsspSchedulerConfiguration. If the Pod does not have wssp / schedulefragment, it means that the Pod will use the default scheduling logic.

[0023] As a preferred solution of the method for extending the Kubernetes scheduler based on WebAssembly of the present invention, wherein:

[0024] The scheduling logic process calls WSSP SI through extension points, then calls the scheduling logic (WASM extension code), executes and returns the result;

[0025] The returned result includes forming a scheduling closed loop and outputting a scheduling result.

[0026] As a preferred solution of the present invention, a device for extending the Kubernetes scheduler based on WebAssembly, wherein:

[0027] Manager module, used to monitor scheduling logic and monitor scheduling logic data;

[0028] The VM module is used to encapsulate functions and is responsible for loading and executing .wasm files and managing sandbox resources;

[0029] The SI module is used to connect the interaction between the scheduling plug-in and the scheduling logic.

[0030] As a preferred solution of the present invention, a device for extending the Kubernetes scheduler based on WebAssembly, wherein:

[0031] The Manager module is used to monitor the scheduling logic WsspSchedulerConfiguration's request to change CR and maintain the WASM sandbox deployment logic;

[0032] The WASM sandbox uses WASM technology to allow users to choose multiple languages ​​to write scheduling logic;

[0033] The scheduling logic is enhanced through Pod annotations.

[0034] As a preferred solution of the present invention, a device for extending the Kubernetes scheduler based on WebAssembly, wherein:

[0035] The VM module provides a unified encapsulation of the WASM Runtime, responsible for loading and executing .wasm files, and resource management of the WASM sandbox instance;

[0036] The resource management of the WASM sandbox instance is to load the .wasm file into the WASM sandbox instance if WSSP watch requests to change the CR.

[0037] As a preferred solution of the present invention, a device for extending the Kubernetes scheduler based on WebAssembly, wherein:

[0038] The SI module provides an external usage interface and can be regarded as the glue layer for interaction between the scheduling plug-in and the scheduling logic. The plug-in implements the extension point of the Scheduling Framework and transfers the call to the scheduling logic through the SI module.

[0039] As a preferred solution of the present invention, a device for extending the Kubernetes scheduler based on WebAssembly, wherein:

[0040] The call is transferred to the scheduling logic through the SI module. If the Pod does not have wssp / schedulefragment, the Pod will use the default scheduling logic to execute and call the SI module through extension points, that is, use the default logic scheduling.

[0041] Beneficial effects of the present invention:

[0042] This invention solves the problems of scheduling extension and supporting hot updates. Secondly, in terms of scenarios, this patent is mainly aimed at Kubernetes-general scheduling scenarios such as public clouds and private clouds, allowing users to customize scheduling logic. Especially in public cloud scenarios, it is generally impossible for users to replace the kube-scheduler image. Finally, this patent is suitable for scenarios that extend the scheduling capabilities of Kubernetes. Based on the scheduling framework, the request for the scheduling extension point is transferred to the WASM sandbox instance for execution, which conveniently and safely supports the user to extend the scheduling logic. At the same time, WSSP supports hot-changing scheduling logic and supports different Pods to use different scheduling logic. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0044] FIG1 is a system flow chart of a method for extending a Kubernetes scheduler based on WebAssembly according to the present invention;

[0045] FIG2 is a diagram showing the overall system composition of an apparatus for automated testing of CDN customer service requirements according to the present invention;

[0046] FIG3 is a diagram illustrating an architecture of a scheduling solution based on WebAssembly-based extended Kubernetes scheduler according to the present invention;

[0047] FIG4 is a flowchart of a method for extending a Kubernetes scheduler based on WebAssembly according to the present invention;

[0048] FIG5 is a flowchart of a Pod annotation device for extending a Kubernetes scheduler based on WebAssembly according to the present invention; DETAILED DESCRIPTION

[0049] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0050] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0051] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0052] Furthermore, the present invention is described in detail with reference to schematic diagrams. For ease of illustration, when describing the embodiments of the present invention, cross-sectional views illustrating device structures may be partially enlarged and not to scale. Furthermore, the schematic diagrams are merely illustrative and should not limit the scope of protection of the present invention. Furthermore, in actual production, the three-dimensional dimensions of length, width, and depth should be included.

[0053] Example 1

[0054] As shown in Figure 1, a method for extending the Kubernetes scheduler based on WebAssembly includes:

[0055] S1. Create a sandbox scheduling plug-in based on kube-scheduler source code.

[0056] Among them, the sandbox scheduling plug-in is recorded as wssp;

[0057] Furthermore, wssp is based on the kube-scheduler source code and inputs the sandbox scheduling plug-in by modifying NewSchedulerCommand;

[0058] S2. Modify NewSchedulerCommand as the configuration file for kube-scheduler;

[0059] WSSP itself is configured through the schedulefragment parameter in KubeSchedulerConfigurations and args. The schedulefragment value defaults to default:v1.0.0, and the schedulefragment status is not required. The default scheduling logic default-v1.0.0.wasm is loaded into the WASM sandbox when the plug-in is initialized.

[0060] As shown in Figure 5, after S3,configuration is completed, the user develops the scheduling logic using,the relevant language, compiles it into a wasm file, uploads it to the storage system,,writes and submits the WsspSchedulerConfiguration request change CR;

[0061] The request to change CR is an output instruction issued by the scheduling logic, and the name of the request to change CR follows the format of {name}-{versionNo};

[0062] After the configuration is complete, the user develops the scheduling logic using the relevant language, compiles it into a wasm file, uploads it to the storage system, and writes and submits the WsspSchedulerConfiguration change request CR. When wssp watch receives the change request CR, it loads the .wasm file into the WASM sandbox.

[0063] Furthermore, Pod annotations are used for wssp / schedulefragment, and the logical scheduling is executed by changing the annotation value;

[0064] If the Pod has wssp / schedulefragment, it means that the Pod will use the scheduling logic in WsspSchedulerConfiguration. If the Pod does not have wssp / schedulefragment, it means that the Pod will use the default scheduling logic.

[0065] Furthermore, Pod annotations are enhanced. Pods support the annotation wssp / schedulefragment, where the annotation value is the name of the WsspSchedulerConfiguration CR, indicating that the execution of this logic segment schedules this Pod.

[0066] S4. When the CR request is changed, the Pod annotation enhancement is started, and the wssp and schedulefragment are annotated. The annotation value is changed to the name of the CR request change in WsspSchedulerConfiguration.

[0067] Among them, when the Pod has wssp / schedulefragment, it means that the Pod will use the scheduling logic in WsspSchedulerConfiguration;

[0068] If the wssp / schedulefragment does not exist in the Pod, it means that the Pod will be executed using the default scheduling logic. This step calls the WSSP SI through extension points, and then calls the scheduling logic (WASM extension code). After execution, the result is returned. This step has the following conventions:

[0069] First, WSSP determines which scheduling logic to use based on the default configuration and the Pod's annotations. The scheduling logic specified by the Pod annotations takes precedence over the default configuration.

[0070] Second, when producing a WASM sandbox instance, scheduling will be blocked by a mutex lock.

[0071] S5. After the request change is completed, the scheduler is scheduled, in which the WSSP SI is called through extension points, and then the scheduling logic is called. After execution, the result is returned and output.

[0072] Among them, the scheduling logic processing calls WSSP SI through extension points, then calls the scheduling logic (WASM extension code), and returns the result after execution;

[0073] Furthermore, returning the result includes forming a scheduling closed loop and outputting the scheduling result.

[0074] As shown in Figure 4, the overall method process is scheduled.

[0075] Example 2

[0076] As shown in Figure 2, a device for extending the Kubernetes scheduler based on WebAssembly includes:

[0077] Manager module, used to monitor scheduling logic and monitor scheduling logic data;

[0078] The Manager module is used to monitor the scheduling logic WsspSchedulerConfiguration's request to change CR and maintain the WASM sandbox deployment logic.

[0079] The WASM sandbox uses WASM technology to allow users to choose multiple languages ​​to write scheduling logic;

[0080] Scheduling logic is enhanced through Pod annotations.

[0081] The VM module is used to encapsulate functions and is responsible for loading and executing .wasm files and managing sandbox resources;

[0082] The VM module provides a unified encapsulation of the WASM Runtime, responsible for loading and executing .wasm files, and managing the resources of the WASM sandbox instance.

[0083] Resource management of the WASM sandbox instance. If WSSP watches a request to change the CR, it loads the .wasm file into the WASM sandbox instance.

[0084] The SI module is used to connect the interaction between the scheduling plug-in and the scheduling logic.

[0085] Among them, the SI module provides an external usage interface, which can be regarded as the glue layer for interaction between the scheduling plug-in and the scheduling logic. The plug-in implements the extension point of the Scheduling Framework and transfers the call to the scheduling logic through the SI module.

[0086] Furthermore, the call is transferred to the scheduling logic through the SI module. If the Pod does not have wssp / schedulefragment, the Pod will use the default scheduling logic to execute and call the SI module through extension points, that is, use the default logic scheduling.

[0087] For example: prepare 5 virtual machines configured with 4-core CPUs and 8G memory, and deploy a high-availability Kubernetes cluster (3 Master nodes).

[0088] As shown in Figure 3, place the default scheduling logic .wasm file on the master host, edit, and submit the CR WsspSchedulerConfiguration. Set the CR name to default-v1, storageType to hostPath, and address to the .wasm file path.

[0089] Edit and submit KubeSchedulerConfigurations. Add an entry to pluginConfig with the name wssp-scheduler and the args parameter containing schedulefragment=default-v1. Replace kube-scheduler with the scheduler containing WSSP.

[0090] Write the scheduling logic in Go and compile it into a .wasm file named sample-v1. Place it on the master host, edit, and submit the CR WsspSchedulerConfiguration. Set the CR name to sample-v1, the storageType to hostPath, and the address to the .wasm file path.

[0091] Release the Pod and observe the logs to see whether the default-v1 scheduling logic is used.

[0092] Publish a pod with the annotation "wssp / schedulefragment=sample-v1" and observe the logs to see whether the sample-v1 scheduling logic is used.

[0093] It is important to note that the configuration and arrangement of the present application shown in a number of different exemplary embodiments are merely illustrative. Although only two embodiments are described in detail in this disclosure, it should be readily understood by those reading this disclosure that many modifications are possible, without substantially departing from the novel teachings and advantages of the subject matter described in this application. For example, the size, scale, structure, shape, and proportion of various components, as well as parameter values ​​(e.g., temperature, pressure, etc.), mounting arrangements, use of materials, color, changes in orientation, etc. For example, an element shown as integrally formed may be composed of multiple parts or elements, the position of an element may be inverted or otherwise changed, and the nature, number, or position of discrete elements may be altered or changed. Therefore, all such modifications are intended to be included within the scope of the present invention. The order or sequence of any process or method steps may be changed or reordered according to alternative embodiments. In the claims, any "means-plus-function" clause is intended to cover structures that perform the functions described herein, and not only structural equivalence but also equivalent structures. Other substitutions, modifications, changes, and omissions may be made in the design, operating conditions, and arrangement of the exemplary embodiments without departing from the scope of the present invention. Therefore, the invention is not limited to the specific embodiments, but extends to various modifications that still fall within the scope of the appended claims.

[0094] Additionally, in order to provide a concise description of exemplary embodiments, all features of an actual embodiment (ie, those features that are not relevant to the best mode presently contemplated for carrying out the invention or those that are not relevant to implementing the invention) may not be described.

[0095] It should be understood that in the development of any actual embodiment, as in any engineering or design project, numerous implementation-specific decisions may be made. Such a development effort may be complex and time-consuming, but for those of ordinary skill having the benefit of this disclosure, the development effort will be a routine task of design, fabrication, and production without undue experimentation.

[0096] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for extending a Kubernetes scheduler based on WebAssembly, characterized in that: include, S1. Use kube-scheduler as the source code to create a sandbox scheduling plug-in; S2. Modify NewSchedulerCommand as the configuration file for kube-scheduler; S3. After the configuration is completed, the user develops the scheduling logic in the relevant language, compiles it into a wasm file, uploads it to the storage system, and writes and submits the WsspSchedulerConfiguration request change CR; S4. When the output request changes the CR, the Pod annotation enhancement is started, the wssp and schedulefragment are annotated, and the annotation value is changed to the name of the request change CR of WsspSchedulerConfiguration; S5. After the request change is completed, the scheduler is scheduled, in which WSSP SI is called through extension points, and then the scheduling logic is called. After execution, the result is returned and output.

2. According to a method for extending a Kubernetes scheduler based on WebAssembly according to claim 1, it is characterized in that: The sandbox scheduling plug-in is denoted as wssp; The wssp is based on the source code of kube-scheduler, and the sandbox scheduling plug-in is input by modifying NewSchedulerCommand; WSSP itself is configured through the parameters schedulefragment in KubeSchedulerConfigurations and args, and the schedulefragment value defaults to default:v1.0.

0. The schedulefragment status is not required, and the default scheduling logic default-v1.0.0.wasm is loaded into the WASM sandbox when the plug-in is initialized.

3. According to a method for extending a Kubernetes scheduler based on WebAssembly according to claim 1, it is characterized in that: The request to change CR is an output instruction issued by the scheduling logic, and the name of the request to change CR follows the format of {name}-{versionNo}; After the configuration is completed, the user develops the scheduling logic in the relevant language, compiles it into a wasm file, uploads it to the storage system, writes and submits the request change CR of WsspSchedulerConfiguration, and when wssp watch receives the request change CR, it loads the .wasm file into the WASM sandbox.

4. A method for extending a Kubernetes scheduler based on WebAssembly according to claim 3, characterized in that The Pod annotation is used for wssp / schedulefragment, and the logical scheduling is executed by changing the annotation value; If the Pod has wssp / schedulefragment, it means that the Pod will be processed using the scheduling logic in WsspSchedulerConfiguration. If the Pod does not have wssp / schedulefragment, it means that the Pod will be executed using the default scheduling logic.

5. According to a method for extending a Kubernetes scheduler based on WebAssembly according to claim 4, it is characterized in that: The scheduling logic process calls WSSP SI through extension points, then calls the scheduling logic (WASM extension code), and returns the result after execution; The returning result includes forming a scheduling closed loop and outputting a scheduling result.

6. A Kubernetes scheduler device based on WebAssembly, characterized in that: Manager module, used to monitor the scheduling logic and monitor the scheduling logic data; The VM module is used to encapsulate functions and is responsible for loading and executing .wasm files and managing sandbox resources; SI module is used to connect the interaction between scheduling plug-ins and scheduling logic.

7. A WebAssembly-based Kubernetes scheduler extension device according to claim 6, characterized in that: The Manager module is used to monitor the request of the scheduling logic WsspSchedulerConfiguration to change the CR and maintain the WASM sandbox deployment logic; The WASM sandbox uses WASM technology to allow users to choose multiple languages ​​to write scheduling logic; The scheduling logic is enhanced through Pod annotations.

8. The WebAssembly-based Kubernetes scheduler extension device according to claim 6, characterized in that: The VM module provides a unified encapsulation of WASM Runtime and is responsible for loading and executing .wasm files and resource management of WASM sandbox instances. The resource management of the WASM sandbox instance, if WSSP watch requests to change CR, the .wasm file is loaded into the WASM sandbox instance.

9. The WebAssembly-based Kubernetes scheduler extension device according to claim 8, characterized in that: The SI module provides an external usage interface, which can be regarded as the glue layer for interaction between the scheduling plug-in and the scheduling logic. The plug-in implements the extension point of the Scheduling Framework and transfers the call to the scheduling logic through the SI module.

10. The WebAssembly-based Kubernetes scheduler extension device according to claim 9, characterized in that: The call is transferred to the scheduling logic via the SI module. If the Pod does not have wssp / schedulefragment, the Pod will be executed using the default scheduling logic and call the SI module through extension points, that is, the default logic scheduling will be used.

Images