A method and apparatus for extending a Kubernetes scheduler based on WebAssembly

Abstract

The present application relates to Kubernetes cloud platform technical field, especially relate to a kind of method and device for extending Kubernetes scheduler based on WebAssembly, by managing and configuring deployment logic, load WASM Runtime, generate WASM Module instruction after loading, and then generate WASM sandbox instance, finally prepare to execute scheduling, by the mutual cooperation between Manager module, VM module and SI module to complete the realization of scheduling logic, while solving the problems of scheduling extension, support hot update, it is convenient and safe to support user extension scheduling logic.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Kubernetes cloud platform, and in particular to a method and device for extending Kubernetes scheduler based on WebAssembly.

[0002] The present application relates to the technical field of Kubernetes cloud platform, and in particular to a method and device for extending Kubernetes scheduler based on WebAssembly. BACKGROUND

[0003] Kubernetes is a powerful container orchestration platform, and its scheduler is responsible for assigning Pods to available nodes. The default scheduler kube-scheduler is a component provided by Kubernetes, which adopts a scheduling strategy based on Predicates and Priorities to ensure that Pods are arranged on nodes that meet resource requirements, affinity constraints, and other conditions. Although kube-scheduler already provides a default scheduling strategy, this strategy may not meet the needs of users with special requirements. For example, users may need to schedule Pods according to custom business rules or optimize specific types of workloads. Therefore, Kubernetes v1.15 onwards allows users to implement their own scheduling strategies through the Scheduling Framework. However, users still face some pain points, 1) the threshold for developing a complete Scheduling Framework extension program is high, and users need to have a good understanding of the development, deployment, and other aspects of the scheduler extension mechanism, and the process from development to debugging to online is complicated and prone to errors; 2) in some scenarios, such as public cloud hosting versions of Kubernetes, users almost have no possibility to deploy Scheduling Framework extension programs on machines.

[0004] Kubernetes, users almost have no possibility to deploy Scheduling Framework extension programs on machines. This patent attempts to provide a lighter and simpler way to extend scheduling, making it easier for users to extend Kubernetes' scheduling strategy and solve the above problems.

[0005] Kubernetes' scheduling strategy, thereby solving the above problems.

[0006] For example, the existing patent with application publication number CN116841718A provides a method and device for extending Kubernetes scheduler based on WebAssembly.

[0007] The physical resource scheduling method and the scheduler of Kubernetes obtain scheduling information in the Kubernetes cluster, the scheduling information including Pod information of a to-be-scheduled Pod and load information of each node; then, each node is traversed, the Pod information of the to-be-scheduled Pod and the load information of the current node are input into a scheduling evaluation model, and evaluation information output by the scheduling evaluation model when the to-be-scheduled Pod is scheduled to the current node is obtained; preferably, the to-be-scheduled Pod is scheduled based on the evaluation information corresponding to each node. The method introduces the load information of the current node, avoids the abnormal situation of the physical resource on the node caused by only considering the hardware resource, guarantees the safe operation of the Kubernetes cluster, and improves the stability of the Kubernetes cluster.

[0008] However, the above patent has the following problems: the scheduling Pod control scheduling logic includes Pod information and load information of each node, and does not solve the problems of scheduling extension and hot update; secondly, although three ways of customizing a scheduler, a scheduler extension program and a scheduling framework are provided, the implementation of the scheme is relatively complicated, and hot update is not supported.

[0009] The present application solves the problems of scheduling extension and supports hot update. Secondly, for the scene, the present application mainly faces the general scheduling scene of Kubernetes in public clouds, private clouds and the like, so that users can customize scheduling logic. In particular in the public cloud scene, users generally cannot replace the image of kube-scheduler, and finally the present application is applicable to the scene of extending the scheduling capability of Kubernetes, converts the request of the scheduling extension point into a WASM sandbox instance for execution based on the scheduling framework, supports users to extend scheduling logic conveniently and safely. Meanwhile, WSSP supports hot update of scheduling logic and supports different Pods using different scheduling logic. SUMMARY

[0010] This section aims to summarize some aspects of the embodiments of the present application and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of the specification of the present application to avoid obscuring the purpose of this section, the abstract and the title, and such simplifications or omissions cannot be used to limit the scope of the present application.

[0011] To solve the above technical problems, the main purpose of the present application is to provide a method for extending Kubernetes scheduler based on WebAssembly, which comprises the steps of:

[0012] The method for extending Kubernetes scheduler based on WebAssembly comprises the steps of:

[0013] S1, establishing a sandbox scheduling plug-in with kube-scheduler as the source code;

[0014] S2, modify NewSchedulerCommand as a configuration file for kube-scheduler;

[0015] S3, after the configuration is completed, the user develops the scheduling logic by supporting the programming language compiled into WebAssembly bytecode, compiles it into a.wasm file, and uploads it to the storage system, writes and submits the request change CR of WsspSchedulerConfiguration;

[0016] S4, when the request change CR is output, the Pod annotation enhanced startup is annotated with wssp and schedulefragment, and the annotation value is changed to the name of the request change CR of WsspSchedulerConfiguration;

[0017] S5, after the request change is completed, the scheduler is scheduled, wherein the WSSPSI is called through the extension points, then the scheduling logic is called, the result is returned after execution, and the output is output.

[0018] As a preferred scheme of the method for extending the Kubernetes scheduler based on WebAssembly, wherein:

[0019] The sandbox scheduling plug-in is denoted as wssp;

[0020] The wssp is based on the source code of kube-scheduler, and the NewSchedulerCommand is modified

[0021] The NewSchedulerCommand inputs the sandbox scheduling plug-in;

[0022] The configuration of the wssp itself is implemented through the parameters schedulefragment in KubeSchedulerConfigurations and args, and the default value of schedulefragment is set to defalut:v1.0.0, and the schedulefragment state is non-required, and the default scheduling logic default-v1.0.0.wasm is loaded into the WASM sandbox during the initialization of the plug-in.

[0023] As a preferred scheme of the method for extending the Kubernetes scheduler based on WebAssembly, wherein:

[0024] The request change CR is a scheduling logic publishing output instruction, and the name of the request change CR follows the format of {name}-{versionNo};

[0025] After the configuration is completed, the user develops the scheduling logic through the relevant language, compiles it into a wasm file, and uploads it to the storage system, writes and submits a request change CR of WsspSchedulerConfiguration, and when the wssp watch receives the request change CR, loads the.wasm file into the WASM sandbox.

[0026] As a preferred scheme of the method for extending the Kubernetes scheduler based on WebAssembly, in the method:

[0027] The Pod annotation is used for wssp / schedulefragment, and the scheduling logic is executed by changing the annotation value;

[0028] If the Pod exists wssp / schedulefragment, it indicates that the Pod will use

[0029] The scheduling logic processing in WsspSchedulerConfiguration, and if the Pod does not exist wssp / schedulefragment, it indicates that the Pod will use the default scheduling logic for execution.

[0030] As a preferred scheme of the method for extending the Kubernetes scheduler based on WebAssembly, in the method:

[0031] The scheduling logic processing, the WSSP SI is called through the extension points, then the scheduling logic is called, the scheduling logic is implemented by the WASM extension code, and a result is returned after execution;

[0032] The returned result includes forming a scheduling closed loop and outputting a scheduling result.

[0033] As a preferred scheme of the device for extending the Kubernetes scheduler based on WebAssembly, in the device:

[0034] A Manager module is configured to listen to the scheduling logic and monitor the scheduling logic data;

[0035] A VM module is configured to encapsulate the function, and is responsible for loading, executing the.wasm file, and managing the resources of the sandbox;

[0036] An SI module is configured to connect the interaction between the scheduling plug-in and the scheduling logic.

[0037] As a preferred scheme of the device for extending the Kubernetes scheduler based on WebAssembly, in the device:

[0038] The Manager module is used for listening to the request change CR of the scheduling logic WsspSchedulerConfiguration and maintaining the WASM sandbox deployment logic.

[0039] The WASM sandbox is used for writing the scheduling logic by a user in multiple languages through the WASM technology.

[0040] The scheduling logic is enhanced through a Pod annotation.

[0041] As a preferred scheme of the device for extending the Kubernetes scheduler based on WebAssembly, wherein:

[0042] The VM module provides uniform encapsulation of the WASM Runtime, is responsible for loading and executing the.wasm file, and resource management of the WASM sandbox instance.

[0043] The resource management of the WASM sandbox instance loads the.wasm file into the WASM sandbox instance when the WSSP watch requests the change CR.

[0044] As a preferred scheme of the device for extending the Kubernetes scheduler based on WebAssembly, wherein:

[0045] The SI module provides an external use interface, which can be regarded as a glue layer for interaction between the scheduling plug-in and the scheduling logic, the plug-in implements the extension points of the Scheduling Framework, and calls the scheduling logic via the SI module.

[0046] As a preferred scheme of the device for extending the Kubernetes scheduler based on WebAssembly, wherein:

[0047] The call via the SI module is called into the scheduling logic, if the Pod does not exist wssp / schedulefragment, the Pod will be executed by using the default scheduling logic, the SI module is called and run through the extension points, that is, the default scheduling logic is used.

[0048] The present application has the following beneficial effects:

[0049] The application solves the problems of scheduling expansion and supporting hot update, and secondly, for the scene, the patent mainly faces the general scheduling scene of Kubernetes such as public cloud and private cloud, so that users can customize scheduling logic. Especially in the public cloud scene, users generally cannot replace the image of kube-scheduler, and finally the patent is suitable for the scene of expanding the scheduling capability of Kubernetes, based on the scheduling framework, the request of the scheduling extension point is transferred to the WASM sandbox instance for execution, which supports users to extend the scheduling logic conveniently and safely. At the same time, WSSP supports hot updating of scheduling logic and supports different Pods using different scheduling logic. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings. Among them:

[0051] Figure 1 A system flowchart of a method for implementing the Kubernetes scheduler based on WebAssembly according to the application;

[0052] Figure 2 A whole system composition diagram of the device for CDN customer business demand automatic test according to the application;

[0053] Figure 3 A scheduling scheme architecture diagram of the Kubernetes scheduler based on WebAssembly according to the application;

[0054] Figure 4 A use flowchart of the Kubernetes scheduler based on WebAssembly according to the application;

[0055] Figure 5 A device Pod annotation flowchart of the Kubernetes scheduler based on WebAssembly according to the application; DETAILED DESCRIPTION

[0056] In order to make the above-mentioned purposes, features and advantages of the application more obvious and easy to understand, the specific embodiments of the application will be described in detail below with reference to the drawings of the specification.

[0057] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without the specific details set forth in this description. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the present application.

[0058] Second, the "one embodiment" or "an embodiment" described herein as containing various features, structures, or characteristics can be combined with one or more other embodiments to form new embodiments, where the new embodiments do not include all of the features, structures, or characteristics of the one or more other embodiments.

[0059] Third, the present application is described in connection with exemplary diagrams. The exemplary diagrams are intended to aid in the description of the present application and are not intended to limit the scope of the present application. The present application should be construed to encompass all those variations and modifications that fall within the scope of the present application. In addition, the actual dimensions of the structures can be varied depending on the actual implementation.

[0060] Embodiment One

[0061] As shown in the figure, a method for extending Kubernetes scheduler based on WebAssembly includes: Figure 1

[0062] S1, establishing a sandbox scheduling plug-in based on the source code of kube-scheduler;

[0063] Wherein, the sandbox scheduling plug-in is denoted as wssp;

[0064] Further, the wssp is based on the source code of kube-scheduler, and the

[0065] NewSchedulerCommand input sandbox scheduling plug-in;

[0066] S2, modifying NewSchedulerCommand as a configuration file for kube-scheduler;

[0067] Wherein, the configuration of wssp itself is realized through the parameters schedulefragment in KubeSchedulerConfigurations and args, and the default value of schedulefragment is set as defalut:v1.0.0, and the schedulefragment state is non-required, and the default scheduling logic default-v1.0.0.wasm is loaded into the WASM sandbox during the initialization of the plug-in. ​

[0068] As Figure 5 shown, S3, after configuration is completed, the user develops scheduling logic through relevant language and compiles it into a.wasm file, and uploads it to the storage system, writes and submits a request change CR of WsspSchedulerConfiguration;

[0069] The request change CR is an output instruction for scheduling logic, and the name of the request change CR follows the format of {name} - {versionNo};

[0070] After configuration is completed, the user develops scheduling logic through a programming language that supports compilation into WebAssembly bytecode, and compiles it into a.wasm file, and uploads it to the storage system, writes and submits a request change CR of WsspSchedulerConfiguration;

[0071] When the wssp watch receives the request change CR, the.wasm file is loaded into the WASM sandbox.

[0072] Further, the Pod annotation is used for wssp / schedulefragment, and the scheduling logic is executed by changing the annotation value;

[0073] If the Pod has wssp / schedulefragment, it means that the Pod will use

[0074] the scheduling logic processing in WsspSchedulerConfiguration, and if the Pod does not have wssp / schedulefragment, it means that the Pod will use the default scheduling logic for execution.

[0075] Further, the Pod annotation enhancement supports annotating wssp / schedulefragment for the Pod, and the annotation value is the name of the WsspSchedulerConfiguration CR, indicating the Pod that executes the scheduling logic fragment.

[0076] S4, when the output request change CR is started, the Pod annotation enhancement is started, the wssp and schedulefragment are annotated, and the annotation value is changed to the name of the request change CR of WsspSchedulerConfiguration;

[0077] When the Pod has wssp / schedulefragment, it means that the Pod will use

[0078] The scheduling logic processing in WsspSchedulerConfiguration;

[0079] When the Pod does not exist wssp / schedulefragment, it indicates that the Pod will use the default scheduling logic for execution, that is, use the default logic scheduling. This step calls WSSP SI through extension points, and then calls the scheduling logic (WASM extension code). After execution, the result is returned. This step has the following conventions:

[0080] First, wssp determines which scheduling logic to use according to the default configuration and the Pod's annotation. The scheduling logic specified by the Pod's annotation has a higher priority than the default configuration.

[0081] Second, the scheduling will be blocked by a mutex when the WASM sandbox instance is produced.

[0082] S5, after the request change is completed, the scheduler is scheduled, wherein WSSP SI is called through extension points, and then the scheduling logic is called. After execution, the result is returned and output.

[0083] The scheduling logic processing calls WSSP SI through extension points, and then calls the scheduling logic (WASM extension code). After execution, the result is returned.

[0084] Further, the returned result includes forming a scheduling closed loop and outputting the scheduling result.

[0085] As shown in Figure 4 , the scheduling is performed through the overall method flow.

[0086] Embodiment Two

[0087] As shown in Figure 2 , an apparatus for extending a Kubernetes scheduler based on WebAssembly includes:

[0088] A Manager module for listening to scheduling logic and monitoring scheduling logic data;

[0089] The Manager module is configured to listen to request change CR of the scheduling logic WsspSchedulerConfiguration and maintain WASM sandbox scheduling logic.

[0090] The WASM sandbox provides users with multiple languages for writing scheduling logic through WASM technology.

[0091] The scheduling logic is enhanced through Pod annotation.

[0092] VM module, used to encapsulate functions, and responsible for loading, executing of.wasm file, and resource management of the sandbox;

[0093] The VM module provides a unified encapsulation of WASM Runtime, is responsible for loading, executing of.wasm file, and resource management of the WASM sandbox instance.

[0094] The resource management of the WASM sandbox instance, if the WSSP watch requests to change CR, the.wasm file is loaded into the WASM sandbox instance.

[0095] SI module, used to connect the interaction between the scheduling plug-in and the scheduling logic.

[0096] The SI module provides an external use interface, which can be regarded as a glue layer for interaction between the scheduling plug-in and the scheduling logic. The plug-in implements the extension points of the Scheduling Framework, and calls the scheduling logic through the SI module.

[0097] Further, the call is made to the scheduling logic through the SI module, and if the Pod does not exist wssp / schedulefragment, the Pod will use the default scheduling logic to execute, and the SI module is called through the extension points to run, that is, the default scheduling logic is used.

[0098] For example, prepare 5 virtual machines configured with 4-core CPU and 8G memory, and deploy a high-availability (3 Master nodes) Kubernetes cluster.

[0099] As shown in Figure 3 , the default scheduling logic.wasm is placed on the master host, and the CR WsspSchedulerConfiguration is edited and submitted. The CR name is default-v1, the storageType is hostPath, and the address is the.wasm file path.

[0100] Edit and submit KubeSchedulerConfigurations, pluginConfig adds an item, name is wssp-scheduler, args contains schedulefragment=default-v1, and replace kube-scheduler with wssp scheduler.

[0101] Write the go scheduling logic and compile to a.wasm file, named sample-v1, put it on the master host, edit and submit a CR WsspSchedulerConfiguration. The CR name is sample-v1, storageType is hostPath, and address is the path to the.wasm file.

[0102] Publish a Pod, observe the logs to see if the default-v1 scheduling logic is used.

[0103] Publish a Pod with annotations wssp / schedulefragment=sample-v1, observe the logs to see if the sample-v1 scheduling logic is used.

[0104] It is important to note that the construction and arrangement of the application shown in the various exemplary embodiments is illustrative only. Although only two embodiments have been described in detail in this disclosure, those skilled in the art who review this disclosure will readily appreciate that many modifications are possible in the light of the novel teachings provided herein. For example, the dimensions, scaling, proportions, and other parameters (e.g., temperature, pressure, etc.) of the various elements, including the size, shape, and orientation of the various components, the mounting arrangements, the use of materials, the colors, the orientation, etc. can be varied. For example, elements shown as integrally formed can be constructed of multiple parts or elements, the positions of elements can be reversed or otherwise changed, and the nature or number of elements or positions can be modified or changed. Therefore, all such modifications are intended to be included within the scope of the application. The order or sequence of any process or method steps can be varied or re-sequenced without departing from the scope of the application. Any "means plus function" clauses are intended to cover the structures described herein as performing the recited functions and not only structural equivalents but also equivalent structures. Other substitutions, modifications, changes, and omissions can be made in the design, operating conditions, and arrangement of the exemplary embodiments without departing from the scope of the application. Accordingly, the present application is not limited to the particular embodiments described and shown herein, but extends to equivalents of what is claimed, as well as to implementations that fall within the scope of the appended claims.

[0105] Furthermore, in order to provide a concise description of the exemplary embodiments, not all features of an actual implementation can be described (i.e., those pertaining to the best mode for carrying out the application, or those that are not necessary for an understanding of the application).

[0106] It is to be understood that the development of the particular implementations described herein was motivated by the desire to solve real-world problems, and as such the claimed implementations can be susceptible to further implementation while still being generically consistent with the descriptions provided herein. Specifically, although many of the examples provided herein describe one or more implementations with any particular feature, an individual feature can be replaced by alternative features within the scope of the application. Thus, features discussed in one example can be interchanged with features in another example. Any implementation of more than one feature disclosed herein is specifically referenced within the scope of the application.

[0107] It should be noted that the above examples are merely used to illustrate the technical solutions of the present application, rather than limit the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the present application, and all should be covered in the scope of the claims of the present application.

Claims

1. A method for extending the Kubernetes scheduler based on WebAssembly, characterized in that: include, S1. Build a sandbox scheduling plugin based on kube-scheduler source code; The sandbox scheduling plugin is denoted as wssp; The wssp is based on the kube-scheduler source code, modified by the NewSchedulerCommand input sandbox scheduling plugin; The configuration of WSSP itself is implemented through the parameter schedulefragment in KubeSchedulerConfigurations and args, and the default value of schedulefragment is set to default:v1.0.

0. The status of schedulefragment is optional. The default scheduling logic default-v1.0.0.wasm is loaded into the WASM sandbox during plugin initialization. S2. Modify NewSchedulerCommand as the configuration file for kube-scheduler; Pod annotations are used in wssp / schedulefragment, and the scheduling logic is executed by changing the annotation value. If the Pod has wssp / schedulefragment, it means that the Pod will use the scheduling logic in WsspSchedulerConfiguration. If the Pod does not have wssp / schedulefragment, it means that the Pod will use the default scheduling logic. S3. After configuration, users develop scheduling logic using a programming language that supports compilation to WebAssembly bytecode, compile it into a .wasm file, upload it to the storage system, and write and submit a request change CR for WsspSchedulerConfiguration. The request to change the CR is a scheduling logic output instruction, and the name of the request to change the CR follows the format {name}-{versionNo}; When WSSP watch receives a request to change the CR, it loads the .wasm file into the WASM sandbox; S4. When the output request changes the CR, the Pod starts with enhanced annotations, annotates wssp and schedulefragment, and changes the annotation value to the name of the request change CR in WsspSchedulerConfiguration. S5. After the request for change is completed, the scheduler is scheduled, which calls WSSP SI through extension points, then calls the scheduling logic, executes it, and returns and outputs the result.

2. The method for extending the Kubernetes scheduler based on WebAssembly according to claim 1, characterized in that: The scheduling logic process calls WSSP SI through extension points, which in turn calls the scheduling logic, which is implemented by WASM extension code. After execution, the result is returned. The returned results include forming a scheduling closed loop and outputting the scheduling results.

3. A WebAssembly-based Kubernetes scheduler extension device, used to implement the method of the WebAssembly-based Kubernetes scheduler extension according to any one of claims 1-2, characterized in that: The Manager module is used to listen to the scheduling logic and monitor the scheduling logic data. The VM module is used to encapsulate functions and is responsible for loading and executing .wasm files, as well as managing sandbox resources. The SI module is used to connect the interaction between the scheduling plugin and the scheduling logic.

4. A WebAssembly-based Kubernetes scheduler device as described in claim 3, characterized in that: The Manager module is used to monitor requests for changes to the scheduling logic WsspSchedulerConfiguration (CR) and maintain the WASM sandbox scheduling logic. The WASM sandbox allows users to choose from multiple languages ​​to write scheduling logic using WASM technology; The scheduling logic is enhanced through Pod annotations.

5. The WebAssembly-based Kubernetes scheduler device according to claim 4, characterized in that: The VM module provides a unified encapsulation of the WASM Runtime, and is responsible for loading and executing .wasm files, as well as managing the resources of WASM sandbox instances; Load the .wasm file into the WASM sandbox instance.

6. The WebAssembly-based Kubernetes scheduler device according to claim 5, characterized in that: The SI module provides an external user interface and can be seen as a glue layer for interaction between the scheduling plugin and the scheduling logic. The plugin implements the extension point of the Scheduling Framework and calls the scheduling logic through the SI module.

7. The WebAssembly-based Kubernetes scheduler device according to claim 6, characterized in that: The call is routed to the scheduling logic via the SI module. If the Pod does not have wssp / schedulefragment, the Pod will execute using the default scheduling logic, which is to call the SI module through extension points, i.e., use the default scheduling logic.

Images