Container scheduling module and scheduling method based on user-defined index configuration file

Abstract

The invention discloses a container scheduling module based on a user-defined index configuration file, which comprises an analysis module, an index acquisition module, a scheduling algorithm module, a data information file and a configuration file, and is characterized in that the analysis module analyzes the configuration file to obtain user-defined index related information; the index acquisition module acquires corresponding numerical values according to the self-defined index and the built-in scheduling index and generates a data information file; the scheduling algorithm module obtains built-in scheduling indexes and a scoring algorithm rule of each index, obtains a final scheduling index and a corresponding scheduling algorithm rule in combination with a self-defined scheduling index provided by the analysis module and the corresponding scoring algorithm rule, and then analyzes a data information file of each node; the analyzed data is substituted into a final scheduling index and a corresponding scheduling algorithm rule, the score value of each node in the cluster is calculated, the node scores are sorted, finally, the node with the highest score is selected, and the container is scheduled to the node with the highest score to complete the scheduling process.

Description

Technical Field

[0001] The present invention relates to a container scheduling module and a scheduling method based on a custom indicator configuration file in a container (such as Docker and other containers that comply with the OCI standard) cluster environment. Background Art

[0002] With the development of cloud computing technology, the market demand for efficient and intelligent container scheduling algorithms is growing. However, existing algorithms have problems of low efficiency and high cost when scheduling containers for custom indicators. This paper proposes a new container scheduling algorithm, which aims to improve the flexibility and efficiency of container scheduling, reduce development and operation and maintenance costs, and improve the efficiency of container scheduling.

[0003] In existing container scheduling engines (such as Kubernetes), when creating a container application (such as Docker and other containers that comply with the OCI standard), the optimal schedulable node will be calculated based on the container's built-in scheduling algorithm. The existing container scheduling algorithm needs to develop extension plug-ins to implement container scheduling based on custom indicators (such as task priority, battery power, height, length, speed, etc.). The process is cumbersome and complex, with low efficiency and poor versatility. Summary of the invention

[0004] The purpose of the present invention is to provide a container scheduling module and scheduling method based on a custom indicator configuration file. Based on a built-in container scheduling engine, the module obtains effective custom indicators, calculation expressions and weights of each indicator by reading and parsing the configuration file, and scores all nodes that meet the scheduling conditions in combination with built-in scheduling indicators and algorithm rules. When executing the scheduling strategy, the container is scheduled to the node with the highest score, so as to achieve an efficient and fast container scheduling strategy.

[0005] The object of the present invention is achieved through the following technical solutions:

[0006] A container scheduling module based on a custom indicator configuration file includes a parsing module, an indicator collection module, a scheduling algorithm module, a data information file and a configuration file. The parsing module and the scheduling algorithm module are functional modules on the main node of the container cluster, and the indicator collection module is a functional module of all nodes.

[0007] The parsing module is triggered when a container is created. It reads and parses the configuration file, sends the obtained custom indicator name, custom indicator weight, and custom indicator score calculation expression to the scheduling algorithm module, and also sends the custom indicator name to the indicator collection module of other nodes in the cluster. The parsing module supports reading and parsing json and xml data files.

[0008] After receiving the custom indicator name sent by the master node parsing module, the indicator collection module triggers the indicator collection function to obtain the latest scheduling indicator value of each node, including the value corresponding to the custom indicator name obtained from the parsing module and the node built-in indicator;

[0009] The indicator collection module on the cluster master node also needs to obtain container status information and node information in the cluster;

[0010] The indicator collection module saves the collection results in the form of data information files and sends them to the scheduling algorithm module of the master node;

[0011] The scheduling algorithm module obtains the built-in scheduling indicators and the scoring algorithm rules for each indicator, and combines the custom scheduling indicators and the corresponding scoring algorithm rules provided by the parsing module to obtain the final scheduling indicators and the corresponding scheduling algorithm rules. Then, it parses the data information files of each node, substitutes the parsed data into the final scheduling indicators and the corresponding scheduling algorithm rules, calculates the score value of each node in the cluster, and sorts the node scores. Finally, it selects the node with the highest score and schedules the container to the node with the highest score to complete the scheduling process.

[0012] Preferably, the parsing module saves the obtained custom indicator name, the weight of the custom indicator and the score calculation expression of the custom indicator in a "key-value" format, and sends the saved "key-value" data to the scheduling algorithm module.

[0013] Preferably, the indicator collection module on the master node of the cluster obtains the container status information in the cluster including the container name, bound nodes, required CPU and memory resource size; and obtains the node information in the cluster including the node status, node name, and whether the node is abnormal.

[0014] Preferably, the data information file is saved in the form of a json file, and is saved in a "key-value" format after being interpreted by the scheduling algorithm module.

[0015] Preferably, a container scheduling method of a container scheduling module based on a custom indicator configuration file comprises the following steps:

[0016] Step 1: When a new container is deployed, the scheduling algorithm module and parsing module on the master node are triggered;

[0017] Step 2: The scheduling algorithm module obtains the built-in scheduling indicators and the scoring algorithm rules for each indicator; the parsing module reads and parses the configuration file to obtain the customized scheduling indicators and the corresponding scoring algorithm rules;

[0018] Step 3: The scheduling algorithm module integrates all scheduling indicators obtained in step 2 to obtain the final scheduling indicators and corresponding scheduling algorithm rules;

[0019] Step 4: The indicator collection module collects the latest information of all nodes and indicators in the cluster;

[0020] Step 5: The scheduling algorithm module selects a scheduling indicator for which the score has not been calculated;

[0021] Step 6: The scheduling algorithm module calculates the score of the indicator at each node according to the algorithm rules of the scheduling indicator;

[0022] Step 7: The scheduling algorithm module determines whether all scheduling indicator scores have been calculated. If yes, execute step 8; if not, execute step 5;

[0023] Step 8: The scheduling algorithm module obtains all indicator scores on a node;

[0024] Step 9: The scheduling algorithm module calculates the final score of the current node based on the weight and calculation expression corresponding to each indicator;

[0025] Step 10: The scheduling algorithm module determines whether all node indicator scores have been calculated. If yes, execute step 11; if not, execute step 8;

[0026] Step 11: The scheduling algorithm module sorts the nodes according to their index scores;

[0027] Step 12: Deploy the container to the node with the highest score.

[0028] The beneficial effects of the present invention are:

[0029] The present invention obtains the name, weight, and score calculation expression of the custom indicator by reading and parsing the custom indicator configuration file when creating a container. In combination with the built-in indicator information of the node, a scheduling algorithm for the custom indicator (such as task priority, battery power, height, length, speed, etc.) can be quickly defined, which changes the current situation in which the prior art can only calculate the optimal schedulable node based on the built-in scheduling algorithm. Compared with the built-in scheduling algorithm in the existing container scheduling engine (such as kubernetes) and the original implementation method through the development of extension plug-ins, the container scheduling algorithm based on the custom indicator configuration file provided by the present invention has obvious advantages in flexibility, convenience, and efficiency. The specific advantages are as follows:

[0030] Reduced development costs: The present invention uses a method based on a custom indicator configuration file to implement the expansion of the container scheduling method, which can avoid the development cost of custom indicator extension plug-ins and reduce R&D cost investment;

[0031] Reduced resource usage: The present invention uses a method based on a custom indicator configuration file to implement the expansion of the container scheduling method, without the need to run additional extension plug-ins, thereby reducing the resource usage of the plug-ins;

[0032] Improved resource utilization: The present invention uses a method based on a custom indicator configuration file to expand the container scheduling method. The spare computing resources can be allocated to other urgently needed task applications, effectively improving resource utilization;

[0033] Enhanced scalability: The present invention uses a method based on a custom indicator configuration file to implement the expansion of the container scheduling method. R&D personnel or operation and maintenance personnel can meet actual business needs by expanding or modifying custom indicator algorithm rules. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Scheduling module composition diagram;

[0035] Figure 2 Scheduling algorithm flow chart;

[0036] Figure 3 Scheduling module deployment diagram. DETAILED DESCRIPTION

[0037] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0038] A container scheduling module based on a custom indicator configuration file shown in this embodiment is a scheduling method for scheduling the container to the optimal node after the user creates a container application. Figure 1 As shown in the figure, it includes parsing module, indicator collection module, scheduling algorithm module, data information file and configuration file, etc. Figure 3 As shown in the figure, the container cluster nodes are divided into master nodes and working nodes. The master node is the cluster control center, responsible for managing the status and configuration information of the entire cluster. The parsing module and the scheduling algorithm module are functional modules on the master node of the container cluster. The indicator collection module is a functional module of all nodes. The configuration file is a file used on the master node, and there is only one. Each node will generate a data information file.

[0039] The parsing module defines the method of reading and parsing the custom indicator configuration file, the storage method of the parsed data, and the content and method of sending the data.

[0040] The parsing module is triggered in the main node of the container cluster when the user creates a container application. It reads and parses the configuration file and supports reading and parsing json and xml data files. The custom indicator name, the weight of the custom indicator, and the score calculation expression of the custom indicator obtained by parsing the configuration file are saved in a "key-value" manner, and the saved "key-value" data is sent to the scheduling algorithm module. At the same time, the parsed custom indicator name is stored separately and sent to the indicator collection module of other nodes in the cluster.

[0041] The indicator collection module defines the content of the collected data, including the data corresponding to the custom indicator name obtained from the analysis module, the node information in the cluster, and the container information, and defines the data storage method, the content and method of sending data.

[0042] After the parsing module parses the configuration file, all nodes in the cluster receive the custom indicator name sent by the master node parsing module and trigger the indicator collection function to obtain the latest scheduling indicator value of each node, including the value corresponding to the custom indicator name obtained from the parsing module and the node built-in indicators, such as CPU, memory, disk usage percentage, etc.

[0043] In addition to obtaining the above data, the indicator collection module on the cluster master node also obtains the container status information in the cluster, including the container name, the bound node (if bound), the required CPU and memory resource size, etc.; obtains the node information in the cluster, including the node status, node name, whether the node is abnormal, etc.

[0044] The indicator collection module of each node in the cluster saves the collection results in the form of data information files and sends the data information files to the scheduling algorithm module of the master node.

[0045] The scheduling algorithm module defines the reading and parsing of data information files and data processing algorithms.

[0046] The scheduling algorithm module obtains the built-in scheduling indicators and the scoring algorithm rules for each indicator, and combines the custom scheduling indicators and corresponding scoring algorithm rules provided by the parsing module to obtain the final scheduling indicators and corresponding scheduling algorithm rules; then parses the data information files of each node, and obtains the latest scheduling indicator values ​​of each node in the cluster collected by the indicator collection module, the container status information in the cluster, and the node information in the cluster, which are saved in a "key-value" manner, and the parsed data is substituted into the final scheduling indicator and the corresponding scheduling algorithm rules, and the score values ​​of each node in the cluster are calculated respectively, and the node scores are sorted. Finally, the node with the highest score is selected, and the container is scheduled to the node with the highest score to complete the scheduling process.

[0047] The data information file stores the latest scheduling index value of each node in the cluster, the container status information in the cluster, and the node information in the cluster collected by the index collection module. They are saved in the form of json files and sent to the scheduling algorithm module of the master node. Among them, the latest scheduling index value of the node is collected by each node, and the container status information in the cluster and the node information in the cluster are collected by the master node.

[0048] The configuration file stores the custom indicator name, the weight of the custom indicator, and the score calculation expression of the custom indicator.

[0049] Before triggering the scheduling, the configuration file describes the custom indicator name, the weight of the custom indicator, and the score calculation expression of the custom indicator in the form of a file. The configuration file can be edited and modified when the cluster is running, but it is necessary to ensure that the content is consistent with the expected content each time the scheduling is triggered. The configuration file supports json or xml format.

[0050] See also Figure 2 As shown in the flowchart of the scheduling algorithm, the specific process of the scheduling module processing is as follows:

[0051] Step 1: When a new container is deployed, the scheduling algorithm module and parsing module on the master node are triggered.

[0052] Step 2: The scheduling algorithm module obtains the built-in scheduling indicators and the scoring algorithm rules for each indicator. The algorithm rules are divided into score calculation expressions and weights; the parsing module reads and parses the configuration file to obtain the customized scheduling indicators and corresponding scoring algorithm rules.

[0053] Step 3: The scheduling algorithm module integrates all the scheduling indicators obtained in step 2, obtains the final scheduling indicators and corresponding scheduling algorithm rules, and temporarily stores them in the form of data.

[0054] Step 4: The indicator collection module collects the latest information of all nodes and indicators in the cluster.

[0055] Step 5: The scheduling algorithm module selects a scheduling indicator without calculating the score.

[0056] Step 6: The scheduling algorithm module calculates the score of the indicator at each node according to the algorithm rules of the scheduling indicator.

[0057] Step 7: The scheduling algorithm module determines whether all scheduling indicator scores have been calculated. If yes, execute step 8; if not, execute step 5.

[0058] Step 8: The scheduling algorithm module obtains all indicator scores on a node.

[0059] Step 9: The scheduling algorithm module calculates the final score of the current node based on the weight and calculation expression corresponding to each indicator.

[0060] Step 10: The scheduling algorithm module determines whether all node indicator scores have been calculated. If yes, execute step 11; if not, execute step 8.

[0061] Step 11: The scheduling algorithm module sorts nodes according to their index scores.

[0062] Step 12: Deploy the container to the node with the highest score.

[0063] It is understandable that those skilled in the art can make equivalent substitutions or changes based on the technical solution and inventive concept of the present invention, and all these changes or substitutions should fall within the protection scope of the claims attached to the present invention.

Claims

1. A container scheduling module based on a custom indicator configuration file, including a parsing module, an indicator collection module, a scheduling algorithm module, a data information file and a configuration file. The parsing module and the scheduling algorithm module are functional modules on the main node of the container cluster, and the indicator collection module is a functional module of all nodes, characterized in that: The parsing module is triggered when a container is created. It reads and parses the configuration file, sends the obtained custom indicator name, custom indicator weight, and custom indicator score calculation expression to the scheduling algorithm module, and also sends the custom indicator name to the indicator collection module of other nodes in the cluster. The parsing module supports reading and parsing json and xml data files. After receiving the custom indicator name sent by the master node parsing module, the indicator collection module triggers the indicator collection function to obtain the latest scheduling indicator value of each node, including the value corresponding to the custom indicator name obtained from the parsing module and the node built-in indicator; The indicator collection module on the cluster master node also needs to obtain container status information and node information in the cluster; The indicator collection module saves the collection results in the form of data information files and sends them to the scheduling algorithm module of the master node; The scheduling algorithm module obtains the built-in scheduling indicators and the scoring algorithm rules for each indicator, and combines the custom scheduling indicators and the corresponding scoring algorithm rules provided by the parsing module to obtain the final scheduling indicators and the corresponding scheduling algorithm rules. Then, it parses the data information files of each node, substitutes the parsed data into the final scheduling indicators and the corresponding scheduling algorithm rules, calculates the score value of each node in the cluster, and sorts the node scores. Finally, it selects the node with the highest score and schedules the container to the node with the highest score to complete the scheduling process.

2. According to claim 1, a container scheduling module based on a custom indicator configuration file is characterized in that The parsing module saves the obtained custom indicator name, custom indicator weight and custom indicator score calculation expression in "key-value" mode, and sends the saved "key-value" data to the scheduling algorithm module. The parsing module supports the reading and parsing of configuration files in json or xml format.

3. According to claim 1, a container scheduling module based on a custom indicator configuration file is characterized in that The indicator collection module on the master node of the cluster obtains the container status information in the cluster, including the container name, bound node, required CPU and memory resource size; obtains the node information in the cluster, including the node status, node name, and whether the node is abnormal.

4. According to claim 1, a container scheduling module based on a custom indicator configuration file is characterized in that The data information file is saved in the form of a json file and saved in a "key-value" format after being interpreted by the scheduling algorithm module.

5. A container scheduling method of a container scheduling module based on a custom indicator configuration file according to any one of claims 1 to 4, comprising the following steps: Step 1: When a new container is deployed, the scheduling algorithm module and parsing module on the master node are triggered; Step 2: The scheduling algorithm module obtains the built-in scheduling indicators and the scoring algorithm rules for each indicator; the parsing module reads and parses the configuration file to obtain the customized scheduling indicators and the corresponding scoring algorithm rules; Step 3: The scheduling algorithm module integrates all scheduling indicators obtained in step 2 to obtain the final scheduling indicators and corresponding scheduling algorithm rules; Step 4: The indicator collection module collects the latest information of all nodes and indicators in the cluster; Step 5: The scheduling algorithm module selects a scheduling indicator for which the score has not been calculated; Step 6: The scheduling algorithm module calculates the score of the indicator at each node according to the algorithm rules of the scheduling indicator; Step 7: The scheduling algorithm module determines whether all scheduling indicator scores have been calculated. If yes, execute step 8; if not, execute step 5; Step 8: The scheduling algorithm module obtains all indicator scores on a node; Step 9: The scheduling algorithm module calculates the final score of the current node based on the weight and calculation expression corresponding to each indicator; Step 10: The scheduling algorithm module determines whether all node indicator scores have been calculated. If yes, execute step 11; if not, execute step 8; Step 11: The scheduling algorithm module sorts the nodes according to their index scores; Step 12: Deploy the container to the node with the highest score.

Images