Method for realizing multi-tenant task scheduling by using XXL-JOB based on Kubernetes
By using Kubernetes and XXL-JOB in a multi-tenant environment for data governance, the problems of insufficient resource isolation and complex management are solved, efficient management of task scheduling and reasonable allocation of resources are achieved, and task execution efficiency and isolation in a multi-tenant environment are improved. sex.
Patent Information
- Application Number
- CN202411347124.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-26
- Publication Date
- 2025-01-10
- Estimated Expiration
- 2044-09-26
AI Technical Summary
Traditional task scheduling has problems such as insufficient resource isolation and complex management in the multi-tenant environment of data governance, resulting in unreasonable resource allocation and failed task scheduling.
Using Kubernetes cluster to build a foundation for high availability and elastic scaling, combined with the XXL-JOB scheduling platform, realize resource monitoring and allocation through the Kubernetes cluster management center, dynamically configure task executor resources, and use XXL-JOB's lightweight distributed task scheduling Functions to achieve efficient management and isolated execution of tasks.
It improves the efficiency and reliability of task scheduling in a multi-tenant environment, realizes the rational use of resources and the isolated execution of tasks under different tenants, reduces the inclination of node resource use, and improves resource utilization.
Smart Images

Figure CN119292743A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of task scheduling, and in particular to a method for implementing multi-tenant task scheduling based on Kubernetes using XXL-JOB. Background Art
[0002] In the era of digital economy, the rise of big data platforms and industrial Internet has led to the continuous mining of the value of data. Governments and enterprises are accelerating the construction of digital transformation, and by building data lake platforms, they are unifying data governance, integrated development, services and visualization. This helps implement big data and data middle-end strategies, realize data-driven business intelligence, and promote digital transformation and upgrading of governments and enterprises and business model innovation. Effective management and efficient concurrent execution of tasks such as metadata collection, quality inspection, aggregation, data aggregation, data labeling, data security desensitization and data identification have become the key to data governance. However, traditional task scheduling has problems such as insufficient resource isolation and complex management in a multi-tenant environment of data governance.
[0003] To solve these problems, Kubernetes, as a powerful container orchestration platform, provides efficient and flexible solutions for resource management, deployment and expansion. XXL-JOB is a lightweight distributed task scheduling platform, which has been widely used in the field of task scheduling for its ease of use, high performance and scalability.
[0004] Based on this, by combining the advantages of Kubernetes and XXL-JOB, a method is involved to better solve the problems of insufficient resource isolation and complex management of traditional task scheduling in a data governance multi-tenant environment, and improve the efficiency and reliability of task scheduling. At the same time, by leveraging the development of big data platforms and the industrial Internet, the value of data can be further explored to promote the digital transformation and upgrading of governments and enterprises and innovation of business models. Summary of the invention
[0005] In view of the needs and shortcomings of current technological development, the present invention provides a method for implementing multi-tenant task scheduling using XXL-JOB based on Kubernetes, which solves many problems of traditional task scheduling in a data governance multi-tenant environment, such as unreasonable resource allocation and insufficient resource isolation, insufficient task isolation and scheduling failure, and complex management.
[0006] The present invention provides a method for implementing multi-tenant task scheduling based on Kubernetes using XXL-JOB, and the technical solution adopted to solve the above technical problems is as follows:
[0007] A method for implementing multi-tenant task scheduling using XXL-JOB based on Kubernetes, comprising the following steps:
[0008] S1. Build and configure a highly available and elastically scalable Kubernetes cluster to provide basic support for XXL-JOB deployment and task scheduling; label the nodes that execute tasks in the Kubernetes cluster and deploy application modules, XXL-JOB scheduling center and executors that the data governance platform depends on, and integrate the Liquibase tool to automatically create the database table structure to support executor registration;
[0009] S2. Realize resource monitoring and allocation through the Kubernetes cluster management center, use XXL-JOB to create and manage executor instances, and write executor information into the scheduling resource configuration table;
[0010] S3. Create computing tasks and select resources through the front-end interface of the Kubernetes cluster management center, bind the computing tasks to the selected resources and generate a scheduling policy, and finally register the scheduling policy information to the scheduling center to manage and execute computing tasks.
[0011] Optionally, the step S1 specifically includes the following operations:
[0012] S110. Build and configure the Kubernetes cluster to ensure that the cluster has high availability and elastic scalability. Then, configure the network and storage resources of the Kubernetes cluster according to actual needs to provide basic support for the deployment and task scheduling of XXL-JOB on the Kubernetes cluster.
[0013] S120. In the Kubernetes cluster, label the nodes used to execute tasks with the datalake-scheduler label. When creating task executors for different tenants, deploy the created executors to the nodes with the datalake-scheduler label.
[0014] S130. In the Kubernetes cluster, deploy the application modules, XXL-JOB scheduling center, and executors that the data governance platform depends on, including:
[0015] When deploying the XXL-JOB scheduling center, a XXL-JOB scheduling center instance is automatically created. The XXL-JOB scheduling center instance is deployed as an independent Pod, and uses the deployment type and multiple copies. The name of each Pod is prefixed with "idlf-scheduler";
[0016] When deploying an executor, the executor is deployed as an independent Pod, using the deployment type and multiple replicas. The Pod name of each executor is prefixed with "idlf-executor-depl";
[0017] When the executor is registered to the XXL-JOB dispatch center, the executor is registered using its corresponding APPNAME;
[0018] In the process of deploying the XXL-JOB scheduling center, the Liquibase tool is integrated to automatically create the database table structure. At the same time, the default APPNAME of the application module that the data governance platform depends on is inserted into the database table for automatic registration by the executor.
[0019] Further optionally, the step S2 specifically includes the following operations:
[0020] S210. The platform administrator creates a workspace in the tenant management module of the Kubernetes cluster management center and assigns an administrator to the workspace.
[0021] S220, adding task scheduling resources, specifically including:
[0022] S221. The administrator or user initiates the creation of a task executor through the front-end interface of the Kubernetes cluster management center. The Kubernetes cluster management center calls the Prometheus interface to obtain the CPU and memory usage of the nodes with the datalake-scheduler label and calculates the CPU usage and memory usage of each node within one day.
[0023] S222. The Kubernetes cluster management center calculates the remaining CPU and memory resources of each node based on the CPU usage and memory usage of each node, and displays them on the front-end interface for user reference;
[0024] S223. The user checks the remaining CPU and memory resources of each node on the front-end interface, and configures resources for various task executor instances in the executor according to actual needs;
[0025] S224. When the user completes filling in the resource configuration, the filled information is returned to the management center backend; the backend first performs resource verification based on the filled information and the remaining resources of the node, and creates APPNAME and a separate namespace for the executor after the verification passes. Then, based on the filled information and creation information, the ConfigMap and deployment of the executor to be created are rendered, and the k8s interface is called to create the Pod application of the executor;
[0026] S225. Write the executor information into the scheduling resource configuration table for use by the application module deployed under the Kubernetes cluster to create tasks.
[0027] Preferably, step S223 is executed to configure resources for various task executor instances in the executor according to actual needs, and the configured resources include specifying the CPU specifications and memory specifications of each executor, specifying the number of copies of each executor, and specifying the workspace, wherein the workspace corresponds to at least one task executor instance, at least one task executor instance exclusively enjoys the resources within its workspace, or at least two task executor instances share the resources within their workspace.
[0028] Further preferably, step S224 is executed, and the Kubernetes cluster management center backend first performs resource verification based on the filled-in specification information and the remaining resources of the node:
[0029] If the verification fails, a prompt message will be returned to the front-end interface.
[0030] If the verification passes, the XXL-JOB interface is called through the management center to create an APPNAME for the executor, and the K8S interface is called through the management center to create a separate namespace for the executor, with the format of ns-idlf-scheduler-yyyyMMdd+6-digit serial number as the name of the namespace;
[0031] Then, according to the filled-in specification information, number of replicas, created APPNAME and namespace, freemarker.template.Template is used to render the ConfigMap and deployment of the executor to be created according to the template, and the k8s interface is called through the Kubernetes API or the kubectl command line tool to create the Pod application of the executor.
[0032] Further preferably, step S224 is executed, and the background performs resource verification according to the filled-in specification information and the remaining resources of the node to verify whether the resource allocation meets the following resource verification rules:
[0033] a) The remaining CPU resources of a single node are greater than the total CPU resources required by a single replica of the executor multiplied by the number of replicas, and the remaining memory resources of a single node are greater than the total memory resources required by a single replica of the executor multiplied by the number of replicas;
[0034] b) The CPU resources of node A are greater than the total CPU required by a single replica of the executor, and the memory resources of node A are greater than the total memory required by a single replica of the executor. At the same time, the CPU resources of node B are greater than the total CPU required by a single replica of the executor, and the memory resources of node B are greater than the total memory required by a single replica of the executor.
[0035] The resource allocation satisfies at least one of the resource verification rules in a) and b), that is, passes the resource verification.
[0036] Optionally, execute step S225, after writing the executor information into the scheduling resource configuration table, start a scheduled task to regularly check the Pod status of the executor:
[0037] If all containers in the Pod become Ready within the set time, the Pod is considered normal;
[0038] If there are still containers in the Pod that are not Ready after the set time, the Pod status is considered abnormal.
[0039] Optionally, step S3 specifically includes the following operations:
[0040] S310. The user creates a computing task through the front-end interface of the Kubernetes cluster management center and selects the workspace ID and task type of the computing task.
[0041] S320, querying the scheduling resource configuration table according to the workspace ID and task type of the computing task, obtaining a list of available task scheduling resources that match the workspace and task type, the list including the CPU specification, memory specification, number of replicas, and namespace of each available task scheduling resource;
[0042] S330, the user selects the required resources from the available task scheduling resource list, binds the computing task to the selected resources, and the selected resources constitute a virtual resource pool of the computing task for subsequent task execution;
[0043] S340. After the computing task is successfully bound to the selected resource, the corresponding scheduling policy information is automatically generated, and the scheduling policy information is registered in the designated task group of the scheduling center so that the scheduling center can manage and execute the scheduling task according to the scheduling policy.
[0044] The method of implementing multi-tenant task scheduling using XXL-JOB based on Kubernetes of the present invention has the following beneficial effects compared with the prior art:
[0045] 1. The present invention allocates the resources of the task executor to different tenants according to the tenant task situation through the Kubernetes cluster management center, utilizes the resource isolation, elastic scaling and automatic management characteristics of Kubernetes, combines Prometheus to monitor the CPU and memory usage of each task node of K8s, and dynamically allocates the resources of the task executor. Through the lightweight distributed task scheduling function of XXL-JOB, the effective management, efficient execution and rational use of resources of task scheduling in a multi-tenant environment, as well as the isolated execution of tasks under different tenants, are realized;
[0046] 2. The present invention enables users to perform task scheduling resource configuration and automatic deployment of task executors according to task usage, effectively preventing and reducing the problem of node resource usage tilt in the Kubernetes platform, achieving multi-node load balancing, and improving the utilization of node resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Attached Figure 1 It is a flow chart of the method described in Embodiment 1 of the present invention. DETAILED DESCRIPTION
[0048] In order to make the technical solution, the technical problem solved and the technical effect of the present invention more clearly understood, the technical solution of the present invention is clearly and completely described below in conjunction with specific embodiments.
[0049] The English words involved in the embodiment are now explained:
[0050] Kubernetes is an open source container orchestration platform for automating the deployment, expansion and management of containerized applications. It provides an efficient way to manage microservice architectures, enabling applications to run efficiently in clusters by providing load balancing, automatic expansion and fault tolerance mechanisms. Kubernetes also provides powerful resource management and scheduling capabilities that can automatically allocate computing resources according to application needs.
[0051] XXL-JOB is a lightweight distributed task scheduling platform, which is mainly used to solve the problem of task execution on multiple machines. It supports the functions of scheduled execution, task dependency, task slicing, etc., which can help developers realize complex task scheduling requirements.
[0052] APPNAME refers to the name of the application module, which is predefined and used to identify the executor instance of each application module.
[0053] Pod is the smallest deployable unit in Kubernetes, which contains one or more containers. Each container runs in the same environment and shares the namespace.
[0054] Embodiment 1:
[0055] Combined with Figure 1 This embodiment proposes a method for implementing multi-tenant task scheduling based on Kubernetes using XXL-JOB, which includes the following steps:
[0056] S1. Build and configure a highly available and elastically scalable Kubernetes cluster to provide basic support for XXL-JOB deployment and task scheduling; label the nodes that execute tasks in the Kubernetes cluster and deploy application modules, XXL-JOB scheduling center and executors that the data governance platform depends on, and integrate the Liquibase tool to automatically create the database table structure to support executor registration.
[0057] To implement step S1, you need to perform the following operations:
[0058] S110. Build and configure the Kubernetes cluster to ensure that the cluster has high availability and elastic scalability. Then, configure the network and storage resources of the Kubernetes cluster according to actual needs to provide basic support for the deployment and task scheduling of XXL-JOB on the Kubernetes cluster.
[0059] S120. In the Kubernetes cluster, label the nodes used to execute tasks with the datalake-scheduler label. When creating task executors for different tenants, deploy the created executors to the nodes with the datalake-scheduler label.
[0060] S130. In the Kubernetes cluster, deploy the application modules, XXL-JOB scheduling center, and executors that the data governance platform depends on, including:
[0061] When deploying the XXL-JOB scheduling center, a XXL-JOB scheduling center instance is automatically created. The XXL-JOB scheduling center instance is deployed as an independent Pod, and uses the deployment type and multiple copies. The name of each Pod is prefixed with "idlf-scheduler";
[0062] When deploying an executor, it is deployed as an independent Pod, using the deployment type and multiple replicas. The Pod name of each executor is prefixed with "idlf-executor-depl".
[0063] When the executor is registered to the XXL-JOB dispatch center, the executor is registered using its corresponding APPNAME.
[0064] In the process of deploying the XXL-JOB scheduling center, the Liquibase tool is integrated to automatically create the database table structure. At the same time, the default APPNAME of the application module that the data governance platform depends on is inserted into the database table for automatic registration by the executor.
[0065] S2. Through the Kubernetes cluster management center, resource monitoring and allocation are realized, XXL-JOB is used to create and manage executor instances, and the executor information is written into the scheduling resource configuration table.
[0066] To implement step S2, you need to perform the following operations:
[0067] S210. The platform administrator creates a workspace in the tenant management module of the Kubernetes cluster management center and assigns an administrator to the workspace.
[0068] S220, adding task scheduling resources, specifically including:
[0069] S221. The administrator or user initiates the creation of a task executor through the front-end interface of the Kubernetes cluster management center. The Kubernetes cluster management center calls the Prometheus interface to obtain the CPU and memory usage of the nodes with the datalake-scheduler label, and calculates the CPU usage and memory usage of each node within one day.
[0070] It should be added here that what is usually calculated is the average CPU usage and average memory usage of each node within a day.
[0071] S222. The Kubernetes cluster management center calculates the remaining CPU and memory resources of each node based on the CPU usage and memory usage of each node, and displays them on the front-end interface for user reference.
[0072] S223. The user views the remaining CPU and memory resources of each node on the front-end interface, and configures resources for various task executor instances in the executor according to actual needs; the configured resources include specifying the CPU specifications and memory specifications of each executor, specifying the number of copies of each executor, and specifying the workspace, wherein the workspace corresponds to at least one task executor instance, at least one task executor instance exclusively enjoys the resources within its workspace, or at least two task executor instances share the resources within their workspace.
[0073] S224. After the user completes filling in the resource configuration, the filled information is returned to the Kubernetes cluster management center backend;
[0074] The backend first performs a resource check based on the filled-in specification information and the remaining resources of the node.
[0075] a) If the verification fails, a prompt message will be returned to the front-end interface.
[0076] b) If the verification passes, the XXL-JOB interface is called through the management center to create an APPNAME for the executor, and the K8S interface is called through the management center to create a separate namespace for the executor, with the format of ns-idlf-scheduler-yyyyMMdd+6-digit serial number as the name of the namespace;
[0077] Then, according to the filled-in specification information, number of replicas, created APPNAME and namespace, freemarker.template.Template is used to render the ConfigMap and deployment of the executor to be created according to the template, and the k8s interface is called through the Kubernetes API or the kubectl command line tool to create the Pod application of the executor.
[0078] It should be added that the background performs resource verification based on the filled-in specification information and the remaining resources of the node to verify whether the resource allocation meets the following resource verification rules:
[0079] a) The remaining CPU resources of a single node are greater than the total CPU resources required by a single replica of the executor multiplied by the number of replicas, and the remaining memory resources of a single node are greater than the total memory resources required by a single replica of the executor multiplied by the number of replicas;
[0080] b) The CPU resources of node A are greater than the total CPU required by a single replica of the executor, and the memory resources of node A are greater than the total memory required by a single replica of the executor. At the same time, the CPU resources of node B are greater than the total CPU required by a single replica of the executor, and the memory resources of node B are greater than the total memory required by a single replica of the executor.
[0081] The resource allocation satisfies at least one of the resource verification rules in a) and b), that is, passes the resource verification.
[0082] S225. Write the executor information of step S224 into the scheduling resource configuration table for use by the application module created under the Kubernetes cluster.
[0083] After writing the executor information of step S224 into the scheduling resource configuration table, start a scheduled task to regularly check the Pod status of the executor:
[0084] If all containers in the Pod become Ready within 20 minutes of the set time, the Pod is considered normal;
[0085] If there are still containers in the Pod that are not Ready after 20 minutes, the Pod status is considered abnormal, and the Pod status is then updated, and a message reminder is sent to the administrator or user.
[0086] S3. Create computing tasks and select resources through the front-end interface of the Kubernetes cluster management center, bind the computing tasks to the selected resources and generate a scheduling policy, and finally register the scheduling policy information to the scheduling center to manage and execute computing tasks.
[0087] To implement step S3, you need to perform the following operations:
[0088] S310. The user creates a computing task through the front-end interface of the Kubernetes cluster management center and selects the workspace ID and task type of the computing task.
[0089] S320, querying the scheduling resource configuration table according to the workspace ID and task type of the computing task, obtaining a list of available task scheduling resources that match the workspace and task type, the list including the CPU specification, memory specification, number of replicas, and namespace of each available task scheduling resource;
[0090] S330, the user selects the required resources from the available task scheduling resource list, binds the computing task to the selected resources, and the selected resources constitute a virtual resource pool of the computing task for subsequent task execution;
[0091] S340. After the computing task is successfully bound to the selected resource, the corresponding scheduling policy information is automatically generated, and the scheduling policy information is registered in the designated task group of the scheduling center so that the scheduling center can manage and execute the scheduling task according to the scheduling policy.
[0092] In summary, the method of implementing multi-tenant task scheduling based on Kubernetes using XXL-JOB of the present invention can realize dynamic allocation of resources for task executors under different tenants according to different requirements of different tenants for tasks, and utilize the resource isolation, elastic scaling and automatic management characteristics of k8s. The remaining CPU and memory of each task node are calculated through the resource monitoring of Promethues for user reference. Combined with the lightweight distributed task scheduling function of XXL-JOB, the reasonable allocation of resources, effective management, high efficiency and isolated execution of task scheduling in a multi-tenant environment are effectively realized.
[0093] The above specific examples are used to explain the principles and implementation methods of the present invention in detail. These examples are only used to help understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, any improvements and modifications made by technicians in this technical field without departing from the principles of the present invention should fall within the scope of patent protection of the present invention.
Claims
1. A method for implementing multi-tenant task scheduling based on Kubernetes using XXL-JOB, characterized in that: The steps include: S1. Build and configure a highly available and elastically scalable Kubernetes cluster to provide basic support for XXL-JOB deployment and task scheduling; label the nodes that execute tasks in the Kubernetes cluster and deploy application modules, XXL-JOB scheduling center and executors that the data governance platform depends on, and integrate the Liquibase tool to automatically create the database table structure to support executor registration; S2. Realize resource monitoring and allocation through the Kubernetes cluster management center, use XXL-JOB to create and manage executor instances, and write executor information into the scheduling resource configuration table; S3. Create computing tasks and select resources through the front-end interface of the Kubernetes cluster management center, bind the computing tasks to the selected resources and generate a scheduling policy, and finally register the scheduling policy information to the scheduling center to manage and execute computing tasks.
2. According to a method for implementing multi-tenant task scheduling based on Kubernetes using XXL-JOB according to claim 1, it is characterized in that: The step S1 specifically includes the following operations: S110. Build and configure the Kubernetes cluster to ensure that the cluster has high availability and elastic scalability. Then, configure the network and storage resources of the Kubernetes cluster according to actual needs to provide basic support for the deployment and task scheduling of XXL-JOB on the Kubernetes cluster. S120. In the Kubernetes cluster, label the nodes used to execute tasks with the datalake-scheduler label. When creating task executors for different tenants, deploy the created executors to the nodes with the datalake-scheduler label. S130. In the Kubernetes cluster, deploy the application modules, XXL-JOB scheduling center, and executors that the data governance platform depends on, including: When deploying the XXL-JOB scheduling center, a XXL-JOB scheduling center instance is automatically created. The XXL-JOB scheduling center instance is deployed as an independent Pod, and uses the deployment type and multiple copies. The name of each Pod is prefixed with "idlf-scheduler"; When deploying an executor, the executor is deployed as an independent Pod, using the deployment type and multiple replicas. The Pod name of each executor is prefixed with "idlf-executor-depl"; When the executor is registered to the XXL-JOB dispatch center, the executor is registered using its corresponding APPNAME; In the process of deploying the XXL-JOB scheduling center, the Liquibase tool is integrated to automatically create the database table structure. At the same time, the default APPNAME of the application module that the data governance platform depends on is inserted into the database table for automatic registration by the executor.
3. According to claim 2, a method for implementing multi-tenant task scheduling based on Kubernetes using XXL-JOB is characterized in that: The step S2 specifically includes the following operations: S210. The platform administrator creates a workspace in the tenant management module of the Kubernetes cluster management center and assigns an administrator to the workspace. S220, adding task scheduling resources, specifically including: S221. The administrator or user initiates the creation of a task executor through the front-end interface of the Kubernetes cluster management center. The Kubernetes cluster management center calls the Prometheus interface to obtain the CPU and memory usage of the nodes with the datalake-scheduler label and calculates the CPU usage and memory usage of each node within one day. S222. The Kubernetes cluster management center calculates the remaining CPU and memory resources of each node based on the CPU usage and memory usage of each node, and displays them on the front-end interface for user reference; S223. The user checks the remaining CPU and memory resources of each node on the front-end interface, and configures resources for various task executor instances in the executor according to actual needs; S224. When the user completes filling in the resource configuration, the filled information is returned to the management center backend; the backend first performs resource verification based on the filled information and the remaining resources of the node, and creates APPNAME and a separate namespace for the executor after the verification passes. Then, based on the filled information and creation information, the ConfigMap and deployment of the executor to be created are rendered, and the k8s interface is called to create the Pod application of the executor; S225. Write the executor information into the scheduling resource configuration table for use by the application module deployed under the Kubernetes cluster to create tasks.
4. According to claim 3, a method for implementing multi-tenant task scheduling based on Kubernetes using XXL-JOB is characterized in that: Execute step S223 to configure resources for various task executor instances in the executor according to actual needs. The configured resources include specifying the CPU specifications and memory specifications of each executor, specifying the number of copies of each executor, and specifying the workspace, wherein the workspace corresponds to at least one task executor instance, at least one task executor instance exclusively enjoys the resources within its workspace, or at least two task executor instances share the resources within their workspace.
5. According to claim 4, a method for implementing multi-tenant task scheduling based on Kubernetes using XXL-JOB is characterized in that: Execute step S224, the Kubernetes cluster management center backend first performs resource verification based on the filled-in specification information and the remaining resources of the node: If the verification fails, a prompt message will be returned to the front-end interface. If the verification passes, the XXL-JOB interface is called through the management center to create an APPNAME for the executor, and the K8S interface is called through the management center to create a separate namespace for the executor, with the format of ns-idlf-scheduler-yyyyMMdd+6-digit serial number as the name of the namespace; Then, according to the filled-in specification information, number of replicas, created APPNAME and namespace, freemarker.template.Template is used to render the ConfigMap and deployment of the executor to be created according to the template, and the k8s interface is called through the Kubernetes API or the kubectl command line tool to create the Pod application of the executor.
6. According to claim 5, a method for implementing multi-tenant task scheduling based on Kubernetes using XXL-JOB is characterized in that: Execute step S224, the background performs resource verification based on the filled specification information and the remaining resources of the node to verify whether the resource allocation meets the following resource verification rules: a) The remaining CPU resources of a single node are greater than the total CPU resources required by a single replica of the executor multiplied by the number of replicas, and the remaining memory resources of a single node are greater than the total memory resources required by a single replica of the executor multiplied by the number of replicas; b) The CPU resources of node A are greater than the total CPU required by a single replica of the executor, and the memory resources of node A are greater than the total memory required by a single replica of the executor. At the same time, the CPU resources of node B are greater than the total CPU required by a single replica of the executor, and the memory resources of node B are greater than the total memory required by a single replica of the executor. The resource allocation satisfies at least one of the resource verification rules in a) and b), that is, passes the resource verification.
7. According to claim 3, a method for implementing multi-tenant task scheduling based on Kubernetes using XXL-JOB is characterized in that: Execute step S225, after writing the executor information into the scheduling resource configuration table, start a scheduled task to regularly check the Pod status of the executor: If all containers in the Pod become Ready within the set time, the Pod is considered normal; If there are still containers in the Pod that are not Ready after the set time, the Pod status is considered abnormal.
8. According to claim 3, a method for implementing multi-tenant task scheduling using XXL-JOB based on Kubernetes is characterized in that: The step S3 specifically includes the following operations: S310. The user creates a computing task through the front-end interface of the Kubernetes cluster management center and selects the workspace ID and task type of the computing task. S320, querying the scheduling resource configuration table according to the workspace ID and task type of the computing task, obtaining a list of available task scheduling resources that match the workspace and task type, the list including the CPU specification, memory specification, number of replicas, and namespace of each available task scheduling resource; S330, the user selects the required resources from the available task scheduling resource list, binds the computing task to the selected resources, and the selected resources constitute a virtual resource pool of the computing task for subsequent task execution; S340. After the computing task is successfully bound to the selected resource, the corresponding scheduling policy information is automatically generated, and the scheduling policy information is registered in the designated task group of the scheduling center so that the scheduling center can manage and execute the scheduling task according to the scheduling policy.
Citation Information
Patent Citations
Task scheduling method and system for multi-tenant mode SaaS service cluster environment
CN110532074A
Big data analysis and processing management middle platform based on catering information management system
CN110737643A
Gateway-based ElasticSearch search service method and system, medium and terminal
CN112714018A
Cloud resource scheduling method and system based on Kubernetes
CN113918270A
Task scheduling method and device, equipment and storage medium
CN114168305A
Cited By
Multi-tenant management method and system based on container and Kubernetes
CN119668886A
Intelligent computing equipment access management method and device based on k8s cluster
CN122372417A