Task allocation method and device based on Kubernetes and storage medium

Abstract

The invention provides a container cluster management system Kubernetes-based task allocation method and device and a storage medium, and relates to the technical field of computers. The task allocation method based on Kubernetes is executed by a main node determined in a cluster through a Kubernetes-based election mechanism, and comprises the steps that task types subscribed by a plurality of working nodes in the cluster are determined, and the working nodes are nodes except the main node in the cluster; and under the condition of determining that the to-be-allocated task belongs to the task type subscribed by any one of the plurality of working nodes, allocating the to-be-allocated task to at least one of the plurality of working nodes according to the task type to which the to-be-allocated task belongs and the subscribed task type.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a task allocation method, device and storage medium based on a container cluster management system Kubernetes. Background Art

[0002] Kubernetes is an open source platform designed to automate the deployment, expansion, and management of containerized applications. It provides a powerful framework for running distributed systems elastically, ensuring high availability, and seamlessly scaling applications. By abstracting the underlying infrastructure, Kubernetes enables developers to focus on building applications without having to manage hardware.

[0003] Kubernetes' election mechanism is used to select a master node or leader in the cluster to coordinate tasks and manage resources. It is particularly important in high-availability scenarios to ensure that the system can run smoothly even if a node fails. Common application scenarios include database master-slave synchronization, distributed lock services, and task scheduling. Through the election mechanism, Kubernetes can effectively avoid single points of failure and improve the reliability and stability of the system. Summary of the invention

[0004] One purpose of the present disclosure is to improve the efficiency of task processing and reduce the performance requirements for a single node.

[0005] According to one aspect of some embodiments of the present disclosure, a Kubernetes-based task allocation method is proposed, which is executed by a master node determined in a cluster based on a Kubernetes election mechanism, including: determining the task type subscribed to by multiple working nodes in the cluster, where the multiple working nodes are nodes other than the master node in the cluster; when it is determined that the task to be allocated belongs to the task type subscribed to by any one of the multiple working nodes, allocating the task to be allocated to at least one of the multiple working nodes according to the task type to which the task to be allocated belongs and the subscribed task type.

[0006] In some embodiments, the method further includes: determining the health status of multiple working nodes through a monitoring mechanism of Kubernetes.

[0007] In some embodiments, the method further includes: determining, through a monitoring mechanism of Kubernetes, tasks in which the records of the assigned working nodes are empty as tasks to be assigned.

[0008] In some embodiments, according to the task type to which the task to be assigned belongs and the subscribed task type, allocating the task to be assigned to at least one of a plurality of working nodes includes: among the working nodes in a healthy state, determining a working node whose subscribed task type includes the task type of the task to be assigned as a candidate working node; determining a target working node among the candidate working nodes according to the allocation strategy, and allocating the task to be assigned to the target working node.

[0009] In some embodiments, the allocation strategy includes at least one of round-robin, random allocation, or selecting the working node with the least load.

[0010] In some embodiments, the method also includes: determining a working node that switches from a healthy state to an unhealthy state as a first working node to be processed; treating tasks that have been assigned to the first working node to be processed as tasks to be assigned, and clearing tasks that have been assigned to the first working node to be processed.

[0011] In some embodiments, the method also includes: obtaining the load degrees of multiple working nodes; when the difference in the load degrees of two working nodes is greater than a load threshold, determining the working node with a higher load degree among the two working nodes as the second working node to be processed; determining at least one target task among the tasks assigned to the second working node to be processed; using the target task as the task to be assigned, and deleting the target task from the tasks assigned to the second working node to be processed.

[0012] In some embodiments, determining the health status of multiple working nodes through the monitoring mechanism of Kubernetes includes: determining that the working node has switched from a healthy state to an unhealthy state when it is determined that the interval between the time when the working node last reported its own health status and the current moment exceeds a predetermined multiple of the heartbeat interval; determining that the working node is in a healthy state when it is determined that the interval between the time when the working node last reported its own health status and the current moment is less than or equal to a predetermined multiple of the heartbeat interval.

[0013] In some embodiments, at least one of the plurality of worker nodes is in a different group than the master node.

[0014] According to one aspect of some embodiments of the present disclosure, a Kubernetes-based task allocation device is proposed, which is located at a master node determined in a cluster based on a Kubernetes election mechanism, and includes: a monitoring unit, configured to determine the task type subscribed to by multiple working nodes in the cluster, where the multiple working nodes are nodes other than the master node in the cluster; and an allocation unit, configured to allocate the task to be allocated to at least one of the multiple working nodes according to the task type to which the task to be allocated belongs and the subscribed task type when it is determined that the task to be allocated belongs to the task type subscribed to by any one of the multiple working nodes.

[0015] According to one aspect of some embodiments of the present disclosure, a Kubernetes-based task allocation device is proposed, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute any one of the above Kubernetes-based task allocation methods based on instructions stored in the memory.

[0016] According to one aspect of some embodiments of the present disclosure, a non-transitory computer-readable storage medium is provided, on which computer program instructions are stored, and when the instructions are executed by a processor, any one of the above-mentioned task allocation methods based on Kubernetes is implemented.

[0017] According to one aspect of some embodiments of the present disclosure, a computer program product is proposed, including a computer program or an instruction, and when the computer program or the instruction is executed by a processor, any one of the above-mentioned task allocation methods based on Kubernetes is implemented.

[0018] According to one aspect of some embodiments of the present disclosure, a task processing system based on Kubernetes is proposed, including: a master node determined in a cluster based on a Kubernetes election mechanism, configured to execute any one of the task allocation methods described above; and a working node, configured to: listen to tasks of a subscribed task type through a Kubernetes listening mechanism; and, upon listening to a task assigned by the master node and the processing status of the assigned task being unprocessed, process the assigned task and update the processing status of the assigned task to processing.

[0019] In some embodiments, the working node is also configured to: after startup, report the heartbeat interval and subscribed task type, and report the working node's own status periodically with the heartbeat interval; the master node is also configured to determine that the working node has switched from a healthy state to an unhealthy state when it is determined that the interval between the time when the working node last reported its own health status and the current moment exceeds a predetermined multiple of the heartbeat interval; determine that the working node is in a healthy state when it is determined that the interval between the time when the working node last reported its own health status and the current moment is less than or equal to a predetermined multiple of the heartbeat interval. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide further understanding of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute improper limitations on the present disclosure.

[0021] Figure 1 The present invention is a flowchart of some embodiments of the Kubernetes-based task allocation method disclosed in the present invention.

[0022] Figure 2 Flowcharts of other embodiments of the Kubernetes-based task allocation method disclosed in the present invention.

[0023] Figure 3 Flowcharts of some further embodiments of the Kubernetes-based task allocation method disclosed in the present invention.

[0024] Figure 4 It is a schematic diagram of some embodiments of the Kubernetes-based task allocation device disclosed in the present invention.

[0025] Figure 5 Schematic diagrams of other embodiments of the Kubernetes-based task allocation device disclosed in the present invention.

[0026] Figure 6 Schematic diagrams of some further embodiments of the Kubernetes-based task allocation device disclosed in the present invention.

[0027] Figure 7 Schematic diagram of some embodiments of the Kubernetes-based task processing system disclosed in the present invention. DETAILED DESCRIPTION

[0028] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments.

[0029] The Kubernetes election mechanism is usually used to ensure that only one instance is active and takes the leadership role among multiple instances to avoid conflicts and repeated operations. If the master is not elected, all nodes will perform the same action, resulting in repeated execution and waste of resources.

[0030] The inventors found that when a task consumes a large amount of resources, based on the Kubernetes election mechanism, the elected master node will execute the task. When the master node fails (for example, it crashes), a new master node will be elected to execute the task. Therefore, the elected new node also needs to have the amount of resources that can meet the requirements of the task. This method requires at least two working nodes to have the amount of resources that can meet the resource consumption of the task, and the amount and difficulty of resource allocation are high.

[0031] In response to the above problems, the present invention proposes a task allocation method, device and storage medium based on Kubernetes. On the basis of the Kubernetes-based election mechanism, it supports allocating tasks to different nodes for execution, each node processes part of the tasks, and the elected master node performs task allocation and scheduling, thereby reducing the task processing pressure of each working node, and can also reduce the resource allocation amount and allocation difficulty for each node, thereby improving the execution efficiency of tasks.

[0032] The flowcharts of some embodiments of the task allocation method based on Kubernetes disclosed in the present invention are as follows: Figure 1 shown. Figure 1 The executor of the method shown is the master node elected based on the Kubernetes election mechanism. The election process can be the same as in the related art and will not be repeated here. In some embodiments, the node in the cluster that serves as the master node can be replaced, for example, when the current master node is abnormal, a new master node is elected.

[0033] In step S11, the task types subscribed by multiple working nodes in the cluster are determined.

[0034] In some embodiments, the working node can report the subscribed task type by itself after startup. The master node obtains the information reported by the working node, thereby determining the task type subscribed by each working node.

[0035] In some embodiments, the master node uses the set of task types subscribed by all working nodes as monitoring objects, and monitors whether tasks of corresponding types appear in the cluster, so that when tasks of types that can be executed by the current working nodes appear, tasks can be assigned in time to improve the reliability of task assignment. For tasks of a task type that is not subscribed by any working node, the master node may not monitor, thereby reducing the workload of the master node and improving the effective utilization of the master node resources.

[0036] In some embodiments, a working node may subscribe to one or more task types, and the specific task type is determined based on demand or resource configuration of the working node, thereby improving the flexibility and controllability of the working node in carrying tasks.

[0037] In some embodiments, the working nodes may be located in the same or different working groups (Groups), for example, there are one or more working nodes that are not located in the same group as the master node, and there are one or more working nodes that are located in the same group as the master node; the number of groups involved in the working nodes may be greater than or equal to 2. This method can facilitate cross-group task allocation and resource scheduling, further improving the flexibility and execution efficiency of task allocation.

[0038] In step S13, according to the task type, the master node allocates the task to be allocated to one or more working nodes.

[0039] In some embodiments, the master node may assign the task to be assigned to the worker node that has subscribed to the task of this type according to the task type of the task to be assigned, thereby ensuring that the task can be processed correctly.

[0040] In some embodiments, the master node can determine, through the monitoring mechanism of Kubernetes, the task whose record of the assigned work node is empty as the task to be assigned. Then in step S13, the task node is assigned to the task whose record of the assigned work node is empty. Based on the method in the embodiment shown above, the monitoring mechanism of Kubernetes can be fully utilized to identify unprocessed tasks, avoiding missed processing or repeated processing of tasks.

[0041] In some embodiments, Figure 2 As shown in , the Kubernetes-based task allocation method disclosed in the present invention also includes step 221.

[0042] In step 221, the master node can also determine the status of each working node through the monitoring mechanism of Kubernetes. In some embodiments, the above status includes a health status.

[0043] Step 221 can be executed in real time after the node is elected as the master node.

[0044] Further, in step S13, step 231 and step 232 are executed.

[0045] In step 231, among the working nodes in a healthy state, working nodes whose subscribed task types include the task type of the task to be assigned are determined as candidate working nodes.

[0046] In some embodiments, the master node marks the health status of the working node according to the status of the monitored working node, and then selects the working node that has subscribed to the corresponding task type from the working nodes in the healthy state. In some embodiments, the master node can determine the working node that has subscribed to the task type according to the task type of the task to be assigned, and then select the working node in the healthy state. The above two methods can be flexibly configured or switched as needed.

[0047] In some embodiments, the master node can determine the health status of the working node based on the time when the working node reports its status. The working node can report its own status based on its heartbeat cycle. If the master node determines that the interval between the time when the working node last reported its own status and the current moment exceeds a predetermined multiple (for example, 3 to 5 times) of the heartbeat interval, the working node is determined to be switched to an unhealthy state, otherwise, the working node is maintained in a healthy state. In some embodiments, the status reported by the working node can be only a heartbeat message, thereby reducing the burden of message delivery. In some embodiments, the status reported by the working node can include specific health information, thereby increasing the effective amount of monitoring information that the master node can obtain.

[0048] In step 232, a target working node is determined from the candidate working nodes according to the allocation strategy, and the task to be allocated is allocated to the target working node. In some embodiments, if the number of candidate working nodes is 1, the candidate working node is the target working node.

[0049] In some embodiments, the allocation strategy includes polling, random allocation or selecting the work node with the smallest load for allocation. The above-mentioned polling includes: sorting the work nodes in advance, and the master node allocates tasks to the candidate work nodes in a predetermined order. The above-mentioned random allocation includes: the master node randomly selects the target work node from the candidate work nodes. The above-mentioned selection of the work node with the smallest load for allocation includes: the master node can obtain the load or load rate of each work node in real time or regularly, and then select the work node with the smallest load or load rate from the candidate work nodes to allocate the task.

[0050] Through the method in the embodiment shown above, it is possible to ensure that the working node to which the task is assigned is a node that is capable of processing the task and is in a healthy state, thereby improving the reliability of task execution.

[0051] Based on the method in the embodiment shown above, the master node determined based on the Kubernetes election mechanism does not need to execute tasks by itself, but acts as a task allocator to allocate tasks to working nodes with corresponding processing capabilities, thereby supporting the allocation of tasks to different nodes for execution. Each node processes part of the tasks, reducing the task processing pressure of each working node, and can also reduce the amount and difficulty of resource allocation to each node, thereby improving the execution efficiency of tasks.

[0052] The flowcharts of other embodiments of the task allocation method based on Kubernetes disclosed in the present invention are as follows: Figure 2 As shown above Figure 1 On the basis of the illustrated embodiment, after the master node determines the status of each working node through the monitoring mechanism of Kubernetes in step 221, in addition to executing step 231, steps 241 and 251 may also be executed synchronously or asynchronously.

[0053] In step 241, the master node determines whether there is a working node that changes from a healthy state to an unhealthy state. If a working node that switches to an unhealthy state is found, step 242 is executed; otherwise, the master node can continue to execute step 241 and make a judgment based on the detection result in step 221.

[0054] In some embodiments, as mentioned above, the master node may determine the health status of the working node based on the time when the working node reports its status. The working node may report its own status based on its heartbeat cycle. If the master node determines that the interval between the time when the working node last reported its own status and the current time exceeds a predetermined multiple (e.g., 3 to 5 times) of the heartbeat interval, the working node is determined to be switched to an unhealthy state.

[0055] In step 242-step 243, the master node will switch to an unhealthy working node as the first to-be-processed working node, and then treat the task assigned to the first to-be-processed working node as the task to be assigned, and clear the record of the task assigned to the first to-be-processed working node. In some embodiments, the working node of the task assigned to the first to-be-processed working node can be cleared, so that when the task is monitored based on the Kubernetes monitoring mechanism, the task can be reallocated as a task to be assigned.

[0056] Based on the method in the embodiment shown above, the master node can promptly discover the working node with problems and assign the tasks assigned to the working node to other working nodes for execution, thereby avoiding task execution failure and improving the reliability and timeliness of task execution.

[0057] In step 251, the master node obtains the load rate of the working node. In some embodiments, the master node can obtain the load rate of each working node regularly (at a certain time interval or at a preset time point). Step 252 is further executed.

[0058] In step 252, it is determined whether the difference between the loads of two working nodes is greater than the load threshold. If the difference between the loads of two working nodes is greater than the load threshold, it is determined that the load is unbalanced and the tasks assigned to the working nodes need to be adjusted, and step 253 is executed; otherwise, no processing is performed.

[0059] In some embodiments, the load degree may be a load rate or a load amount, or other information for identifying the load degree.

[0060] In step 253, it is determined that, among the two working nodes whose load difference is greater than the load threshold, the working node with the higher load is used as the second working node to be processed.

[0061] In step 254, at least one target task is determined from the tasks assigned to the second work node to be processed. In some embodiments, the target task and the number of target tasks can be determined based on the current load of the second work node to be processed and the amount of resources required for the tasks it carries, for example, so that the load of the second work node to be processed reaches the average load of the work nodes in the current cluster after it does not carry the target task.

[0062] In step 255, the target task is used as a task to be assigned, and the target task is deleted from the record of tasks assigned to the second working node to be processed. In some embodiments, the target task is made a task to be assigned by marking the working node of the target task as empty.

[0063] Based on the method in the embodiment shown above, the master node can select a suitable worker node for unassigned tasks based on the monitoring mechanism of Kubernetes, and adjust the tasks assigned to each worker node, so as to make full use of the resources of the worker nodes in the cluster, avoid delays in task execution, and improve the reliability and efficiency of task execution.

[0064] The flowcharts of some other embodiments of the task allocation method based on Kubernetes disclosed in the present invention are as follows Figure 3 shown.

[0065] In step 301, when task scheduling is started, based on the Kubernetes election mechanism, the node participates in the election of the leader.

[0066] In step 302, the node that is successfully elected as the master node is selected, and step 303 is executed.

[0067] In steps 303-304, the master node monitors all working nodes, monitors these task type instances according to the task types subscribed by the working nodes, and marks the health status of the working nodes according to the heartbeat cycle of the working nodes and the time of the most recent status report.

[0068] In some embodiments, if the time when the working node A last reported its status is more than a predetermined heartbeat period from the current time, the working node will be marked as unhealthy.

[0069] Based on the above steps, steps 311, 312, and 331 can be executed synchronously or asynchronously, and the order of execution is not important.

[0070] In steps 311-313, when the master node monitors changes in the task instance and there are tasks with empty working nodes, it selects a batch of working nodes according to the subscribed task type, health status and other information of the working nodes. Then, the task is assigned to a working node in each working group according to the task allocation strategy. In some embodiments, the allocation strategy can be polling, random allocation, least load (i.e., assigning the task to the working node with the smallest current load), etc. The allocation strategy can also be any one of the related technologies, which can be set or adjusted as needed, and will not be repeated here.

[0071] In steps 321-323, when the master node determines that a work node is in an unhealthy state, it will mark the work node as unhealthy, and set the work nodes of all tasks assigned to the work node to empty and the processing status to unprocessed. These tasks will be assigned again.

[0072] In steps 331-333, the master node will also periodically check the load of all working nodes (the load is related to the allocation strategy and is positively correlated with the workload of the working node, that is, the higher the load, the heavier the workload of the working node). When the load difference of the working nodes exceeds the specified threshold, the working nodes of some tasks on the working nodes with high load will be set to empty and the processing status will be set to unprocessed, so that these tasks will be reassigned.

[0073] In addition, after the above steps 323 and 333, when the working node monitors that a task is assigned to itself and the processing status is unprocessed, it will start processing the task and set the processing status of the task to processing.

[0074] Based on the method in the embodiment shown above, the availability of the task allocation system can be improved with the help of the election mechanism and monitoring mechanism of Kubernetes; tasks of the same type can be assigned to working nodes in different working groups; the task distribution can be checked regularly to see if it is balanced, so as to reallocate the tasks and improve the efficiency and reliability of task execution.

[0075] Schematic diagrams of some embodiments of the task allocation device 41 based on Kubernetes disclosed in the present invention are as follows Figure 4As shown in , the device is deployed on the master node determined by the Kubernetes election mechanism. In some embodiments, considering that the master node is determined by election and is not fixed, a task allocation device based on Kubernetes can be deployed on each working node to improve the startup efficiency, or after the master node is determined, a task allocation device based on Kubernetes can be deployed on the master node to reduce resource usage.

[0076] The monitoring unit 411 can determine the task types subscribed by multiple working nodes in the cluster. In some embodiments, each working node can report the subscribed task type by itself after startup. The master node obtains the information reported by the working node, thereby determining the task type subscribed by each working node.

[0077] In some embodiments, the master node uses the set of task types subscribed by all working nodes as monitoring objects, and monitors whether tasks of corresponding types appear in the cluster, so that when tasks of types that can be executed by the current working nodes appear, tasks can be assigned in time to improve the reliability of task assignment. For tasks of a task type that is not subscribed by any working node, the master node may not monitor, thereby reducing the workload of the master node and improving the effective utilization of the master node resources.

[0078] In some embodiments, a working node may subscribe to one or more task types, and the specific task type is determined based on demand or resource configuration of the working node, thereby improving the flexibility and controllability of the working node in carrying tasks.

[0079] In some embodiments, the working nodes may be located in the same or different working groups (Groups), for example, there are one or more working nodes that are not located in the same group as the master node, and there are one or more working nodes that are located in the same group as the master node; the number of groups involved in the working nodes may be greater than or equal to 2. This method can facilitate cross-group task allocation and resource scheduling, further improving the flexibility and execution efficiency of task allocation.

[0080] In some embodiments, the monitoring unit 411 can also determine, through the monitoring mechanism of Kubernetes, that the assigned working node is an empty task as a task to be assigned.

[0081] The allocation unit 412 can allocate the task to be allocated to one or more working nodes according to the task type. In some embodiments, the master node can allocate the task to be allocated to the working nodes that have subscribed to the task of this type according to the task type of the task to be allocated, thereby ensuring that the task can be processed correctly.

[0082] Based on the device in the embodiment shown above, the master node does not need to execute tasks by itself, but acts as a task distributor to distribute tasks to working nodes with corresponding processing capabilities, thereby supporting the distribution of tasks to different nodes for execution. Each node handles part of the task, reducing the task processing pressure of each working node, and can also reduce the amount and difficulty of resource allocation to each node, thereby improving the execution efficiency of tasks.

[0083] In some embodiments, the monitoring unit 411 can also determine the status of each working node through the monitoring mechanism of Kubernetes. The allocation unit 412 then determines, among the working nodes in a healthy state, working nodes whose subscribed task types include the task types of the tasks to be assigned, as candidate working nodes, determines the target working node among the candidate working nodes according to the allocation strategy, and assigns the tasks to be assigned to the target working node.

[0084] Such a device can ensure that the working node to which the task is assigned is a node that is capable of processing the task and is in a healthy state, thereby improving the reliability of task execution.

[0085] In some embodiments, the monitoring unit 411 can also timely determine the working node that changes from a healthy state to an unhealthy state, and then the allocation unit 412 will switch to the working node in the unhealthy state as the first working node to be processed, and the task assigned to the first working node to be processed will be used as the task to be assigned, and the record of the task assigned to the first working node to be processed will be cleared. Such a device can timely find the working node with problems, and assign the tasks assigned to the working node to other working nodes for execution, avoiding task execution failure, and improving the reliability and timeliness of task execution.

[0086] In some embodiments, the monitoring unit 411 is also capable of obtaining the load degree of the working node and identifying whether there is a situation where the difference in the load degree of two working nodes is greater than the load threshold. When it is determined that the difference in the load degree of two working nodes is greater than the load threshold, the triggering allocation unit 412 determines the working node with a higher load degree among the two working nodes where the difference in the load degree of the two working nodes is greater than the load threshold as the second working node to be processed, and determines at least one target task among the tasks assigned to the second working node to be processed, uses the target task as the task to be assigned, and deletes the target task from the record of tasks assigned to the second working node to be processed.

[0087] Such a device can select suitable working nodes for unassigned tasks based on the monitoring mechanism of Kubernetes, and adjust the tasks assigned to each working node, so as to make full use of the resources of the working nodes in the cluster, avoid delays in task execution, and improve the reliability and efficiency of task execution.

[0088] The structural diagram of an embodiment of the task allocation device based on Kubernetes disclosed in the present invention is as follows Figure 5 As shown. The Kubernetes-based task allocation device includes a memory 501 and a processor 502. Wherein: the memory 501 can be a disk, a flash memory, or any other non-volatile storage medium. The memory is used to store the instructions in the corresponding embodiment of the task allocation method based on Kubernetes above. The processor 502 is coupled to the memory 501 and can be implemented as one or more integrated circuits, such as a microprocessor or a microcontroller. The processor 502 is used to execute instructions stored in the memory, which can improve the efficiency of task processing and reduce the performance requirements for a single node.

[0089] In one embodiment, it is also possible to Figure 6 As shown, the Kubernetes-based task allocation device 600 includes a memory 601 and a processor 602. The processor 602 is coupled to the memory 601 via a BUS bus 603. The Kubernetes-based task allocation device 600 can also be connected to an external storage device 605 via a storage interface 604 to call external data, and can also be connected to a network or another computer system (not shown) via a network interface 606. No further detailed description will be given here.

[0090] In this embodiment, by storing data instructions in a memory and then processing the instructions through a processor, the efficiency of task processing can be improved and the performance requirements for a single node can be reduced.

[0091] In another embodiment, a computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the steps of the method in the corresponding embodiment of the task allocation method based on Kubernetes. It should be understood by those skilled in the art that the embodiments of the present disclosure may be provided as methods, devices, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transient storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0092] Schematic diagrams of some embodiments of the task processing system based on Kubernetes disclosed in the present invention are as follows Figure 7 as shown in .

[0093] A master node 71 is elected in the cluster through an election mechanism based on Kubernetes. In addition, there are multiple working nodes 721-724 in the cluster. In some embodiments, Figure 7 As shown in the figure, the master node and the working node can be located in the same or different working groups, such as group 1 and group 2 in the figure. The figure takes two working groups as an example, and there is no limit on the actual number of working groups and the number of working nodes in each working group.

[0094] The master node 71 can execute any of the Kubernetes-based task allocation methods mentioned above.

[0095] Working nodes 721~724 can monitor tasks of the task type they have subscribed to through the monitoring mechanism of Kubernetes, and when they monitor that tasks are assigned by the master node and the processing status of the assigned tasks is unprocessed, they process the assigned tasks and update the processing status of the assigned tasks to processing.

[0096] Based on the Kubernetes-based task processing system in the embodiment shown above, the master node determined by the Kubernetes election mechanism does not need to execute tasks by itself, but acts as a task allocator to allocate tasks to working nodes with corresponding processing capabilities, thereby supporting the allocation of tasks to different nodes for execution. Each node processes part of the tasks, reducing the task processing pressure of each working node, and can also reduce the amount and difficulty of resource allocation to each node, thereby improving the execution efficiency of tasks.

[0097] In some embodiments, the working nodes 721-724 can also report their own heartbeat intervals and subscribed task types after startup. Subsequently, the working nodes report their own health status with their own heartbeat intervals as a period. In some embodiments, the working nodes can register with the API Server (application programming interface server) to declare their own heartbeat intervals and the task types to be subscribed. In some embodiments, the working nodes can also report the workgroup to which they belong.

[0098] When the master node 71 determines that the interval between the time when the working node last reported its own status and the current time exceeds a predetermined multiple of the heartbeat interval, the master node 71 determines that the working node has switched to an unhealthy state.

[0099] Based on the system in the embodiment shown above, the working node can report its own heartbeat interval and subscribed task type in a timely manner, which is conducive to setting differentiated working nodes and improving the master node's grasp of each working node. The information, which in turn is conducive to making correct decisions in subsequent task allocation and node monitoring, and improving the system's stable operation capability.

[0100] In some embodiments, the working nodes 721-724 can also become the new master node through the Kubernetes election mechanism when the current master node is abnormal, and then execute any of the Kubernetes-based task allocation methods mentioned above.

[0101] Based on the system in the embodiment shown above, it is possible to ensure that there is a master node in the system that can execute task scheduling, further improving the stable operation capability of the system.

[0102] In some embodiments, the task processing system based on Kubernetes also includes a Kubernetes controller, which can manage the state of resources in the cluster to ensure that the actual state meets the expected state. The controller achieves this goal by constantly monitoring changes in resources and taking corresponding actions. The Kubernetes controller can realize the automatic expansion and contraction function based on Kubernetes. Kubernetes elastic scaling refers to automatically adjusting the number of application copies or the number of cluster nodes according to changes in application load to ensure efficient use of resources and stable performance of applications. Elastic scaling can be divided into two types: horizontal scaling and vertical scaling, among which horizontal scaling refers to responding to load changes by increasing or decreasing the number of application copies (number of Pods); vertical scaling refers to responding to load changes by adjusting the resource requests and restrictions (such as CPU and memory) of a single Pod. In some embodiments, when the load of a working node is high, the working node is automatically expanded, and the tasks of other working nodes are dynamically and automatically assigned to the new working node; when the load of a working node is low, the working node is automatically reduced, and the tasks on the reduced working node are dynamically and automatically assigned to other working nodes. In addition, if the node that is cancelled by reduction is the master node, a new master node is selected through the election mechanism, and the system can operate normally again.

[0103] Based on the system in the embodiment shown above, it is possible to fully utilize the functions of Kubernetes, improve the stable operation capability of the equipment, and improve the effective utilization of resources.

[0104] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0107] So far, the present disclosure has been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed here.

[0108] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0109] It should be noted that the terms "first", "second", etc. in the specification, claims, and drawings of the present disclosure are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present disclosure rather than to limit it. Although the present disclosure has been described in detail with reference to the preferred embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present disclosure can still be modified or some technical features can be replaced by equivalents without departing from the spirit of the technical solution of the present disclosure, which should be included in the scope of the technical solution for protection of the present disclosure.

Claims

1. A task allocation method based on a container cluster management system Kubernetes, executed by a master node determined in a cluster based on an election mechanism of the Kubernetes, comprising: Determine the task types subscribed by multiple working nodes in the cluster, where the multiple working nodes are nodes in the cluster other than the master node; In the case where it is determined that the task to be assigned belongs to the task type subscribed by any one of the multiple working nodes, According to the task type to which the task to be assigned belongs and the subscribed task type, the task to be assigned is assigned to at least one of the multiple working nodes.

2. The task allocation method according to claim 1, further comprising: The health status of the multiple working nodes is determined through the monitoring mechanism of the Kubernetes.

3. The task allocation method according to claim 1 or 2, further comprising: Through the monitoring mechanism of Kubernetes, the task whose record of the assigned working node is empty is determined as the task to be assigned.

4. The task allocation method according to claim 2, wherein: The allocating the task to be allocated to at least one of the plurality of working nodes according to the task type to which the task to be allocated belongs and the subscribed task type comprises: Among the working nodes in a healthy state, determine working nodes whose subscribed task types include the task type of the task to be assigned as candidate working nodes; A target working node is determined from the candidate working nodes according to an allocation strategy, and the task to be allocated is allocated to the target working node.

5. The task allocation method according to claim 4, wherein: The allocation strategy includes at least one of polling, random allocation, or selecting the working node with the smallest load.

6. The task allocation method according to claim 2, further comprising: Determine a working node that switches from a healthy state to an unhealthy state as a first working node to be processed; The task assigned to the first to-be-processed working node is used as a task to be assigned, and the task assigned to the first to-be-processed working node is cleared.

7. The task allocation method according to any one of claims 1 to 3, further comprising: Obtaining the load degrees of the multiple working nodes; In the case where the difference between the loads of two working nodes is greater than the load threshold, determining the working node with the higher load among the two working nodes as the second working node to be processed; Determine at least one target task among the tasks assigned to the second work node to be processed; The target task is used as a task to be assigned, and the target task is deleted from the tasks assigned to the second working node to be processed.

8. The task allocation method according to claim 2, wherein: Determining the health status of the plurality of working nodes through the monitoring mechanism of Kubernetes includes: When it is determined that the interval between the time when the working node last reported its own state and the current time exceeds a predetermined multiple of the heartbeat interval, determining that the working node is switched from a healthy state to an unhealthy state; When it is determined that the interval between the time when the working node last reported its own status and the current time is less than or equal to a predetermined multiple of the heartbeat interval, it is determined that the working node is in a healthy state.

9. The task allocation method according to claim 1, wherein: At least one of the multiple working nodes is located in a different group from the master node.

10. A task allocation device based on a container cluster management system Kubernetes, located on a master node determined in a cluster based on an election mechanism of the Kubernetes, comprising: A monitoring unit is configured to determine a task type subscribed to by a plurality of working nodes in a cluster, wherein the plurality of working nodes are nodes in the cluster other than the master node; The allocation unit is configured to allocate the task to be allocated to at least one of the multiple working nodes according to the task type to which the task to be allocated belongs and the subscribed task type when it is determined that the task to be allocated belongs to the task type subscribed to by any one of the multiple working nodes.

11. A task allocation device based on a container cluster management system Kubernetes, comprising: Memory; as well as A processor coupled to the memory, the processor being configured to execute the method according to any one of claims 1 to 9 based on instructions stored in the memory.

12. A non-transitory computer-readable storage medium having computer program instructions stored thereon, which implement the method of any one of claims 1 to 9 when executed by a processor.

13. A computer program product, comprising a computer program or instructions, wherein when the computer program or instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

14. A task processing system based on a container cluster management system Kubernetes, comprising: A master node determined in the cluster based on the Kubernetes election mechanism is configured to execute the method described in any one of claims 1 to 9; and Multiple worker nodes, configured as: Using the Kubernetes monitoring mechanism, monitor tasks of the subscribed task type; When monitoring that a task is assigned by the master node and the processing status of the assigned task is unprocessed, the assigned task is processed and the processing status of the assigned task is updated to being processed.

15. The task processing system according to claim 14, wherein: The working node is further configured to: after startup, report the heartbeat interval and the subscribed task type, and report the working node's own state periodically with the heartbeat interval; The master node is also configured to determine that the working node has switched from a healthy state to an unhealthy state when it is determined that the interval between the time when the working node last reported its own status and the current moment exceeds a predetermined multiple of the heartbeat interval; and to determine that the working node is in a healthy state when it is determined that the interval between the time when the working node last reported its own status and the current moment is less than or equal to a predetermined multiple of the heartbeat interval.

Images