Multi-cluster scheduling method and platform, computer readable storage medium and electronic equipment

Abstract

The invention relates to the technical field of cluster scheduling, and provides a multi-cluster scheduling method, a multi-cluster management platform, a computer readable storage medium and electronic device.The multi-cluster scheduling method is applied to the multi-cluster management platform, the multi-cluster management platform is used for managing multiple clusters, and the multi-cluster scheduling method comprises the steps that a cluster distribution request is received; the cluster allocation request comprises computing power resource demand information of a service requester; at least one target cluster meeting the computing power resource demand information is selected from the multiple clusters according to the operation condition of each cluster; the operation condition comprises a computing power resource use condition; and generating a cluster allocation response according to the cluster information of each target cluster and the computing power resource use condition of each target cluster, and feeding back the cluster allocation response to the service requester, so that the service requester schedules at least one target cluster according to the cluster allocation response. According to the invention, the resource utilization rate of each cluster can be balanced, and network congestion is reduced.

Description

Background Art

[0002] As the demand for cloud computing by enterprises or governments continues to grow, single-cluster systems can no longer meet the growing business needs due to their limited carrying capacity and disaster recovery capabilities. Therefore, multi-cluster orchestration has become the preferred solution to this problem.

[0003] In actual multi-cluster application scenarios, since the resources and configurations of each cluster may be different, resource allocation and management become complicated. Therefore, how to effectively manage resources and load balance in a multi-cluster environment to meet the application needs in large-scale scenarios has become a technical problem that needs to be solved urgently.

[0004] In view of this, there is an urgent need to develop a new multi-cluster scheduling method in this field.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure. Summary of the invention

[0006] The purpose of the present disclosure is to provide a multi-cluster scheduling method, a multi-cluster management platform, a computer-readable storage medium and an electronic device, thereby at least to a certain extent overcoming the technical problem of being unable to effectively schedule resources in a multi-cluster environment due to limitations of related technologies.

[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0008] According to a first aspect of the present disclosure, a multi-cluster scheduling method is provided, which is applied to a multi-cluster management platform, wherein the multi-cluster management platform is used to manage multiple clusters, and a cluster monitoring component is deployed in each cluster, and the cluster monitoring component is used to monitor the operating status of each cluster. The method includes:

[0009] Receive a cluster allocation request; the cluster allocation request includes computing resource demand information of the service requester;

[0010] Selecting at least one target cluster that meets the computing resource demand information from the multiple clusters according to the operating status of each cluster; the operating status includes computing resource usage;

[0011] A cluster allocation response is generated according to the cluster information of each target cluster and the computing resource usage of each target cluster, and the cluster allocation response is fed back to the service requester, so that the service requester schedules the at least one target cluster according to the cluster allocation response.

[0012] In an exemplary embodiment of the present disclosure, the receiving a cluster allocation request includes:

[0013] Receiving the cluster allocation request sent by the service requester through the service open platform, and performing identity authentication on the service requester;

[0014] After the identity authentication is passed, the cluster allocation request forwarded by the service open platform is received.

[0015] In an exemplary embodiment of the present disclosure, the selecting at least one target cluster that meets the computing resource demand information from the multiple clusters according to the operating status of each cluster includes:

[0016] According to the operating status of each cluster, selecting at least one target cluster that meets the computing resource demand information from the multiple clusters based on a pre-configured cluster screening strategy;

[0017] The cluster screening strategy uses at least one capability requirement parameter as a key consideration factor.

[0018] In an exemplary embodiment of the present disclosure, the at least one target cluster includes a target cluster;

[0019] After feeding back the cluster allocation response to the service requester, the method further includes:

[0020] Receive a cluster scheduling request; the cluster scheduling request is used to request scheduling of the target cluster, the cluster scheduling request includes task data of a computing task, and the task data includes a task description and data required to execute the computing task;

[0021] Dispatching the task data to the master node of the target cluster, executing the computing task through the master node of the target cluster and generating a task execution result;

[0022] The task execution result is returned to the service open platform, so that the service open platform feeds back the task execution result to the business requester.

[0023] In an exemplary embodiment of the present disclosure, executing the computing task through the master node of the target cluster and generating a task execution result includes:

[0024] Sending the task data to the master node of the target cluster;

[0025] In response to the computing resource usage of the master node meeting the preset computing resource usage standard, executing the computing task through the master node of the target cluster and generating a task execution result;

[0026] In response to the computing resource usage of the master node not meeting the preset computing resource usage standard, the computing task is executed by the secondary node of the target cluster and a task execution result is generated.

[0027] In an exemplary embodiment of the present disclosure, the at least one target cluster includes a plurality of target clusters;

[0028] After feeding back the cluster allocation response to the service requester, the method further includes:

[0029] Receive a cluster scheduling request; the cluster scheduling request is used to request scheduling of the target cluster, the cluster scheduling request includes task data of a computing task, and the task data includes a task description and data required to execute the computing task;

[0030] Splitting the task data into multiple subtask data, and sending each subtask data to the master node of each target cluster, executing each subtask through the master node of each target cluster and generating a subtask execution result;

[0031] The execution results of the plurality of subtasks are integrated, and the obtained comprehensive execution result is returned to the service open platform, so that the service open platform feeds back the comprehensive execution result to the service requester.

[0032] In an exemplary embodiment of the present disclosure, executing each subtask through the master node of each target cluster and generating a subtask execution result includes:

[0033] In response to the computing resource usage of the master node meeting the preset computing resource usage standard, executing each of the subtasks through the master node of each target cluster and generating the subtask execution result;

[0034] In response to the computing resource usage of the master node not satisfying the preset computing resource usage standard, each subtask is executed by the secondary node of each target cluster and the subtask execution result is generated.

[0035] According to a second aspect of the present disclosure, a multi-cluster management platform is provided, wherein the multi-cluster management platform is used to manage multiple clusters, wherein a cluster monitoring component is deployed in each cluster, and wherein the cluster monitoring component is used to monitor the operating status of each cluster, and the platform includes:

[0036] A request receiving module, used to receive a cluster allocation request; the cluster allocation request includes computing resource demand information of the service requester;

[0037] A cluster selection module, configured to select at least one target cluster that meets the computing power resource demand information from the multiple clusters according to the operating status of each cluster; the operating status includes computing power resource usage;

[0038] The request response module is used to generate a cluster allocation response according to the cluster information of each target cluster and the computing resource usage of each target cluster, and feed back the cluster allocation response to the service requester so that the service requester schedules at least one target cluster according to the cluster allocation response.

[0039] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the multi-cluster scheduling method described in the first aspect is implemented.

[0040] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the multi-cluster scheduling method described in the first aspect above by executing the executable instructions.

[0041] It can be seen from the above technical solutions that the multi-cluster scheduling method, multi-cluster management platform, computer-readable storage medium and electronic device in the exemplary embodiments of the present disclosure have at least the following advantages and positive effects:

[0042] In the technical solutions provided by some embodiments of the present disclosure, on the one hand, by monitoring the operating status of each cluster (including the usage of computing resources), the method can intelligently select the cluster that best suits the current business request, avoiding the situation where some clusters are overloaded while other clusters are idle, thereby efficiently performing resource scheduling in a multi-cluster environment; further, it can also quickly analyze the status of each cluster based on the cluster monitoring component, and make the best choice accordingly, thereby shortening the task allocation time and improving the response speed and service quality of the overall system; on the other hand, when one or several clusters fail, the scheduling mechanism in the present disclosure can automatically adjust the task allocation according to the latest cluster operating status to ensure that key businesses are not affected, thereby enhancing the stability and reliability of the entire system.

[0043] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0045] Figure 1 A schematic diagram showing a flow chart of a multi-cluster scheduling method in an embodiment of the present disclosure;

[0046] Figure 2 A schematic diagram showing a flow chart of how to generate a cluster allocation response in an embodiment of the present disclosure;

[0047] Figure 3 A flowchart showing how to obtain a task execution result when at least one target cluster includes one target cluster in an embodiment of the present disclosure;

[0048] Figure 4 A flowchart showing how to obtain a task execution result when at least one target cluster includes multiple target clusters in an embodiment of the present disclosure;

[0049] Figure 5 A flowchart showing how to generate a task execution result in an embodiment of the present disclosure;

[0050] Figure 6 The overall architecture diagram of the multi-cluster scheduling method in the embodiment of the present disclosure is shown;

[0051] Figure 7 A schematic diagram showing the structure of a multi-cluster management platform in an exemplary embodiment of the present disclosure is shown;

[0052] Figure 8 A schematic structural diagram of an electronic device in an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0053] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0054] The terms "a", "an", "the" and "said" are used in this specification to indicate the presence of one or more elements / components / etc.; the terms "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first" and "second" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0055] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and their repeated descriptions will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0056] Related technologies usually only support the scheduling process for a single cluster (including multiple computing nodes). These technologies generally use unified scheduling load distribution technology, which collects the operating status and load conditions of all computing nodes by defining one or more scheduling nodes, and then performs real-time computing analysis according to the predetermined scheduling strategy and allocates computing tasks to each computing node. However, the above solution has limitations in multi-cluster application scenarios because it cannot solve the problem of cross-cluster resource scheduling well.

[0057] Therefore, the present disclosure proposes a cross-cluster resource balancing scheduling system and method, which supports intelligent computing resource scheduling and end-to-end capability opening of multiple clusters. By selecting a node as the master node in each cluster, deploying a cluster monitoring component in the node, and monitoring the interface bandwidth utilization and resource status of the cluster, the resource utilization of each cluster can be balanced and network congestion can be reduced.

[0058] In the embodiments of the present disclosure, a multi-cluster scheduling method is firstly provided, which at least to some extent overcomes the defect in the related art that resources cannot be effectively scheduled in a multi-cluster environment.

[0059] Figure 1 A schematic flow chart of a multi-cluster scheduling method in an embodiment of the present disclosure is shown. The execution subject of the multi-cluster scheduling method may be a multi-cluster management platform.

[0060] refer to Figure 1 According to an embodiment of the present disclosure, a multi-cluster scheduling method includes the following steps:

[0061] Step S110, receiving a cluster allocation request; the cluster allocation request includes computing resource demand information of the service requester;

[0062] Step S120, selecting at least one target cluster that meets the computing resource demand information from multiple clusters according to the operating status of each cluster; the operating status includes the computing resource usage;

[0063] Step S130: Generate a cluster allocation response according to the cluster information of each target cluster and the computing resource usage of each target cluster, and feed back the cluster allocation response to the service requester, so that the service requester schedules at least one target cluster according to the cluster allocation response.

[0064] exist Figure 1 In the technical solution provided by the illustrated embodiment, on the one hand, by monitoring the operating status of each cluster (including the usage of computing resources), the method can intelligently select the cluster that best suits the current business request, avoiding the situation where some clusters are overloaded while other clusters are idle, thereby efficiently performing resource scheduling in a multi-cluster environment; further, it can also quickly analyze the status of each cluster based on the cluster monitoring component, and make the best choice based on this, thereby shortening the task allocation time and improving the response speed and service quality of the overall system; on the other hand, when one or several clusters fail, the scheduling mechanism in the present disclosure can automatically adjust the task allocation according to the latest cluster operating status to ensure that key businesses are not affected, thereby enhancing the stability and reliability of the entire system.

[0065] The following Figure 1 The specific implementation process of each step is described in detail:

[0066] Before step S110, it should be noted that the cluster monitoring component can be deployed in each cluster. In an optional implementation, a master node can be selected in each cluster, and other nodes except the master node can be used as auxiliary nodes, and then the cluster monitoring component can be deployed on the master node to monitor the operating status of each node in the cluster and report the operating status to the multi-cluster management platform; in another optional implementation, the cluster monitoring component can also be deployed on any auxiliary node, which can be set according to actual conditions, and the present disclosure does not make special restrictions on this.

[0067] Exemplarily, the operation status of the cluster may include the service status of each node in the cluster and the computing resource usage of each node. Specifically, the service status of each node may be the availability of each node in the cluster. Exemplarily, it may be whether each node is currently idle or occupied, and whether it is currently available. The computing resource usage of each node may be the utilization efficiency and usage of the computing resources on each node. Exemplarily, the computing resources may be CPU (Central Processing Unit), GPU (Graphics Processing Unit), memory, disk I / O (Input / Output), network bandwidth, etc.

[0068] Exemplarily, the operating status of the cluster may also be the overall operating status of the cluster summarized by the cluster monitoring components, such as the number of nodes in the cluster that are available, statistical data of all available computing resources in the cluster, etc., which can be set according to actual conditions, and the present disclosure does not make any special limitations on this.

[0069] Therefore, the multi-cluster management platform in the present disclosure can receive the operating status of each cluster reported by the above cluster monitoring component, and perform cross-cluster scheduling and management according to the operating status of each cluster.

[0070] Next reference Figure 1 , in step S110 , a cluster allocation request is received.

[0071] In this step, the service requester can first send a cluster allocation request to the service open platform. After receiving the above cluster allocation request, the service open platform can authenticate the service requester. After the identity authentication is passed, the above cluster allocation request can be forwarded to the multi-cluster management platform, so that the multi-cluster management platform can receive the cluster allocation request. Exemplarily, the service open platform can authenticate the above service requester based on username + password, digital certificate, API key and token, etc., which can be set according to actual conditions, and this disclosure does not make special restrictions on this.

[0072] Among them, the above-mentioned cluster allocation request can be a request for applying for the allocation of a cluster to perform batch computing tasks, and the cluster allocation request can include computing power resource demand information, such as: storage space requirements, response time requirements, etc., which can be set according to actual conditions, and the present disclosure does not make special restrictions on this. Exemplarily, the above-mentioned cluster allocation request can be an algorithm training service request (used to request the allocation of a cluster to perform algorithm training and reasoning services), and the algorithm training service request can include data sets, model file information, algorithm templates, storage space requirements, CPU core number requirements, GPU card number, network bandwidth requirements, etc.

[0073] It should be noted that, in the case where the identity authentication fails, a prompt message may also be sent to the service requester to inform the reason for the identity authentication failure.

[0074] In step S120, at least one target cluster that meets the computing resource demand information is selected from multiple clusters according to the operating status of each cluster.

[0075] In this step, after receiving the cluster allocation request forwarded by the above-mentioned service open platform, the multi-cluster management platform can check the operating status of each cluster, that is, check the service status of each node in each cluster and its computing power resource usage, and then select at least one target cluster that meets the computing power resource demand information from multiple clusters according to the operating status of each cluster. Exemplarily, multiple idle clusters in an idle state can be first determined from multiple clusters, and then at least one target cluster whose available resources meet the above-mentioned computing power resource demand can be selected from the above-mentioned multiple idle clusters. It can be set according to the actual situation, and this disclosure does not make any special restrictions on this.

[0076] In an optional implementation, the multi-cluster management platform can select at least one target cluster that meets the computing resource demand information from multiple clusters based on the pre-configured cluster screening strategy according to the operating status of each cluster, wherein the above cluster screening strategy uses at least one capability requirement parameter as a key consideration factor. Exemplarily, the above capability requirement parameters may include: cost, latency, energy consumption, error rate and security, etc. Thus, exemplarily, the above cluster screening strategy may include a cost priority strategy, a green and low-carbon strategy, a performance priority strategy, etc. Among them, the cost priority strategy will mainly consider the lowest cost of computing resources and network transmission resources, the green and low-carbon strategy will mainly consider the cluster with the smallest PUE (Power Usage Effectiveness), and the performance priority strategy will mainly consider comprehensive latency, cost and energy consumption and other factors.

[0077] In step S130, a cluster allocation response is generated according to the cluster information of each target cluster and the computing resource usage of each target cluster, and the cluster allocation response is fed back to the service requester, so that the service requester schedules at least one target cluster according to the cluster allocation response.

[0078] In this step, after determining at least one target cluster, a cluster allocation response can be generated based on the cluster information of each target cluster and the computing resource usage of each target cluster, and the cluster allocation response can be fed back to the above-mentioned service open platform, so that the service open platform can feed back the above-mentioned cluster allocation response to the business requester, so that the business requester can schedule at least one target cluster according to the above-mentioned cluster allocation response.

[0079] Exemplarily, the above cluster allocation response may include the following information: cluster information of each target cluster (for example, cluster ID, cluster name, API server address, cluster status), cluster allocation response of each target cluster (for example, number of PODs, CPU allocatable amount, memory allocatable amount, GPU allocatable amount, etc.).

[0080] refer to Figure 2 , Figure 2 A flow chart showing how to generate a cluster allocation response in an embodiment of the present disclosure includes steps S201 to S204:

[0081] In step S201, the business requester sends an algorithm training request to the service open platform;

[0082] In step S202, the service open platform forwards the algorithm training request to the multi-cluster management platform to initiate batch computing power calls;

[0083] In step S203, the multi-cluster management platform calls the resource control platform and selects at least one target cluster that meets the computing resource demand information from multiple clusters according to the pre-configured cluster screening strategy;

[0084] In step S204, the resource control platform generates a cluster allocation response according to the cluster information of each target cluster and the computing resource usage of each target cluster, and the multi-cluster management platform forwards the cluster allocation response to the service open platform.

[0085] After receiving the above cluster allocation response, the service requester can send a cluster scheduling request to the service open platform. The cluster scheduling request can contain the task data of the computing task, and the task data includes the task description (for example, task target) and the data required to execute the computing task (for example, task data set, model file information, algorithm template, etc.). Afterwards, the service open platform can forward the above cluster scheduling request to the multi-cluster management platform so that the multi-cluster management platform schedules at least one target cluster to execute the computing task and obtain the task execution result.

[0086] In an optional implementation manner, when the at least one target cluster includes a target cluster, the service requester may Figure 3 The implementation method in obtains the task execution result. Figure 3 , Figure 3 A flowchart showing how to obtain a task execution result when at least one target cluster includes a target cluster in an embodiment of the present disclosure includes steps S301 to S303:

[0087] In step S301, a cluster scheduling request is received.

[0088] In this step, the multi-cluster management platform may receive the cluster scheduling request forwarded by the service open platform, where the cluster scheduling request is used to schedule the target cluster.

[0089] In step S302, the task data is dispatched to the master node of the target cluster, and the computing task is executed by the master node of the target cluster to generate the task execution result.

[0090] In this step, the multi-cluster management platform can dispatch the task data of the above computing task to the master node of the above target cluster, so that the master node executes the above computing task and generates a task execution result, which can include

[0091] It should be noted that after scheduling the task data of the above-mentioned computing task to the master node of the above-mentioned target cluster, the multi-cluster management platform can also obtain the computing resource usage of the master node. If the computing resource usage of the master node meets the preset computing resource usage standard (for example: bandwidth utilization is less than or equal to 70%), the computing task is executed through the master node of the target cluster and the task execution result is generated; if the computing resource usage of the master node does not meet the preset computing resource usage standard (bandwidth utilization is greater than 70%), the cluster scheduling request can be re-initiated to request that the above-mentioned task data be scheduled to the auxiliary node of the target cluster, so as to execute the above-mentioned computing task through the auxiliary node of the target cluster and generate the task execution result.

[0092] In step S303, the task execution result is returned to the service open platform, so that the service open platform feeds back the task execution result to the business requester.

[0093] In this step, the multi-cluster management platform may return the task execution result to the service open platform, so that the service open platform feeds back the task execution result to the service requester.

[0094] Exemplarily, in addition to the above-mentioned task execution results, the multi-cluster management platform can also return the above-mentioned task ID, cluster ID of the target cluster, cluster name, current memory usage of the target cluster, current CPU usage, current CPU usage, etc. to the above-mentioned service open platform, so that the service open platform can also feedback the above-mentioned information to the business requester.

[0095] In another optional implementation, when the at least one target cluster includes multiple target clusters, the service requester may Figure 4 The implementation method in obtains the task execution result. Figure 4 , Figure 4 A flowchart showing how to obtain a task execution result when at least one target cluster includes multiple target clusters in an embodiment of the present disclosure includes steps S401 to S403:

[0096] In step S401, a cluster scheduling request is received.

[0097] In this step, the multi-cluster management platform may receive the cluster scheduling request forwarded by the service open platform, where the cluster scheduling request is used to schedule the target cluster.

[0098] In step S402, the task data is split into a plurality of subtask data, and each subtask data is sent to the master node of each target cluster, and each subtask is executed by the master node of each target cluster and a subtask execution result is generated.

[0099] In this step, illustratively, the multi-cluster management platform may split the task data into multiple subtask data, and send each subtask data to the master node of each target cluster, and execute each subtask through the master node of each target cluster and generate a subtask execution result.

[0100] Specifically, if the computing power resource usage of the main node of a target cluster meets the preset computing power resource usage standard (for example, bandwidth utilization is less than or equal to 70%), the corresponding subtask can be executed through the main node of the target cluster and the subtask execution result can be generated. If the computing power resource usage of the main node of a target cluster does not meet the above-mentioned preset computing power resource usage standard (bandwidth utilization is greater than 70%), a cluster scheduling request can be re-initiated to request that the corresponding subtask data be scheduled to the auxiliary node of the target cluster, so that the corresponding subtask can be executed through the auxiliary node of the target cluster and the subtask execution result can be generated.

[0101] In step S403, the execution results of the multiple subtasks are integrated, and the obtained comprehensive execution result is returned to the service open platform, so that the service open platform feeds back the comprehensive execution result to the service requester.

[0102] In this step, the multi-cluster management platform can integrate the execution results of the above multiple subtasks, and return the obtained comprehensive execution result to the service open platform, so that the service open platform feeds back the comprehensive execution result to the service requester.

[0103] refer to Figure 5 , Figure 5 A flowchart showing how to generate a task execution result in an embodiment of the present disclosure includes steps S501 to S505:

[0104] In step S501, the service requester sends a cluster scheduling request to the service open platform;

[0105] In step S502, the service open platform forwards the cluster scheduling request to the multi-cluster management platform, so that the multi-cluster management platform calls the resource control platform to initiate cluster scheduling;

[0106] In step S503, the task data is dispatched to the master node of the target cluster;

[0107] In step S504, if the primary node does not meet the demand, the computing power call is re-initiated and dispatched to the secondary node of the target cluster;

[0108] In step S505, the cluster monitoring component deployed on the cluster master node feeds back the task execution result to the service open platform.

[0109] The overall process of the multi-cluster scheduling method in the present disclosure is schematically described below in combination with a specific business scenario:

[0110] First, the service requester can initiate an algorithm training service request. The request parameters include data set, model file information algorithm template, CPU core number requirement, GPU card number, storage requirement, network bandwidth requirement, etc.

[0111] Second, the business requester sends an algorithm training service request through the service open platform, applies for batch computing resource calls, and executes batch computing tasks;

[0112] Third, the service open platform forwards the algorithm training service request to the multi-cluster management platform, applies for batch computing resource calls, and executes batch computing tasks;

[0113] Fourth, the multi-cluster management platform checks the corresponding resources and selects the target cluster that meets the computing resource requirements based on factors such as latency, energy consumption, cost, and performance;

[0114] Fifth, a cluster allocation response is generated according to the cluster information of each target cluster and the computing resource usage of each target cluster, and the cluster allocation response is fed back to the service requester. The cluster allocation response includes: cluster ID, cluster name, API server address, cluster status, number of PODs, CPU allocatable amount, memory allocatable amount, GPU allocatable amount and other information for the service requester to select;

[0115] Sixth, the multi-cluster management platform calls the target cluster to perform algorithm training and promotion tasks;

[0116] Seventh, dispatch the task data to the master node of the target cluster. If the Agent monitors the bandwidth utilization of the master node interface of the cluster and the bandwidth utilization rate is less than 70% and the computing resource requirements are met, the master node will execute the algorithm training task.

[0117] Eighth, dispatch the task data to the master node of the target cluster. If the Agent monitors the bandwidth utilization of the master node interface of the cluster and the utilization rate is greater than 70%, the Agent sets the node as a tainted node (i.e., sets the node device to an unschedulable state) and re-initiates the computing power call to dispatch it to the slave node of the target cluster.

[0118] Ninth, the multi-cluster management platform sends the algorithm training service call results (including task id, cluster id, cluster name, current memory usage, current CPU usage, current CPU usage, etc.) to the business requester through the service open platform.

[0119] refer to Figure 6 , Figure 6 The overall architecture diagram of the multi-cluster scheduling method in the embodiment of the present disclosure is shown as follows: Figure 6 As shown:

[0120] The service requester can send a service request to the service open platform;

[0121] The service open platform can authenticate the identity of the service requester, and after the identity authentication is passed, the service request can be forwarded to the multi-cluster management platform;

[0122] The multi-cluster management platform can call the resource control platform, select the master node in each cluster (the nodes other than the master node are auxiliary nodes) through the scheduling and balancing control module of the resource control platform, and deploy the cluster monitoring component Agent on each master node to select the target cluster according to the computing power resource usage reported by the cluster monitoring component, so as to balance the computing power resource usage of each cluster and reduce network congestion.

[0123] Based on the above technical solutions, the present disclosure has at least the following technical effects:

[0124] First, improve resource utilization: Through cross-cluster load balancing, tasks can be more reasonably allocated to nodes in different clusters, avoiding overloading of some clusters while idling resources in other clusters, thereby improving overall resource utilization;

[0125] Second, enhance system reliability: In a multi-cluster environment, by assigning tasks to different clusters, the risk of single point failure can be reduced. Even if a cluster has a problem, other clusters can still continue to provide services, which enhances the reliability and stability of the system.

[0126] Third, support business expansion: As the business develops, a single cluster may not be able to meet all business needs. The cross-cluster load balancing method provided by the present disclosure can support smooth business expansion without large-scale changes to the existing architecture.

[0127] The present disclosure also provides a multi-cluster management platform, which is used to manage multiple clusters. A cluster monitoring component is deployed in each cluster, and the cluster monitoring component is used to monitor the operating status of each cluster. Figure 7 A schematic diagram showing the structure of a multi-cluster management platform in an exemplary embodiment of the present disclosure is shown; Figure 7 As shown, the multi-cluster management platform 700 may include a request receiving module 710, a cluster selection module 720, and a request response module 730. Among them:

[0128] The request receiving module 710 is used to receive a cluster allocation request; the cluster allocation request includes computing resource demand information of the service requester;

[0129] A cluster selection module 720 is used to select at least one target cluster that meets the computing resource demand information from the multiple clusters according to the operating status of each cluster; the operating status includes computing resource usage;

[0130] The request response module 730 is used to generate a cluster allocation response according to the cluster information of each target cluster and the computing resource usage of each target cluster, and feed back the cluster allocation response to the service requester so that the service requester schedules the at least one target cluster according to the cluster allocation response.

[0131] In an exemplary embodiment of the present disclosure, the request receiving module 710 receives a cluster allocation request, including:

[0132] Receiving the cluster allocation request sent by the service requester through the service open platform, and performing identity authentication on the service requester;

[0133] After the identity authentication is passed, the cluster allocation request forwarded by the service open platform is received.

[0134] In an exemplary embodiment of the present disclosure, the cluster selection module 720 selects at least one target cluster that meets the computing resource demand information from the multiple clusters according to the operating status of each cluster, including:

[0135] According to the operating status of each cluster, selecting at least one target cluster that meets the computing resource demand information from the multiple clusters based on a pre-configured cluster screening strategy;

[0136] The cluster screening strategy uses at least one capability requirement parameter as a key consideration factor.

[0137] In an exemplary embodiment of the present disclosure, the at least one target cluster includes a target cluster;

[0138] After feeding back the cluster allocation response to the service requester, the request response module 730 is configured to:

[0139] Receive a cluster scheduling request; the cluster scheduling request is used to request scheduling of the target cluster, the cluster scheduling request includes task data of a computing task, and the task data includes a task description and data required to execute the computing task;

[0140] Dispatching the task data to the master node of the target cluster, executing the computing task through the master node of the target cluster and generating a task execution result;

[0141] The task execution result is returned to the service open platform, so that the service open platform feeds back the task execution result to the business requester.

[0142] In an exemplary embodiment of the present disclosure, the request response module 730 executes the computing task through the master node of the target cluster and generates a task execution result, including:

[0143] Sending the task data to the master node of the target cluster;

[0144] In response to the computing resource usage of the master node meeting the preset computing resource usage standard, executing the computing task through the master node of the target cluster and generating a task execution result;

[0145] In response to the computing resource usage of the master node not meeting the preset computing resource usage standard, the computing task is executed by the secondary node of the target cluster and a task execution result is generated.

[0146] In an exemplary embodiment of the present disclosure, the at least one target cluster includes a plurality of target clusters;

[0147] After feeding back the cluster allocation response to the service requester, the request response module 730 is configured to:

[0148] Receive a cluster scheduling request; the cluster scheduling request is used to request scheduling of the target cluster, the cluster scheduling request includes task data of a computing task, and the task data includes a task description and data required to execute the computing task;

[0149] Splitting the task data into multiple subtask data, and sending each subtask data to the master node of each target cluster, executing each subtask through the master node of each target cluster and generating a subtask execution result;

[0150] The execution results of the plurality of subtasks are integrated, and the obtained comprehensive execution result is returned to the service open platform, so that the service open platform feeds back the comprehensive execution result to the service requester.

[0151] In an exemplary embodiment of the present disclosure, the request response module 730 executes each subtask through the master node of each target cluster and generates a subtask execution result, including:

[0152] In response to the computing resource usage of the master node meeting the preset computing resource usage standard, executing each of the subtasks through the master node of each target cluster and generating the subtask execution result;

[0153] In response to the computing resource usage of the master node not satisfying the preset computing resource usage standard, each subtask is executed by the secondary node of each target cluster and the subtask execution result is generated.

[0154] The specific details of each module in the above multi-cluster management platform have been described in detail in the corresponding multi-cluster scheduling method, so they will not be repeated here.

[0155] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0156] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0157] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0158] The present disclosure also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist independently without being assembled into the electronic device.

[0159] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0160] Computer-readable storage media can send, propagate or transmit programs for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0161] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiments.

[0162] In addition, an electronic device capable of implementing the above method is also provided in an embodiment of the present disclosure.

[0163] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0164] Refer to the following Figure 8 The electronic device 800 according to this embodiment of the present disclosure is described. Figure 8 The electronic device 800 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0165] like Figure 8 As shown, the electronic device 800 is in the form of a general computing device. The components of the electronic device 800 may include but are not limited to: at least one processor 810, at least one memory 820, a bus 830 connecting different system components (including the memory 820 and the processor 810), and a display 840.

[0166] The memory stores program codes, which can be executed by the processor 810, so that the processor 810 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processor 810 can perform the following steps: Figure 1As shown in: Step S110, receiving a cluster allocation request; the cluster allocation request includes computing power resource demand information of the business requester; Step S120, selecting at least one target cluster that meets the computing power resource demand information from multiple clusters according to the operating status of each cluster; the operating status includes computing power resource usage; Step S130, generating a cluster allocation response according to the cluster information of each target cluster and the computing power resource usage of each target cluster, and feeding back the cluster allocation response to the business requester, so that the business requester schedules at least one target cluster according to the cluster allocation response.

[0167] The memory 820 may include a readable medium in the form of a volatile storage, such as a random access memory (RAM) 8201 and / or a cache memory 8202 , and may further include a read-only memory (ROM) 8203 .

[0168] The memory 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0169] Bus 830 may be a bus representing one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0170] The electronic device 800 may also communicate with one or more external devices 900 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 800, and / or communicate with any device that enables the electronic device 800 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 850. Furthermore, the electronic device 800 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 860. As shown, the network adapter 860 communicates with other modules of the electronic device 800 via a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0171] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A multi-cluster scheduling method, characterized in that: Applied to a multi-cluster management platform, the multi-cluster management platform is used to manage multiple clusters, a cluster monitoring component is deployed in each cluster, and the cluster monitoring component is used to monitor the operating status of each cluster, the method includes: Receive a cluster allocation request; the cluster allocation request includes computing resource demand information of the service requester; Selecting at least one target cluster that meets the computing resource demand information from the multiple clusters according to the operating status of each cluster; the operating status includes computing resource usage; A cluster allocation response is generated according to the cluster information of each target cluster and the computing resource usage of each target cluster, and the cluster allocation response is fed back to the service requester, so that the service requester schedules the at least one target cluster according to the cluster allocation response.

2. The method according to claim 1, characterized in that The receiving a cluster allocation request comprises: Receiving the cluster allocation request sent by the service requester through the service open platform, and performing identity authentication on the service requester; After the identity authentication is passed, the cluster allocation request forwarded by the service open platform is received.

3. The method according to claim 2, characterized in that The selecting at least one target cluster that meets the computing resource demand information from the multiple clusters according to the operating status of each cluster includes: According to the operating status of each cluster, selecting at least one target cluster that meets the computing resource demand information from the multiple clusters based on a pre-configured cluster screening strategy; The cluster screening strategy uses at least one capability requirement parameter as a key consideration factor.

4. The method according to any one of claims 1 to 3, characterized in that: The at least one target cluster comprises a target cluster; After feeding back the cluster allocation response to the service requester, the method further includes: Receive a cluster scheduling request; the cluster scheduling request is used to request scheduling of the target cluster, the cluster scheduling request includes task data of a computing task, and the task data includes a task description and data required to execute the computing task; Dispatching the task data to the master node of the target cluster, executing the computing task through the master node of the target cluster and generating a task execution result; The task execution result is returned to the service open platform, so that the service open platform feeds back the task execution result to the business requester.

5. The method according to claim 4, characterized in that The executing the computing task through the master node of the target cluster and generating a task execution result includes: Sending the task data to the master node of the target cluster; In response to the computing resource usage of the master node meeting the preset computing resource usage standard, executing the computing task through the master node of the target cluster and generating a task execution result; In response to the computing resource usage of the master node not meeting the preset computing resource usage standard, the computing task is executed by the secondary node of the target cluster and a task execution result is generated.

6. The method according to any one of claims 1 to 3, characterized in that: The at least one target cluster includes a plurality of target clusters; After feeding back the cluster allocation response to the service requester, the method further includes: Receive a cluster scheduling request; the cluster scheduling request is used to request scheduling of the target cluster, the cluster scheduling request includes task data of a computing task, and the task data includes a task description and data required to execute the computing task; Splitting the task data into multiple subtask data, and sending each subtask data to the master node of each target cluster, executing each subtask through the master node of each target cluster and generating a subtask execution result; The execution results of the plurality of subtasks are integrated, and the obtained comprehensive execution result is returned to the service open platform, so that the service open platform feeds back the comprehensive execution result to the service requester.

7. The method according to claim 6, characterized in that The executing each subtask through the master node of each target cluster and generating a subtask execution result includes: In response to the computing resource usage of the master node meeting the preset computing resource usage standard, executing each of the subtasks through the master node of each target cluster and generating the subtask execution result; In response to the computing resource usage of the master node not satisfying the preset computing resource usage standard, each subtask is executed by the secondary node of each target cluster and the subtask execution result is generated.

8. A multi-cluster management platform, the multi-cluster management platform is used to manage multiple clusters, each cluster is deployed with a cluster monitoring component, the cluster monitoring component is used to monitor the operation status of each cluster, the platform comprises: A request receiving module, used for receiving a cluster allocation request; The cluster allocation request includes computing resource demand information of the service requester; A cluster selection module, configured to select at least one target cluster that meets the computing power resource demand information from the multiple clusters according to the operating status of each cluster; the operating status includes computing power resource usage; The request response module is used to generate a cluster allocation response according to the cluster information of each target cluster and the computing resource usage of each target cluster, and feed back the cluster allocation response to the service requester so that the service requester schedules at least one target cluster according to the cluster allocation response.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-cluster scheduling method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the multi-cluster scheduling method according to any one of claims 1 to 7 by executing the executable instructions.

Images