Resource scheduling method and device

Abstract

The embodiment of the invention provides a resource scheduling method and device, computer equipment, a computer readable storage medium, a computer program product and a system, and belongs to the field of distributed computing. The method comprises the steps that a target creation request is received, the target creation request is used for creating a target computing network, and the target computing network comprises a head node and a working node group; creating a head node according to the target creation request, wherein node information of the head node is stored in a target storage position; under the condition that the head node is successfully created, a working node group is created, and each working node group comprises one or more working nodes; wherein the one or more working nodes establish communication connection with the head node according to the node information read from the target storage position. According to the technical scheme provided by the embodiment of the invention, cross-cluster resource scheduling and integration can be realized, the resource bottleneck limitation of a single cluster is avoided, and the resource utilization rate and the expandability of the computing cluster are further improved.

Description

Technical Field

[0001] This application relates to the field of distributed computing technology, and in particular to a resource scheduling method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] When using distributed computing frameworks for large-scale computations, task deployment across multiple computing clusters is often necessary to improve task throughput. However, the computing network built using distributed computing frameworks only supports networking between head nodes and worker nodes within a single cluster. This means that once the head node is deployed, its address information is limited to the DNS / CoreDNS scope of the local cluster. Worker nodes in other computing clusters cannot complete registration because they cannot resolve the address across clusters. Ultimately, this forces the entire computing network to be locked within a single cluster resource pool, making it impossible to utilize the idle resources of other computing clusters or meet the elastic resource scaling requirements of large-scale tasks.

[0003] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the Invention

[0004] This application provides a resource scheduling method, apparatus, computer device, computer-readable storage medium, computer program product, and resource scheduling system to solve or alleviate one or more of the technical problems mentioned above.

[0005] One aspect of this application provides a resource scheduling method for use in resource scheduling nodes of a multi-computing cluster, the method comprising: Receive a target creation request, the target creation request being used to create a target computing network, the target computing network including a head node and a group of worker nodes; The head node is created according to the target creation request, and the node information of the head node is stored in the target storage location; If the head node is successfully created, the working node group is created, and each working node group includes one or more working nodes; The one or more working nodes establish a communication connection with the head node based on the node information read from the target storage location.

[0006] Optionally, creating the head node according to the target creation request includes: A first computing cluster is determined from the multiple computing clusters, and the first computing cluster has a first reserved resource pool; Create the head node in the first reserved resource pool.

[0007] Optionally, the method further includes: If the head node is successfully created, the head node will be started through the first computing cluster. In response to the successful startup of the head node, the node information of the head node is stored in the target storage location through the first computing cluster. The node information includes the service name and network address of the head node.

[0008] Optionally, creating the working node group includes: Determine the minimum number of nodes corresponding to the working node group based on the target creation request; A second computing cluster is determined from the multiple computing clusters based on the minimum number of nodes, and the second computing cluster has a second reserved resource pool; In the second reserved resource pool, create one or more of the working nodes corresponding to the minimum number of nodes; The first computing cluster may be the same as or different from the second computing cluster.

[0009] Optionally, the node information includes the node name of the head node and the network address of the head node; The one or more working nodes establish a communication connection with the head node based on the node information read from the target storage location, including: When one or more worker nodes are started, the started worker node obtains the network address from the target storage location based on the node name of the head node, and establishes a communication connection with the head node through the network address.

[0010] Optionally, the method further includes: The head node receives a scaling request in response to a target computing task, where the resource requirement of the target computing task is greater than the resource supply of the worker node group, and the resource supply is determined based on the minimum number of nodes in the worker node group. Based on the idle resources of each of the multiple computing clusters, determine the expansion computing cluster from the multiple computing clusters; Create one or more elastic worker nodes in the expanded computing cluster.

[0011] Optionally, the method further includes: Receive a scaling-down request from the head node, the scaling-down request being used to indicate that the idle time of the target worker node exceeds a preset time threshold, the target worker node being one of the plurality of worker nodes and the one or more elastic worker nodes; Based on the target worker node, a scaled-down computing cluster is determined from the multiple computing clusters, wherein the scaled-down computing cluster includes one or more elastic worker nodes; Send a scaling-down command to the scaling-down computing cluster to delete one or more elastic worker nodes from the scaling-down computing cluster.

[0012] Optionally, each computing cluster is configured with a cluster resource control layer; The cluster resource control layer is configured as follows: Obtain the resource status of the computing cluster, including the node status of the head node and / or worker nodes; The resource status is reported to the resource scheduling node.

[0013] Optionally, the cluster resource control layer is further configured as follows: Receive the raw configuration information sent by the resource scheduling node, the raw configuration information being used to describe the target computing network; Based on the environment information of the computing cluster and the original configuration information, target configuration information is generated. The target configuration information is used to create nodes in the computing cluster, and the nodes include the head node and / or the worker nodes.

[0014] Another aspect of this application provides a resource scheduling method for use in a computing cluster, the method comprising: In response to the startup of a worker node, the node information of the head node is read from the target storage location, the node information including the network address; Based on the network address, establish a communication connection between the working node and the head node; The head node is located in the computing cluster or in another computing cluster that is different from the computing cluster.

[0015] Another aspect of this application provides a resource scheduling apparatus for use in resource scheduling nodes of a multi-computing cluster, the apparatus comprising: The receiving module receives a target creation request, which is used to create a target computing network, the target computing network including a head node and a group of worker nodes; The head node creation module is used to create the head node according to the target creation request, and the node information of the head node is stored in the target storage location; The working node group creation module is used to create the working node group when the head node is successfully created. Each working node group includes one or more working nodes. The one or more working nodes establish a communication connection with the head node based on the node information read from the target storage location. Another aspect of this application provides a resource scheduling apparatus for use in a computing cluster, the apparatus comprising: A read module is used to read node information of the head node from the target storage location in response to the startup of the worker node, the node information including the network address; A connection module is used to establish a communication connection between the working node and the head node based on the network address; The head node is located in the computing cluster or in another computing cluster that is different from the computing cluster.

[0016] Another aspect of this application provides a computer device, including: At least one processor; and A memory that is communicatively connected to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0017] Another aspect of this application provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method described above.

[0018] Another aspect of this application provides a computer program product including a computer program that, when executed by a processor, implements the method described above.

[0019] Another aspect of this application provides a resource scheduling system for use in a multi-computing cluster, the system comprising: A multi-cluster resource scheduling layer is configured to: receive a target creation request, the target creation request being used to create a target computing network, the target computing network including a head node and a group of worker nodes; create the head node according to the target creation request; and, if the head node is successfully created, create the group of worker nodes, each group of worker nodes including one or more worker nodes. A single cluster resource controller is configured to initialize the head node and the worker nodes; The cluster control layer is configured to: store the node information of the head node in a target storage location when the head node starts; read the node information from the target storage location according to the node name of the head node when the worker nodes start; and establish communication connections between the multiple worker nodes and the head node according to the node information.

[0020] The embodiments of this application employing the above-described technical solution may have the following advantages: By establishing the association between the head node and the worker nodes through the node information stored in the target storage location, stable communication between the head node and worker nodes in different computing clusters is achieved. This enables cross-cluster resource scheduling and integration, avoids the resource bottleneck limitations of a single cluster, and further improves resource utilization and the scalability of the computing cluster. Attached Figure Description

[0021] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0022] Figure 1 The diagram illustrates the operating environment of the resource scheduling method according to Embodiment 1 of this application. Figure 2 A flowchart illustrating a resource scheduling method according to Embodiment 1 of this application is shown schematically; Figure 3 The flowchart of the sub-steps of step S202 is shown schematically; Figure 4 This schematically illustrates another flowchart of the resource scheduling method according to Embodiment 1 of this application; Figure 5 The flowchart of the sub-steps of step S204 is shown schematically; Figure 6 This schematically illustrates another flowchart of the resource scheduling method according to Embodiment 1 of this application; Figure 7 This schematically illustrates another flowchart of the resource scheduling method according to Embodiment 1 of this application; Figure 8 This schematically illustrates a workflow diagram of the cluster resource control layer according to Embodiment 1 of this application; Figure 9 This schematically illustrates another workflow diagram of the cluster resource control layer according to Embodiment 1 of this application; Figure 10 This diagram illustrates the working environment of a cluster resource control layer. Figure 11 An exemplary application flowchart of the resource scheduling method according to Embodiment 1 of this application is illustrated; Figure 12 A flowchart illustrating a resource scheduling method according to Embodiment 2 of this application is shown schematically; Figure 13 A block diagram of a resource scheduling apparatus according to Embodiment 3 of this application is shown schematically; Figure 14A block diagram of a resource scheduling apparatus according to Embodiment 4 of this application is shown schematically; Figure 15 This illustration schematically shows a hardware architecture diagram of a computer device according to Embodiment 5 of this application. Figure 16 The schematic diagram illustrates the architecture of a resource scheduling system according to Embodiment 8 of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0024] It should be noted that the descriptions involving "first," "second," etc., in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0025] It should be noted that, in any stage of this application involving the collection, storage, use, transmission, and processing of data, each stage strictly adheres to the laws, regulations, industry standards, and regulatory requirements of the data source, usage location, and relevant countries and regions to ensure the legality and compliance of data activities. In the collection stage, the purpose, method, and scope of collection are clearly communicated to the data subject in a prominent manner. Collection is conducted only after obtaining the data subject's legal authorization, ensuring that the collection process follows the "minimum necessary" principle and does not exceed the scope of data collection. In the storage stage, storage periods are limited, and data is promptly deleted or anonymized / encrypted after the storage purpose is achieved. In the usage stage, a strict data security protection mechanism is implemented, using field-level desensitization technology and processing the original data according to preset desensitization rules. For different types of data, multiple desensitization strategies, such as data generalization, data anonymization, and data encryption, are employed to effectively mitigate the risk of sensitive information leakage and ensure that all data used is securely processed and desensitized, comprehensively protecting the rights and interests of data subjects and data security. In the transmission and processing stages, the confidentiality and security of data are ensured during transmission and processing.

[0026] In the description of this application, it should be understood that the numerical labels before the steps do not indicate the order of the steps, but are only used to facilitate the description of this application and to distinguish each step, and therefore should not be construed as a limitation of this application.

[0027] First, a definition of the terminology used in this application is provided: Distributed computing: a computing paradigm that divides massive computing tasks into multiple subtasks, distributes them across multiple networked computers (nodes) for parallel processing, and then aggregates the results to shorten the overall computing time and overcome the performance bottleneck of a single machine.

[0028] Cluster: A system that interconnects and manages multiple independent computers (nodes) through a network, presenting itself to the outside world as a single, elastically scalable pool of high-performance computing and storage resources.

[0029] GCS (Global Control Service): A core component within the head node of the computing network, responsible for maintaining the cluster's metadata.

[0030] Autoscaler: The built-in elastic brain of the computing network, continuously monitors task resource requirements and node load, and issues scaling-up or scaling-down commands.

[0031] Head node: The central controller of the computing network, responsible for coordinating and scheduling tasks across the entire computing network, managing the status of worker nodes, and maintaining the computing network metadata. It contains multiple components such as autoscaler and GCS.

[0032] Worker node: A node in a computing network that is responsible for executing specific computing tasks. It receives and executes task instructions issued by the head node and is the core computing carrier of the computing network.

[0033] CoreDNS: The default DNS server in the cluster, used to discover services within the cluster. It can resolve cluster service names to their corresponding Pod IP addresses, ensuring communication between components within the cluster.

[0034] OOM (Out Of Memory): refers to the "memory overflow" state triggered by the system running out of available memory.

[0035] Secondly, to facilitate understanding of the technical solutions provided in the embodiments of this application by those skilled in the art, the relevant technologies are described below: In distributed computing, the head node information of the distributed computing network is only stored within the scope of a single cluster's DNS or CoreDNS. Worker nodes in other clusters cannot resolve across domains, causing the computing network to be locked to a single cluster resource pool, and the head node lacks a globally visible anchor point. Simultaneously, the total amount of resources in a single cluster is fixed, and the upper limit of schedulable resources directly restricts the task scale. Requesters must manually split ultra-large-scale jobs and deploy them separately, increasing operation and maintenance costs. Furthermore, idle resources in each cluster are scattered and their real-time status is not visible, making horizontal comparisons impossible for the scheduling system. Fragmented computing power remains idle for extended periods, and the bottleneck of a single cluster cannot be overcome. After a task ends, offline nodes lack a unified recycling mechanism, and idle instances continue to occupy resources, causing secondary waste. In addition, there is a lack of real-time reporting channels for cross-cluster resource levels and node health information. The scheduling side can only rely on delayed or missing data for decision-making, further exacerbating resource waste and task waiting.

[0036] Therefore, this application provides a resource scheduling technical solution. In this technical solution, (1) the node information of the created head node is stored in a specific target storage location, which can realize the global visibility of the head node, so that it can be perceived and discovered by the subsequently created worker nodes, providing a unique and stable anchor point for subsequent cross-cluster resource scheduling; (2) the single cluster resource pool is expanded into a multi-cluster unified resource pool, which improves the total amount of schedulable resources for large-scale computing tasks and the overall utilization rate of multi-cluster computing resources. Meanwhile, the increase in the total amount of schedulable resources has led to an increase in the scale of tasks carried out, which has also reduced the workload of manual task splitting when facing ultra-large-scale computing tasks, and reduced the operation and maintenance costs of multiple computing clusters; (3) In multiple computing clusters, the global idle resource status is compared in real time, and elastic work nodes are created in the computing cluster with the most abundant idle resources, which breaks through the bottleneck of computing resources in a single computing cluster. At the same time, the fragmented idle computing power in multiple computing clusters is utilized, which improves the resource utilization rate of multiple computing clusters and the schedulable scale of target computing tasks; (4) By triggering the shrinkage command when the node is idle, the idle elastic work nodes can be recycled from the computing cluster in a timely manner, realizing the rapid release of global resources, avoiding the fragmentation of idle resources, reducing the resource waste caused by the long-term idle operation of elastic work nodes after the offline task ends, and improving the resource utilization rate of multiple computing clusters; (5) By continuously collecting and reporting the real-time status of the head node and work nodes in each computing cluster, the resource scheduling node can perceive the cross-cluster resource level and node health status in real time, which can provide a real-time and accurate data basis for subsequent expansion and shrinkage decisions, reducing resource waste and task waiting caused by scheduling delays or misjudgments. See below for details.

[0037] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0038] like Figure 1 As shown in the diagram, the runtime environment includes: requester 2 and resource pool. The resource pool can include multiple clusters 4.

[0039] Requestor 2 is the party requesting the computing task, which can be a local application, server, microservice in a cloud platform, distributed data processing system, scheduling system, etc.

[0040] Cluster 4 can be multiple service clusters, or it can be an automatically managed cluster formed by distributing multiple containers across multiple physical or virtual computers using a container orchestration platform. Cluster 4 is used to provide distributed computing services.

[0041] Resource pools can provide distributed computing services via a network. The network can include various network devices such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and / or similar devices. The network can include physical links, such as coaxial cable links, twisted-pair cable links, fiber optic links, or combinations thereof, or wireless links, such as cellular links, satellite links, and Wi-Fi links.

[0042] It should be noted that the number of claimants 2 and clusters 4 shown in the diagram is merely illustrative and is not intended to limit the scope of patent protection of this application. Depending on the actual situation, there may be any number of claimants 2 and clusters 4.

[0043] The technical solutions of this application are described below through several embodiments. It should be understood that these embodiments can be implemented in many different forms and should not be construed as being limited to the embodiments set forth herein.

[0044] Example 1 This method can be used in resource scheduling nodes of multiple computing clusters. The resource scheduling node is the single execution entity of this process.

[0045] Figure 2 A flowchart illustrating a resource scheduling method according to Embodiment 1 of this application is shown schematically.

[0046] like Figure 2 As shown, it may include steps S200~S204, wherein: Step S200: Receive a target creation request, the target creation request being used to create a target computing network, the target computing network including a head node and a group of worker nodes; Step S202: Create the head node according to the target creation request, and store the node information of the head node in the target storage location; Step S204: If the head node is successfully created, create the working node group, each working node group including one or more working nodes; wherein, the one or more working nodes establish a communication connection with the head node based on the node information read from the target storage location.

[0047] The resource scheduling method provided in this embodiment stores the head node's node information in a target storage location after creating the head node. This allows worker nodes to establish an association (e.g., communication connection) with the head node after creation, based on the node information stored in the target storage location. This enables stable communication between the head node and worker nodes in different computing clusters, thereby achieving cross-cluster resource scheduling and integration. This avoids the resource bottleneck limitations of a single cluster and further improves resource utilization and the scalability of the computing cluster.

[0048] The following combination Figure 2 The steps in steps S200 to S204, as well as other optional steps, are described in detail.

[0049] Step S200 The system receives a target creation request, which is used to create a target computing network, the target computing network including a head node and a group of worker nodes.

[0050] The target creation request can be initiated by the requester through a scheduling service. The scheduling service can refer to a service used to manage and schedule tasks or workloads in a distributed computing, containerized environment, or computing cluster. In some embodiments, the target creation request can also be issued by the requester through a global scheduler, which can run in a management cluster independent of the multiple computing clusters and issue the target creation request to the resource scheduling nodes through relevant interfaces.

[0051] The target creation request can include the definition and configuration of worker node groups, elastic scaling strategies, etc. There can be one or more worker node groups, and each worker node group can be configured with independent resource specifications, minimum number of nodes, and maximum number of nodes.

[0052] Step S202 The head node is created according to the target creation request, and the node information of the head node is stored in the target storage location.

[0053] The head node is the central controller of the target computing network and can contain multiple core components, such as the Global Control Service (GCS) and an autoscaler component for elastic resource scaling. In some embodiments, the host cluster or resource pool for the head node can be selected based on factors such as cluster load, operating costs, network latency, data locality, or data structure. After creation, the head node can be instantiated in the selected resource pool. After instantiation, health checks and other operations can be performed on the head node, and the health results and node status can be reported to the resource scheduling node.

[0054] The target storage location can be a specific cluster within the multi-computing cluster, or an external registry center in a management cluster independent of the multi-computing cluster. In some embodiments, the target storage location can be configured to be read and written only by resource scheduling nodes or other authorized components, with worker nodes having only read permissions and no write permissions. Sensitive fields in the node information of the head node can also be stored encrypted or only as references, with worker nodes only able to retrieve them through controlled channels.

[0055] In this embodiment, storing the node information of the created head node in a specific target storage location can achieve global visibility of the head node, enabling it to be perceived and discovered by subsequently created worker nodes, thus providing a unique and stable anchor point for subsequent cross-cluster resource scheduling.

[0056] As mentioned earlier, the host cluster or resource pool of the head node can be selected based on various methods or conditions. The following provides an example selection method.

[0057] In optional embodiments, such as Figure 3 As shown, step S202 includes: S300, determine a first computing cluster from the multiple computing clusters, the first computing cluster having a first reserved resource pool.

[0058] S302, Create the head node in the first reserved resource pool.

[0059] The first computing cluster suitable for hosting the head node can be filtered from multiple computing clusters based on strategies such as minimum cost, minimum queuing latency, highest success rate, load balancing (minimum utilization / minimum fragmentation), and data locality (proximity to data source / storage). This first reserved resource pool can be declared during cluster initialization or dynamically generated by each computing cluster during operation based on long-term unused idle computing resources within that cluster. When creating a head node in the first reserved resource pool, specific resource items (such as instance specifications, GPU card type, memory, disk type / capacity, IP / subnet, etc.) can be allocated from the reserved resource pool, and a resource allocation record is generated. In some embodiments, a higher priority than the system default priority can be set for the head node during or after creation, and over-provisioning of head node resources can be rejected.

[0060] In this embodiment, the head node is created in the first reserved resource pool, which can physically isolate the head node from the tidal mixed and preemptible elastic resources, ensuring that the head node runs in a stable resource position, reducing control layer jitter caused by cross-cluster resource competition or preemption, thereby providing a continuously available control and scheduling center for the tasks of multiple computing clusters.

[0061] In optional embodiments, such as Figure 4 As shown, the method further includes: S400, if the head node is successfully created, start the head node through the first computing cluster.

[0062] S402, in response to the successful startup of the head node, the node information of the head node is stored in the target storage location through the first computing cluster. The node information includes the service name and network address of the head node.

[0063] Header nodes can be started as virtual machine instances, containers / pods, or launched as jobs and kept resident. After starting the head node, health checks (such as port scanning, HTTP / gRPC health interfaces, etc.), successful registration of critical services, and completion of initialization scripts can be used to determine whether the head node has started successfully. In some embodiments, in addition to service names and network addresses (such as node IP addresses), node information may also include cluster ID, protocol and port, version number, tags, certificate fingerprints, etc. Certain important information in the node information can be stored using encrypted fields, and access control measures such as allowing only authorized worker nodes or the control plane to read it can be implemented.

[0064] In this embodiment, the service name and network address of the successfully started head node are written to the target storage location. This allows the lifecycle status and routable information of the head node to be provided to all nodes in the target computing network, ensuring that subsequently created worker nodes can obtain valid head node access information at any time, thus improving the flexibility and reliability of the resource scheduling process.

[0065] Step S204 If the head node is successfully created, the working node group is created, and each working node group includes one or more working nodes; wherein, the one or more working nodes establish a communication connection with the head node based on the node information read from the target storage location.

[0066] Worker node groups can be of various types, such as CPU-based, GPU-based, or memory-based, and the target computing network may include one or more worker node groups. Different worker node groups can be configured with different node instance specifications, priorities, and minimum / maximum / desired number of nodes. In some embodiments, the resources required to create a worker node group can be obtained from a reserved resource pool, an on-demand pool, a bidding pool, or a preemptible pool, based on weights. In other embodiments, when there are multiple worker nodes in a worker node group, the resources required by the multiple worker nodes can also be obtained through distributed scheduling across multiple computing clusters or multiple availability zones within the same computing cluster.

[0067] Worker nodes can be instantiated or initialized in any of the multiple compute clusters. In some embodiments, a worker node can establish a communication connection with the head node immediately after instantiation or initialization. Alternatively, a worker node can be scheduled and started by a lower-level scheduler within the cluster before reading node information from the target storage location and establishing a communication connection with the head node.

[0068] In this embodiment, by utilizing the target storage location, worker nodes first read node information and then register themselves with the head node. This allows worker nodes to perceive head nodes in other computing clusters and join the target computing network without relying on components such as CoreDNS within the computing cluster. This expands the single-cluster resource pool into a unified multi-cluster resource pool, improving the total schedulable resources for large-scale computing tasks and the overall utilization rate of multi-cluster computing resources. Simultaneously, the increased task capacity resulting from the increased schedulable resources reduces the workload of manual task splitting when facing ultra-large-scale computing tasks, thus reducing the operation and maintenance costs of multiple computing clusters.

[0069] In optional embodiments, such as Figure 5 As shown, step S204 includes: S500, determine the minimum number of nodes corresponding to the working node group based on the target creation request.

[0070] S502, a second computing cluster is determined from the multiple computing clusters based on the minimum number of nodes, the second computing cluster having a second reserved resource pool.

[0071] S504, In the second reserved resource pool, create one or more working nodes corresponding to the minimum number of nodes; wherein the first computing cluster is the same as or different from the second computing cluster.

[0072] The minimum number of nodes can be the number of node instances in the worker node group specified in the target creation request, or it can be the number of nodes calculated from the resource requirements (such as total CPU, total memory, number of GPUs, etc.) or task expectations (such as parallelism, expected throughput, maximum queuing latency, etc.) read from the target creation request. In some embodiments, the minimum number of nodes can include the minimum number of nodes for cold start and the minimum number of nodes for steady state, where the minimum number of nodes for cold start is less than or equal to the minimum number of nodes for steady state. After the head node is successfully created and started, a small number of worker nodes can be created immediately based on the minimum number of nodes for cold start to quickly establish a usable computing network. Then, the remaining worker nodes can be created in a rolling manner based on the minimum number of nodes for steady state to supplement the network and achieve long-term keep-alive capability.

[0073] The second computing cluster can be determined by weighting multiple conditions such as minimum cost, shortest startup latency, and highest available capacity. The second reserved resource pool can be a pre-locked static reserved pool, a dynamic reserved pool for rapid expansion based on a strategy, a GPU pool or high memory pool reserved by specification family, or a dedicated pool reserved by requesting object, etc. In some embodiments, the second reserved resource pool can set quotas, priorities, and preemption rules for requesting objects or projects. When the first and second computing clusters are the same, the first and second reserved resource pools can also be set as different resource pools; for example, the first reserved resource pool can be a control layer resource pool, and the second reserved resource pool can be a computing resource pool.

[0074] In this embodiment, by establishing a minimum number of working nodes in the second reserved resource pool to form a working node group, the minimum availability of the target computing network can be guaranteed, thereby ensuring that the target computing network always maintains a minimum amount of computing resources to cope with computing tasks, thus improving the reliability and stability of the target computing network.

[0075] As mentioned earlier, node information can include various components, therefore worker nodes can also use different connection methods to establish communication connections with the head node based on different component formats. The following provides an example of a component format and its corresponding connection method.

[0076] In an optional embodiment, the node information includes the node name and network address of the head node, and the one or more worker nodes establish a communication connection with the head node based on the node information read from the target storage location, including: When one or more worker nodes are started, the started worker node obtains the network address from the target storage location based on the node name of the head node, and establishes a communication connection with the head node through the network address.

[0077] The startup of a worker node can be triggered in response to a request from an upper-level scheduler such as a resource scheduling node, or it can be triggered by scheduling by a lower-level scheduler within the cluster. In some embodiments, the network address can be the virtual service address and corresponding port within the cluster where the head node resides, or it can be the host IP address and node port of the head node.

[0078] In this embodiment, the worker nodes dynamically pull the network address of the head node directly from the target storage location based on the node name. This avoids the impact of DNS invisibility between different clusters, directly reads the real IP of the head node to establish a connection, reduces connection mismatch and scheduling delay caused by changes in the head node configuration, and improves the stability and reliability of the target computing network.

[0079] In the preceding steps, the minimum target computing network has been created with the minimum number of nodes. In practical use, the target computing network can be expanded to meet the resource requirements of the computing task. An exemplary resource expansion method is provided below.

[0080] In optional embodiments, such as Figure 6 As shown, the method further includes: S600, receive the expansion request sent by the head node in response to the target computing task, wherein the resource requirement of the target computing task is greater than the resource supply of the working node group, and the resource supply is determined according to the minimum number of nodes in the working node group.

[0081] S602, Based on the idle resources of each of the multiple computing clusters, determine the expansion computing cluster from the multiple computing clusters.

[0082] S604, Create one or more elastic worker nodes in the expanded computing cluster.

[0083] The resource types in the resource demand and supply quantities can include one or more types such as CPU, GPU, memory, or video memory. When the resource supply quantity of any type cannot meet the resource demand quantity, the head node will issue an expansion request. Idle resources can include the reported but unreserved CPU allocation quantity, GPU allocation quantity, and video memory / memory capacity in each computing cluster. In some embodiments, a computing cluster is only identified as an expansion computing cluster when all types of resource demands (such as CPU, GPU, memory, or video memory) of the target computing task can be fully met by the idle resources of a certain computing cluster. In other embodiments, multiple computing clusters can be identified as expansion computing clusters simultaneously, and different elastic worker nodes can be set according to the types of idle resources that each expansion computing cluster can provide. In some embodiments, the priority of the elastic worker nodes can be set to the default priority or a lower priority than the default priority.

[0084] In this embodiment, the global idle resource status is compared in real time across multiple computing clusters, and elastic worker nodes are created in the computing cluster with the most abundant idle resources. This breaks through the bottleneck limitation of computing resources in a single computing cluster, while also utilizing fragmented idle computing power in multiple computing clusters, thereby improving the resource utilization rate of multiple computing clusters and the schedulable scale of resources for the target computing task.

[0085] After the target computation task is completed, some nodes in the expanded target computation network may become idle for an extended period. In this case, idle resources can be reclaimed in various ways. The following provides an exemplary resource reclamation method.

[0086] In optional embodiments, such as Figure 7 As shown, the method further includes: S700, receive a scaling-down request from the head node, the scaling-down request being used to indicate that the idle time of the target worker node exceeds a preset time threshold, the target worker node being one of the plurality of worker nodes and the one or more elastic worker nodes.

[0087] S702, determine a scaled-down computing cluster from the multiple computing clusters based on the target working node, wherein the scaled-down computing cluster includes one or more elastic working nodes.

[0088] S704, send a scaling-down command to the scaling-down computing cluster to delete one or more elastic worker nodes in the scaling-down computing cluster.

[0089] A node can be considered idle when there is no active workload on a worker node or elastic worker node. In some embodiments, the idle time can be timed uniformly by the head node, or each worker node or elastic worker node can autonomously report the idle start time to the head node. When determining the scaling down computing cluster, it is also necessary to ensure that the number of nodes in each worker node group is still not less than its minimum number of nodes after scaling down.

[0090] In this embodiment, by triggering a scaling-down command when a node is idle, idle elastic worker nodes can be promptly reclaimed from the computing cluster, thereby achieving rapid release of global resources, avoiding fragmentation of idle resources, reducing resource waste caused by long-term idling of elastic worker nodes after offline tasks are completed, and improving the resource utilization of multiple computing clusters.

[0091] In a multi-computing cluster, in addition to the resource scheduling node, various other components and modules can be configured. The following provides an example component configuration.

[0092] In an optional embodiment, each computing cluster is configured with a cluster resource control layer, such as... Figure 8 As shown, the cluster resource control layer is configured as follows: S800, obtain the resource status of the computing cluster, including the node status of the head node and / or worker nodes.

[0093] S802, the resource status is reported to the resource scheduling node.

[0094] Node status can include hardware parameters such as the number of CPU cores, GPU model and number of GPUs, memory capacity, and RDMA network cards, or software parameters such as cluster version, container runtime version, and driver version. It can also include operational parameters such as the number of assigned tasks, GPU memory utilization, network RTT, node temperature, and power consumption. Resource status can also include other workloads in the computing cluster besides the head node and worker nodes, such as deployments and jobs. The cluster resource control layer can periodically report resource status to the resource scheduling node at preset time intervals. In some embodiments, the cluster resource control layer can also trigger incremental reporting of resource status when a node experiences events such as node creation / deletion / failure, node NotReady, GPU Xid error, or OOM. In other embodiments, the cluster resource control layer can also dynamically adjust the reporting time interval based on the node load change rate; for example, shortening the time interval by 5 seconds when the node load change rate exceeds a threshold.

[0095] In this embodiment, by continuously collecting the resource status of each computing cluster locally, the resource scheduling node can perceive the cross-cluster resource level and node health status in real time. This provides a real-time and accurate data foundation for subsequent scaling decisions, reducing resource waste and task waiting caused by scheduling delays or misjudgments.

[0096] In optional embodiments, such as Figure 9 As shown, the cluster resource control layer is also configured as follows: S900, receive the original configuration information sent by the resource scheduling node, the original configuration information being used to describe the target computing network.

[0097] S902, Based on the environment information of the computing cluster and the original configuration information, generate target configuration information. The target configuration information is used to create nodes in the computing cluster, and the nodes include the head node and / or the worker nodes.

[0098] In some embodiments, the original configuration information may include various elements such as image fields (e.g., image name, image repository address), GPU model and quantity, service discovery domain name, elasticity policies (e.g., minimum number of nodes, scaling priority), CPU request / limit, and memory request / limit. The computing cluster's environment information may include the cluster provider type (e.g., public cloud, private cloud, edge cluster), region and availability zone, cluster version number, and container runtime type. In some embodiments, the cluster resource control layer can use escaping techniques to transform the original configuration information into target configuration information.

[0099] For example, such as Figure 10 As shown, in a public cloud scenario, after the cluster resource control layer 3 obtains the original configuration information sent by the resource scheduling node, it finds that the environment information indicates that this cluster uses an image repository in the public cloud. Therefore, it will modify the image source in the original configuration information to the image source of the public cloud.

[0100] For example, if the environment information indicates that the GPU model used by this cluster is X, and this model of GPU needs to mount some devices when requesting resources, the cluster resource control layer will modify the original configuration information and add the environment variables corresponding to these mounting operations to obtain the target configuration information.

[0101] In this embodiment, the globally unified original configuration information is automatically translated into target configuration information that can be executed by this cluster through the cluster resource control layer. This reduces the workload of manually modifying the configuration for each cluster and ensures that cross-cluster deployment can be defined once and run everywhere, thereby improving the delivery efficiency and portability of the target computing network.

[0102] To make this application easier to understand, the following is combined with... Figure 11 An example application is provided. Wherein: Step 1: The requester (S) submits a request to the multi-cluster resource scheduling layer (A) to create the target computing network.

[0103] Step 2: The multi-cluster resource scheduling layer (A) requests stable resources from the single-cluster resource control layer (B) according to the policy to create a head instance of the target computing network.

[0104] Step 3: The single cluster resource control layer (B) performs the initialization of the head node container on the target cluster.

[0105] Step 4: At the same time, the cluster control layer (C) starts the Autoscaler process on the head node.

[0106] Step 5: After the head node passes the health check, its status is collected and reported to the multi-cluster resource scheduling layer (A), marking that the head node is ready.

[0107] Step 6: The multi-cluster resource scheduling layer (A) triggers the worker node expansion process.

[0108] Step 7: A requests stable resources from the single cluster resource control layer (B) to create a predetermined number of initial worker node instances.

[0109] Step 8: B performs the initialization of the initial worker node containers on the cluster.

[0110] Step 9: After the worker node process starts, it registers with the head node.

[0111] Step 10: The status of worker nodes (such as health status and successful registration) is collected and reported. Autoscaler continuously monitors the cluster status. If a worker node fails to start, the system will detect and report the failure, and then repeat step 7. If the health check finds that a worker node is not alive, step 7 will also be repeated.

[0112] Steps 7 through 10 continue until the number of surviving worker node instances reaches the minimum number of nodes.

[0113] Step 11: Submit computing tasks to the created cluster in the request direction.

[0114] Step 12: The task may require the creation of multiple actors. If insufficient cluster resources cause an actor to be in a pending state, the Autoscaler on the head node will detect this situation.

[0115] Step 13: Autoscaler calls the ScaleUp interface of the multi-cluster resource scheduling layer (A). A decides to schedule to the optimal cluster based on the idle resources of multiple clusters.

[0116] Step 14: A sends an instruction to the single cluster resource control layer (B) of the target cluster to request the creation of elastic worker nodes.

[0117] Step 15: B creates and initializes the Elastic Worker Node container on the specified cluster.

[0118] Step 16: The elastic worker node starts up, registers with the head node, reports its status, and then begins to execute tasks.

[0119] Step 17: Once all computational tasks are completed, the Actor is released.

[0120] Step 18: Autoscaler detects that some worker node resources have been idle for a long time and triggers the scaling-down process.

[0121] Step 19: Autoscaler calls the ScaleDown interface of the multi-cluster resource scheduling layer (A).

[0122] Step 20: A issues a command to the single cluster resource control layer (B) to delete a specific worker node. B triggers the destruction of the corresponding elastic worker node container.

[0123] Step 21: After receiving the instruction, the worker node process exits normally, and its status (deleted) is reported.

[0124] Step 22: Finally, only the head node or a small number of basic worker nodes may remain in the cluster, and all temporary resources created for elastic computing have been released.

[0125] Example 2 This method embodiment can be used in a computing cluster. The computing cluster is used as the single execution entity in this process.

[0126] Figure 12 A flowchart illustrating a resource scheduling method according to Embodiment 2 of this application is shown schematically.

[0127] like Figure 12 As shown, it may include steps S1200~S1202, wherein: S1200, in response to the startup of the worker node, reads the node information of the head node from the target storage location, the node information including the network address.

[0128] S1202, establish a communication connection between the working node and the head node according to the network address; wherein the head node is located in the computing cluster or in another computing cluster different from the computing cluster.

[0129] In this embodiment, the association between the head node and the worker node is established by storing node information in the target storage location, which realizes stable communication between the head node and worker nodes in different computing clusters. This enables cross-cluster resource scheduling and integration, avoids the resource bottleneck limitation of a single cluster, and further improves resource utilization and the scalability of the computing cluster.

[0130] Example 3 Figure 13 The diagram schematically illustrates a resource scheduling apparatus according to Embodiment 3 of this application. This apparatus, used in resource scheduling nodes of a multi-computing cluster, can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiments of this application. The program module referred to in the embodiments of this application refers to a series of computer program instruction segments capable of performing a specific function. The following description will specifically introduce the functions of each program module in this embodiment. For example... Figure 13 As shown, the device 1000 may include: a receiving module 1100, a head node creation module 1200, and a worker node group creation module 1300, wherein: The receiving module 1100 receives a target creation request, which is used to create a target computing network, the target computing network including a head node and a group of worker nodes; The head node creation module 1200 is used to create the head node according to the target creation request, and the node information of the head node is stored in the target storage location; The worker node group creation module 1300 is used to create the worker node group when the head node is successfully created. The worker node group includes one or more worker nodes. The one or more worker nodes establish a communication connection with the head node based on the node information read from the target storage location.

[0131] As an optional embodiment, the head node creation module is also used for: A first computing cluster is determined from the multiple computing clusters, and the first computing cluster has a first reserved resource pool; Create the head node in the first reserved resource pool.

[0132] As an optional embodiment, the head node creation module is also used for: If the head node is successfully created, the head node will be started through the first computing cluster. In response to the successful startup of the head node, the node information of the head node is stored in the target storage location through the first computing cluster. The node information includes the service name and network address of the head node.

[0133] As an optional embodiment, the worker node group creation module is also used for: Determine the minimum number of nodes corresponding to the working node group based on the target creation request; A second computing cluster is determined from the multiple computing clusters based on the minimum number of nodes, and the second computing cluster has a second reserved resource pool; In the second reserved resource pool, create one or more of the working nodes corresponding to the minimum number of nodes; The first computing cluster may be the same as or different from the second computing cluster.

[0134] As an optional embodiment, the node information includes the node name of the head node and the network address of the head node; the device further includes a connection module for: When one or more worker nodes are started, the started worker node obtains the network address from the target storage location based on the node name of the head node, and establishes a communication connection with the head node through the network address.

[0135] As an optional embodiment, the device is further used for: The head node receives a scaling request in response to a target computing task, where the resource requirement of the target computing task is greater than the resource supply of the worker node group, and the resource supply is determined based on the minimum number of nodes in the worker node group. Based on the idle resources of each of the multiple computing clusters, determine the expansion computing cluster from the multiple computing clusters; Create one or more elastic worker nodes in the expanded computing cluster.

[0136] As an optional embodiment, the device is further used for: Receive a scaling-down request from the head node, the scaling-down request being used to indicate that the idle time of the target worker node exceeds a preset time threshold, the target worker node being one of the plurality of worker nodes and the one or more elastic worker nodes; Based on the target worker node, a scaled-down computing cluster is determined from the multiple computing clusters, wherein the scaled-down computing cluster includes one or more elastic worker nodes; Send a scaling-down command to the scaling-down computing cluster to delete one or more elastic worker nodes from the scaling-down computing cluster.

[0137] As an optional embodiment, each computing cluster is configured with a cluster resource control layer, which is configured as follows: Obtain the resource status of the computing cluster, including the node status of the head node and / or worker nodes; The resource status is reported to the resource scheduling node.

[0138] As an optional embodiment, the cluster resource control layer is further configured to: Receive the raw configuration information sent by the resource scheduling node, the raw configuration information being used to describe the target computing network; Based on the environment information of the computing cluster and the original configuration information, target configuration information is generated. The target configuration information is used to create nodes in the computing cluster, and the nodes include the head node and / or the worker nodes.

[0139] Example 4 Figure 14 The diagram schematically illustrates a resource scheduling apparatus according to Embodiment 4 of this application. This apparatus, used in a computing cluster, can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiments of this application. The program module referred to in the embodiments of this application refers to a series of computer program instruction segments capable of performing a specific function. The following description will specifically introduce the functions of each program module in this embodiment. For example... Figure 14 As shown, the device 2000 may include: a reading module 2100 and a connection module 2200, wherein: The reading module 2100 is used to read the node information of the head node from the target storage location in response to the startup of the worker node, the node information including the network address; The connection module 2200 is used to establish a communication connection between the working node and the head node according to the network address; wherein the head node is located in the computing cluster or in another computing cluster different from the computing cluster.

[0140] Example 5 Figure 15 This illustration schematically depicts the hardware architecture of a computer device 10000 suitable for implementing a resource scheduling method according to Embodiment 5 of this application. In some embodiments, the computer device 10000 may be a terminal device such as a smartphone, wearable device, tablet computer, personal computer, in-vehicle terminal, game console, virtual device, workbench, digital assistant, set-top box, or robot. In other embodiments, the computer device 10000 may be a rack server, blade server, tower server, or cabinet server (including independent servers or server clusters composed of multiple servers), etc. Figure 15 As shown, the computer device 10000 includes, but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate and be linked with each other via a system bus. Wherein: The memory 10010 includes at least one type of computer-readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 may be an internal storage module of a computer device 10000, such as the hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 may also be an external storage device of the computer device 10000, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 10000. Of course, the memory 10010 may also include both the internal storage module and the external storage device of the computer device 10000. In this embodiment, the memory 10010 is typically used to store the operating system and various application software installed on the computer device 10000, such as program code for resource scheduling methods. In addition, the memory 10010 can also be used to temporarily store various types of data that have been output or will be output.

[0141] In some embodiments, processor 10020 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other chip. Processor 10020 is typically used to control the overall operation of computer device 10000, such as performing control and processing related to data interaction or communication with computer device 10000. In this embodiment, processor 10020 is used to run program code stored in memory 10010 or process data.

[0142] Network interface 10030 may include a wireless network interface or a wired network interface, which is typically used to establish a communication link between computer device 10000 and other computer devices. For example, network interface 10030 is used to connect computer device 10000 to an external terminal via a network, establishing a data transmission channel and communication link between computer device 10000 and the external terminal. The network may be an intranet, the Internet, Global System for Mobile Communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks.

[0143] It should be pointed out that, Figure 15 Only computer devices with components 10010-10030 are shown; however, it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead.

[0144] In this embodiment, the resource scheduling method stored in memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of this application.

[0145] Example 6 This application also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the resource scheduling method in the embodiments.

[0146] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device. Of course, the computer-readable storage medium can also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the computer-readable storage medium is typically used to store the operating system and various application software installed on the computer device, such as the program code of the resource scheduling method in the embodiment. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or will be output.

[0147] Example 7 This application also provides a computer program product, including a computer program that, when executed by a processor, implements the methods described in the above embodiments.

[0148] Example 8 Figure 16 The schematic diagram illustrates the architecture of a resource scheduling system according to Embodiment 8 of this application.

[0149] like Figure 16 As shown in the illustration, this application also provides a resource scheduling system for use in multiple computing clusters. The implementation and technical details of this system can be found above. The resource scheduling system includes a multi-cluster resource scheduling layer, a single-cluster resource controller, and a cluster control layer, wherein: A multi-cluster resource scheduling layer is configured to: receive a target creation request, the target creation request being used to create a target computing network, the target computing network including a head node and a group of worker nodes; create the head node according to the target creation request; and, if the head node is successfully created, create the group of worker nodes, each group of worker nodes including one or more worker nodes. A single cluster resource controller is configured to initialize the head node and the worker nodes; The cluster control layer is configured to: store the node information of the head node in a target storage location when the head node starts; read the node information from the target storage location according to the node name of the head node when the worker nodes start; and establish communication connections between the multiple worker nodes and the head node according to the node information.

[0150] like Figure 16 As shown, the multi-cluster resource scheduling layer creates the target computing network based on the requester's cluster creation request. The request includes the relevant configuration of the target computing network (e.g., the target computing network includes a head node requiring 8 CPU cores and 25 GPUs, and two worker nodes requiring 30 CPU cores and 60 GPUs. The resources occupied by these head nodes and worker nodes are given high priority and cannot be preempted by other processes or system components). The multi-cluster resource scheduling layer creates the target computing network based on these relevant configurations, including creating a head node in one cluster and multiple worker nodes (worker node0, 1) in another computing cluster.

[0151] When the computing resources provided by the current target computing network cannot meet the needs of large-scale computing tasks (such as needing a worker node with 30 CPU cores and 60 GPUs in addition to the existing worker nodes), available resources will be selected from multiple computing clusters based on the list of currently active elastic nodes and the resource distribution of the current computing cluster. Elastic worker nodes (elastic workers, which have low priority and whose resources can be preempted) will be created in the selected computing cluster, such as worker node3-5.

[0152] Obviously, those skilled in the art should understand that the modules or steps of the embodiments of this application described above can be implemented using general-purpose computer devices. They can be centralized on a single computer device or distributed across a network of multiple computer devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computer device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of this application are not limited to any particular combination of hardware and software.

[0153] It should be noted that the above are merely preferred embodiments of this application and do not limit the scope of patent protection of this application. Any equivalent structural or procedural changes made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.

Claims

1. A resource scheduling method, characterized in that, In a resource scheduling node used in a multi-computing cluster, the method includes: Receive a target creation request, the target creation request being used to create a target computing network, the target computing network including a head node and a group of worker nodes; The head node is created according to the target creation request, and the node information of the head node is stored in the target storage location; If the head node is successfully created, the working node group is created, and each working node group includes one or more working nodes; The one or more working nodes establish a communication connection with the head node based on the node information read from the target storage location.

2. The method according to claim 1, characterized in that, Creating the head node according to the target creation request includes: A first computing cluster is determined from the multiple computing clusters, and the first computing cluster has a first reserved resource pool; Create the head node in the first reserved resource pool.

3. The method according to claim 2, characterized in that, The method further includes: If the head node is successfully created, the head node will be started through the first computing cluster. In response to the successful startup of the head node, the node information of the head node is stored in the target storage location through the first computing cluster. The node information includes the service name and network address of the head node.

4. The method according to claim 2, characterized in that, Creating the work node group includes: Determine the minimum number of nodes corresponding to the working node group based on the target creation request; A second computing cluster is determined from the multiple computing clusters based on the minimum number of nodes, and the second computing cluster has a second reserved resource pool; In the second reserved resource pool, create one or more of the working nodes corresponding to the minimum number of nodes; The first computing cluster may be the same as or different from the second computing cluster.

5. The method according to claim 1, characterized in that, The node information includes the node name of the head node and the network address of the head node; The one or more working nodes establish a communication connection with the head node based on the node information read from the target storage location, including: When one or more worker nodes are started, the started worker node obtains the network address from the target storage location based on the node name of the head node, and establishes a communication connection with the head node through the network address.

6. The method according to claim 1, characterized in that, The method further includes: The head node receives a scaling request in response to a target computing task, where the resource requirement of the target computing task is greater than the resource supply of the worker node group, and the resource supply is determined based on the minimum number of nodes in the worker node group. Based on the idle resources of each of the multiple computing clusters, determine the expansion computing cluster from the multiple computing clusters; Create one or more elastic worker nodes in the expanded computing cluster.

7. The method according to claim 6, characterized in that, The method further includes: Receive a scaling-down request from the head node, the scaling-down request being used to indicate that the idle time of the target worker node exceeds a preset time threshold, the target worker node being one of the plurality of worker nodes and the one or more elastic worker nodes; Based on the target worker node, a scaled-down computing cluster is determined from the multiple computing clusters, wherein the scaled-down computing cluster includes one or more elastic worker nodes; Send a scaling-down command to the scaling-down computing cluster to delete one or more elastic worker nodes from the scaling-down computing cluster.

8. The method according to claim 1, characterized in that, Each computing cluster is configured with a cluster resource control layer; The cluster resource control layer is configured as follows: Obtain the resource status of the computing cluster, including the node status of the head node and / or worker nodes; The resource status is reported to the resource scheduling node.

9. The method according to claim 8, characterized in that, The cluster resource control layer is also configured to: Receive the raw configuration information sent by the resource scheduling node, the raw configuration information being used to describe the target computing network; Based on the environment information of the computing cluster and the original configuration information, target configuration information is generated. The target configuration information is used to create nodes in the computing cluster, and the nodes include the head node and / or the worker nodes.

10. A resource scheduling method, characterized in that, For use in a computing cluster, the method includes: In response to the startup of a worker node, the node information of the head node is read from the target storage location, the node information including the network address; Based on the network address, establish a communication connection between the working node and the head node; The head node is located in the computing cluster or in another computing cluster that is different from the computing cluster.

11. A resource scheduling device, characterized in that, In a resource scheduling node for a multi-computing cluster, the device includes: The receiving module receives a target creation request, which is used to create a target computing network, the target computing network including a head node and a group of worker nodes; The head node creation module is used to create the head node according to the target creation request, and the node information of the head node is stored in the target storage location; The working node group creation module is used to create the working node group when the head node is successfully created. Each working node group includes one or more working nodes. The one or more working nodes establish a communication connection with the head node based on the node information read from the target storage location.

12. A resource scheduling device, characterized in that, For use in a computing cluster, the device includes: A read module is used to read node information of the head node from the target storage location in response to the startup of the worker node, the node information including the network address; A connection module is used to establish a communication connection between the working node and the head node based on the network address; The head node is located in the computing cluster or in another computing cluster that is different from the computing cluster.

13. A computer device, characterized in that, include: At least one processor; and A memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 10.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 10.

15. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 10.

16. A resource scheduling system, characterized in that, For use in multiple computing clusters, the system includes: A multi-cluster resource scheduling layer is configured to: receive a target creation request, the target creation request being used to create a target computing network, the target computing network including a head node and a group of worker nodes; create the head node according to the target creation request; and, if the head node is successfully created, create the group of worker nodes, each group of worker nodes including one or more worker nodes. A single cluster resource controller is configured to initialize the head node and the worker nodes; The cluster control layer is configured to: store the node information of the head node in a target storage location when the head node starts; read the node information from the target storage location according to the node name of the head node when the worker nodes start; and establish communication connections between the multiple worker nodes and the head node according to the node information.

Images