Resource scheduling method and device, computing cluster and database

Abstract

The embodiment of the present specification provides a resource scheduling method, device, computing cluster and database, the resource scheduling method comprises: taking resource supply information corresponding to a plurality of resource pools, the resource supply information comprises resource information corresponding to resource attributes of a resource pool, one of the resource pool comprises one or more computing nodes; determine the resource requirement information corresponding to the to-be-scheduled computing engine, the resource requirement information comprises the resource information corresponding to the preset resource attribute; according to the resource requirement information corresponding to the to-be-scheduled computing engine and the resource supply information corresponding to the plurality of resource pools, determine the resource configuration corresponding to the to-be-scheduled computing engine in the plurality of resource pools respectively; according to the resource configuration corresponding to the to-be-scheduled computing engine in the plurality of resource pools respectively, the to-be-scheduled computing engine is scheduled to one or more resource pools for execution.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to a resource scheduling method. Background Technology

[0002] Cloud-native products built on cloud infrastructure can leverage cloud computing power to provide users with instantaneous, massive data processing capabilities, enhancing user experience. When using cloud computing power for data processing, resources must be provided according to the requirements of cloud-native products, such as network resources, storage resources, processor specifications, memory specifications, etc.

[0003] Currently, homogeneous resource pools are commonly used to provide resources for cloud-native products. Resource pools are categorized based on the commonalities in their resource information structure; multiple resource pools with the same resource information structure are called homogeneous resource pools. However, using homogeneous resource pools to provide resources for cloud-native products can lead to low resource utilization in scenarios involving heterogeneous resource pools, such as cross-cloud environments. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a resource scheduling method. One or more embodiments of this specification also relate to a resource scheduling device, a computing cluster, a database, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a resource scheduling method is provided, comprising: acquiring resource supply information corresponding to multiple resource pools, the resource supply information including resource information corresponding to resource attributes of the resource pools, and one resource pool including one or more computing nodes; determining resource demand information corresponding to a computing engine to be scheduled, the resource demand information including resource information corresponding to preset resource attributes; determining resource configurations of the computing engine to be scheduled in the multiple resource pools according to the resource demand information of the computing engine to be scheduled and the resource supply information corresponding to the multiple resource pools; and scheduling the computing engine to be scheduled to be executed in one or more resource pools according to the resource configurations of the computing engine to be scheduled in the multiple resource pools.

[0006] According to a second aspect of the embodiments of this specification, a resource scheduling apparatus is provided, comprising: an information acquisition module configured to acquire resource supply information corresponding to multiple resource pools, the resource supply information including resource information corresponding to resource attributes of the resource pools, and one resource pool including one or more computing nodes; a demand determination module configured to determine resource demand information corresponding to a computing engine to be scheduled, the resource demand information including resource information corresponding to preset resource attributes; a resource configuration module configured to determine the resource configuration of the computing engine to be scheduled in each of the multiple resource pools based on the resource demand information of the computing engine to be scheduled and the resource supply information corresponding to the multiple resource pools; and a scheduling execution module configured to schedule the computing engine to be scheduled to one or more resource pools for execution according to the resource configuration of the computing engine to be scheduled in each of the multiple resource pools.

[0007] According to a third aspect of the embodiments of this specification, a computing cluster is provided, comprising: multiple resource pools and a master node applying the resource scheduling method as described in any embodiment of this specification, wherein one of the resource pools includes one or more computing nodes.

[0008] According to a fourth aspect of the embodiments of this specification, a database is provided, including: a parsing module, multiple computing engines, and a computing cluster as described in any embodiment of this specification; the parsing module is configured to receive a database query request, parse the database query request to determine the computing engine to be scheduled corresponding to the database query request and the resource requirement information corresponding to the computing engine to be scheduled, wherein the computing engine to be scheduled is one or more of the multiple computing engines, and send the resource requirement information corresponding to the computing engine to be scheduled to the master node of the computing cluster, so that the master node schedules the computing engine to be scheduled to a computing node in a resource pool for execution according to the resource scheduling method described in any embodiment of this specification.

[0009] According to a fifth aspect of the embodiments of this specification, a computing device is provided, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, wherein the computer-executable instructions, when executed by the processor, implement the steps of the resource scheduling method described above.

[0010] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions that, when executed by a processor, implement the steps of the resource scheduling method described above.

[0011] According to a seventh aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the resource scheduling method described above.

[0012] One embodiment of this specification implements a resource scheduling method. Since this method obtains resource supply information corresponding to multiple resource pools, including resource information corresponding to the resource attributes of the resource pools, it effectively achieves self-awareness of heterogeneous resource information across multiple resource pools. Thus, after determining the resource demand information corresponding to the computing engine to be scheduled, it can determine the resource configuration of the computing engine in each of the multiple resource pools based on the resource demand information of the computing engine to be scheduled and the resource supply information corresponding to the multiple resource pools. That is, it achieves self-awareness of the heterogeneous resource configuration of the computing engine in multiple resource pools. Therefore, according to the resource configuration of the computing engine in each of the multiple resource pools, the computing engine to be scheduled is scheduled to one or more resource pools for execution. This achieves the goal of supporting large-scale resource supply for cloud-native products by integrating multiple heterogeneous resource pools, maximizing the utilization of the infrastructure of multiple heterogeneous resource pools, fully utilizing the resources of the resource pools, and meeting the needs of heterogeneous resource pool integration and cross-cloud scenarios. Attached Figure Description

[0013] Figure 1 This is a schematic diagram illustrating an application scenario of a resource scheduling method provided in one embodiment of this specification;

[0014] Figure 2 This is a flowchart illustrating a resource scheduling method provided in one embodiment of this specification;

[0015] Figure 3 This is a flowchart illustrating the processing procedure of a resource scheduling method provided in one embodiment of this specification.

[0016] Figure 4 This is a schematic diagram of a four-layer architecture of a cloud-native data warehouse provided in one embodiment of this specification;

[0017] Figure 5 This is a schematic diagram of the structure of a resource scheduling device provided in one embodiment of this specification;

[0018] Figure 6 This is a schematic diagram of the structure of a computing cluster provided in one embodiment of this specification;

[0019] Figure 7 This is a schematic diagram of the structure of a database provided in one embodiment of this specification;

[0020] Figure 8 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0021] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0022] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0023] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0024] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0025] Cloud-native data warehouse: A data warehouse built on cloud infrastructure with unlimited resource and storage elasticity.

[0026] Resource pools are composed of compute nodes and are used to provide resource provisioning for cloud-native products. For example, in cloud-native data warehouse application scenarios, they can provide resource provisioning for cloud-native data warehouse queries. There are no restrictions on how resource pools are constructed. For example, they can be self-supported resource pools built on physical machines, or they can utilize cloud elastic services as resource pools.

[0027] Resource Configuration: The execution of a computing engine consumes a set of computing resources, including storage, network, specifications, logs, cloud environment, and other resources. The descriptive information of the combination of these resources is called the resource configuration of that set of computing resources. For example, in a container-based computing cluster, the container group used to run the computing engine is bound to the computing nodes of the resource pool for execution. The container group can be understood as a set of computing resources.

[0028] Scheduling technology: an allocation mechanism that finds resource configurations that meet the needs of the computing engine and optimize execution efficiency.

[0029] With the development of cloud infrastructure, major cloud vendors are vigorously promoting cloud-native products. For example, cloud-native data warehouses provide users with instantaneous massive data analysis capabilities by deeply utilizing cloud elastic computing and cross-cloud elastic computing capabilities, meeting users' needs for a multi-cloud integrated user experience. However, when providing resource supply for cloud-native products based on resource pools, the resources of the resource pools are not fully utilized. For example, some cloud vendors have built self-owned homogeneous resource pools for elastic resource supply, eliminating the problem of resource heterogeneity. However, the elasticity scale is limited by the self-owned resource pool, and in cross-cloud scenarios, resource pools in different clouds cannot achieve resource reuse independently. Other cloud vendors use cross-cloud resources for cross-cloud scenarios, but each resource pool is used independently, the elasticity scale is limited by each independent resource pool, and the independent resources of each resource pool are difficult to reuse.

[0030] Taking cloud-native data warehouses as an example, they need excellent capabilities in performance, cost, and other dimensions to provide better services. However, current cloud-native data warehouses have multiple isolated resource pools. When the computing engine of a data warehouse instance runs on physical computing nodes in a solid-state resource pool and virtual computing nodes in an elastic resource pool, the resource usage differs between the two pools. The same instance can only run in one resource pool, making resource sharing difficult. Furthermore, the configurations of instances running in different resource pools vary, exhibiting diverse characteristics. Because resource configuration cannot be automatically integrated, the data warehouse platform needs to perform special resource configuration planning for different resource pools, resulting in significant maintenance and scalability costs.

[0031] In view of this, this specification provides a resource scheduling method. This method obtains resource supply information corresponding to multiple resource pools. The resource supply information includes resource information corresponding to the resource attributes of the resource pools. This is equivalent to realizing the self-awareness of heterogeneous resource information of multiple resource pools. Thus, after determining the resource demand information corresponding to the computing engine to be scheduled, since the resource demand information includes the resource information corresponding to the preset resource attributes, the resource configuration of the computing engine to be scheduled in each of the multiple resource pools can be determined according to the resource demand information of the computing engine to be scheduled and the resource supply information of the multiple resource pools. That is, the self-awareness of the heterogeneous resource configuration of the computing engine in multiple resource pools is realized. Therefore, according to the resource configuration of the computing engine to be scheduled in each of the multiple resource pools, the computing engine to be scheduled is scheduled to one or more resource pools for execution. This achieves the goal of supporting the large-scale resource supply of cloud-native products by integrating multiple heterogeneous resource pools. It can maximize the utilization of the infrastructure of multiple heterogeneous resource pools, make full use of the resources of the resource pools, and meet the needs of heterogeneous multiple resource pool integrated resource supply and cross-cloud scenarios. It is evident that this method achieves the goal of providing resources in a fusion manner across heterogeneous multi-resource pools. According to the scheduling of this method, the resource configuration of the computing engine running in different resource pools can be heterogeneous, realizing a unified scheduling scheme for self-aware heterogeneous resource configuration.

[0032] Specifically, this specification provides a resource scheduling method, and also relates to a resource scheduling device, a computing cluster, a database, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0033] See Figure 1 , Figure 1 A schematic diagram illustrating an application scenario of a resource scheduling method according to an embodiment of this specification is shown. For example... Figure 1As shown, the resource scheduling method can be applied to the master node 104 in a master-slave computing cluster 102. For example, on the master node 104, the resource scheduling method can manifest as a resource scheduling service. The computing cluster 102 is a master-slave architecture, specifically composed of the master node 104 and multiple computing nodes, such as computing nodes 106a, 106b, etc. The resource scheduling service 1042 in the master node 104 is used to determine the resource configuration of the computing engine in each resource pool and schedule the computing engine to one or more resource pools for execution. Computing nodes such as computing nodes 106a and 106b can respectively deploy resource management services 1062a and 1062b to manage the resources of their respective nodes. In addition, to support the addition of cloud elastic resources to the computing cluster, virtual nodes can also be supported, which manage the cloud elastic resources. In the computing cluster, virtual nodes and physical nodes have equal status; the difference is that virtual nodes do not require an actual physical server as a deployment carrier. Both virtual nodes and physical nodes are referred to as computing nodes. Figure 1 The computing cluster architecture shown in the diagram includes a resource scheduling service 1042 in the master node 104, which receives resources from the resource management services of each computing node (such as...). Figure 1 The resource management services 1062a and 1062b shown acquire resource supply information corresponding to multiple resource pools, determine resource demand information corresponding to the computing engine to be scheduled, and, based on the resource demand information of the computing engine to be scheduled and the resource supply information corresponding to the multiple resource pools, determine the resource configuration of the computing engine to be scheduled in each of the multiple resource pools. According to the resource configuration of the computing engine to be scheduled in each of the multiple resource pools, the computing engine to be scheduled is scheduled to one or more resource pools for execution. In addition, the master node and computing nodes of the computing cluster may also include other components, such as interface services, persistent storage, controller managers, cloud controller managers, cloud provider interfaces, network proxies, etc. Their specific functions can be found in the functions of corresponding components in a typical container cluster, and will not be described in detail here.

[0034] The computing nodes can be physical servers or cloud servers providing various services in the cloud. For example, they can be servers providing communication services, data processing services, computing services, storage services, database services, etc., to multiple clients. It should be noted that computing nodes in the cloud can be implemented as a distributed server cluster composed of multiple servers, or as a single server. Servers can also be servers in a distributed system, or servers integrated with blockchain. Servers can also be cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or intelligent cloud computing servers or intelligent cloud hosts with artificial intelligence technology.

[0035] It is understood that the above application scenarios are only used to illustrate the methods provided in the embodiments of this specification and do not constitute a limitation on the methods provided in the embodiments of this specification. The methods provided in the embodiments of this specification can perform resource scheduling for cloud-native products of various master-slave architecture computing clusters. For example, the methods provided in the embodiments of this specification can be applied to a multi-tenant elastic pooling scheduling platform to schedule elastic resources of a multi-tenant cloud-native data warehouse; as another example, the methods provided in the embodiments of this specification can be applied to resource scheduling of multi-cloud databases to meet the needs of data warehouse analysis and computing.

[0036] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0037] See Figure 2 , Figure 2 A flowchart of a resource scheduling method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0038] Step 202: Obtain resource supply information corresponding to multiple resource pools. The resource supply information includes resource information corresponding to the resource attributes of the resource pools. Each resource pool includes one or more computing nodes.

[0039] Resource attributes, also known as resource types, are abstract descriptions of various types of resources. For example, resource attributes may include network resources, storage resources, processor specifications, memory specifications, logging policies, security isolation policies, and so on.

[0040] In one or more embodiments of this specification, resource pools are categorized according to whether their resources are elastic. These resource pools may include fixed resource pools and elastic resource pools. A fixed resource pool refers to a resource pool where, after creation and deployment, resources do not dynamically increase or decrease with usage; the resources remain constant. A dynamic resource pool refers to a resource pool where, after creation and deployment, resources are dynamically adjusted based on usage; the resources change dynamically.

[0041] For example, a cloud database has multiple resource pools, including: "Solid State Resource Pool", "ECI-based Elastic Resource Pool", and other types of elastic resource pools.

[0042] For example, the resource information corresponding to the resource attributes of a certain solid-state resource pool may include: storage capacity corresponding to cloud disks and shared LVM (Logical Volume Manager), network capacity corresponding to ENI (Elastic Network Interface) and ENI cache, log capacity corresponding to SideCar mode (dividing the functions of the application into separate processes, called SideCar mode) collection, operation and maintenance capacity corresponding to self-healing operation and maintenance, and so on.

[0043] For example, the resource information corresponding to the resource attributes of a certain elastic resource pool may include: storage capacity corresponding to cloud disks and NAS (Network Attached Storage), network capacity corresponding to ENI and CIP (Common Industrial Protocol), log capacity corresponding to built-in collection mode, operation and maintenance capacity corresponding to fully managed mode, and so on.

[0044] Step 204: Determine the resource requirement information corresponding to the computing engine to be scheduled. The resource requirement information includes resource information corresponding to preset resource attributes.

[0045] The source of the computing engines to be scheduled is not limited; they can be computing engines from various cloud-based products. When a computing engine of a cloud product needs to be scheduled to execute in a resource pool, that computing engine can be understood as a computing engine to be scheduled. A computing engine refers to a program used to process data.

[0046] For example, in cloud database application scenarios, the computing engine may include database engines used for querying and other processing of database data. Suppose a user sends a query request to the cloud database. After receiving the query request, the cloud database parses it to determine the database engine required to complete the query and the resource information required by the database engine, including resource information corresponding to resource attributes such as storage capacity and network capacity required when executing the database engine.

[0047] In one or more embodiments of this specification, to facilitate subsequent determination of resource configuration, the resource requirement information, which is directed to different computing engines and different resource pools, may have preset unified resource attributes, similar to the resource configuration. These preset resource attributes may be unified resource attributes abstracted from multiple computing engines and multiple resource pools.

[0048] Step 206: Based on the resource demand information corresponding to the computing engine to be scheduled and the resource supply information corresponding to the multiple resource pools, determine the resource configuration of the computing engine to be scheduled in each of the multiple resource pools.

[0049] Determining the resource configuration refers to the process of selecting resources from the resource pool that meet the needs of the computing engine and assigning these selected resources to the computing engine for use. The resource configuration can also be understood as descriptive information about the resources that can meet the runtime requirements of the computing engine to be scheduled within the corresponding resource pool.

[0050] In one or more embodiments of this specification, the resource configuration may include resource information corresponding to unified resource attributes abstracted from multiple resource pools. The resource configuration has unified resource attributes, thereby providing a standardized resource configuration abstraction at the service layer. When facing different computing engines and different resource pools, the resource attributes in their resource configurations are the same; the difference lies in the resource information corresponding to the resource attributes. For example, different computing engines within a cloud-native data warehouse include analytics computing engines, storage engines, big data analytics computing engines, etc. For these engines, their runtime resource configurations can be abstracted, and the resource attributes in the resource configurations are the same across different resource pools / multi-cloud environments.

[0051] Because resource information across multiple resource pools can be heterogeneous—with varying resource attributes such as storage capacity, network capacity, and logging capacity—the resource information in the resource configuration of the same computing engine may differ across different resource pools. This can be understood as different resource configurations. For example, a database engine's resource configuration in a solid-state resource pool might include: using local LVM disks for temporary storage, using node-level collection services for logging, and using rund mode (a container runtime mode) for security isolation, etc. As another example, a database engine's resource configuration in an elastic resource pool might include: using persistent cloud disks for storage, using SideCar mode for logging, and using cloud services for security isolation, etc.

[0052] Step 208: According to the resource configurations of the computing engine to be scheduled in the multiple resource pools, schedule the computing engine to be scheduled to one or more resource pools for execution.

[0053] As in the example of the database engine above, according to the resource configurations of the database engine in the solid-state resource pool and the elastic resource pool respectively, the database engine can be scheduled to execute concurrently in the solid-state resource pool and the elastic resource pool.

[0054] It should be noted that, given the resource configurations of the computing engine in the various resource pools, the heterogeneous resource pools can be integrated to provide resource support for the computing engine. Specifically, resources can be provided by multiple resource pools simultaneously, or one or more resource pools can be selected as needed to provide resource support. This specification does not impose any restrictions on this.

[0055] The methods provided in the embodiments of this specification can be used in various cloud computing scenarios involving the integration of multiple resource pools, such as database computing engine computing, function computing, and stream computing. For example, the methods provided in the embodiments of this specification can be used to meet the elastic resource assurance needs of multi-tenant cloud-native data warehouses and the analytical computing needs of multi-cloud data warehouses, and to solve the problems of multi-resource pool integration and heterogeneous resource pool consolidation.

[0056] As can be seen, since this method obtains resource supply information corresponding to multiple resource pools, including resource information corresponding to the resource attributes of the resource pools, it is equivalent to realizing the self-awareness of heterogeneous resource information of multiple resource pools. Thus, after determining the resource demand information corresponding to the computing engine to be scheduled, it can determine the resource configuration of the computing engine to be scheduled in each of the multiple resource pools based on the resource demand information of the computing engine to be scheduled and the resource supply information of the multiple resource pools. That is, it realizes the self-awareness of the heterogeneous resource configuration of the computing engine in multiple resource pools. Therefore, according to the resource configuration of the computing engine to be scheduled in each of the multiple resource pools, the computing engine to be scheduled is scheduled to one or more resource pools for execution. This achieves the goal of using multiple heterogeneous resource pools to support the large-scale resource supply of cloud-native products, maximizes the use of the infrastructure of multiple heterogeneous resource pools, makes full use of the resources of the resource pools, and meets the needs of heterogeneous multiple resource pools for integrated resource supply and cross-cloud scenarios.

[0057] Taking the method provided in the embodiments of this specification applied to the resource scheduling service on the master node of a computing cluster as an example, obtaining the resource supply information corresponding to the multiple resource pools includes:

[0058] The resource scheduling service receives resource supply information reported by the resource management service on the computing nodes of the computing cluster. The resource management service is used to manage the resource information corresponding to the resource attributes of the computing nodes.

[0059] The computing cluster is a master-slave architecture computing cluster, including a master node and several computing nodes.

[0060] According to the above embodiment, a resource scheduling service runs on the master node of the computing cluster. This resource scheduling service, also understood as a resource scheduler, is responsible for selecting container groups and binding compute nodes within the entire container cluster. Therefore, scheduling the compute engines to be scheduled to the resource pool according to the determined resource configuration can also be understood as the process of selecting container groups and binding compute nodes. When performing resource scheduling, the resource scheduling service can consider comprehensive factors including node affinity, container group affinity, node resource filtering, scheduling priority, and fair scheduling. Binding compute nodes can be understood as updating the node name in the container group resource definition to the target compute node to be scheduled.

[0061] A resource management service runs on the compute node, which reports resource provisioning information on the node to the resource scheduling service on the master node. This resource management service can be understood as a proxy component on the compute node, periodically receiving new or modified container group specifications from the master node and ensuring that the container group and its containers run under the expected specifications. It also acts as a monitoring component for the compute node, reporting its operational status to the master node. Compute nodes can include physical nodes and / or virtual nodes. Virtual machine nodes simulate node resource objects and are responsible for managing the lifecycle of container groups after they are scheduled to the node.

[0062] As can be seen, the above embodiments, when applied to the master node of the computing cluster, utilize the method provided in the embodiments of this specification to not only provide resources in a fusion manner using heterogeneous multi-resource pools, but also fully utilize the scheduling capabilities of the original scheduler on the master node of the computing cluster. For example, scheduling is performed by comprehensively considering factors such as node affinity scheduling priority and fair scheduling, so as to achieve the purpose of fully and effectively utilizing resources.

[0063] Resource scheduling is generally divided into multiple scheduling phases. For example, these multiple scheduling phases, from beginning to end, typically include at least a filtering phase, a scoring phase, and a binding phase. The filtering phase selects compute nodes that meet the criteria for adding to a container group; the scoring phase scores the compute nodes that meet the criteria; and the binding phase allocates the container group to the compute node with the highest score. In one or more embodiments of this specification, to better filter resources that meet the requirements of each scheduling phase, the step of determining the resource configuration of the compute engine to be scheduled in each of the multiple resource pools, based on the resource requirement information corresponding to the compute engine to be scheduled and the resource information of each of the multiple resource pools, includes:

[0064] Matching rules are pre-set for each of the multiple scheduling stages, and the matching rules include resource information filtering conditions corresponding to the resource attributes of the computing engine to be scheduled.

[0065] In any scheduling phase, the resource configuration of the computing engine to be scheduled in each of the multiple resource pools is determined according to the resource requirement information corresponding to the computing engine to be scheduled, the resource information of each of the multiple resource pools, and the matching rules corresponding to the scheduling phase.

[0066] In conjunction with the above embodiments, in one or more embodiments of this specification, determining the resource configuration of the computing engine to be scheduled in each of the multiple resource pools in any scheduling stage, according to the resource requirement information corresponding to the computing engine to be scheduled, the resource information of each of the multiple resource pools, and the matching rules corresponding to the scheduling stage, includes:

[0067] In the current scheduling phase, according to the resource requirement information corresponding to the computing engine to be scheduled, the resource information of each of the multiple resource pools, and the matching rules corresponding to the current scheduling phase, the resource configurations of the computing engine to be scheduled in the multiple resource pools under the current scheduling phase are selected from the resource configurations determined in the previous scheduling phase.

[0068] If there is a next scheduling phase after the current scheduling phase, then the next scheduling phase of the current scheduling phase is taken as the updated current scheduling phase, and the step of filtering out the resource configurations of the computing engine to be scheduled in the multiple resource pools under the current scheduling phase from the resource configurations determined in the previous scheduling phase of the current scheduling phase according to the resource requirement information corresponding to the computing engine to be scheduled, the resource information of each of the multiple resource pools, and the matching rules corresponding to the current scheduling phase is continued.

[0069] If there is no next scheduling phase after the current scheduling phase, then the resource configuration of the computing engine to be scheduled is determined for each of the multiple resource pools.

[0070] For example, in a series of scheduling phases, the first scheduling phase is designated as the first current scheduling node. After the first scheduling phase is completed, the next scheduling phase of the first scheduling node can be designated as the next current scheduling phase, and so on, until all scheduling phases are completed.

[0071] Through the above embodiments, pre-set resource information filtering conditions can be used in each scheduling stage from beginning to end to select matching resource configurations for the computing engine to be scheduled. This can more effectively shield the heterogeneous capabilities of different resource pools, quickly achieve self-awareness of heterogeneous resource configurations, and quickly complete the scheduling of computing engines to different resource pools, thereby improving scheduling efficiency.

[0072] In one or more embodiments of this specification, in order to more conveniently extend the newly added resource pool to the resource scheduling management of multi-resource pool integration, the method further includes:

[0073] When a new resource pool is added, the resource management service is set up on the computing node of the new resource pool, so that the resource management service reports the resource information corresponding to the resource attributes of the computing node to the resource scheduling service.

[0074] Through the above embodiments, the newly added resource pool can be directly extended to the resource scheduling management of multiple resource pools based on the deployment resource management service, so that the newly added resource pool can provide resources for cloud products in an integrated manner with other resource pools. Therefore, the method provided in the embodiments of this specification can well support the processing capabilities of cloud products in multiple clouds.

[0075] The following is in conjunction with the appendix Figure 3 Taking the application of the resource scheduling method provided in this specification in a cloud-based database as an example, the resource scheduling method will be further explained. In this embodiment, determining the resource requirement information corresponding to the computing engine to be scheduled includes: receiving a database query request; parsing the database query request to determine the computing engine to be scheduled corresponding to the database query request and the resource requirement information corresponding to the computing engine to be scheduled. Specifically, Figure 3 A flowchart illustrating the processing steps of a resource scheduling method according to an embodiment of this specification is shown, specifically including the following steps.

[0076] Step 302: The resource management service in each computing node of the multiple resource pools reports the resource supply information of its node to the resource scheduling service of the master node.

[0077] After the resource management service in each computing node reports the resource supply information of its node to the resource scheduling service of the master node, the resource scheduling service will summarize the resource supply information reported by the computing nodes of each resource pool in order to perform resource scheduling in a unified manner.

[0078] Step 304: The database receives the query request, parses the database query request, and determines the scheduled computing engine corresponding to the database query request and the resource requirement information corresponding to the scheduled computing engine.

[0079] Step 306: The database calls the resource scheduling service of the master node in the computing cluster and passes in the resource requirement information corresponding to the computing engine to be scheduled.

[0080] Step 308: The resource scheduling service determines the resource configuration of the computing engine to be scheduled in each of the multiple resource pools based on the resource demand information corresponding to the computing engine to be scheduled and the resource supply information corresponding to the multiple resource pools.

[0081] In this step, the resource scheduling service can filter resources one by one based on the matching rules corresponding to each scheduling stage, and automatically assemble heterogeneous resource configurations for different resource pools in combination with resource demand information.

[0082] Step 310: The resource scheduling service schedules the computing engine to be scheduled to one or more resource pools for execution according to the resource configurations corresponding to the computing engine to be scheduled in the multiple resource pools.

[0083] In the above embodiments, the database, based on the resource scheduling service in the master node of the computing cluster, realizes the self-awareness of heterogeneous resource configuration from the database computing engine to different resource pools. That is, it realizes a multi-resource pool fusion scheduling scheme and supports the scheduling of multiple computing engines across multiple resource pools at the query level. This provides fusion capability for resource scheduling of multiple resource pools in the database scenario, effectively improving the processing efficiency and resource utilization of the cloud-based database.

[0084] To make the methods provided in the embodiments of this specification easier to understand, the following will further explain... Figure 4 The diagram illustrating the four-layer architecture of a cloud-native data warehouse serves as an illustrative example of the resource scheduling method provided in the embodiments of this specification. Figure 4 As shown, the four-layer architecture of the cloud-native data warehouse includes: cloud-native data warehouse engine resource configuration, unified scheduling of heterogeneous resources, heterogeneous capability management of resource pools, and data warehouse execution configuration. This four-layer architecture demonstrates the scheduling framework from the abstraction of cloud-native data warehouse resources to the actual execution resource configuration. The four-layer architecture will be described in detail below:

[0085] The cloud-native data warehouse engine resource configuration layer abstracts the runtime resource configurations for different computing engines within the cloud-native data warehouse. This configuration is consistent across different resource pools / multi-cloud environments, providing a standardized resource abstraction at the service layer. Figure 4 The three computing engines A, B, and C shown have the same resource requirements and resource configurations, and possess unified resource attributes, including storage, network, and affinity.

[0086] The unified scheduling of heterogeneous resources is used for resource scheduling services on the master node of a container cluster. For the computing engines to be scheduled, it filters resources in each of the three stages of scheduling—filtering, scoring, and binding—using matching rules corresponding to each stage. Combined with resource demand information, it automatically assembles heterogeneous resource configurations for different resource pools. This layer effectively shields the heterogeneous capabilities of different resource pools, quickly achieving self-awareness of heterogeneous resource configuration. Additionally, a resource pool capability acquisition module can be set up in the resource scheduling service to collect resource supply information reported by the resource management services of each computing node. This allows the resource pool capability acquisition module to aggregate the resource supply information reported by each computing node, obtain and manage the resource supply information of different resource pools. Figure 4The three scheduling phases—filtering, scoring, and binding—each have corresponding matching rules. These rules include resource information filtering conditions corresponding to one or more resource attributes for each of computing engines A, B, and C. For example, suppose the matching rules for the filtering phase include: computing engine A has no corresponding resource pool filtering condition; computing engine B's resource pool filtering condition is "either a fixed resource pool or an elastic resource pool"; and computing engine C has no corresponding resource pool filtering condition. Similarly, suppose the matching rules for the scoring phase include: computing engine A's resource pool filtering condition is "prioritize fixed resource pools"; computing engine B's resource pool filtering condition is "fixed pool three-replica physical distribution"; and computing engine C has no corresponding resource pool filtering condition. The matching rules for the binding phase are as follows... Figure 4 As shown, I will not go into details here.

[0087] The heterogeneous resource pool capability management layer can include solid-state resource pools and elastic resource pools. Of course, as the product evolves, this layer can include various types of resource pools, and the methods provided in this specification are not limited to this. This layer can set up resource management services on each computing node of different resource pools to manage the heterogeneous resource capabilities of that node in terms of storage, network, logs, and operations and maintenance. Figure 4 As shown, the resource provisioning information reported by the resource management service of a compute node includes: storage capacity corresponding to cloud disks and shared LVM (Logical Volume Manager); network capacity corresponding to ENI (Elastic Network Interface) and ENI caching; log capacity corresponding to Sidecar mode (dividing application functions into separate processes, called Sidecar mode) collection; and operation and maintenance capacity corresponding to self-healing operation and maintenance. The resource provisioning information reported by the resource management services of other compute nodes will not be described in detail here.

[0088] The data warehouse execution configuration: This layer is used to schedule the computing engine to be scheduled to one or more resource pools for execution according to the resource configurations corresponding to the computing engines to be scheduled in the multiple resource pools respectively.

[0089] Based on the above four-layer architecture, when applying the resource scheduling method provided in the embodiments of this specification to a cloud-native data warehouse, the scheduling process may include:

[0090] Step 1: The resource management service set up in each compute node of multiple resource pools reports the network, storage, specifications and other capabilities of this node;

[0091] Step 2: When a user of the cloud-native data warehouse issues a query request, the cloud-native data warehouse parses the query request, determines the computing engine to be scheduled and the corresponding resource requirement information of the computing engine to be scheduled, and sends the resource requirement information of the computing engine to be scheduled to the resource scheduling service so as to apply for resources in batches for the data warehouse query.

[0092] Step 3: The resource scheduling service automatically assembles the resource configurations of different resource pools based on the resource demand information of the computing engine to be scheduled, the matching rules corresponding to each scheduling stage, and the resource supply information of each resource pool, so as to complete the automatic perception and scheduling of heterogeneous resources of the rule-based data warehouse computing engine.

[0093] Step 4: The resource scheduling service schedules the computing engine to be scheduled to one or more resource pools for execution according to the resource configurations corresponding to the computing engines to be scheduled in the multiple resource pools.

[0094] like Figure 4 As shown, the assembled computing engines A, B, and C have different resource configurations in the solid-state resource pool and the elastic resource pool. After the assembled computing engines are scheduled to the corresponding resource pools, they will be executed according to their configured resource configurations. In the solid-state resource pool, computing engine A uses a system disk built on a shared disk, while computing engines B and C use data disks built on LVM shared disks. In the elastic resource pool, computing engines A, B, and C each use independent system disks, and computing engines B and C use independent cloud disks for their data disks.

[0095] As can be seen from the above scheduling process, the resource scheduling method provided in the embodiments of this specification enables cloud-based data warehouses to support self-awareness of resource configurations from data warehouse computing engines to different resource pools. This allows data warehouse computing engines to maximize the utilization of the infrastructure advantages of different resource pools, building end-to-end optimization and customization capabilities, including fine-grained computing resource specifications, local storage sharing and reuse of computing resources, and automatic specification inventory awareness. Furthermore, because the data warehouse parses query requests to determine the computing engines to be scheduled and their resource requirements, it achieves the integrated provision of resources to data warehouse instances at the query level across multiple heterogeneous / cross-cloud resource pools, providing unified resource elasticity capabilities for multiple computing engines at the query level across multiple resource pools. In addition, by abstracting a common resource configuration embodiment for different computing engines in the data warehouse, when a new resource pool is added, only a resource management service needs to be added to the computing nodes of the new resource pool, making expansion easier.

[0096] It should be noted that the above-described application scenarios of cloud-based data warehouses are merely illustrative examples of the methods provided in the embodiments of this specification. The multiple resource pools provided in the embodiments of this specification may include solid-state resource pools, dynamic resource pools, and other types of resource pools to improve the resource supply for cross-cloud analytics needs.

[0097] Corresponding to the above method embodiments, this specification also provides embodiments of a resource scheduling device. Figure 5 A schematic diagram of a resource scheduling device according to one embodiment of this specification is shown. Figure 5 As shown, the device includes:

[0098] The information acquisition module 502 is configured to acquire resource supply information corresponding to multiple resource pools. The resource supply information includes resource information corresponding to the resource attributes of the resource pools, and each resource pool includes one or more computing nodes.

[0099] The demand determination module 504 is configured to determine the resource demand information corresponding to the computing engine to be scheduled, wherein the resource demand information includes resource information corresponding to preset resource attributes.

[0100] The resource configuration module 506 is configured to determine the resource configuration of the computing engine to be scheduled in the multiple resource pools based on the resource demand information corresponding to the computing engine to be scheduled and the resource supply information corresponding to the multiple resource pools.

[0101] The scheduling execution module 508 is configured to schedule the computing engine to be scheduled to one or more resource pools for execution according to the resource configurations corresponding to the computing engine to be scheduled in the multiple resource pools respectively.

[0102] In one or more embodiments of this specification, the device is configured as a resource scheduler on the master node of the computing cluster. The information acquisition module shown is configured to receive resource supply information reported by the resource management service on the computing nodes of the computing cluster, wherein the resource management service is used to manage resource information corresponding to the resource attributes of the computing nodes.

[0103] In one or more embodiments of this specification, the requirement determination module includes: a query receiving submodule configured to receive database query requests; and a parsing submodule configured to parse the database query requests to determine the scheduled computing engine corresponding to the database query request and the resource requirement information corresponding to the scheduled computing engine.

[0104] In one or more embodiments of this specification, the resource configuration module includes:

[0105] The rule setting submodule is configured to pre-set corresponding matching rules for multiple scheduling stages. The multiple scheduling stages, from front to back, include at least a filtering stage, a scoring stage, and a binding stage. The matching rules include resource information filtering conditions corresponding to the resource attributes of the computing engine to be scheduled.

[0106] The stage matching submodule is configured to determine, in any scheduling stage, the resource configuration of the computing engine to be scheduled in each of the multiple resource pools, according to the resource requirement information of the computing engine to be scheduled, the resource information of each of the multiple resource pools, and the matching rules corresponding to the scheduling stage.

[0107] In one or more embodiments of this specification, the stage matching submodule includes:

[0108] The current matching submodule is configured to, in the current scheduling phase, according to the resource requirement information corresponding to the computing engine to be scheduled, the resource information of each of the multiple resource pools, and the matching rules corresponding to the current scheduling phase, filter out the resource configurations of the computing engine to be scheduled in the multiple resource pools under the current scheduling phase from the resource configurations determined in the previous scheduling phase of the current scheduling phase.

[0109] The phase update submodule is configured to, if there is a next scheduling phase after the current scheduling phase, take the next scheduling phase of the current scheduling phase as the updated current scheduling phase, and continue to execute the step of filtering out the resource configurations of the computing engine to be scheduled in the multiple resource pools under the current scheduling phase from the resource configurations determined in the previous scheduling phase of the current scheduling phase, according to the resource requirement information corresponding to the computing engine to be scheduled, the resource information of each of the multiple resource pools, and the matching rules corresponding to the current scheduling phase;

[0110] The configuration determination submodule is configured to determine the resource configuration of the computing engine to be scheduled in each of the multiple resource pools if there is no next scheduling phase after the current scheduling phase.

[0111] In one or more embodiments of this specification, the apparatus further includes: an extension module configured to, when a new resource pool exists, set up the resource management service on the computing node of the new resource pool, so that the resource management service reports the resource information corresponding to the resource attributes of the computing node to the resource scheduling service.

[0112] The above is an illustrative scheme of a resource scheduling device according to this embodiment. It should be noted that the technical solution of this resource scheduling device and the technical solution of the resource scheduling method described above belong to the same concept. For details not described in detail in the technical solution of the resource scheduling device, please refer to the description of the technical solution of the resource scheduling method described above.

[0113] Corresponding to the above method embodiments, this specification also provides computing cluster embodiments. Figure 6 A schematic diagram of the structure of a computing cluster provided in one embodiment of this specification is shown. Figure 6 As shown, the computing cluster includes: multiple resource pools 602 ( Figure 6 The diagram uses resource pools 602A and 602B to illustrate multiple resource pools (more resource pools can be set up in actual applications) and a master node 604 that applies the resource scheduling method as described in any embodiment of this specification. Each resource pool includes one or more computing nodes.

[0114] The above is an illustrative scheme of a computing cluster according to this embodiment. It should be noted that the technical solution of this computing cluster and the technical solution of the resource scheduling method described above belong to the same concept. For details not described in detail in the technical solution of the computing cluster, please refer to the description of the technical solution of the resource scheduling method described above.

[0115] Corresponding to the above method embodiments, this specification also provides database embodiments. Figure 7 A schematic diagram of the structure of a database provided in one embodiment of this specification is shown. For example... Figure 7 As shown, the database includes: a parsing module 702, and multiple computing engines 706. Figure 7 The diagram uses computing engines 706A and 706B to illustrate multiple computing engines (more computing engines can be set in actual applications) and computing cluster 704 as described in any embodiment of this specification.

[0116] The parsing module 702 is configured to receive a database query request, parse the database query request to determine the scheduled computing engine corresponding to the database query request and the resource requirement information corresponding to the scheduled computing engine, wherein the scheduled computing engine is one or more of the plurality of computing engines, and send the resource requirement information corresponding to the scheduled computing engine to the master node of the computing cluster, so that the master node schedules the scheduled computing engine to the computing node of the resource pool for execution according to the resource scheduling method described in any embodiment of this specification.

[0117] The above is an illustrative database scheme according to this embodiment. It should be noted that the technical solution of this database belongs to the same concept as the technical solution of the resource scheduling method described above. For details not described in detail in the technical solution of the database, please refer to the description of the technical solution of the resource scheduling method described above.

[0118] Figure 8 A structural block diagram of a computing device 800 according to one embodiment of this specification is shown. The components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0119] The computing device 800 also includes an access device 840, which enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 840 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0120] In one embodiment of this specification, the above-described components of the computing device 800 and Figure 8 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 8 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0121] The computing device 800 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 800 can also be a mobile or stationary server.

[0122] The processor 820 is used to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the resource scheduling method described above.

[0123] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the resource scheduling method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the resource scheduling method described above.

[0124] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the resource scheduling method described above.

[0125] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the resource scheduling method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the resource scheduling method described above.

[0126] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the resource scheduling method described above.

[0127] The above is an illustrative example of a computer program according to this embodiment. It should be noted that the technical solution of this computer program and the technical solution of the resource scheduling method described above belong to the same concept. Details not described in detail in the technical solution of the computer program can be found in the description of the technical solution of the resource scheduling method described above.

[0128] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0129] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0130] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0131] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0132] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A resource scheduling method, comprising: Obtain resource supply information corresponding to multiple resource pools. The resource supply information includes resource information corresponding to the resource attributes of the resource pools. One resource pool includes one or more computing nodes. The multiple resource pools include solid-state resource pools and elastic resource pools. Determine the resource requirement information corresponding to the computing engine to be scheduled, wherein the resource requirement information includes resource information corresponding to preset resource attributes; Based on the resource demand information corresponding to the computing engine to be scheduled and the resource supply information corresponding to the multiple resource pools, the resource configuration of the computing engine to be scheduled in the multiple resource pools is determined. The resource configuration is the description information of the resources that meet the runtime capability requirements of the computing engine to be scheduled in the corresponding resource pool. When different computing engines are in different resource pools, the resource attributes in the resource configuration are the same, but the resource information corresponding to the resource attributes is different. According to the resource configurations of the computing engines to be scheduled in the multiple resource pools, the computing engines to be scheduled are scheduled to one or more resource pools for concurrent execution.

2. The method according to claim 1, wherein the method is applied to resource scheduling services on the master node of a computing cluster; The step of obtaining the resource supply information corresponding to the multiple resource pools includes: The resource scheduling service receives resource supply information reported by the resource management service on the computing nodes of the computing cluster. The resource management service is used to manage the resource information corresponding to the resource attributes of the computing nodes.

3. The method according to claim 1, wherein determining the resource requirement information corresponding to the computing engine to be scheduled includes: Receive database query requests; The database query request is parsed to determine the scheduled computing engine corresponding to the database query request and the resource requirement information corresponding to the scheduled computing engine.

4. The method according to claim 1, wherein determining the resource configuration of the computing engine to be scheduled in each of the plurality of resource pools based on the resource requirement information corresponding to the computing engine to be scheduled and the resource information of each of the plurality of resource pools includes: Matching rules are pre-set for multiple scheduling stages, which, from front to back, include at least a filtering stage, a scoring stage, and a binding stage. The matching rules include resource information filtering conditions corresponding to the resource attributes of the computing engine to be scheduled. In any scheduling phase, the resource configuration of the computing engine to be scheduled in each of the multiple resource pools is determined according to the resource requirement information corresponding to the computing engine to be scheduled, the resource information of each of the multiple resource pools, and the matching rules corresponding to the scheduling phase.

5. The method according to claim 4, wherein in any scheduling stage, determining the resource configuration of the computing engine to be scheduled in the multiple resource pools according to the resource requirement information corresponding to the computing engine to be scheduled, the resource information of each of the multiple resource pools, and the matching rule corresponding to the scheduling stage, includes: In the current scheduling phase, according to the resource requirement information corresponding to the computing engine to be scheduled, the resource information of each of the multiple resource pools, and the matching rules corresponding to the current scheduling phase, the resource configurations of the computing engine to be scheduled in the multiple resource pools under the current scheduling phase are selected from the resource configurations determined in the previous scheduling phase. If there is a next scheduling phase after the current scheduling phase, then the next scheduling phase of the current scheduling phase is taken as the updated current scheduling phase, and the step of filtering out the resource configurations of the computing engine to be scheduled in the multiple resource pools under the current scheduling phase from the resource configurations determined in the previous scheduling phase of the current scheduling phase according to the resource requirement information corresponding to the computing engine to be scheduled, the resource information of each of the multiple resource pools, and the matching rules corresponding to the current scheduling phase is continued. If there is no next scheduling phase after the current scheduling phase, then the resource configuration of the computing engine to be scheduled is determined for each of the multiple resource pools.

6. The method according to claim 3, further comprising: When a new resource pool is added, the resource management service is set on the computing node of the new resource pool, so that the resource management service reports the resource information corresponding to the resource attributes of the computing node to the resource scheduling service. Resource supply information corresponding to multiple resource pools.

7. A computing cluster, comprising: Multiple resource pools and a master node that applies the resource scheduling method as described in any one of claims 1-6, wherein one of the resource pools includes one or more computing nodes.

8. A database, comprising: Parsing module, multiple computing engines, and computing clusters; The parsing module is configured to receive a database query request, parse the database query request to determine the scheduled computing engine corresponding to the database query request and the resource requirement information corresponding to the scheduled computing engine, wherein the scheduled computing engine is one or more of the plurality of computing engines, and send the resource requirement information corresponding to the scheduled computing engine to the computing cluster, so that the master node in the computing cluster schedules the scheduled computing engine to the computing node in the resource pool for execution according to the resource scheduling method as described in any one of claims 1-6.

9. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the resource scheduling method according to any one of claims 1 to 6.

10. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the resource scheduling method according to any one of claims 1 to 6.

Images