Resource scheduling method and apparatus, and electronic device and storage medium

Abstract

Provided in the embodiments of the present application are a resource scheduling method and apparatus, and an electronic device and a storage medium, which are applied to the technical field of cloud services. The method comprises: acquiring time overheads corresponding to a data task; on the basis of the time overheads and an elastic efficiency, screening out, from resource pools of a cloud end, a target resource pool corresponding to the data task; and scheduling, from the target resource pool, an elastic computing resource for completing the data task, so as to complete the data task on the basis of the elastic computing resource.

Description

Resource scheduling method, device, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 15, 2023, with application number 202311027651.5 and application name “Resource Scheduling Method, Device, Electronic Device and Storage Medium”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of cloud service technology, and in particular to a resource scheduling method, a resource scheduling device, an electronic device, and a computer-readable storage medium. Background Art

[0003] Cloud-native data warehouses are built on cloud infrastructure, offering unlimited resource and storage elasticity. Due to varying usage scenarios, processing time requirements for cloud-native data warehouses vary significantly. In actual resource scheduling, improper resource scheduling can lead to resource waste. This reduces cloud server resource utilization, increases operating costs, and can also lead to a poor user experience when using cloud services.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a resource scheduling method, device, electronic device and computer-readable storage medium to solve or partially solve the problem that unreasonable resource scheduling leads to resource waste and high costs in the resource scheduling process.

[0006] The present application discloses a resource scheduling method, which is applied to a cloud. The cloud is configured with a plurality of resource pools corresponding to different elastic efficiencies. The elastic efficiency is used to describe the time required for the resource pool to provide resources that meet a set elasticity requirement. The method includes:

[0007] The time overhead corresponding to the data acquisition task;

[0008] Filtering a target resource pool corresponding to the data task from a cloud resource pool according to the time cost and the elastic efficiency;

[0009] The elastic computing resources for completing the data task are scheduled from the target resource pool to complete the data task based on the elastic computing resources.

[0010] Optionally, the selecting a target resource pool corresponding to the data task from a resource pool in the cloud based on the time cost and the elastic efficiency includes:

[0011] Analyzing resource load of the data task using the time overhead to obtain a target elastic efficiency corresponding to the data task;

[0012] According to a matching result between the target elastic efficiency and the elastic efficiencies corresponding to the resource pools, a target resource pool corresponding to the data task is screened out from the resource pools.

[0013] Optionally, the resource pool is divided into at least a hot elastic resource pool, a warm elastic resource pool, and a cold elastic resource pool according to elastic efficiency, and selecting a target resource pool corresponding to the data task from each resource pool based on a matching result between the target elastic efficiency and the elastic efficiency corresponding to each resource pool includes:

[0014] Matching the target elastic efficiency with the elastic efficiency corresponding to each of the resource pools in sequence;

[0015] If the target elastic efficiency successfully matches the first elastic efficiency corresponding to the thermal elastic resource pool, the thermal elastic resource pool is used as the target resource pool for processing the data task;

[0016] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool, but successfully matches the second elastic efficiency corresponding to the warm elastic resource pool, the warm elastic resource pool is used as the target resource pool for processing the data task;

[0017] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool and fails to match the second elastic efficiency corresponding to the warm elastic resource pool, the cold elastic resource pool is used as the target resource pool for processing the data task.

[0018] Optionally, the method further includes:

[0019] Obtaining pool status information corresponding to each of the resource pools, the pool status information at least including a resource water level;

[0020] If the resource level of the target resource pool is higher than the preset level threshold, resources are replenished in the target resource pool.

[0021] Optionally, the time overhead corresponding to the data acquisition task includes:

[0022] Obtaining the access layer duration, computing resource startup duration, and execution duration of the data task corresponding to the data task;

[0023] The access layer duration, the computing resource startup duration, and the execution duration are used to calculate the duration overhead to obtain the time overhead for completing the data task.

[0024] Optionally, analyzing the resource load of the data task using the time overhead to obtain a target elastic efficiency corresponding to the data task includes:

[0025] Obtaining the percentage of the computing resource startup duration to the time overhead;

[0026] The resource load of the data task is analyzed using the percentage and the time overhead to obtain a target elastic efficiency corresponding to the data task.

[0027] Optionally, it also includes:

[0028] Pre-starting computing resource units of different specifications in the thermal elastic resource pool;

[0029] A cloud disk and a network card are pre-created on the device to which the warm elastic resource pool belongs, and the cloud disk and the network card are mounted.

[0030] Optionally, the scheduling the elastic computing resources for completing the data task from the target resource pool, so as to complete the data task based on the elastic computing resources, includes:

[0031] If the target resource pool is a thermal elastic resource pool, the elastic computing resources required to complete the data task are obtained, and a number of first computing resource units corresponding to the elastic computing resources are scheduled from the thermal elastic resource pool, and the data task is completed through the first computing resource units.

[0032] Optionally, the scheduling the elastic computing resources for completing the data task from the target resource pool, so as to complete the data task based on the elastic computing resources, includes:

[0033] If the target resource pool is a warm elastic resource pool, obtaining elastic computing resources required to complete the data task, and creating a number of second computing resource units corresponding to the elastic computing resources in the warm elastic resource pool based on the cloud disk and the network card;

[0034] The data task is completed by the second computing resource unit.

[0035] Optionally, the scheduling the elastic computing resources for completing the data task from the target resource pool, so as to complete the data task based on the elastic computing resources, includes:

[0036] Calculate the required cloud disks and network cards based on elastic computing resources, create them, and mount them separately.

[0037] According to the cloud disk and the network card, create a number of third computing resource units corresponding to the elastic computing resources in the cold elastic resource pool;

[0038] The data task is completed by the third computing resource unit.

[0039] The present application also discloses a resource scheduling method, which is applied to a cloud-native data warehouse. The cloud-native data warehouse includes at least a workload manager, a workload scheduler, and a plurality of resource pools corresponding to different elastic efficiencies. The elastic efficiency is used to describe the time required for the resource pool to provide resources that meet the set elastic requirements. The method includes:

[0040] Obtaining the time overhead corresponding to the data task through the business load manager;

[0041] The load scheduler selects a target resource pool corresponding to the data task from a cloud resource pool according to the time cost and the elastic efficiency;

[0042] The elastic computing resources for completing the data task are scheduled by the target resource pool, and the data task is completed based on the elastic computing resources.

[0043] Optionally, the selecting, by the load scheduler, a target resource pool corresponding to the data task from a resource pool in the cloud according to the time cost and the elastic efficiency includes:

[0044] The load scheduler uses the time overhead to analyze the resource load of the data task to obtain a target elastic efficiency corresponding to the data task;

[0045] The load scheduler selects a target resource pool corresponding to the data task from each resource pool based on a matching result between the target elastic efficiency and the elastic efficiency corresponding to each resource pool.

[0046] Optionally, the resource pool is divided into at least a hot elastic resource pool, a warm elastic resource pool, and a cold elastic resource pool according to elastic efficiency, and the load scheduler matches the target elastic efficiency with the elastic efficiency corresponding to each of the resource pools in sequence, and uses the successfully matched resource pool as the target resource pool corresponding to the data task, including:

[0047] Matching the target elastic efficiency with the elastic efficiency corresponding to each of the resource pools in sequence through the load scheduler;

[0048] If the target elastic efficiency successfully matches the first elastic efficiency corresponding to the thermal elastic resource pool, the load scheduler uses the thermal elastic resource pool as the target resource pool for processing the data task;

[0049] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool, but successfully matches the second elastic efficiency corresponding to the warm elastic resource pool, the load scheduler uses the warm elastic resource pool as the target resource pool for processing the data task;

[0050] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool and fails to match the second elastic efficiency corresponding to the warm elastic resource pool, the load scheduler uses the cold elastic resource pool as the target resource pool for processing the data task.

[0051] Optionally, the method further includes:

[0052] Obtaining pool status information corresponding to each of the resource pools through the load scheduler, wherein the pool status information at least includes a resource water level;

[0053] If the resource level of the target resource pool is higher than the preset water level threshold, the target resource pool is supplemented with resources through the load scheduler.

[0054] Optionally, obtaining the time overhead corresponding to the data task through the service load manager includes:

[0055] Obtaining, through the business load manager, the access layer duration, computing resource startup duration, and execution duration of the data task corresponding to the data task;

[0056] The service load manager uses the access layer duration, the computing resource startup duration, and the execution duration to perform duration overhead calculation to obtain the time overhead for completing the data task.

[0057] Optionally, the analyzing the resource load of the data task using the time overhead by the service load manager to obtain a target elastic efficiency corresponding to the data task includes:

[0058] Obtaining, by the business load manager, the percentage of the computing resource startup duration to the time overhead;

[0059] The service load manager uses the percentage and the time overhead to analyze the resource load of the data task and obtain a target elastic efficiency corresponding to the data task.

[0060] Optionally, the method further includes:

[0061] Pre-starting computing resource units of different specifications in the thermal elastic resource pool through the load scheduler;

[0062] The load scheduler pre-creates a cloud disk and a network card on the device to which the warm elastic resource pool belongs, and mounts the cloud disk and the network card.

[0063] Optionally, it also includes:

[0064] If the target resource pool is a thermal elastic resource pool, the elastic computing resources required to complete the data task are obtained through the thermal elastic resource pool, and the first computing resource units corresponding to the number of the elastic computing resources are scheduled from the thermal elastic resource pool, and the data task is completed through the first computing resource units.

[0065] Optionally, it also includes:

[0066] If the target resource pool is a warm elastic resource pool, obtaining the elastic computing resources required to complete the data task through the warm elastic resource pool, and creating a number of second computing resource units corresponding to the elastic computing resources in the warm elastic resource pool based on the cloud disk and the network card;

[0067] The data task is completed by the second computing resource unit.

[0068] Optionally, scheduling the elastic computing resources for completing the data task through the target resource pool, and completing the data task based on the elastic computing resources, includes:

[0069] Obtaining the elastic computing resources required to complete the data task through the cold elastic resource pool;

[0070] Calculate the cloud disk and network card that need to be created according to the elastic computing resources in the cold elastic resource pool, create the cloud disk and network card, and mount the created cloud disk and network card respectively;

[0071] Creating, by using the cold elastic resource pool according to the cloud disk and the network card, a number of third computing resource units corresponding to the elastic computing resources in the cold elastic resource pool;

[0072] The data task is completed by the cold elastic resource pool and the third computing resource unit.

[0073] The present application also discloses a resource scheduling device, which is applied to a cloud. The cloud is configured with a plurality of resource pools corresponding to different elastic efficiencies. The elastic efficiency is used to describe the time required for the resource pool to provide resources that meet a set elasticity requirement. The device includes:

[0074] Data acquisition module, used to obtain the time overhead corresponding to the data task;

[0075] A resource pool screening module, configured to screen a target resource pool corresponding to the data task from a resource pool in the cloud based on the time cost and the elastic efficiency;

[0076] The task processing module is used to schedule elastic computing resources for completing the data task from the target resource pool, so as to complete the data task based on the elastic computing resources.

[0077] Optionally, the resource pool screening module is specifically configured to:

[0078] Analyzing resource load of the data task using the time overhead to obtain a target elastic efficiency corresponding to the data task;

[0079] According to a matching result between the target elastic efficiency and the elastic efficiencies corresponding to the resource pools, a target resource pool corresponding to the data task is screened out from the resource pools.

[0080] Optionally, the resource pool is divided into at least a hot elastic resource pool, a warm elastic resource pool, and a cold elastic resource pool according to elasticity efficiency, and the resource pool screening module is specifically configured to:

[0081] Matching the target elastic efficiency with the elastic efficiency corresponding to each of the resource pools in sequence;

[0082] If the target elastic efficiency successfully matches the first elastic efficiency corresponding to the thermal elastic resource pool, the thermal elastic resource pool is used as the target resource pool for processing the data task;

[0083] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool, but successfully matches the second elastic efficiency corresponding to the warm elastic resource pool, the warm elastic resource pool is used as the target resource pool for processing the data task;

[0084] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool and fails to match the second elastic efficiency corresponding to the warm elastic resource pool, the cold elastic resource pool is used as the target resource pool for processing the data task.

[0085] Optionally, the device further comprises:

[0086] A status information acquisition module, configured to acquire pool status information corresponding to each resource pool, wherein the pool status information includes at least a resource water level;

[0087] The resource replenishment module is configured to replenish resources in the target resource pool if the resource level of the target resource pool is higher than a preset level threshold.

[0088] Optionally, the data acquisition module is specifically used to:

[0089] Obtaining the access layer duration, computing resource startup duration, and execution duration of the data task corresponding to the data task;

[0090] The access layer duration, the computing resource startup duration, and the execution duration are used to calculate the duration overhead to obtain the time overhead for completing the data task.

[0091] Optionally, the data acquisition module is specifically used to:

[0092] Obtaining the percentage of the computing resource startup duration to the time overhead;

[0093] The resource load of the data task is analyzed using the percentage and the time overhead to obtain a target elastic efficiency corresponding to the data task.

[0094] Optionally, it also includes:

[0095] A first resource pool startup module, configured to pre-start computing resource units of different specifications in the thermal elastic resource pool;

[0096] The second resource pool startup module is used to pre-create a cloud disk and a network card on the device to which the warm elastic resource pool belongs, and mount the cloud disk and the network card.

[0097] Optionally, the task processing module is specifically configured to:

[0098] If the target resource pool is a thermal elastic resource pool, the elastic computing resources required to complete the data task are obtained, and a number of first computing resource units corresponding to the elastic computing resources are scheduled from the thermal elastic resource pool, and the data task is completed through the first computing resource units.

[0099] Optionally, the task processing module is specifically configured to:

[0100] If the target resource pool is a warm elastic resource pool, obtaining elastic computing resources required to complete the data task, and creating a number of second computing resource units corresponding to the elastic computing resources in the warm elastic resource pool based on the cloud disk and the network card;

[0101] The data task is completed by the second computing resource unit.

[0102] Optionally, the task processing module is specifically configured to:

[0103] Obtaining the elastic computing resources required to complete the data task;

[0104] Calculate the required cloud disks and network cards based on elastic computing resources, create them, and mount them separately.

[0105] According to the cloud disk and the network card, create a number of third computing resource units corresponding to the elastic computing resources in the cold elastic resource pool;

[0106] The data task is completed by the third computing resource unit.

[0107] The embodiment of the present application further discloses an electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0108] The memory is used to store computer programs;

[0109] The processor is used to implement the method described in the embodiment of the present application when executing the program stored in the memory.

[0110] The embodiment of the present application further discloses a computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, causes the processors to execute the method described in the embodiment of the present application.

[0111] The embodiments of the present application include the following advantages:

[0112] In an embodiment of the present application, it can be applied to the cloud, where several resource pools corresponding to different elastic efficiencies are configured. The elastic efficiency is used to describe the time required for the resource pool to provide resources that meet the set elastic requirements. The data tasks submitted by the user are processed through the cloud. During the processing, the time overhead corresponding to the data task is obtained, and then the target resource pool corresponding to the data task is screened from the resource pool in the cloud based on the time overhead and elastic efficiency. Then, the elastic computing resources for completing the data task are scheduled from the target resource pool to complete the data task, so as to complete the data task based on the elastic computing resources. On the one hand, by constructing a resource pool with multi-level elastic efficiency, the processing requirements of different data tasks can be met, thereby reducing the operating cost. On the other hand, the resource elasticity is analyzed based on the time overhead corresponding to the data task, and the appropriate resource pool is selected for task processing based on the analysis results, thereby improving the rationality of resource scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0113] FIG1 is a flowchart of a resource scheduling method provided in an embodiment of the present application;

[0114] FIG2 is a schematic diagram of a cloud provided in an embodiment of the present application;

[0115] FIG3 is a schematic diagram of a scenario provided in an embodiment of the present application;

[0116] FIG4 is a schematic diagram of a thermal elastic resource pool provided in an embodiment of the present invention;

[0117] FIG5 is a schematic diagram of a warm elastic resource pool provided in an embodiment of the present invention;

[0118] FIG6 is a schematic diagram of a cold elastic resource pool provided in an embodiment of the present invention;

[0119] FIG7 is a flowchart of the steps of a resource scheduling method provided in an embodiment of the present application;

[0120] FIG8 is a schematic diagram of a cloud-native data warehouse provided in an embodiment of the present application;

[0121] FIG9 is a structural block diagram of a resource scheduling device provided in an embodiment of the present application;

[0122] FIG10 is a block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0123] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0124] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0125] As an example, for cloud-native data warehouses, for data tasks submitted by users, different data tasks correspond to different requirements for resource elasticity and efficiency. In related technologies, there are situations of over-configuration and under-configuration of resources for handling different resource elasticity and efficiency requirements. This not only easily affects the processing of data tasks and reduces user experience, but also, due to unreasonable resource configuration, it is easy to cause resource waste in cloud-native data warehouses, resulting in high operating costs.

[0126] In this regard, one of the core invention points of this application is to configure resource pools with different elastic efficiencies in the cloud. During the processing process, by obtaining the time overhead corresponding to the data task, and then based on the time overhead and elastic efficiency, the target resource pool corresponding to the data task is screened out from the resource pool in the cloud, and then the elastic computing resources for completing the data task are scheduled from the target resource pool to complete the data task based on the elastic computing resources. On the one hand, by constructing a multi-level elastic efficiency resource pool, the processing requirements of different data tasks can be met, reducing the operating costs. On the other hand, the resource elasticity is analyzed based on the time overhead corresponding to the data task, and the appropriate resource pool is selected for task processing based on the analysis results, thereby improving the rationality of resource scheduling.

[0127] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical features involved in the embodiments of the present application are explained and illustrated below:

[0128] Cloud-native data warehouse (CDN): A data warehouse built on cloud infrastructure with unlimited resource elasticity and storage elasticity.

[0129] Query (data task): A user initiates a query, analysis, calculation, offline processing, or other tasks that require cloud native data warehouse processing.

[0130] Query-level resources: Resources applied for a separate query in the cloud-native data warehouse. The resources are released after the query ends, that is, the computing resources required to complete a data task. They refer to the hardware and software used to calculate and process data, including hardware devices such as processors, memory, hard disks, networks, and software systems such as operating systems, programming languages, applications, and frameworks. Different data tasks require different computing resources to complete, and different computing resources correspond to different hardware and software. In this application, computing resource units are used as the "carrier" of computing resources to process data tasks. The computing resource units can be Pods (hereinafter uniformly described as computing resource Pods), and this application does not impose any restrictions on this.

[0131] Load QoS (Quality of Service): The processing time requirements for queries (data tasks) of different priorities, demand scenarios, data volumes, and computational complexities. For example, the processing time requirement for simple query tasks is less than 1 second; the processing time requirement for complex data aggregation tasks is less than 5 minutes; and the processing time requirement for load ETL tasks is less than 1 hour. For ETL, it can be Extract, Transform, Load) is a common data processing method that extracts data from different sources (Extract), transforms the data (Transform), and finally loads (Load) to the target system.

[0132] Resource pool: A resource pool used to meet the computing resource requirements of cloud-native data warehouse queries.

[0133] 1 , a flowchart of a resource scheduling method provided in an embodiment of the present application is shown. The method is applied to a cloud, where the cloud is configured with a plurality of resource pools corresponding to different elastic efficiencies. The elastic efficiency is used to describe the time required for a resource pool to provide resources that meet a set elasticity requirement. Specifically, the method may include the following steps:

[0134] Step 101, obtaining the time overhead corresponding to the data task;

[0135] With reference to FIG2, a schematic diagram of the cloud provided in an embodiment of the present application is shown, wherein the cloud can be a cloud native data warehouse, and resource pools with different elastic efficiencies can be configured in the cloud native data warehouse. It should be noted that for elasticity, it can prevent the over-configuration of computing resources and the under-configuration of resources, while elastic efficiency refers to the efficiency of data processing, which is expressed by describing the time required for the resource pool to provide resources that meet the set elastic requirements. The higher the elastic efficiency, the shorter the time level of data tasks that can be supported. Correspondingly, the higher the elastic efficiency, the higher the configuration cost of the resource pool. To this end, by configuring resource pools with different elastic efficiencies and calling the appropriate resource pool on demand for task processing during the processing of data tasks, the rationality of resource scheduling can be improved while reducing operating costs. With reference to FIG3, a schematic diagram of a scenario provided in an embodiment of the present application is shown. A user can send a corresponding data processing instruction to the cloud native data warehouse through a user terminal. The cloud native data warehouse determines the corresponding data task based on the data processing instruction, then obtains the time overhead corresponding to the data task, and selects the appropriate target resource pool based on the matching result between the time overhead and each resource pool to process the data task, such as completing data query, data analysis, offline ETL, etc.

[0136] In a specific implementation, the user can send the corresponding data processing instructions to the cloud through the user terminal, and the cloud native data warehouse can determine at least one data task corresponding to the data processing instructions based on the data processing instructions. Among them, the data task includes at least one of a data query task, a data analysis task, a data calculation task, and an offline data task. At the same time, after determining the data task, the time overhead corresponding to the data task can be further calculated. The time overhead can be used to characterize the overall time required to complete the data task. Different data tasks can correspond to different time overheads. For example, for a data task, the corresponding time overhead can cover multiple delay scenarios such as 100ms level, second level, minute level, ten-minute level, hour level, etc.

[0137] Step 102: Filtering a target resource pool corresponding to the data task from a cloud resource pool based on the time cost and the elastic efficiency;

[0138] In specific implementations, the cloud can use time overhead to analyze the resource load of data tasks to obtain the target elastic efficiency corresponding to the data tasks. Then, based on the matching results between the target elastic efficiency and the elastic efficiency corresponding to each resource pool, the target resource pool corresponding to the data task is screened out from each resource pool.

[0139] For data tasks, you can first obtain the access layer duration, computing resource startup duration, and execution duration corresponding to the data task, and then use the access layer duration, computing resource startup duration, and execution duration to calculate the completion duration to generate the time overhead required to complete the data task.

[0140] Among them, the access layer time, which can be the time from the user terminal initiating a request to the backend service responding to the request, is also called the end-to-end response time or user experience time, which is generally between 10ms and 100ms; the computing resource startup time is related to the elasticity of different resource pools in the cloud. For example, resource pool ① takes 1s to elastically start 300 computing resources; resource pool ② takes 5s to elastically start 300 computing resources; resource pool ③ takes 30s to elastically start 300 computing resources; resource pool ④ takes 1 minute to elastically start 300 computing resources, and so on. The cost of building resource pools with different elasticity capabilities is different. Generally, the stronger the elasticity, the greater the cost; the execution time, which can be the time required for the data task to actually execute on the computing resources, is related to the server's processing power, data volume, and the computational complexity of the task.

[0141] Optionally, for the access layer duration, computing resource startup duration and execution duration, the access layer duration can be basically ignored, the execution duration is strongly related to the user's input, the computing resource startup duration is a constant overhead, and for the computing resource startup duration, high elastic efficiency also increases the cost. In the coverage range of the delay duration of the cloud native data warehouse, the delay duration corresponding to different data tasks is also sensitive to the computing resource startup duration. The longer the delay duration, the lower the sensitivity. The computing resources of the data task can be scheduled to a resource pool with low cost and slow elastic efficiency. In this regard, in this application, the load situation of the data task is analyzed by defining "Elastic_Resource_Time (i.e. elastic efficiency duration)". The longer the elastic efficiency duration corresponding to the data task, the lower the elastic efficiency of the resource pool used to complete the data task. Conversely, the higher the elastic efficiency of the resource pool used for the data task.

[0142] Based on the above description, the load of data tasks can be analyzed through elastic efficiency duration. In the process of load analysis of data tasks, we can first obtain the percentage of computing resource startup duration to time overhead, and then use the percentage and time overhead to analyze the resource load of the data task to obtain the target elastic efficiency corresponding to the data task, so that we can schedule it to a matching resource pool based on the elastic efficiency duration and process the data task in a reasonable and low-cost manner.

[0143] In one example, the elastic efficiency duration can be generated by the following formula:

[0144] Elastic_Resource_Time=Query_RT*Elastic_Tolerate_Threshold

[0145] Elastic_Resource_Time is the elastic efficiency duration; Query_RT is the time overhead, that is, the overall time to complete the data task; Elastic_Tolerate_Threshold is the percentage of computing resource startup time to time overhead.

[0146] The time cost can be calculated by the following formula:

[0147] Query_RT = access layer duration + computing resource startup duration + execution duration

[0148] For example, assuming Elastic_Tolerate_Threshold = 10%, a data task with a time overhead of 10s will be scheduled to a resource pool with Elastic_Resource_Time = 1s; a query with a minute-level delay can be scheduled to a resource pool with Elastic_Resource_Time = 6s; and a query with a ten-minute-level delay can be scheduled to a resource pool with Elastic_Resource_Time = 60s. This allows us to analyze resource elasticity based on the time overhead corresponding to the data task, and select a suitable resource pool for task processing based on the analysis results, thereby improving the rationality of resource scheduling.

[0149] In the embodiment of the present application, while matching a resource pool with a data task, the cloud can also obtain real-time pool status information reported by each resource pool while calculating the elastic efficiency duration corresponding to the data task. This pool status information can be used to represent the current usage status of the resource pool. The pool status information can include information such as resource level, elastic efficiency, and cost. The resource level can be the resource utilization rate of the resource pool; the elastic efficiency can be the efficiency of data processing; and the cost can be the hardware configuration cost of the resource pool.

[0150] In a specific implementation, the target elastic efficiency can be matched with the elastic efficiency corresponding to each resource pool in turn, and the resource pool that is successfully matched is used as the target resource pool corresponding to the data task. Specifically, different resource pools correspond to different elastic efficiencies, and the elastic efficiency is expressed as the startup time required to start a preset number of computing resource units. The shorter the time required to start the same number of computing resource units, the higher the elastic efficiency. For example, a resource pool of 300 computing resource units is started in 1 second, a resource pool of 300 computing resource units is started in 5 seconds, and a resource pool of 300 computing resource units is started in 30 seconds. Among them, the computing resource unit can be the smallest unit running in the resource pool, which can include one or more containers and share the same network for performing corresponding processing on data tasks, such as completing data query, data analysis, offline ETL, etc.

[0151] In an optional embodiment, resource pools in the cloud can be divided into at least hot elastic resource pools, warm elastic resource pools, and cold elastic resource pools based on elastic efficiency. The target elastic efficiency corresponding to a data task can be compared with the elastic efficiencies corresponding to each elastic resource pool to select an appropriate resource pool for processing the data task. If the target elastic efficiency successfully matches the elastic efficiency corresponding to the hot elastic resource pool, the hot elastic resource pool is used as the target resource pool for processing the data task. If the target elastic efficiency fails to match the elastic efficiency corresponding to the hot elastic resource pool but successfully matches the elastic efficiency corresponding to the warm elastic resource pool, the warm elastic resource pool is used as the target resource pool for processing the data task. If the target elastic efficiency fails to match the elastic efficiency corresponding to the hot elastic resource pool and fails to match the elastic efficiency corresponding to the warm elastic resource pool, the cold elastic resource pool is used as the target resource pool for processing the data task. Thus, by constructing resource pools with multiple levels of elastic efficiency, the processing requirements of different data tasks can be met, reducing operating costs. Furthermore, resource elasticity analysis is performed based on the time overhead corresponding to the data task, and an appropriate resource pool is selected for task processing based on the analysis results, improving the rationality of resource scheduling.

[0152] In a specific implementation, resource pools such as the hot elastic resource pool, warm elastic resource pool, and cold elastic resource pool each correspond to different startup duration ranges (i.e., elastic efficiency). The resource pool that belongs to the startup duration range within which the target elastic efficiency falls can be used as the target resource pool.

[0153] Among them, for the thermal elastic resource pool, in order to improve the elastic efficiency of the computing resource unit, computing resource Pods of different rules can be pre-started in the thermal elastic resource pool. When the data task requires computing resources, the computing resource Pods of the corresponding specifications can be directly obtained from the thermal elastic resource pool. In the process of obtaining the computing resource Pod, since the computing resource Pod has been started in advance, the elastic efficiency of its computing resources can reach the 1s level. For example, since the cloud has multiple specifications for computing resources, computing resources of different specifications need to pre-start a batch of cache pools in advance. In one example, different computing resources can be started in the thermal elastic resource pool according to three different specifications: large, medium and small, including starting 12 small-specification computing resources, 9 medium-specification computing resources and 6 large-specification computing resources. It should be noted that for the specifications, it can be divided according to computing power, and this application does not limit this.

[0154] In a warm elastic resource pool, the startup time of a computing resource pod primarily includes the creation and mounting time of the cloud disk and network interface card. In this case, the cloud disk and network interface card can be created and mounted in advance. This saves the creation and mounting time of the cloud disk and network interface card when computing resources are requested, thus optimizing elastic efficiency. Compared to a "hot elastic resource pool," this architecture does not require pre-starting computing resources. All computing resources are created when a query arrives, allowing the entire machine to reuse computing resources of various specifications.

[0155] For cold elastic resource pools, resource requirements for data tasks are applied for on-demand. Once the application is successful, cloud disks and network cards are created and mounted in the cold elastic resource pool. Therefore, the elasticity efficiency of cold elastic resource pools is low.

[0156] It should be noted that the embodiments of this application are illustrated by taking hot elastic resource pool, warm elastic resource pool and cold elastic resource pool as examples. Under the guidance of the concept of this application, other types of resource pools can also be set according to actual needs.

[0157] In one example, assume that the startup time for a hot elastic resource pool is 1 second to start 300 pods, with a startup time interval of less than 5 seconds; the startup time for a warm elastic resource pool is 5 seconds to start 300 pods, with a startup time interval of (5s, 30s); and the startup time for a cold elastic resource pool is 30 seconds to start 300 pods, with a startup time interval of greater than 30s. If the target elastic efficiency for a data task is 3 seconds, the hot elastic resource pool can be used as the target resource pool for processing the data task; if the target elastic efficiency is 10 seconds, the warm elastic resource pool can be used as the target resource pool for processing the data task; and if the target elastic efficiency is 1 minute, the cold elastic resource pool can be used as the target resource pool for processing the data task. This allows resource pools with multiple levels of elastic efficiency to meet the processing requirements of different data tasks and reduce operating costs. Furthermore, resource elasticity analysis is performed based on the time overhead of the data task, and the appropriate resource pool is selected for task processing based on the analysis results, improving the rationality of resource scheduling.

[0158] In addition, during the operation of the resource pool, the cloud can judge the current resource pool utilization rate in real time based on the resource water level reported by the resource pool, and replenish resources for resource pools whose resource water level is lower than the preset water level threshold, ensuring that each resource pool has sufficient computing resources to complete the corresponding data tasks.

[0159] Step 103: Schedule elastic computing resources for completing the data task from the target resource pool, so as to complete the data task based on the elastic computing resources.

[0160] After selecting the target resource pool that matches the data task, the elastic computing resources required to complete the data task can be scheduled from the target resource pool, and the data task can be completed based on the elastic computing resources. On the one hand, by building a multi-level elastic and efficient resource pool, the processing requirements of different data tasks can be met, reducing operating costs. On the other hand, based on the time overhead corresponding to the data task, the resource elasticity is analyzed, and based on the results of the analysis, the appropriate resource pool is selected for task processing, thereby improving the rationality of resource scheduling.

[0161] In a specific implementation, after determining the elastic computing resources required to complete a data task, if the target resource pool is a hot elastic resource pool, since computing resource units of different specifications are pre-started in the hot elastic resource pool, during the processing of the data task, a first computing resource unit corresponding to the number of elastic computing resources can be directly obtained from the hot elastic resource pool, and the data task can be completed using the first computing resource unit. If the target resource pool is a warm elastic resource pool, since cloud disks and network cards have been pre-installed and mounted on the device to which the warm elastic resource pool belongs, a second computing resource unit corresponding to the number of elastic computing resources can be created in the warm elastic resource pool based on the cloud disks and network cards, and the data task can then be completed using the second computing resource unit. When the target resource pool is a cold elastic resource pool, the cloud disk and network card that need to be created can be calculated according to the elastic computing resources, and the cloud disk and network card can be created and mounted respectively. Then, according to the cloud disk and network card, a third computing resource unit corresponding to the elastic computing resources is created in the cold elastic resource pool. Then, the data task is completed through the third computing resource unit. On the one hand, by building a multi-level elastic efficiency resource pool, the processing requirements of different data tasks can be met, reducing the operating cost. On the other hand, the resource elasticity is analyzed based on the time overhead corresponding to the data task, and the appropriate resource pool is selected for task processing based on the analysis results, thereby improving the rationality of resource scheduling.

[0162] In some examples, referring to FIG4 , a schematic diagram of a thermal elastic resource pool provided in an embodiment of the present invention is shown. The thermal elastic resource pool may be composed of several machines, each of which includes four small-scale computing resources, three medium-scale computing resources, and two large-scale computing resources. Each computing resource is mounted with a corresponding disk and network card. Query 1 is completed by eight small-scale computing resources, Query 2 is completed by four small-scale computing resources, and Query 3 is completed by four large-scale computing resources.

[0163] Figure 5 shows a schematic diagram of a warm elastic resource pool provided in an embodiment of the present invention. The warm elastic resource pool can be composed of several machines. Machine 1 includes four small-scale computing resources and two large-scale computing resources, each with a corresponding disk and network interface card (NIC). Machine 2 includes four small-scale computing resources and two large-scale computing resources, and each machine is equipped with a disk cache pool and a network interface card (NIC) cache pool. Query 1 is completed by the eight small-scale computing resources of the two machines, Query 2 is completed by the four small-scale computing resources, and Query 3 is completed by the four large-scale computing resources of the two machines.

[0164] Referring to Figure 6, a schematic diagram of a cold elastic resource pool provided in an embodiment of the present invention is shown. The cold elastic resource pool may include 12 small-scale computing resources and 4 large-scale computing resources. Each computing resource is mounted with a corresponding disk and network card. Query 1 is completed by 8 small-scale computing resources, Query 2 is completed by 4 small-scale computing resources, and Query 3 is completed by 4 large-scale computing resources.

[0165] It should be noted that the embodiments of the present application include but are not limited to the above examples. It can be understood that under the guidance of the ideas of the embodiments of the present application, settings can be made according to actual needs, and the present application does not impose any restrictions on this.

[0166] In an embodiment of the present application, it can be applied to the cloud, where several resource pools corresponding to different elastic efficiencies are configured. The elastic efficiency is used to describe the time required for the resource pool to provide resources that meet the set elastic requirements. The data tasks submitted by the user are processed through the cloud. During the processing, the time overhead corresponding to the data task is obtained, and then the target resource pool corresponding to the data task is screened from the resource pool in the cloud based on the time overhead and elastic efficiency. Then, the elastic computing resources for completing the data task are scheduled from the target resource pool to complete the data task, so as to complete the data task based on the elastic computing resources. On the one hand, by constructing a resource pool with multi-level elastic efficiency, the processing requirements of different data tasks can be met, thereby reducing the operating cost. On the other hand, the resource elasticity is analyzed based on the time overhead corresponding to the data task, and the appropriate resource pool is selected for task processing based on the analysis results, thereby improving the rationality of resource scheduling.

[0167] 7 , a flowchart of a resource scheduling method provided in an embodiment of the present application is shown. The method is applied to a cloud-native data warehouse. The cloud-native data warehouse includes at least a workload manager, a workload scheduler, and a plurality of resource pools corresponding to different elastic efficiencies. The elastic efficiency is used to describe the time required for a resource pool to provide resources that meet a set elasticity requirement. Specifically, the method may include the following steps:

[0168] Step 701: Obtain the time overhead corresponding to the data task through the business load manager;

[0169] Step 702: The load scheduler selects a target resource pool corresponding to the data task from the cloud resource pool according to the time cost and the elastic efficiency.

[0170] Step 703: Schedule the elastic computing resources for completing the data task through the target resource pool, and complete the data task based on the elastic computing resources.

[0171] In some feasible implementations, the load scheduler screening out a target resource pool corresponding to the data task from a cloud resource pool based on the time cost and the elastic efficiency includes:

[0172] The load scheduler uses the time overhead to analyze the resource load of the data task to obtain a target elastic efficiency corresponding to the data task;

[0173] The load scheduler selects a target resource pool corresponding to the data task from each resource pool based on a matching result between the target elastic efficiency and the elastic efficiency corresponding to each resource pool.

[0174] In some feasible implementations, the resource pool is divided into at least a hot elastic resource pool, a warm elastic resource pool, and a cold elastic resource pool according to elastic efficiency, and the load scheduler matches the target elastic efficiency with the elastic efficiency corresponding to each of the resource pools in sequence, and uses the successfully matched resource pool as the target resource pool corresponding to the data task, including:

[0175] Matching the target elastic efficiency with the elastic efficiency corresponding to each of the resource pools in sequence through the load scheduler;

[0176] If the target elastic efficiency successfully matches the first elastic efficiency corresponding to the thermal elastic resource pool, the load scheduler uses the thermal elastic resource pool as the target resource pool for processing the data task;

[0177] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool, but successfully matches the second elastic efficiency corresponding to the warm elastic resource pool, the load scheduler uses the warm elastic resource pool as the target resource pool for processing the data task;

[0178] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool and fails to match the second elastic efficiency corresponding to the warm elastic resource pool, the load scheduler uses the cold elastic resource pool as the target resource pool for processing the data task.

[0179] In some feasible implementations, the method further includes:

[0180] Obtaining pool status information corresponding to each of the resource pools through the load scheduler, wherein the pool status information at least includes a resource water level;

[0181] If the resource level of the target resource pool is higher than the preset water level threshold, the target resource pool is supplemented with resources through the load scheduler.

[0182] In some feasible implementations, obtaining the time overhead corresponding to the data task by the service load manager includes:

[0183] Obtaining, through the business load manager, the access layer duration, computing resource startup duration, and execution duration of the data task corresponding to the data task;

[0184] The service load manager uses the access layer duration, the computing resource startup duration, and the execution duration to perform duration overhead calculation to obtain the time overhead for completing the data task.

[0185] In some feasible implementations, analyzing the resource load of the data task using the time overhead by the service load manager to obtain a target elastic efficiency corresponding to the data task includes:

[0186] Obtaining, by the business load manager, the percentage of the computing resource startup duration to the time overhead;

[0187] The service load manager uses the percentage and the time overhead to analyze the resource load of the data task and obtain a target elastic efficiency corresponding to the data task.

[0188] In some feasible implementations, the method further includes:

[0189] Pre-starting computing resource units of different specifications in the thermal elastic resource pool through the load scheduler;

[0190] The load scheduler pre-creates a cloud disk and a network card on the device to which the warm elastic resource pool belongs, and mounts the cloud disk and the network card.

[0191] In some feasible implementations, the following further comprises:

[0192] If the target resource pool is a thermal elastic resource pool, the elastic computing resources required to complete the data task are obtained through the thermal elastic resource pool, and the first computing resource units corresponding to the number of the elastic computing resources are scheduled from the thermal elastic resource pool, and the data task is completed through the first computing resource units.

[0193] In some feasible implementations, the following further comprises:

[0194] If the target resource pool is a warm elastic resource pool, obtaining the elastic computing resources required to complete the data task through the warm elastic resource pool, and creating a number of second computing resource units corresponding to the elastic computing resources in the warm elastic resource pool based on the cloud disk and the network card;

[0195] The data task is completed by the second computing resource unit.

[0196] In some feasible implementations, scheduling the elastic computing resources for completing the data task through the target resource pool, and completing the data task based on the elastic computing resources, includes:

[0197] Obtaining the elastic computing resources required to complete the data task through the cold elastic resource pool;

[0198] Calculate the cloud disk and network card that need to be created according to the elastic computing resources in the cold elastic resource pool, create the cloud disk and network card, and mount the created cloud disk and network card respectively;

[0199] Creating, by using the cold elastic resource pool according to the cloud disk and the network card, a number of third computing resource units corresponding to the elastic computing resources in the cold elastic resource pool;

[0200] The data task is completed by the cold elastic resource pool and the third computing resource unit.

[0201] In an embodiment of the present application, it can be applied to the cloud, and the data tasks submitted by users are processed through the cloud. During the processing, at least one data task and the time overhead corresponding to the data task are obtained, and then the target resource pool corresponding to the data task is screened out from the resource pool of the cloud based on the time overhead and the elastic efficiency, and then the data task is completed based on the target resource pool. On the one hand, by constructing a resource pool with multi-level elastic efficiency, the processing requirements of different data tasks can be met, thereby reducing the operating cost. On the other hand, the resource elasticity is analyzed based on the time overhead corresponding to the data task, and the appropriate resource pool is selected for task processing based on the analysis results, thereby improving the rationality of resource scheduling.

[0202] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, an example is provided below for illustrative purposes:

[0203] Referring to Figure 8, a schematic diagram of a cloud-native data warehouse provided in an embodiment of the present application is shown. For a cloud-native data warehouse, it may include a business load manager, a load scheduler, and several resource pools (hot elastic resource pool, warm elastic resource pool, and cold elastic resource pool, etc.).

[0204] Among them, for the business load manager (WorkLoadManager), in the cloud native data warehouse, the business load manager can manage the queries issued by users, including queuing, priority, RT estimation and other capabilities. In this example, the RT estimation capability is mainly used; at the same time, the historical RT time of QueryPattern and Query can be used to estimate the RT of the query issued by the current user, and the RT is used as the input of the load scheduler to evaluate the query elastic Qos.

[0205] The load scheduler (also known as the Elastic QoS-aware scheduler or the ElasticQosScheduler) can handle queries with different elastic QoS (Quality of Service, service selection priority) and, under the premise of meeting the elastic QoS, use reasonable resource costs to complete the scheduling of query computing resources in different resource pools. The load scheduler can include four modules: the "resource pool elastic configuration" module is used to receive the pool status information of elastic efficiency, cost overhead, and resource water level reported by each resource pool; the "query elastic tolerance time calculation" module uses the corresponding formula to calculate the tolerance time Elastic_Resource_Time of the query elastic QoS (the calculation process refers to the aforementioned embodiment and will not be repeated here); the "resource pool matching scoring" module can use the elastic QoS tolerance time of the query to match different resource pools. Specifically, the resource pool that meets the quality elastic QoS can be matched in the order of cold resource pool -> warm resource pool -> hot resource pool; the "dispatching to resource pool" module matches the specific resource pool and dispatches the query computing resources to the specific resource pool for execution.

[0206] Each resource pool can manage its own elasticity efficiency, cost, and self-sustaining resources. For example, a hot elastic resource pool can have an elasticity efficiency of 1s, a corresponding cost of "high," and self-sustaining resources of "self-sustaining" (indicating that it can self-sustain computing resources); a warm elastic resource pool can have an elasticity efficiency of 5s, a corresponding cost of "medium," and self-sustaining resources of "self-sustaining"; a cold elastic resource pool can have an elasticity efficiency of 30s, a corresponding cost of "low," and self-sustaining resources of "none" (indicating that it needs to create corresponding computing resources on demand based on data tasks).

[0207] In a specific implementation, data processing can be performed according to the following process:

[0208] 1. Supplement cache machine resources based on already used resources: Different resource pools evaluate their own elasticity and costs, and supplement cache machine resources based on the water level.

[0209] 2. Resource pools report elastic efficiency, cost, and resource level to the central scheduler: Each resource pool reports its own elastic efficiency, cost expenditure, resource level, and other information to the central scheduler.

[0210] 3. User-issued queries with various latency requirements: Users issue queries with various latency characteristics, which may include online analysis, complex calculations, offline ETL, etc.

[0211] 4. WorkLoadManager estimates the RT of different queries based on historical query patterns: The WorkLoadManager module estimates the corresponding RT of the query issued by the user

[0212] 5. Query and estimated RT are sent to the scheduler: WorkLoadManager sends the query and estimated RT information to the ElasticQosScheduler scheduler

[0213] 6. The load-aware scheduler schedules based on elastic QoS matching with the lowest-cost resource pool: In the ElasticQosScheduler, scheduling to minimize resource costs while ensuring elastic QoS is completed through three steps. The computing resources required by the user query are scheduled to a specific resource pool for execution.

[0214] It should be noted that for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the order of the actions described, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0215] 9 shows a block diagram of a resource scheduling device provided in an embodiment of the present application. The device is applied to a cloud. The cloud is configured with a plurality of resource pools corresponding to different elastic efficiencies. The elastic efficiency is used to describe the time required for a resource pool to provide resources that meet a set elasticity requirement. Specifically, the device may include the following modules:

[0216] Data acquisition module 901, used to obtain the time overhead corresponding to the data task;

[0217] A resource pool screening module 902 is configured to screen a target resource pool corresponding to the data task from the cloud resource pool based on the time cost and the elastic efficiency;

[0218] The task processing module 903 is configured to schedule elastic computing resources for completing the data task from the target resource pool, so as to complete the data task based on the elastic computing resources.

[0219] In some feasible embodiments, the resource pool screening module 902 is specifically configured to:

[0220] Analyzing resource load of the data task using the time overhead to obtain a target elastic efficiency corresponding to the data task;

[0221] According to a matching result between the target elastic efficiency and the elastic efficiencies corresponding to the resource pools, a target resource pool corresponding to the data task is screened out from the resource pools.

[0222] In some feasible embodiments, the resource pool is divided into at least a hot elastic resource pool, a warm elastic resource pool, and a cold elastic resource pool according to elasticity efficiency. The resource pool screening module 902 is specifically configured to:

[0223] Matching the target elastic efficiency with the elastic efficiency corresponding to each of the resource pools in sequence;

[0224] If the target elastic efficiency successfully matches the first elastic efficiency corresponding to the thermal elastic resource pool, the thermal elastic resource pool is used as the target resource pool for processing the data task;

[0225] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool, but successfully matches the second elastic efficiency corresponding to the warm elastic resource pool, the warm elastic resource pool is used as the target resource pool for processing the data task;

[0226] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool and fails to match the second elastic efficiency corresponding to the warm elastic resource pool, the cold elastic resource pool is used as the target resource pool for processing the data task.

[0227] In some feasible embodiments, the device further includes:

[0228] A status information acquisition module, configured to acquire pool status information corresponding to each resource pool, wherein the pool status information includes at least a resource water level;

[0229] The resource replenishment module is configured to replenish resources in the target resource pool if the resource level of the target resource pool is higher than a preset level threshold.

[0230] In some feasible embodiments, the data acquisition module 901 is specifically configured to:

[0231] Obtaining the access layer duration, computing resource startup duration, and execution duration of the data task corresponding to the data task;

[0232] The access layer duration, the computing resource startup duration, and the execution duration are used to calculate the duration overhead to obtain the time overhead for completing the data task.

[0233] In some feasible embodiments, the data acquisition module 901 is specifically configured to:

[0234] Obtaining the percentage of the computing resource startup duration to the time overhead;

[0235] The resource load of the data task is analyzed using the percentage and the time overhead to obtain a target elastic efficiency corresponding to the data task.

[0236] In some feasible embodiments, the following further comprises:

[0237] A first resource pool startup module, configured to pre-start computing resource units of different specifications in the thermal elastic resource pool;

[0238] The second resource pool startup module is used to pre-create a cloud disk and a network card on the device to which the warm elastic resource pool belongs, and mount the cloud disk and the network card.

[0239] In some feasible embodiments, the task processing module 903 is specifically configured to:

[0240] If the target resource pool is a thermal elastic resource pool, the elastic computing resources required to complete the data task are obtained, and a number of first computing resource units corresponding to the elastic computing resources are scheduled from the thermal elastic resource pool, and the data task is completed through the first computing resource units.

[0241] In some feasible embodiments, the task processing module 903 is specifically configured to:

[0242] If the target resource pool is a warm elastic resource pool, obtaining elastic computing resources required to complete the data task, and creating a number of second computing resource units corresponding to the elastic computing resources in the warm elastic resource pool based on the cloud disk and the network card;

[0243] The data task is completed by the second computing resource unit.

[0244] In some feasible embodiments, the task processing module 903 is specifically configured to:

[0245] Obtaining the elastic computing resources required to complete the data task;

[0246] Calculate the required cloud disks and network cards based on elastic computing resources, create them, and mount them separately.

[0247] According to the cloud disk and the network card, create a number of third computing resource units corresponding to the elastic computing resources in the cold elastic resource pool;

[0248] The data task is completed by the third computing resource unit.

[0249] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0250] In addition, an embodiment of the present application also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and runnable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned resource scheduling method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0251] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the various processes of the resource scheduling method embodiment described above are implemented, and the same technical effects are achieved. To avoid repetition, the details are not described here. The computer-readable storage medium may be, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0252] FIG10 is a schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present application.

[0253] The electronic device 1000 includes but is not limited to components such as a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, a processor 1010, and a power supply 1011. Those skilled in the art will appreciate that the electronic device structure involved in the embodiments of the present application does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In the embodiments of the present application, the electronic device includes but is not limited to a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted terminal, a wearable device, and a pedometer.

[0254] It should be understood that in the embodiments of the present application, the RF unit 1001 may be used to receive and transmit signals during information transmission or calls. Specifically, it receives downlink data from the base station and transmits it to the processor 1010 for processing; in addition, it transmits uplink data to the base station. Typically, the RF unit 1001 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, and the like. Furthermore, the RF unit 1001 may communicate with the network and other devices via a wireless communication system.

[0255] The electronic device provides users with wireless broadband Internet access through the network module 1002, such as helping users to send and receive emails, browse web pages, and access streaming media.

[0256] The audio output unit 1003 can convert audio data received by the RF unit 1001 or the network module 1002 or stored in the memory 1009 into an audio signal and output it as sound. In addition, the audio output unit 1003 can also provide audio output related to a specific function performed by the electronic device 1000 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 1003 includes a speaker, a buzzer, a receiver, etc.

[0257] The input unit 1004 is used to receive audio or video signals. The input unit 1004 may include a graphics processing unit (GPU) 10041 and a microphone 10042. The graphics processor 10041 processes image data of still pictures or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on the display unit 1006. The image frames processed by the graphics processor 10041 can be stored in the memory 1009 (or other storage medium) or transmitted via the radio frequency unit 1001 or the network module 1002. The microphone 10042 can receive sound and process such sound into audio data. In the case of telephone call mode, the processed audio data can be converted into a format that can be sent to a mobile communication base station via the radio frequency unit 1001 for output.

[0258] The electronic device 1000 also includes at least one sensor 1005, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 10071 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 10071 and / or the backlight when the electronic device 1000 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used to identify the posture of the electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; the sensor 1005 can also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.

[0259] The display unit 1006 is used to display information input by the user or information provided to the user. The display unit 1006 may include a display panel 10071, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0260] The user input unit 1007 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the electronic device. Specifically, the user input unit 1007 includes a touch panel 10071 and other input devices 10072. The touch panel 10071, also known as a touch screen, can collect user touch operations on or near it (such as operations performed by the user using any suitable object or accessory such as a finger, stylus, etc. on or near the touch panel 10071). The touch panel 10071 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch direction, detects the signal caused by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device and converts it into contact point coordinates, which are then sent to the processor 1010, which receives the command sent by the processor 1010 and executes it. In addition, the touch panel 10071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 10071, the user input unit 1007 may also include other input devices 10072. Specifically, the other input devices 10072 may include but are not limited to a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and an operating stick, which will not be described in detail here.

[0261] Furthermore, the touch panel 10071 may be overlaid on the display panel 10071. When the touch panel 10071 detects a touch operation on or near the touch panel 10071, the touch operation is transmitted to the processor 1010 to determine the type of touch event. The processor 1010 then provides a corresponding visual output on the display panel 10071 based on the type of touch event. It will be understood that in one embodiment, the touch panel 10071 and the display panel 10071 are used as two independent components to implement the input and output functions of the electronic device. However, in some embodiments, the touch panel 10071 and the display panel 10071 may be integrated to implement the input and output functions of the electronic device. The specific details are not limited here.

[0262] The interface unit 1008 is an interface for connecting external devices to the electronic device 1000. For example, the external devices may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 1008 may be used to receive input (e.g., data information, power, etc.) from the external device and transmit the received input to one or more elements within the electronic device 1000, or may be used to transmit data between the electronic device 1000 and the external device.

[0263] Memory 1009 can be used to store software programs and various data. Memory 1009 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function or an image playback function); the data storage area may store data generated based on the use of the mobile phone (such as audio data, a phone book, etc.). Furthermore, memory 1009 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0264] The processor 1010 is the control center of the electronic device. It connects the various components of the electronic device using various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 1009 and accessing data stored in the memory 1009, it performs various functions of the electronic device and processes data, thereby monitoring the entire electronic device. The processor 1010 may include one or more processing units; preferably, the processor 1010 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into the processor 1010.

[0265] The electronic device 1000 may also include a power supply 1011 (such as a battery) to supply power to each component. Preferably, the power supply 1011 may be logically connected to the processor 1010 through a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0266] In addition, the electronic device 1000 includes some functional modules not shown, which will not be described here.

[0267] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0268] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0269] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

[0270] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0271] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0272] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0273] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0274] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0275] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0276] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A resource scheduling method, characterized in that: Applied to a cloud, the cloud is configured with a plurality of resource pools corresponding to different elastic efficiencies, the elastic efficiency is used to describe the time required for the resource pool to provide resources that meet the set elastic requirements, the method includes: The time overhead corresponding to the task of obtaining data; According to the time cost and the elastic efficiency, a target resource pool corresponding to the data task is selected from a cloud resource pool; The elastic computing resources for completing the data task are scheduled from the target resource pool to complete the data task based on the elastic computing resources.

2. The method according to claim 1, characterized in that The step of selecting a target resource pool corresponding to the data task from a cloud resource pool according to the time cost and the elastic efficiency includes: Using the time overhead to analyze the resource load of the data task, and obtaining a target elastic efficiency corresponding to the data task; According to the matching result between the target elastic efficiency and the elastic efficiency corresponding to each of the resource pools, a target resource pool corresponding to the data task is screened out from the resource pools.

3. The method according to claim 2, characterized in that The resource pool is divided into at least a hot elastic resource pool, a warm elastic resource pool, and a cold elastic resource pool according to elastic efficiency, and the target resource pool corresponding to the data task is screened out from each of the resource pools according to a matching result between the target elastic efficiency and the elastic efficiency corresponding to each of the resource pools, including: Matching the target elastic efficiency with the elastic efficiency corresponding to each of the resource pools in turn; If the target elastic efficiency successfully matches the first elastic efficiency corresponding to the thermal elastic resource pool, the thermal elastic resource pool is used as the target resource pool for processing the data task; If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool, but successfully matches the second elastic efficiency corresponding to the warm elastic resource pool, the warm elastic resource pool is used as the target resource pool for processing the data task; If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool and fails to match the second elastic efficiency corresponding to the warm elastic resource pool, the cold elastic resource pool is used as the target resource pool for processing the data task.

4. The method according to claim 3, characterized in that The method further comprises: Obtaining pool status information corresponding to each of the resource pools, the pool status information at least including a resource water level; If the resource level of the target resource pool is higher than the preset level threshold, resources are supplemented for the target resource pool.

5. The method according to claim 2, characterized in that: The time overhead corresponding to the task of acquiring data includes: Obtaining the access layer duration, computing resource startup duration, and execution duration of the data task corresponding to the data task; The access layer duration, the computing resource startup duration and the execution duration are used to calculate the duration overhead to obtain the time overhead for completing the data task.

6. The method according to claim 5, characterized in that The adopting the time overhead to analyze the resource load of the data task to obtain a target elastic efficiency corresponding to the data task includes: Obtaining the percentage of the startup time of the computing resource to the time overhead; The resource load of the data task is analyzed using the percentage and the time overhead to obtain a target elastic efficiency corresponding to the data task.

7. The method according to claim 3, characterized in that Also includes: Pre-starting computing resource units of different specifications in the thermal elastic resource pool; A cloud disk and a network card are pre-created on the device to which the warm elastic resource pool belongs, and the cloud disk and the network card are mounted.

8. The method according to claim 7, characterized in that The step of scheduling the elastic computing resources for completing the data task from the target resource pool to complete the data task based on the elastic computing resources includes: If the target resource pool is a thermal elastic resource pool, the elastic computing resources required to complete the data task are obtained, and the first computing resource units corresponding to the number of the elastic computing resources are scheduled from the thermal elastic resource pool, and the data task is completed through the first computing resource units.

9. The method according to claim 7, characterized in that: The step of scheduling the elastic computing resources for completing the data task from the target resource pool to complete the data task based on the elastic computing resources includes: If the target resource pool is a warm elastic resource pool, obtaining elastic computing resources required to complete the data task, and creating a number of second computing resource units corresponding to the elastic computing resources in the warm elastic resource pool based on the cloud disk and the network card; The data task is completed by the second computing resource unit.

10. The method according to claim 7, characterized in that The step of scheduling the elastic computing resources for completing the data task from the target resource pool to complete the data task based on the elastic computing resources includes: Acquire the elastic computing resources required to complete the data task; Calculate the cloud disk and network card to be created according to the elastic computing resources, create the cloud disk and network card, and mount the created cloud disk and network card respectively; According to the cloud disk and the network card, create a number of third computing resource units corresponding to the elastic computing resources in the cold elastic resource pool; The data task is completed by the third computing resource unit.

11. A resource scheduling method, characterized in that: Applied to a cloud native data warehouse, the cloud native data warehouse includes at least a business load manager, a load scheduler, and a plurality of resource pools corresponding to different elastic efficiencies, wherein the elastic efficiency is used to describe the time required for the resource pool to provide resources that meet the set elastic requirements; wherein the method includes: Obtaining the time overhead corresponding to the data task through the business load manager; The load scheduler selects a target resource pool corresponding to the data task from a resource pool in the cloud according to the time cost and the elastic efficiency; The elastic computing resources for completing the data task are scheduled by the target resource pool, and the data task is completed based on the elastic computing resources.

12. A resource scheduling device, characterized in that: Applied to a cloud, the cloud is configured with a plurality of resource pools corresponding to different elastic efficiencies, the elastic efficiency is used to describe the time required for the resource pool to provide resources that meet the set elastic requirements, the device includes: Data acquisition module, used to obtain the time overhead corresponding to the data task; A resource pool screening module, used to screen a target resource pool corresponding to the data task from a resource pool in the cloud according to the time cost and the elastic efficiency; The task processing module is used to schedule the elastic computing resources for completing the data task from the target resource pool, so as to complete the data task based on the elastic computing resources.

13. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 11 when executing the program stored in the memory.

14. A computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 11.

Images