Resource scheduling method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a resource scheduling method and device, electronic equipment and a storage medium, and is applied to the technical field of cloud services. The method comprises the following steps: acquiring time overhead corresponding to a data task; according to the time overhead and the elastic efficiency, screening out a target resource pool corresponding to the data task from a cloud resource pool; and scheduling elastic computing resources for completing the data task from the target resource pool so as to complete the data task based on the elastic computing resources.

Description

Technical Field

[0001] The present application relates to the field of cloud service technology, and in particular to a resource scheduling method, a resource scheduling device, an electronic device, and a computer-readable storage medium. Background Art

[0002] Cloud-native data warehouses are data warehouses with unlimited resource elasticity and storage elasticity built on cloud infrastructure. Due to differences in usage scenarios, different usage scenarios have different requirements for processing time. In the actual resource scheduling process, there is a situation where unreasonable resource scheduling leads to resource waste, which reduces the resource utilization of cloud servers and increases operating costs. On the other hand, it is easy to bring poor user experience to users when using cloud services. Summary of the invention

[0003] The embodiments of the present application provide a resource scheduling method, device, electronic device and computer-readable storage medium to solve or partially solve the problem that unreasonable resource scheduling leads to resource waste and high cost in the resource scheduling process.

[0004] The embodiment of the present application discloses a resource scheduling method, which is applied to a cloud, wherein the cloud is configured with a plurality of resource pools corresponding to different elastic efficiencies, wherein the elastic efficiency is used to describe the time required for the resource pool to provide resources that meet set elastic requirements, and the method includes:

[0005] The time overhead corresponding to the task of obtaining data;

[0006] According to the time cost and the elastic efficiency, a target resource pool corresponding to the data task is selected from a cloud resource pool;

[0007] The elastic computing resources for completing the data task are scheduled from the target resource pool to complete the data task based on the elastic computing resources.

[0008] Optionally, the selecting a target resource pool corresponding to the data task from a resource pool in the cloud according to the time cost and the elastic efficiency includes:

[0009] Using the time overhead to analyze the resource load of the data task, and obtaining a target elastic efficiency corresponding to the data task;

[0010] According to the matching result between the target elastic efficiency and the elastic efficiency corresponding to each of the resource pools, a target resource pool corresponding to the data task is screened out from the resource pools.

[0011] Optionally, the resource pool is divided into at least a hot elastic resource pool, a warm elastic resource pool, and a cold elastic resource pool according to elastic efficiency, and the target resource pool corresponding to the data task is screened out from each of the resource pools according to a matching result between the target elastic efficiency and the elastic efficiency corresponding to each of the resource pools, including:

[0012] Matching the target elastic efficiency with the elastic efficiency corresponding to each of the resource pools in turn;

[0013] If the target elastic efficiency successfully matches the first elastic efficiency corresponding to the thermal elastic resource pool, the thermal elastic resource pool is used as the target resource pool for processing the data task;

[0014] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool, but successfully matches the second elastic efficiency corresponding to the warm elastic resource pool, the warm elastic resource pool is used as the target resource pool for processing the data task;

[0015] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool and fails to match the second elastic efficiency corresponding to the warm elastic resource pool, the cold elastic resource pool is used as the target resource pool for processing the data task.

[0016] Optionally, the method further comprises:

[0017] Obtaining pool status information corresponding to each of the resource pools, the pool status information at least including a resource water level;

[0018] If the resource level of the target resource pool is higher than the preset level threshold, resources are supplemented for the target resource pool.

[0019] Optionally, the time overhead corresponding to the data acquisition task includes:

[0020] Obtaining the access layer duration, computing resource startup duration, and execution duration of the data task corresponding to the data task;

[0021] The access layer duration, the computing resource startup duration and the execution duration are used to calculate the duration overhead to obtain the time overhead for completing the data task.

[0022] Optionally, the adopting the time overhead to perform resource load analysis on the data task to obtain a target elastic efficiency corresponding to the data task includes:

[0023] Obtaining the percentage of the startup time of the computing resource to the time overhead;

[0024] The resource load of the data task is analyzed using the percentage and the time overhead to obtain a target elastic efficiency corresponding to the data task.

[0025] Optionally, it also includes:

[0026] Pre-starting computing resource units of different specifications in the thermal elastic resource pool;

[0027] A cloud disk and a network card are pre-created on the device to which the warm elastic resource pool belongs, and the cloud disk and the network card are mounted.

[0028] Optionally, the scheduling the elastic computing resources for completing the data task from the target resource pool to complete the data task based on the elastic computing resources includes:

[0029] If the target resource pool is a thermal elastic resource pool, the elastic computing resources required to complete the data task are obtained, and the first computing resource units corresponding to the number of the elastic computing resources are scheduled from the thermal elastic resource pool, and the data task is completed through the first computing resource units.

[0030] Optionally, the scheduling the elastic computing resources for completing the data task from the target resource pool to complete the data task based on the elastic computing resources includes:

[0031] If the target resource pool is a warm elastic resource pool, obtaining elastic computing resources required to complete the data task, and creating a number of second computing resource units corresponding to the elastic computing resources in the warm elastic resource pool based on the cloud disk and the network card;

[0032] The data task is completed by the second computing resource unit.

[0033] Optionally, the scheduling the elastic computing resources for completing the data task from the target resource pool to complete the data task based on the elastic computing resources includes:

[0034] Calculate the cloud disk and network card that need to be created according to the elastic computing resources, create the cloud disk and network card, and mount the created cloud disk and network card respectively;

[0035] According to the cloud disk and the network card, create a number of third computing resource units corresponding to the elastic computing resources in the cold elastic resource pool;

[0036] The data task is completed by the third computing resource unit.

[0037] The embodiment of the present application further discloses a resource scheduling method, which is applied to a cloud native data warehouse, wherein the cloud native data warehouse comprises at least a business load manager, a load scheduler, and a plurality of resource pools corresponding to different elastic efficiencies, wherein the elastic efficiency is used to describe the time required for the resource pool to provide resources that meet the set elastic requirements; wherein the method comprises:

[0038] Obtaining the time overhead corresponding to the data task through the business load manager;

[0039] The load scheduler selects a target resource pool corresponding to the data task from a resource pool in the cloud according to the time cost and the elastic efficiency;

[0040] The elastic computing resources for completing the data task are scheduled by the target resource pool, and the data task is completed based on the elastic computing resources.

[0041] Optionally, the selecting, by the load scheduler, a target resource pool corresponding to the data task from a resource pool in the cloud according to the time cost and the elastic efficiency includes:

[0042] The load scheduler uses the time overhead to analyze the resource load of the data task to obtain a target elastic efficiency corresponding to the data task;

[0043] The load scheduler selects a target resource pool corresponding to the data task from each resource pool based on a matching result between the target elastic efficiency and the elastic efficiency corresponding to each resource pool.

[0044] Optionally, the resource pool is divided into at least a hot elastic resource pool, a warm elastic resource pool and a cold elastic resource pool according to elastic efficiency, and the load scheduler matches the target elastic efficiency with the elastic efficiency corresponding to each of the resource pools in sequence, and uses the successfully matched resource pool as the target resource pool corresponding to the data task, including:

[0045] Matching the target elastic efficiency with the elastic efficiency corresponding to each of the resource pools in sequence through the load scheduler;

[0046] If the target elastic efficiency successfully matches the first elastic efficiency corresponding to the thermal elastic resource pool, the thermal elastic resource pool is used as the target resource pool for processing the data task by the load scheduler;

[0047] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool, but successfully matches the second elastic efficiency corresponding to the warm elastic resource pool, the warm elastic resource pool is used as the target resource pool for processing the data task by the load scheduler;

[0048] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool and fails to match the second elastic efficiency corresponding to the warm elastic resource pool, the cold elastic resource pool is used as the target resource pool for processing the data task through the load scheduler.

[0049] Optionally, the method further comprises:

[0050] Obtaining pool status information corresponding to each of the resource pools through the load scheduler, wherein the pool status information at least includes a resource water level;

[0051] If the resource water level of the target resource pool is higher than the preset water level threshold, the target resource pool is supplemented with resources through the load scheduler.

[0052] Optionally, the acquiring, by the service load manager, the time overhead corresponding to the data task includes:

[0053] Obtaining, through the business load manager, the access layer duration corresponding to the data task, the computing resource startup duration, and the execution duration of executing the data task;

[0054] The service load manager uses the access layer duration, the computing resource startup duration, and the execution duration to perform duration overhead calculation to obtain the time overhead for completing the data task.

[0055] Optionally, the analyzing the resource load of the data task by using the time overhead through the service load manager to obtain a target elastic efficiency corresponding to the data task includes:

[0056] Obtaining, by the business load manager, the percentage of the computing resource startup duration to the time overhead;

[0057] The service load manager adopts the percentage and the time overhead to analyze the resource load of the data task and obtain the target elastic efficiency corresponding to the data task.

[0058] Optionally, the method further comprises:

[0059] Pre-starting computing resource units of different specifications in the thermal elastic resource pool through the load scheduler;

[0060] The load scheduler pre-creates a cloud disk and a network card on the device to which the warm elastic resource pool belongs, and mounts the cloud disk and the network card.

[0061] Optionally, it also includes:

[0062] If the target resource pool is a thermal elastic resource pool, the elastic computing resources required to complete the data task are obtained through the thermal elastic resource pool, and the first computing resource units corresponding to the number of the elastic computing resources are scheduled from the thermal elastic resource pool, and the data task is completed through the first computing resource units.

[0063] Optionally, it also includes:

[0064] If the target resource pool is a warm elastic resource pool, the elastic computing resources required to complete the data task are obtained through the warm elastic resource pool, and based on the cloud disk and the network card, a number of second computing resource units corresponding to the elastic computing resources are created in the warm elastic resource pool;

[0065] The data task is completed by the second computing resource unit.

[0066] Optionally, the scheduling of the elastic computing resources for completing the data task through the target resource pool, and completing the data task based on the elastic computing resources, includes:

[0067] Acquire the elastic computing resources required to complete the data task through the cold elastic resource pool;

[0068] The cold elastic resource pool calculates the cloud disk and network card that need to be created according to the elastic computing resources, creates the cloud disk and the network card, and mounts the created cloud disk and the network card respectively;

[0069] Creating, by the cold elastic resource pool according to the cloud disk and the network card, a number of third computing resource units corresponding to the elastic computing resources in the cold elastic resource pool;

[0070] The data task is completed by the cold elastic resource pool and the third computing resource unit.

[0071] The embodiment of the present application further discloses a resource scheduling device, which is applied to a cloud, wherein the cloud is configured with a plurality of resource pools corresponding to different elastic efficiencies, wherein the elastic efficiency is used to describe the time required for the resource pool to provide resources that meet the set elastic requirements, and the device includes:

[0072] Data acquisition module, used to obtain the time overhead corresponding to the data task;

[0073] A resource pool screening module, used to screen a target resource pool corresponding to the data task from a resource pool in the cloud according to the time cost and the elastic efficiency;

[0074] The task processing module is used to schedule the elastic computing resources for completing the data task from the target resource pool, so as to complete the data task based on the elastic computing resources.

[0075] Optionally, the resource pool screening module is specifically used to:

[0076] Using the time overhead to analyze the resource load of the data task, and obtaining a target elastic efficiency corresponding to the data task;

[0077] According to the matching result between the target elastic efficiency and the elastic efficiency corresponding to each of the resource pools, a target resource pool corresponding to the data task is screened out from the resource pools.

[0078] Optionally, the resource pool is divided into at least a hot elastic resource pool, a warm elastic resource pool and a cold elastic resource pool according to elasticity efficiency, and the resource pool screening module is specifically used to:

[0079] Matching the target elastic efficiency with the elastic efficiency corresponding to each of the resource pools in turn;

[0080] If the target elastic efficiency successfully matches the first elastic efficiency corresponding to the thermal elastic resource pool, the thermal elastic resource pool is used as the target resource pool for processing the data task;

[0081] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool, but successfully matches the second elastic efficiency corresponding to the warm elastic resource pool, the warm elastic resource pool is used as the target resource pool for processing the data task;

[0082] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool and fails to match the second elastic efficiency corresponding to the warm elastic resource pool, the cold elastic resource pool is used as the target resource pool for processing the data task.

[0083] Optionally, the device further comprises:

[0084] A status information acquisition module, used to acquire pool status information corresponding to each resource pool, wherein the pool status information at least includes a resource water level;

[0085] The resource replenishment module is used to replenish resources to the target resource pool if the resource water level of the target resource pool is higher than a preset water level threshold.

[0086] Optionally, the data acquisition module is specifically used to:

[0087] Obtaining the access layer duration, computing resource startup duration, and execution duration of the data task corresponding to the data task;

[0088] The access layer duration, the computing resource startup duration and the execution duration are used to calculate the duration overhead to obtain the time overhead for completing the data task.

[0089] Optionally, the data acquisition module is specifically used to:

[0090] Obtaining the percentage of the startup time of the computing resource to the time overhead;

[0091] The resource load of the data task is analyzed using the percentage and the time overhead to obtain a target elastic efficiency corresponding to the data task.

[0092] Optionally, it also includes:

[0093] A first resource pool startup module, used for pre-starting computing resource units of different specifications in the thermal elastic resource pool;

[0094] The second resource pool startup module is used to pre-create a cloud disk and a network card on the device to which the warm elastic resource pool belongs, and mount the cloud disk and the network card.

[0095] Optionally, the task processing module is specifically used to:

[0096] If the target resource pool is a thermal elastic resource pool, the elastic computing resources required to complete the data task are obtained, and the first computing resource units corresponding to the number of the elastic computing resources are scheduled from the thermal elastic resource pool, and the data task is completed through the first computing resource units.

[0097] Optionally, the task processing module is specifically used to:

[0098] If the target resource pool is a warm elastic resource pool, obtaining elastic computing resources required to complete the data task, and creating a number of second computing resource units corresponding to the elastic computing resources in the warm elastic resource pool based on the cloud disk and the network card;

[0099] The data task is completed by the second computing resource unit.

[0100] Optionally, the task processing module is specifically used to:

[0101] Acquire the elastic computing resources required to complete the data task;

[0102] Calculate the cloud disk and network card that need to be created according to the elastic computing resources, create the cloud disk and network card, and mount the created cloud disk and network card respectively;

[0103] According to the cloud disk and the network card, create a number of third computing resource units corresponding to the elastic computing resources in the cold elastic resource pool;

[0104] The data task is completed by the third computing resource unit.

[0105] The embodiment of the present application also discloses an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;

[0106] The memory is used to store computer programs;

[0107] The processor is used to implement the method described in the embodiment of the present application when executing the program stored in the memory.

[0108] The embodiment of the present application also discloses a computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, enables the processors to execute the method described in the embodiment of the present application.

[0109] The embodiments of the present application include the following advantages:

[0110] In an embodiment of the present application, it can be applied to the cloud, where several resource pools corresponding to different elastic efficiencies are configured. The elastic efficiency is used to describe the time required for the resource pool to provide resources that meet the set elastic requirements. The data tasks submitted by the user are processed through the cloud. During the processing, the time overhead corresponding to the data task is obtained, and then the target resource pool corresponding to the data task is screened out from the resource pool in the cloud according to the time overhead and the elastic efficiency. Then, the elastic computing resources for completing the data task are scheduled from the target resource pool to complete the data task, so as to complete the data task based on the elastic computing resources. Therefore, on the one hand, by constructing a resource pool with multi-level elastic efficiency, the processing requirements of different data tasks can be met, thereby reducing the operating cost. On the other hand, the resource elasticity is analyzed based on the time overhead corresponding to the data task, and the appropriate resource pool is selected for task processing based on the analysis results, thereby improving the rationality of resource scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0111] Figure 1 It is a flowchart of the steps of a resource scheduling method provided in an embodiment of the present application;

[0112] Figure 2 is a schematic diagram of the cloud provided in an embodiment of the present application;

[0113] Figure 3 It is a schematic diagram of a scenario provided in an embodiment of the present application;

[0114] Figure 4 is a schematic diagram of a thermal elastic resource pool provided in an embodiment of the present invention;

[0115] Figure 5 is a schematic diagram of a warm elastic resource pool provided in an embodiment of the present invention;

[0116] Figure 6 is a schematic diagram of a cold elastic resource pool provided in an embodiment of the present invention;

[0117] Figure 7 It is a flowchart of the steps of a resource scheduling method provided in an embodiment of the present application;

[0118] Figure 8 is a schematic diagram of a cloud native data warehouse provided in an embodiment of the present application;

[0119] Fig. 9 It is a structural block diagram of a resource scheduling device provided in an embodiment of the present application;

[0120] Fig.10 It is a block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0121] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0122] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0123] As an example, for cloud-native data warehouses, for data tasks submitted by users, different data tasks have different requirements for resource elasticity efficiency. In related technologies, there are over-configuration and under-configuration of resources for handling different resource elasticity efficiency requirements, which not only easily affects the processing of data tasks and reduces user experience, but also easily leads to resource waste in cloud-native data warehouses due to unreasonable resource configuration, resulting in high operating costs.

[0124] In this regard, one of the core invention points of the present application is to configure resource pools with different elastic efficiencies in the cloud. During the processing, the time overhead corresponding to the data task is obtained, and then the target resource pool corresponding to the data task is screened out from the resource pool in the cloud based on the time overhead and elastic efficiency. Then, the elastic computing resources for completing the data task are scheduled from the target resource pool to complete the data task, so as to complete the data task based on the elastic computing resources. Therefore, on the one hand, by constructing a multi-level elastic efficiency resource pool, the processing requirements of different data tasks can be met, thereby reducing the operating cost. On the other hand, the resource elasticity is analyzed based on the time overhead corresponding to the data task, and the appropriate resource pool is selected for task processing based on the analysis results, thereby improving the rationality of resource scheduling.

[0125] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical features involved in the embodiments of the present application are explained and illustrated below:

[0126] Cloud native data warehouse (cloud native data warehouse): a data warehouse built on cloud infrastructure with unlimited resource elasticity and unlimited storage elasticity.

[0127] Query (data task): A user initiates a query, analysis, calculation, offline processing, or other task on the cloud native data warehouse that requires processing by the cloud native data warehouse.

[0128] Query-level resources: resources applied for a query separately in the cloud native data warehouse. The resources are released after the query ends, that is, the computing resources required to complete a data task. They refer to the hardware and software used to calculate and process data, including hardware devices such as processors, memory, hard disks, and networks, as well as software systems such as operating systems, programming languages, applications, and frameworks. Different data tasks need to be completed through different computing resources, and different computing resources correspond to different hardware and software. In this application, computing resource units are used as the "carrier" of computing resources to process data tasks. The computing resource units can be Pods (hereinafter collectively described as computing resource Pods), and this application does not impose any restrictions on this.

[0129] Load Qos (Quality of Service): The processing time requirements of queries (data tasks) with different priorities, demand scenarios, data volumes, and computational complexities. For example, the processing time requirement for simple query tasks is less than 1 second; the processing time requirement for complex data aggregation tasks is less than 5 minutes; the processing time requirement for load ETL tasks is less than 1 hour. For ETL, it can be Extract, Transform, Load) is a common data processing method that extracts data from different sources (Extract), transforms the data (Transform), and finally loads (Load) the target system.

[0130] Resource pool: A resource pool used to meet the computing resource requirements of cloud native data warehouse queries.

[0131] Reference Figure 1 , shows a flowchart of the steps of a resource scheduling method provided in an embodiment of the present application, which is applied to a cloud, and the cloud is configured with a plurality of resource pools corresponding to different elastic efficiencies. The elastic efficiency is used to describe the time required for the resource pool to provide resources that meet the set elastic requirements, and specifically may include the following steps:

[0132] Step 101, obtaining the time overhead corresponding to the data task;

[0133] Reference Figure 2 , showing a schematic diagram of the cloud provided in an embodiment of the present application, wherein the cloud can be a cloud-native data warehouse, and resource pools with different elastic efficiencies can be configured in the cloud-native data warehouse. It should be noted that, for elasticity, it can prevent over-configuration of computing resources and under-configuration of resources, while elastic efficiency refers to the efficiency of data processing, which is expressed by describing the time required for the resource pool to provide resources that meet the set elastic requirements. The higher the elastic efficiency, the shorter the time level of data tasks that can be supported. Correspondingly, the higher the elastic efficiency, the higher the configuration cost of the resource pool. In this regard, by configuring resource pools with different elastic efficiencies and calling the appropriate resource pool on demand to perform task processing during the processing of data tasks, the rationality of resource scheduling can be improved while reducing operating costs. With reference to Figure 3 , showing a scenario diagram provided in an embodiment of the present application, a user can send corresponding data processing instructions to the cloud native data warehouse through a user terminal, and the cloud native data warehouse determines the corresponding data task based on the data processing instruction, and then obtains the time overhead corresponding to the data task, and selects a suitable target resource pool based on the matching results between the time overhead and each resource pool, and processes the data task, such as completing data query, data analysis, offline ETL, etc.

[0134] In a specific implementation, a user can send corresponding data processing instructions to the cloud through a user terminal, and the cloud native data warehouse can determine at least one data task corresponding to the data processing instruction based on the data processing instruction. Among them, the data task includes at least one of a data query task, a data analysis task, a data calculation task, and an offline data task. At the same time, after determining the data task, the time overhead corresponding to the data task can be further calculated. The time overhead can be used to characterize the overall time required to complete the data task. Different data tasks can correspond to different time overheads. For example, for a data task, the corresponding time overhead can cover multiple delay scenarios such as 100ms level, second level, minute level, ten minute level, hour level, etc.

[0135] Step 102, selecting a target resource pool corresponding to the data task from a resource pool in the cloud according to the time cost and the elastic efficiency;

[0136] In the specific implementation, the cloud can use time overhead to analyze the resource load of the data task to obtain the target elastic efficiency corresponding to the data task, and then filter out the target resource pool corresponding to the data task from each resource pool based on the matching result between the target elastic efficiency and the elastic efficiency corresponding to each resource pool.

[0137] For data tasks, you can first obtain the access layer duration, computing resource startup duration, and execution duration corresponding to the data task, and then use the access layer duration, computing resource startup duration, and execution duration to calculate the completion duration to generate the time overhead required to complete the data task.

[0138] Among them, the access layer time, which can be the time from the user terminal initiating a request to the backend service responding to the request, is also called the end-to-end response time or user experience time, which is generally between 10ms and 100ms; the computing resource startup time, which is related to the elasticity of different resource pools in the cloud. For example, resource pool ① takes 1s to elastically start 300 computing resources; resource pool ② takes 5s to elastically start 300 computing resources; resource pool ③ takes 30s to elastically start 300 computing resources; resource pool ④ takes 1 minute to elastically start 300 computing resources, and so on. The cost of building resource pools with different elastic capabilities is different. Generally, the stronger the elasticity, the greater the cost; the execution time, which can be the time required for the data task to actually execute on the computing resources, is related to the server's processing power, data volume, and the computational complexity of the task.

[0139] Optionally, for the access layer duration, computing resource startup duration and execution duration, the access layer duration can be basically ignored, the execution duration is strongly related to the user's input, and the computing resource startup duration is a constant overhead. For the computing resource startup duration, the elastic efficiency is high while the cost is also increased. In the coverage range of the delay duration of the cloud native data warehouse, the delay duration corresponding to different data tasks is also sensitive to the computing resource startup duration. The greater the delay duration, the lower the sensitivity. The computing resources of the data task can be scheduled to a resource pool with low cost overhead and slow elastic efficiency. In this regard, in this application, the load situation of the data task is analyzed by defining "Elastic_Resource_Time (i.e., elastic efficiency duration)". The larger the elastic efficiency duration corresponding to the data task, the lower the elastic efficiency of the resource pool used to complete the data task. Conversely, the higher the elastic efficiency of the resource pool used for the data task.

[0140] Based on the above description, the load situation of the data task can be analyzed through the elastic efficiency duration. In the process of load analysis of the data task, the percentage of the computing resource startup duration to the time overhead can be obtained first, and then the percentage and time overhead can be used to analyze the resource load of the data task to obtain the target elastic efficiency corresponding to the data task, so as to schedule it to a matching resource pool according to the elastic efficiency duration and process the data task in a reasonable and low-cost manner.

[0141] In one example, the elastic efficiency duration can be generated by the following formula:

[0142] Elastic_Resource_Time=Query_RT*Elastic_Tolerate_Threshold

[0143] Among them, Elastic_Resource_Time is the elastic efficiency time; Query_RT is the time overhead, that is, the overall time to complete the data task; Elastic_Tolerate_Threshold is the percentage of computing resource startup time to time overhead.

[0144] As for the time cost, it can be calculated by the following formula:

[0145] Query_RT = access layer duration + computing resource startup duration + execution duration

[0146] For example, assuming Elastic_Tolerate_Threshold = 10%, a data task with a time overhead of 10s is scheduled to a resource pool with Elastic_Resource_Time = 1s; a query with a minute-level delay can be scheduled to a resource pool with Elastic_Resource_Time = 6s; a query with a ten-minute-level delay can be scheduled to a resource pool with Elastic_Resource_Time = 60s. This allows an analysis of resource elasticity based on the time overhead corresponding to the data task, and based on the results of the analysis, a suitable resource pool is selected for task processing, thereby improving the rationality of resource scheduling.

[0147] In the embodiment of the present application, in the process of matching the resource pool suitable for the data task, while calculating the elastic efficiency duration corresponding to the data task, the cloud can also obtain the pool status information reported in real time by each resource pool, and the pool status information can be used to characterize the current usage status information of the resource pool. Among them, the pool status information can include information such as resource water level, elastic efficiency, and cost overhead. The resource water level can be the resource utilization rate of the resource pool; the elastic efficiency can be the efficiency of data processing; and the cost overhead can be the hardware configuration cost of the resource pool.

[0148] In a specific implementation, the target elastic efficiency can be matched with the elastic efficiency corresponding to each resource pool in turn, and the resource pool that is successfully matched is used as the target resource pool corresponding to the data task. Specifically, different resource pools correspond to different elastic efficiencies, and the elastic efficiency is expressed by the startup time required to start a preset number of computing resource units. The shorter the time required to start the same number of computing resource units, the higher the elastic efficiency. For example, a resource pool of 300 computing resource units is started in 1s, a resource pool of 300 computing resource units is started in 5s, and a resource pool of 300 computing resource units is started in 30s. Among them, the computing resource unit can be the smallest unit running in the resource pool, which can include one or more containers and share the same network for corresponding processing of data tasks, such as completing data query, data analysis, offline ETL, etc.

[0149] In an optional embodiment, the resource pool in the cloud can be divided into at least a hot elastic resource pool, a warm elastic resource pool and a cold elastic resource pool according to elastic efficiency. Then, the target elastic efficiency corresponding to the data task can be compared with the elastic efficiency corresponding to each elastic resource pool to select a suitable resource pool to process the data task. If the target elastic efficiency successfully matches the elastic efficiency corresponding to the hot elastic resource pool, the hot elastic resource pool is used as the target resource pool for processing the data task; if the target elastic efficiency fails to match the elastic efficiency corresponding to the hot elastic resource pool, but successfully matches the elastic efficiency corresponding to the warm elastic resource pool, the warm elastic resource pool is used as the target resource pool for processing the data task; if the target elastic efficiency fails to match the elastic efficiency corresponding to the hot elastic resource pool, but fails to match the elastic efficiency corresponding to the warm elastic resource pool, the cold elastic resource pool is used as the target resource pool for processing the data task. Thus, on the one hand, by constructing a resource pool with multi-level elastic efficiency, the processing requirements of different data tasks can be met, thereby reducing the operating cost. On the other hand, the resource elasticity is analyzed based on the time overhead corresponding to the data task, and a suitable resource pool is selected for task processing based on the analysis result, thereby improving the rationality of resource scheduling.

[0150] In a specific implementation, resource pools such as a hot elastic resource pool, a warm elastic resource pool, and a cold elastic resource pool correspond to different startup time intervals (i.e., elastic efficiency), and the resource pool to which the startup time interval in which the target elastic efficiency falls belongs can be taken as the target resource pool.

[0151] Among them, for the thermal elastic resource pool, in order to improve the elastic efficiency of the computing resource unit, computing resource Pods of different rules can be pre-started in the thermal elastic resource pool. When the data task requires computing resources, the computing resource Pods of the corresponding specifications can be directly obtained from the thermal elastic resource pool. In the process of obtaining the computing resource Pod, since the computing resource Pod has been started in advance, the elastic efficiency of its computing resources can reach the 1s level. For example, since the cloud has multiple specifications of computing resources, computing resources of different specifications need to pre-start a batch of cache pools in advance. In one example, different computing resources can be started in the thermal elastic resource pool according to three different specifications of large, medium and small, including starting 12 small-specification computing resources, 9 medium-specification computing resources and 6 large-specification computing resources. It should be noted that for the specifications, it can be divided according to computing power, and this application does not limit this.

[0152] For warm elastic resource pools, since the startup time of a computing resource Pod mainly includes the creation time of cloud disks and network cards, as well as the mounting time of cloud disks and network cards, in warm elastic resource pools, cloud disks and network cards can be created and mounted in advance, so that when computing resources are applied, the creation time and mounting time of cloud disks and network cards can be saved, achieving elastic efficiency optimization. Compared with the "hot elastic resource pool", this architecture does not require pre-starting computing resources. All computing resources are created when a query comes, so that the overall machine can reuse computing resources of various specifications.

[0153] For the cold elastic resource pool, the resource demand for data tasks is applied on demand. Once the application is successful, the cloud disk and network card are created and mounted in the cold elastic resource pool. Therefore, the elasticity efficiency of the cold elastic resource pool is low.

[0154] It should be noted that in the embodiments of the present application, hot elastic resource pools, warm elastic resource pools and cold elastic resource pools are used as examples for illustrative description. Under the guidance of the concept of the present application, other types of resource pools can also be set according to actual needs.

[0155] In an example, assuming that the startup time corresponding to the hot elastic resource pool is 1 second for starting 300 Pods, and the corresponding startup time interval is less than 5 seconds; the startup time corresponding to the warm elastic resource pool is 5 seconds for starting 300 Pods, and the corresponding startup time interval is (5 seconds, 30 seconds); the startup time corresponding to the cold elastic resource pool is 30 seconds for starting 300 Pods, and the corresponding startup time interval is greater than 30 seconds, then when the target elastic efficiency corresponding to the data task is 3 seconds, the hot elastic resource pool can be used as the target resource pool for processing the data task; when the target elastic efficiency is 10 seconds, the warm elastic resource pool can be used as the target resource pool for processing the data task; when the target elastic efficiency is 1 minute, the cold elastic resource pool can be used as the target resource pool for processing the data task. Thus, on the one hand, by constructing a resource pool with multi-level elastic efficiency, the processing requirements of different data tasks can be met, reducing the operating cost; on the other hand, the resource elasticity is analyzed based on the time overhead corresponding to the data task, and the appropriate resource pool is selected for task processing based on the analysis result, thereby improving the rationality of resource scheduling.

[0156] In addition, during the operation of the resource pool, the cloud can determine the current utilization rate of the resource pool in real time based on the resource water level reported by the resource pool, and replenish resources for resource pools whose resource water level is lower than the preset water level threshold, to ensure that each resource pool has sufficient computing resources to complete the corresponding data tasks.

[0157] Step 103: Schedule elastic computing resources for completing the data task from the target resource pool, so as to complete the data task based on the elastic computing resources.

[0158] After selecting the target resource pool that matches the data task, the elastic computing resources required to complete the data task can be scheduled from the target resource pool, and the data task can be completed based on the elastic computing resources. On the one hand, by building a multi-level elastic and efficient resource pool, the processing requirements of different data tasks can be met, reducing operating costs. On the other hand, the resource elasticity is analyzed based on the time overhead corresponding to the data task, and the appropriate resource pool is selected for task processing based on the analysis results, thereby improving the rationality of resource scheduling.

[0159] In a specific implementation, after determining the elastic computing resources required to complete the data task, when the target resource pool is a hot elastic resource pool, since computing resource units of different specifications are pre-started in the hot elastic resource pool, in the process of processing the data task, the first computing resource unit corresponding to the number of elastic computing resources can be directly obtained from the hot elastic resource pool, and the data task can be completed through the first computing resource unit. When the target resource pool is a warm elastic resource pool, since the cloud disk and network card have been pre-installed on the device to which the warm elastic resource pool belongs, and the cloud disk and network card have been mounted, a second computing resource unit corresponding to the number of elastic computing resources can be created in the warm elastic resource pool based on the cloud disk and network card, and then the data task can be completed through the second computing resource unit. When the target resource pool is a cold elastic resource pool, the cloud disk and network card that need to be created can be calculated according to the elastic computing resources, and the cloud disk and network card can be created and mounted respectively. Then, according to the cloud disk and network card, a third computing resource unit corresponding to the elastic computing resources is created in the cold elastic resource pool. Then, the data task is completed through the third computing resource unit. Thus, on the one hand, by constructing a multi-level elastic efficiency resource pool, the processing requirements of different data tasks can be met, thereby reducing the operating cost. On the other hand, the resource elasticity is analyzed based on the time overhead corresponding to the data task, and the appropriate resource pool is selected for task processing based on the analysis results, thereby improving the rationality of resource scheduling.

[0160] In some examples, reference Figure 4 , shows a schematic diagram of a thermal elastic resource pool provided in an embodiment of the present invention. The thermal elastic resource pool can be composed of several machines, each of which includes 4 small-specification computing resources, 3 medium-specification computing resources and 2 large-specification computing resources. Each computing resource is mounted with a corresponding disk and network card. Among them, Query1 is completed by 8 small-specification computing resources, Query2 is completed by 4 small-specification computing resources, and Query3 is completed by 4 large-specification computing resources.

[0161] Reference Figure 5 , shows a schematic diagram of a warm elastic resource pool provided in an embodiment of the present invention. For a warm elastic resource pool, it can be composed of several machines, wherein machine 1 includes 4 small-sized computing resources and 2 large-sized computing resources, each of which is mounted with a corresponding disk and network card; machine 2 includes 4 small-sized computing resources and 2 large-sized computing resources, etc., and each machine is provided with a disk cache pool and a network card cache pool. Among them, Query1 is completed by 8 small-sized computing resources of two machines, Query2 is completed by 4 small-sized computing resources, and Query3 is completed by 4 large-sized computing resources of two machines.

[0162] Reference Figure 6 , shows a schematic diagram of a cold elastic resource pool provided in an embodiment of the present invention. The cold elastic resource pool may include 12 small-sized computing resources and 4 large-sized computing resources. Each computing resource is mounted with a corresponding disk and network card. Query1 is completed by 8 small-sized computing resources, Query2 is completed by 4 small-sized computing resources, and Query3 is completed by 4 large-sized computing resources.

[0163] It should be noted that the embodiments of the present application include but are not limited to the above examples. It can be understood that under the guidance of the ideas of the embodiments of the present application, settings can be made according to actual needs, and the present application does not impose any restrictions on this.

[0164] In an embodiment of the present application, it can be applied to the cloud, where several resource pools corresponding to different elastic efficiencies are configured. The elastic efficiency is used to describe the time required for the resource pool to provide resources that meet the set elastic requirements. The data tasks submitted by the user are processed through the cloud. During the processing, the time overhead corresponding to the data task is obtained, and then the target resource pool corresponding to the data task is screened out from the resource pool in the cloud according to the time overhead and the elastic efficiency. Then, the elastic computing resources for completing the data task are scheduled from the target resource pool to complete the data task, so as to complete the data task based on the elastic computing resources. Therefore, on the one hand, by constructing a resource pool with multi-level elastic efficiency, the processing requirements of different data tasks can be met, thereby reducing the operating cost. On the other hand, the resource elasticity is analyzed based on the time overhead corresponding to the data task, and the appropriate resource pool is selected for task processing based on the analysis results, thereby improving the rationality of resource scheduling.

[0165] Reference Figure 7 , shows a flowchart of the steps of a resource scheduling method provided in an embodiment of the present application, which is applied to a cloud native data warehouse, wherein the cloud native data warehouse includes at least a business load manager, a load scheduler, and a plurality of resource pools corresponding to different elastic efficiencies, wherein the elastic efficiency is used to describe the time required for the resource pool to provide resources that meet the set elastic requirements; specifically, the following steps may be included:

[0166] Step 701, obtaining the time overhead corresponding to the data task through the business load manager;

[0167] Step 702, the load scheduler selects a target resource pool corresponding to the data task from a resource pool in the cloud according to the time cost and the elastic efficiency;

[0168] Step 703: The elastic computing resources for completing the data task are scheduled by the target resource pool, and the data task is completed based on the elastic computing resources.

[0169] In some feasible implementations, the load scheduler selects a target resource pool corresponding to the data task from a cloud resource pool according to the time cost and the elastic efficiency, including:

[0170] The load scheduler uses the time overhead to analyze the resource load of the data task to obtain a target elastic efficiency corresponding to the data task;

[0171] The load scheduler selects a target resource pool corresponding to the data task from each resource pool based on a matching result between the target elastic efficiency and the elastic efficiency corresponding to each resource pool.

[0172] In some feasible implementations, the resource pool is divided into at least a hot elastic resource pool, a warm elastic resource pool, and a cold elastic resource pool according to elastic efficiency, and the load scheduler matches the target elastic efficiency with the elastic efficiency corresponding to each of the resource pools in sequence, and uses the successfully matched resource pool as the target resource pool corresponding to the data task, including:

[0173] Matching the target elastic efficiency with the elastic efficiency corresponding to each of the resource pools in sequence through the load scheduler;

[0174] If the target elastic efficiency successfully matches the first elastic efficiency corresponding to the thermal elastic resource pool, the thermal elastic resource pool is used as the target resource pool for processing the data task by the load scheduler;

[0175] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool, but successfully matches the second elastic efficiency corresponding to the warm elastic resource pool, the warm elastic resource pool is used as the target resource pool for processing the data task by the load scheduler;

[0176] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool and fails to match the second elastic efficiency corresponding to the warm elastic resource pool, the cold elastic resource pool is used as the target resource pool for processing the data task through the load scheduler.

[0177] In some feasible implementations, the method further includes:

[0178] Obtaining pool status information corresponding to each of the resource pools through the load scheduler, wherein the pool status information at least includes a resource water level;

[0179] If the resource water level of the target resource pool is higher than the preset water level threshold, the target resource pool is supplemented with resources through the load scheduler.

[0180] In some feasible implementations, the acquiring, by the service load manager, the time overhead corresponding to the data task includes:

[0181] Obtaining, through the business load manager, the access layer duration corresponding to the data task, the computing resource startup duration, and the execution duration of executing the data task;

[0182] The service load manager uses the access layer duration, the computing resource startup duration, and the execution duration to perform duration overhead calculation to obtain the time overhead for completing the data task.

[0183] In some feasible implementations, the analyzing the resource load of the data task by the service load manager using the time overhead to obtain a target elastic efficiency corresponding to the data task includes:

[0184] Obtaining, by the business load manager, the percentage of the computing resource startup duration to the time overhead;

[0185] The service load manager adopts the percentage and the time overhead to analyze the resource load of the data task and obtain the target elastic efficiency corresponding to the data task.

[0186] In some feasible implementations, the method further includes:

[0187] Pre-starting computing resource units of different specifications in the thermal elastic resource pool through the load scheduler;

[0188] The load scheduler pre-creates a cloud disk and a network card on the device to which the warm elastic resource pool belongs, and mounts the cloud disk and the network card.

[0189] In some feasible implementations, the following further includes:

[0190] If the target resource pool is a thermal elastic resource pool, the elastic computing resources required to complete the data task are obtained through the thermal elastic resource pool, and the first computing resource units corresponding to the number of the elastic computing resources are scheduled from the thermal elastic resource pool, and the data task is completed through the first computing resource units.

[0191] In some feasible implementations, the following further includes:

[0192] If the target resource pool is a warm elastic resource pool, the elastic computing resources required to complete the data task are obtained through the warm elastic resource pool, and based on the cloud disk and the network card, a number of second computing resource units corresponding to the elastic computing resources are created in the warm elastic resource pool;

[0193] The data task is completed by the second computing resource unit.

[0194] In some feasible implementations, the scheduling of the elastic computing resources for completing the data task through the target resource pool, and completing the data task based on the elastic computing resources, includes:

[0195] Acquire the elastic computing resources required to complete the data task through the cold elastic resource pool;

[0196] The cold elastic resource pool calculates the cloud disk and network card that need to be created according to the elastic computing resources, creates the cloud disk and the network card, and mounts the created cloud disk and the network card respectively;

[0197] Creating, by the cold elastic resource pool according to the cloud disk and the network card, a number of third computing resource units corresponding to the elastic computing resources in the cold elastic resource pool;

[0198] The data task is completed by the cold elastic resource pool and the third computing resource unit.

[0199] In an embodiment of the present application, it can be applied to the cloud, and the data tasks submitted by the user are processed through the cloud. During the processing, at least one data task and the time overhead corresponding to the data task are obtained, and then according to the time overhead and the elastic efficiency, the target resource pool corresponding to the data task is screened out from the resource pool of the cloud, and then the data task is completed based on the target resource pool. Therefore, on the one hand, by constructing a resource pool with multi-level elastic efficiency, the processing requirements of different data tasks can be met, thereby reducing the operating cost. On the other hand, the resource elasticity is analyzed based on the time overhead corresponding to the data task, and the appropriate resource pool is selected for task processing based on the analysis result, thereby improving the rationality of resource scheduling.

[0200] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, an example is given below for illustrative purposes:

[0201] Reference Figure 8 , showing a schematic diagram of a cloud-native data warehouse provided in an embodiment of the present application. For a cloud-native data warehouse, it may include a business load manager, a load scheduler, and several resource pools (hot elastic resource pool, warm elastic resource pool, cold elastic resource pool, etc.).

[0202] Among them, for the business load manager (WorkLoadManager), in the cloud native data warehouse, the business load manager can manage the queries issued by users, including queuing, priority, RT estimation and other capabilities. In this example, the RT estimation capability is mainly used; at the same time, the historical RT time of QueryPattern and Query can be used to estimate the RT of the query issued by the current user, and the RT can be used as the input of the load scheduler to evaluate the query elastic Qos.

[0203] For the load scheduler (also known as the elastic Qos-aware scheduler, also known as the ElasticQosScheduler scheduler), it can solve queries with different elastic Qos (Quality of Service, service selection priority), and use reasonable resource costs to complete the scheduling of query computing resources in different resource pools under the premise of meeting the elastic Qos. The load scheduler can include four modules. The "resource pool elastic configuration" module is used to receive the pool status information of elastic efficiency, cost overhead, and resource water level reported by each resource pool; the "query elastic tolerance time calculation" module uses the corresponding formula to calculate the tolerance time Elastic_Resource_Time of the Query elastic Qos (the calculation process refers to the aforementioned embodiment, which will not be repeated here); the "resource pool matching scoring" can use the elastic Qos tolerance time of the Query to match different resource pools. Specifically, the resource pool that meets the Qeury elastic Qos can be matched in the order of cold resource pool -> warm resource pool -> hot resource pool; after the "dispatch to resource pool" matches the specific resource pool, the computing resources of the Query are dispatched to the specific resource pool for execution.

[0204] For each resource pool, different resource pools can manage their own elastic efficiency, cost, and self-sustaining resources. For example, the elastic efficiency of a hot elastic resource pool can be at the 1s level, the corresponding cost is "high", and the self-sustaining resources are "self-sustaining" (indicating that it can self-sustain computing resources); the elastic efficiency of a warm elastic resource pool can be at the 5s level, the corresponding cost is "medium", and the self-sustaining resources are "self-sustaining"; the elastic efficiency of a cold elastic resource pool can be at the 30s level, the corresponding cost is "low", and the self-sustaining resources are "none" (indicating that it needs to create corresponding computing resources on demand according to data tasks), etc.

[0205] In a specific implementation, data processing can be performed according to the following process:

[0206] 1. Supplement cache machine resources based on already used resources: Different resource pools evaluate their own elasticity and costs, and supplement cache machine resources based on the water level.

[0207] 2. Resource pools report elastic efficiency, cost, and resource level to the central scheduler: Each resource pool reports its own elastic efficiency, cost, resource level, and other information to the central scheduler

[0208] 3. Users issue queries with various latency requirements: Users issue queries with various latency characteristics, which may include online analysis, complex calculations, offline ETL, etc.

[0209] 4. WorkLoadManager estimates the RT of different queries based on historical QueryPattern: The WorkLoadManager module estimates the corresponding RT of the query issued by the user

[0210] 5. Query and estimated RT are sent to the scheduler: WorkLoadManager sends the query and estimated RT information to the ElasticQosScheduler scheduler

[0211] 6. The load-aware scheduler schedules based on elastic Qos matching the resource pool with the lowest cost: In ElasticQosScheduler, scheduling to minimize resource costs to ensure elastic Qos is completed through three steps. The computing resources required by the user query are scheduled to a specific resource pool for execution.

[0212] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0213] Reference Fig. 9 , shows a structural block diagram of a resource scheduling device provided in an embodiment of the present application, which is applied to a cloud, and the cloud is configured with a plurality of resource pools corresponding to different elastic efficiencies. The elastic efficiency is used to describe the time required for the resource pool to provide resources that meet the set elastic requirements, and specifically may include the following modules:

[0214] Data acquisition module 901, used to acquire the time overhead corresponding to the data task;

[0215] A resource pool screening module 902, configured to screen a target resource pool corresponding to the data task from a resource pool in the cloud according to the time cost and the elastic efficiency;

[0216] The task processing module 903 is used to schedule the elastic computing resources for completing the data task from the target resource pool, so as to complete the data task based on the elastic computing resources.

[0217] In some feasible embodiments, the resource pool screening module 902 is specifically used to:

[0218] Using the time overhead to analyze the resource load of the data task, and obtaining a target elastic efficiency corresponding to the data task;

[0219] According to the matching result between the target elastic efficiency and the elastic efficiency corresponding to each of the resource pools, a target resource pool corresponding to the data task is screened out from the resource pools.

[0220] In some feasible embodiments, the resource pool is divided into at least a hot elastic resource pool, a warm elastic resource pool, and a cold elastic resource pool according to elasticity efficiency, and the resource pool screening module 902 is specifically used to:

[0221] Matching the target elastic efficiency with the elastic efficiency corresponding to each of the resource pools in turn;

[0222] If the target elastic efficiency successfully matches the first elastic efficiency corresponding to the thermal elastic resource pool, the thermal elastic resource pool is used as the target resource pool for processing the data task;

[0223] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool, but successfully matches the second elastic efficiency corresponding to the warm elastic resource pool, the warm elastic resource pool is used as the target resource pool for processing the data task;

[0224] If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool and fails to match the second elastic efficiency corresponding to the warm elastic resource pool, the cold elastic resource pool is used as the target resource pool for processing the data task.

[0225] In some feasible embodiments, the device further comprises:

[0226] A status information acquisition module, used to acquire pool status information corresponding to each resource pool, wherein the pool status information at least includes a resource water level;

[0227] The resource replenishment module is used to replenish resources to the target resource pool if the resource water level of the target resource pool is higher than a preset water level threshold.

[0228] In some feasible embodiments, the data acquisition module 901 is specifically used for:

[0229] Obtaining the access layer duration, computing resource startup duration, and execution duration of the data task corresponding to the data task;

[0230] The access layer duration, the computing resource startup duration and the execution duration are used to calculate the duration overhead to obtain the time overhead for completing the data task.

[0231] In some feasible embodiments, the data acquisition module 901 is specifically used for:

[0232] Obtaining the percentage of the startup time of the computing resource to the time overhead;

[0233] The resource load of the data task is analyzed using the percentage and the time overhead to obtain a target elastic efficiency corresponding to the data task.

[0234] In some feasible embodiments, it also includes:

[0235] A first resource pool startup module, used for pre-starting computing resource units of different specifications in the thermal elastic resource pool;

[0236] The second resource pool startup module is used to pre-create a cloud disk and a network card on the device to which the warm elastic resource pool belongs, and mount the cloud disk and the network card.

[0237] In some feasible embodiments, the task processing module 903 is specifically used to:

[0238] If the target resource pool is a thermal elastic resource pool, the elastic computing resources required to complete the data task are obtained, and the first computing resource units corresponding to the number of the elastic computing resources are scheduled from the thermal elastic resource pool, and the data task is completed through the first computing resource units.

[0239] In some feasible embodiments, the task processing module 903 is specifically used to:

[0240] If the target resource pool is a warm elastic resource pool, obtaining elastic computing resources required to complete the data task, and creating a number of second computing resource units corresponding to the elastic computing resources in the warm elastic resource pool based on the cloud disk and the network card;

[0241] The data task is completed by the second computing resource unit.

[0242] In some feasible embodiments, the task processing module 903 is specifically used to:

[0243] Acquire the elastic computing resources required to complete the data task;

[0244] Calculate the cloud disk and network card that need to be created according to the elastic computing resources, create the cloud disk and network card, and mount the created cloud disk and network card respectively;

[0245] According to the cloud disk and the network card, create a number of third computing resource units corresponding to the elastic computing resources in the cold elastic resource pool;

[0246] The data task is completed by the third computing resource unit.

[0247] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0248] In addition, an embodiment of the present application also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned resource scheduling method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0249] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned resource scheduling method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0250] Fig.10 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present application.

[0251] The electronic device 1000 includes but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, a processor 1010, and a power supply 1011. Those skilled in the art will appreciate that the electronic device structure involved in the embodiments of the present application does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently. In the embodiments of the present application, the electronic device includes but is not limited to a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted terminal, a wearable device, and a pedometer.

[0252] It should be understood that in the embodiment of the present application, the RF unit 1001 can be used for receiving and sending signals during information transmission or calls. Specifically, after receiving downlink data from the base station, it is sent to the processor 1010 for processing; in addition, uplink data is sent to the base station. Generally, the RF unit 1001 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the RF unit 1001 can also communicate with the network and other devices through a wireless communication system.

[0253] The electronic device provides users with wireless broadband Internet access through the network module 1002, such as helping users to send and receive emails, browse web pages, and access streaming media.

[0254] The audio output unit 1003 can convert the audio data received by the RF unit 1001 or the network module 1002 or stored in the memory 1009 into an audio signal and output it as sound. Moreover, the audio output unit 1003 can also provide audio output related to a specific function performed by the electronic device 1000 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 1003 includes a speaker, a buzzer, a receiver, etc.

[0255] The input unit 1004 is used to receive audio or video signals. The input unit 1004 may include a graphics processor (GPU) 10041 and a microphone 10042. The graphics processor 10041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 1006. The image frame processed by the graphics processor 10041 can be stored in the memory 1009 (or other storage medium) or sent via the radio frequency unit 1001 or the network module 1002. The microphone 10042 can receive sound and can process such sound into audio data. The processed audio data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 1001 in the case of a telephone call mode.

[0256] The electronic device 1000 also includes at least one sensor 1005, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 10071 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 10071 and / or the backlight when the electronic device 1000 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used to identify the posture of the electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; the sensor 1005 can also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.

[0257] The display unit 1006 is used to display information input by the user or information provided to the user. The display unit 1006 may include a display panel 10071, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0258] The user input unit 1007 can be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the electronic device. Specifically, the user input unit 1007 includes a touch panel 10071 and other input devices 10072. The touch panel 10071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 10071 or near the touch panel 10071 using any suitable object or accessory such as a finger, stylus, etc.). The touch panel 10071 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the coordinates of the contact point, and then sends it to the processor 1010, receives the command sent by the processor 1010 and executes it. In addition, the touch panel 10071 can be implemented in various types such as resistive, capacitive, infrared and surface acoustic waves. In addition to the touch panel 10071, the user input unit 1007 may also include other input devices 10072. Specifically, other input devices 10072 may include but are not limited to physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which will not be repeated here.

[0259] Further, the touch panel 10071 may be covered on the display panel 10071. When the touch panel 10071 detects a touch operation on or near it, it is transmitted to the processor 1010 to determine the type of the touch event, and then the processor 1010 provides a corresponding visual output on the display panel 10071 according to the type of the touch event. It can be understood that in one embodiment, the touch panel 10071 and the display panel 10071 are used as two independent components to implement the input and output functions of the electronic device, but in some embodiments, the touch panel 10071 and the display panel 10071 can be integrated to implement the input and output functions of the electronic device, which is not limited here.

[0260] The interface unit 1008 is an interface for connecting an external device to the electronic device 1000. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 1008 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the electronic device 1000 or may be used to transmit data between the electronic device 1000 and an external device.

[0261] The memory 1009 can be used to store software programs and various data. The memory 1009 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 1009 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0262] The processor 1010 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 1009 and calling data stored in the memory 1009, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. The processor 1010 may include one or more processing units; preferably, the processor 1010 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1010.

[0263] The electronic device 1000 may also include a power supply 1011 (such as a battery) for supplying power to each component. Preferably, the power supply 1011 may be logically connected to the processor 1010 via a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0264] In addition, the electronic device 1000 includes some functional modules not shown, which will not be described in detail here.

[0265] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0266] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0267] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0268] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0269] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0270] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0271] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0272] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0273] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0274] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A resource scheduling method, characterized in that: Applied to a cloud, the cloud is configured with a plurality of resource pools corresponding to different elastic efficiencies, the elastic efficiency is used to describe the time required for the resource pool to provide resources that meet the set elastic requirements, the method includes: The time overhead corresponding to the task of obtaining data; According to the time cost and the elastic efficiency, a target resource pool corresponding to the data task is selected from a cloud resource pool; The elastic computing resources for completing the data task are scheduled from the target resource pool to complete the data task based on the elastic computing resources.

2. The method according to claim 1, characterized in that The step of selecting a target resource pool corresponding to the data task from a cloud resource pool according to the time cost and the elastic efficiency includes: Using the time overhead to analyze the resource load of the data task, and obtaining a target elastic efficiency corresponding to the data task; According to the matching result between the target elastic efficiency and the elastic efficiency corresponding to each of the resource pools, a target resource pool corresponding to the data task is screened out from the resource pools.

3. The method according to claim 2, characterized in that The resource pool is divided into at least a hot elastic resource pool, a warm elastic resource pool, and a cold elastic resource pool according to elastic efficiency, and the target resource pool corresponding to the data task is screened out from each of the resource pools according to a matching result between the target elastic efficiency and the elastic efficiency corresponding to each of the resource pools, including: Matching the target elastic efficiency with the elastic efficiency corresponding to each of the resource pools in turn; If the target elastic efficiency successfully matches the first elastic efficiency corresponding to the thermal elastic resource pool, the thermal elastic resource pool is used as the target resource pool for processing the data task; If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool, but successfully matches the second elastic efficiency corresponding to the warm elastic resource pool, the warm elastic resource pool is used as the target resource pool for processing the data task; If the target elastic efficiency fails to match the first elastic efficiency corresponding to the hot elastic resource pool and fails to match the second elastic efficiency corresponding to the warm elastic resource pool, the cold elastic resource pool is used as the target resource pool for processing the data task.

4. The method according to claim 3, characterized in that The method further comprises: Obtaining pool status information corresponding to each of the resource pools, the pool status information at least including a resource water level; If the resource level of the target resource pool is higher than the preset level threshold, resources are supplemented for the target resource pool.

5. The method according to claim 2, characterized in that: The time overhead corresponding to the task of acquiring data includes: Obtaining the access layer duration, computing resource startup duration, and execution duration of the data task corresponding to the data task; The access layer duration, the computing resource startup duration and the execution duration are used to calculate the duration overhead to obtain the time overhead for completing the data task.

6. The method according to claim 5, characterized in that The adopting the time overhead to analyze the resource load of the data task to obtain a target elastic efficiency corresponding to the data task includes: Obtaining the percentage of the startup time of the computing resource to the time overhead; The resource load of the data task is analyzed using the percentage and the time overhead to obtain a target elastic efficiency corresponding to the data task.

7. The method according to claim 3, characterized in that Also includes: Pre-starting computing resource units of different specifications in the thermal elastic resource pool; A cloud disk and a network card are pre-created on the device to which the warm elastic resource pool belongs, and the cloud disk and the network card are mounted.

8. The method according to claim 7, characterized in that The step of scheduling the elastic computing resources for completing the data task from the target resource pool to complete the data task based on the elastic computing resources includes: If the target resource pool is a thermal elastic resource pool, the elastic computing resources required to complete the data task are obtained, and the first computing resource units corresponding to the number of the elastic computing resources are scheduled from the thermal elastic resource pool, and the data task is completed through the first computing resource units.

9. The method according to claim 7, characterized in that: The step of scheduling the elastic computing resources for completing the data task from the target resource pool to complete the data task based on the elastic computing resources includes: If the target resource pool is a warm elastic resource pool, obtaining elastic computing resources required to complete the data task, and creating a number of second computing resource units corresponding to the elastic computing resources in the warm elastic resource pool based on the cloud disk and the network card; The data task is completed by the second computing resource unit.

10. The method according to claim 7, characterized in that The step of scheduling the elastic computing resources for completing the data task from the target resource pool to complete the data task based on the elastic computing resources includes: Acquire the elastic computing resources required to complete the data task; Calculate the cloud disk and network card to be created according to the elastic computing resources, create the cloud disk and network card, and mount the created cloud disk and network card respectively; According to the cloud disk and the network card, create a number of third computing resource units corresponding to the elastic computing resources in the cold elastic resource pool; The data task is completed by the third computing resource unit.

11. A resource scheduling method, characterized in that: Applied to a cloud native data warehouse, the cloud native data warehouse includes at least a business load manager, a load scheduler, and a plurality of resource pools corresponding to different elastic efficiencies, wherein the elastic efficiency is used to describe the time required for the resource pool to provide resources that meet the set elastic requirements; wherein the method includes: Obtaining the time overhead corresponding to the data task through the business load manager; The load scheduler selects a target resource pool corresponding to the data task from a resource pool in the cloud according to the time cost and the elastic efficiency; The elastic computing resources for completing the data task are scheduled by the target resource pool, and the data task is completed based on the elastic computing resources.

12. A resource scheduling device, characterized in that: Applied to a cloud, the cloud is configured with a plurality of resource pools corresponding to different elastic efficiencies, the elastic efficiency is used to describe the time required for the resource pool to provide resources that meet the set elastic requirements, the device includes: Data acquisition module, used to obtain the time overhead corresponding to the data task; A resource pool screening module, used to screen a target resource pool corresponding to the data task from a resource pool in the cloud according to the time cost and the elastic efficiency; The task processing module is used to schedule the elastic computing resources for completing the data task from the target resource pool, so as to complete the data task based on the elastic computing resources.

13. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 11 when executing the program stored in the memory.

14. A computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 11.

Images