A resource quota adjustment method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a resource quota adjustment method and device, electronic equipment and storage medium, the method comprises: obtaining the current cluster resource utilization of big data platform, tenant label, tenant resource usage, period information; if it is busy state, then according to the priority of tenant adjusts the resource quota corresponding to each tenant;If it is idle state, then according to the tenant resource usage adjusts the resource quota corresponding to the tenant, relative to the prior art can only statically provide resource quota for tenants, in the busy state, the priority of tenant is adjusted by period information, to adjust the resource quota corresponding to the tenant, the resource quota of tenant is dynamically adjusted, the execution of the task of important tenant is guaranteed;In the idle state, the resource quota of tenant is adjusted by the tenant resource usage, the waste of the resources of big data platform is avoided, and each tenant can flexibly use the resources of big data platform.

Description

Technical Field

[0001] This invention relates to the field of big data technology, and in particular to a method for adjusting resource quotas, a device for adjusting resource quotas, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Hadoop has become the de facto standard for big data platforms, with large enterprises building multi-tenant shared big data platforms based on Hadoop to meet the big data resource needs of different data application scenarios. Multi-tenant big data platforms centered on Hadoop allocate resources to different tenants using resource quotas. However, the two quota management schemes built into the Hadoop platform—the Fair Scheduler and the Capability Scheduler—only provide static quota management. This can lead to unreasonable resource allocation within the big data platform during actual use. Summary of the Invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a method for adjusting resource quotas, a device for adjusting resource quotas, an electronic device, and a computer-readable storage medium to overcome or at least partially solve the above problems.

[0004] To address the aforementioned problems, this invention discloses a method for adjusting resource quotas, applied to a big data platform with multiple tenants. The method includes:

[0005] Obtain the current cluster resource utilization rate, tenant tags, tenant resource utilization rate, and time period information of the big data platform; the tenant tags include the default priority tags corresponding to the tenants;

[0006] Based on the cluster resource utilization rate, determine the resource load status of the big data platform;

[0007] If the resource load status of the big data platform is busy, the priority of the tenant is adjusted according to the time period information.

[0008] Adjust the resource quotas for each tenant based on their priority.

[0009] If the resource load status of the big data platform is idle, the resource quota corresponding to the tenant is adjusted according to the tenant's resource utilization rate.

[0010] Optionally, determining the resource load status of the big data platform based on the cluster resource utilization rate includes:

[0011] If the cluster resource utilization rate is greater than the first value, then the resource load status of the big data platform is determined to be busy.

[0012] If the cluster resource utilization rate is less than the second value, then the resource load status of the big data platform is determined to be idle; wherein, the first value is greater than the second value.

[0013] Optionally, the time period information includes the early morning period, working hours, and other time periods; the tenant tag includes automatically scheduled tasks and manually scheduled tasks; and adjusting the priority of the tenant according to the time period information includes:

[0014] If the current time period is the early morning period, increase the priority of tenants whose tenant tags include automatically scheduled tasks;

[0015] If the current time period information is a working period, then increase the priority of tenants whose tenant tags include manually scheduled tasks;

[0016] If the current time period information is another time period, the tenant's priority will be the default priority.

[0017] Optionally, adjusting the resource quotas corresponding to each tenant based on their priority includes:

[0018] Obtain the resource quota range for each tenant;

[0019] While keeping the resource quota within the specified range, reduce the resource quota for low-priority tenants;

[0020] After reducing the resource quota for low-priority tenants, the resource quota for high-priority tenants is increased while keeping the resource quota within the specified range.

[0021] Optionally, adjusting the resource quota corresponding to a tenant based on the tenant's resource utilization rate includes:

[0022] Based on the tenant resource utilization rate, determine the resource status corresponding to the tenant;

[0023] Based on the resource status of the tenant, adjust the tenant's resource quota while keeping the resource quota within the resource quota range.

[0024] Optionally, determining the resource status of a tenant based on the tenant's resource utilization rate includes:

[0025] If the tenant's resource utilization rate is less than the third value, then the resource status corresponding to the tenant is determined to be idle.

[0026] If the tenant's resource utilization rate is greater than the fourth value, then the resource status corresponding to the tenant is determined to be busy.

[0027] The fourth value is greater than the third value.

[0028] Optionally, adjusting a tenant's resource quota based on the tenant's resource status, while keeping the resource quota within the specified range, includes:

[0029] While keeping the resource quota within the specified range, reduce the resource quota for tenants that are in an idle state;

[0030] After reducing the resource quota for tenants in an idle state, the resource quota for tenants in a busy state is increased while keeping the resource quota within the specified range.

[0031] This invention also discloses a resource quota adjustment device applied to a big data platform, wherein the big data platform has multiple tenants, and the device includes:

[0032] The information acquisition module is used to acquire the current cluster resource utilization rate, tenant tags, tenant resource utilization rate, and time period information of the big data platform; the tenant tags include the default priority tags corresponding to the tenants;

[0033] The load status determination module is used to determine the resource load status of the big data platform based on the cluster resource utilization rate.

[0034] The priority adjustment module is used to adjust the priority of the tenant according to the time period information if the resource load status of the big data platform is busy.

[0035] The first quota adjustment module is used to adjust the resource quotas corresponding to each tenant according to the tenant's priority.

[0036] The second quota adjustment module is used to adjust the resource quota corresponding to the tenant based on the tenant's resource utilization rate if the resource load status of the big data platform is idle.

[0037] Optionally, the load status determination module includes:

[0038] The first load status determination submodule is used to determine that the resource load status of the big data platform is busy if the cluster resource utilization rate is greater than a first value.

[0039] The second load status determination submodule is used to determine that the resource load status of the big data platform is idle if the cluster resource utilization rate is less than a second value; wherein the first value is greater than the second value.

[0040] Optionally, the time period information includes the early morning period, working hours, and other time periods; the tenant tag includes automatically scheduled tasks and manually scheduled tasks; and the priority adjustment module includes:

[0041] The first priority adjustment submodule is used to increase the priority of tenants whose tenant tags include automatic scheduling tasks if the current time period information is the early morning period.

[0042] The second priority adjustment submodule is used to increase the priority of tenants whose tenant tags include manually scheduled tasks if the current time period information is a working time period.

[0043] The third priority adjustment submodule is used to set the tenant's priority to the default priority if the current time period information is from another time period.

[0044] Optionally, the first quota adjustment module includes:

[0045] The quota range acquisition submodule is used to obtain the resource quota range corresponding to each tenant;

[0046] The first quota reduction submodule is used to reduce the resource quota of low-priority tenants while keeping the resource quota within the resource quota range.

[0047] The first quota increase submodule is used to increase the resource quota of high-priority tenants after reducing the resource quota of low-priority tenants, while keeping the resource quota within the resource quota range.

[0048] Optionally, the second quota adjustment module includes:

[0049] The tenant status determination submodule is used to determine the resource status of a tenant based on the tenant's resource utilization rate.

[0050] The resource quota adjustment submodule is used to adjust the tenant's resource quota based on the tenant's resource status, while keeping the resource quota within the resource quota range.

[0051] Optionally, the tenant status determination submodule includes:

[0052] The first tenant status determination subunit is used to determine that the resource status of the tenant is idle if the tenant's resource utilization rate is less than a third value.

[0053] The second tenant status determination subunit is used to determine that the resource status of the tenant is busy if the tenant resource utilization rate is greater than the fourth value; wherein the fourth value is greater than the third value.

[0054] Optionally, the resource quota adjustment submodule includes:

[0055] The first resource quota reduction unit is used to reduce the resource quota of tenants in an idle state while keeping the resource quota within the resource quota range.

[0056] The second resource quota increase unit is used to increase the resource quota of tenants in a busy state after reducing the resource quota of tenants in an idle state, while keeping the resource quota within the resource quota range.

[0057] This invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0058] The memory is used to store computer programs;

[0059] When the processor executes the program stored in the memory, it implements the resource quota adjustment method as described in the embodiments of the present invention.

[0060] This invention also discloses one or more computer-readable media storing instructions that, when executed by one or more processors, cause the processors to perform the resource quota adjustment method as described in this invention.

[0061] The embodiments of the present invention have the following advantages:

[0062] In this embodiment of the invention, the current cluster resource utilization rate, tenant tags, tenant resource utilization rate, and time period information of the big data platform are obtained. The tenant tags include the default priority tags corresponding to the tenants. The resource load status of the big data platform is determined based on the cluster resource utilization rate. If the resource load status of the big data platform is busy, the priority of the tenants is adjusted according to the time period information. The resource quotas corresponding to each tenant are adjusted according to the tenant priority. If the resource load status of the big data platform is idle, the resource quotas corresponding to the tenants are adjusted according to the tenant resource utilization rate. Compared with the prior art, which can only statically provide resource quotas to tenants and cannot coordinate the resource quotas of tenants, this invention determines the resource load status of the big data platform by obtaining the cluster resource utilization rate. In the busy state, the priority of tenants is adjusted according to the time period information, and the resource quotas corresponding to tenants are adjusted according to the priority, realizing dynamic adjustment of the resource quotas of tenants and ensuring the execution of tasks of important tenants. In the idle state, the resource quotas of tenants are adjusted according to the tenant resource utilization rate, avoiding the waste of big data platform resources and enabling each tenant to flexibly use the resources of the big data platform. Attached Figure Description

[0063] Figure 1 This is a flowchart illustrating the steps of a resource quota adjustment method provided in an embodiment of the present invention;

[0064] Figure 2 This is a flowchart of a quota change method provided in an embodiment of the present invention;

[0065] Figure 3 This is a flowchart of a resource quota adjustment method provided in an embodiment of the present invention;

[0066] Figure 4 This is a structural block diagram of a resource quota adjustment device provided in an embodiment of the present invention. Detailed Implementation

[0067] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0068] In existing technologies, the two quota management schemes that come with the Hadoop platform, the Fair Scheduler and the Function Scheduler, can only provide static quota management schemes. This can lead to unreasonable allocation of resources on the big data platform during actual use.

[0069] One of the core concepts of this invention is to determine the resource load status of the big data platform by obtaining the cluster resource utilization rate. In the busy state, the priority of tenants is adjusted by time period information, and the resource quota of the tenants is adjusted according to the priority. This realizes the dynamic adjustment of the resource quota of tenants and ensures the execution of tasks of important tenants. In the idle state, the resource quota of tenants is adjusted by the tenant resource utilization rate to avoid the waste of big data platform resources and enable each tenant to flexibly use the resources of the big data platform.

[0070] Reference Figure 1 This diagram illustrates a flowchart of a resource quota adjustment method provided by an embodiment of the present invention, applied to a big data platform with multiple tenants. The method may specifically include the following steps:

[0071] Step 101: Obtain the current cluster resource utilization rate, tenant tags, tenant resource utilization rate, and time period information of the big data platform; the tenant tags include the default priority tags corresponding to the tenants;

[0072] In this embodiment of the invention, Hadoop is a distributed system infrastructure that can be used to develop distributed programs to fully utilize the power of clusters for high-speed computing and storage. Specifically, a big data platform can be built using Hadoop, and this big data platform can have multiple tenants.

[0073] In this embodiment of the invention, the current cluster resource utilization rate can be obtained through the cluster monitoring interface, and the resource utilization rate of each tenant can be obtained through the platform monitoring interface. The tenant resource utilization rate is used to represent the resource usage of the tenant under the resource quota limit.

[0074] Tenant tags and time period information can be obtained. Tenant tags can include the default priority tag corresponding to the tenant. The default priority can be represented by a number from 1 to 9; specifically, priorities of 1-3 are low priority, priorities of 4-6 are medium priority, and priorities of 7-9 are high priority.

[0075] For example, tenant tags can also include scheduling type, operation mode, business type, task cycle and business access, etc., and tenant tags can be expanded as needed according to different enterprise resource management goals.

[0076] Step 102: Determine the resource load status of the big data platform based on the cluster resource utilization rate;

[0077] In this embodiment of the invention, the resource load status of the big data platform can be determined based on the cluster resource utilization rate.

[0078] In one embodiment of the present invention, step 102 may include sub-steps S11-S12:

[0079] Sub-step S11: If the cluster resource utilization rate is greater than the first value, then the resource load status of the big data platform is determined to be busy.

[0080] In this embodiment of the invention, if the cluster resource utilization rate is greater than the first value, it can be determined that the resource load status of the big data platform is busy. In actual situations, the first value can be set to 70%. Specifically, the size of the first value can be determined according to the actual situation, and those skilled in the art will not limit it here.

[0081] Sub-step S12: If the cluster resource utilization rate is less than the second value, then the resource load status of the big data platform is determined to be idle; wherein, the first value is greater than the second value.

[0082] In this embodiment of the invention, if the cluster resource utilization rate is less than the second value, the resource load status of the big data platform can be determined to be idle. The first value can be greater than the second value. In practice, the second value can be set to 20%. Specifically, the size of the second value can be determined according to the actual situation, and those skilled in the art will not limit it here.

[0083] Step 103: If the resource load status of the big data platform is busy, then adjust the priority of the tenant according to the time period information.

[0084] In this embodiment of the invention, if the resource load status of the big data platform is busy, the priority of the tenant can be adjusted according to the time period information. The priority can be represented by 1 to 9. Specifically, priority of 1-3 can be low priority, priority of 4-6 can be medium priority, and priority of 7-9 can be high priority.

[0085] In one embodiment of the present invention, the time period information includes the early morning period, the working period, and other time periods, the tenant tag includes automatically scheduled tasks and manually scheduled tasks, and the step of adjusting the priority of the tenant according to the time period information includes: if the current time period information is the early morning period, then increase the priority of the tenant whose tenant tag includes automatically scheduled tasks; if the current time period information is the working period, then increase the priority of the tenant whose tenant tag includes manually scheduled tasks; if the current time period information is other time periods, then the priority of the tenant is the default priority.

[0086] In this embodiment of the invention, the time period information may include the early morning period, the working period and other time periods. The 24 hours of a day can be divided into 0-23 time periods. The early morning period can be from 0:00 to 8:00, the working period can be from 8:00 to 18:00, and other time periods can be from 18:00 to 0:00.

[0087] Tenant tags can include automatically scheduled tasks and manually scheduled tasks.

[0088] If the current time period is the early morning, the priority of tenants whose tenant tags include automatically scheduled tasks can be increased; if the current time period is during working hours, the priority of tenants whose tenant tags include manually scheduled tasks can be increased; if the current time period is any other time period, the priority of the tenant will not be adjusted and can remain at the default priority.

[0089] Step 104: Adjust the resource quotas for each tenant according to their priority.

[0090] In this embodiment of the invention, tenants may have corresponding priorities and resource quotas. If the resource load status of the big data platform is busy, the resource quotas corresponding to each tenant can be adjusted according to the tenant's priority.

[0091] In one embodiment of the present invention, step 102 may include sub-steps S21-S23:

[0092] Sub-step S21: Obtain the resource quota range corresponding to each tenant;

[0093] In this embodiment of the invention, the resource quota range corresponding to each tenant can be obtained, wherein the resource quota range may include an upper limit value and a lower limit value of the resource quota.

[0094] Sub-step S22: While keeping the resource quota within the resource quota range, reduce the resource quota of low-priority tenants;

[0095] In this embodiment of the invention, the resource quota of low-priority tenants can be reduced while keeping the resource quota within the resource quota range. The low-priority tenants can be tenants with a priority of 1-3.

[0096] In one example, a tenant's resource quota can be divided into three tiers. When it is necessary to reduce the resource quota of a low-priority tenant, the resource quota of the low-priority tenant can be lowered by one tier while keeping the resource quota within the resource quota range.

[0097] Sub-step S23: After reducing the resource quota of low-priority tenants, increase the resource quota of high-priority tenants while keeping the resource quota within the specified range.

[0098] In this embodiment of the invention, after reducing the resource quota of low-priority tenants, the resource quota of high-priority tenants can be increased while maintaining the resource quota within the specified range. Here, high-priority tenants can be tenants with a priority level of 7-9.

[0099] Step 105: If the resource load status of the big data platform is idle, adjust the resource quota corresponding to the tenant according to the tenant resource utilization rate.

[0100] In this embodiment of the invention, if the resource load status of the big data platform is idle, it can be determined whether to reduce or increase the resource quota corresponding to the tenant based on the tenant's resource utilization rate.

[0101] In one example, refer to Figure 2 This is a flowchart of a quota change provided in an embodiment of the present invention, including a big data platform, a rule engine, and a resource quota change plugin, wherein the rule engine includes a dynamic quota database.

[0102] Step 1: Obtain rule-driven data from the big data platform. The rule-driven data includes cluster resource utilization, tenant tags, tenant resource utilization, and time period information.

[0103] Step 2: Send the rule-driven data to the rule engine. The rule engine generates a quota change decision based on the rule-driven data and sends the quota change decision to the resource quota change plugin.

[0104] Step 3: The resource quota change plugin implements the resource quota change for tenants based on the quota change decision.

[0105] In one embodiment of the present invention, step 105 may include sub-steps S31-S32:

[0106] Sub-step S31: Determine the resource status of the tenant based on the tenant resource utilization rate;

[0107] In this embodiment of the invention, the resource status of a tenant can be determined based on the tenant resource utilization rate. The tenant resource utilization rate represents the tenant's resource usage under resource quota constraints; specifically, it represents the CPU and memory utilization rate of the tenant under resource quota constraints.

[0108] In one embodiment of the present invention, sub-step S31 may include sub-steps S311-S312:

[0109] Sub-step S311: If the tenant's resource utilization rate is less than the third value, then determine that the resource status corresponding to the tenant is idle.

[0110] In this embodiment of the invention, if the tenant's resource utilization rate is less than the third value, the resource status corresponding to the tenant can be determined to be idle. In actual cases, the third value can be set to 20%. Specifically, the value of the third value can be determined according to the actual situation, and those skilled in the art will not limit it here.

[0111] Sub-step S312: If the tenant's resource utilization rate is greater than the fourth value, then the resource status corresponding to the tenant is determined to be busy; wherein, the fourth value is greater than the third value.

[0112] In this embodiment of the invention, if the tenant's resource utilization rate is greater than the fourth value, the resource status corresponding to the tenant can be determined to be busy. The fourth value can be greater than the third value. In practice, the fourth value can be set to 70%. Specifically, the size of the fourth value can be determined according to the actual situation, and those skilled in the art will not limit it here.

[0113] Sub-step S32: Based on the resource status of the tenant, adjust the tenant's resource quota while keeping the resource quota within the resource quota range.

[0114] In this embodiment of the invention, the resource quota of a tenant can be adjusted according to the resource status of the tenant, while keeping the tenant's resource quota within the resource quota range. Specifically, the resource quota of a tenant in a busy state can be adjusted.

[0115] In one embodiment of the present invention, sub-step S32 may include sub-steps S321-S322:

[0116] Sub-step S321: While keeping the resource quota within the resource quota range, reduce the resource quota of tenants in an idle state;

[0117] Sub-step S322: After reducing the resource quota of tenants in an idle state, increase the resource quota of tenants in a busy state while keeping the resource quota within the resource quota range.

[0118] In this embodiment of the invention, while keeping the resource quota within the resource quota range, the resource quota of tenants in an idle state can be reduced, and after reducing the resource quota of tenants in an idle state, the resource quota of resources in a busy state can be increased.

[0119] In one example, refer to Figure 3 This is a flowchart of a model-based server heat dissipation method provided in an embodiment of the present invention, which may specifically include the following steps:

[0120] Step 1: Determine if the resource load status of the big data platform is busy; if it is busy, proceed to Step 2; if it is idle, proceed to Step 4.

[0121] Step 2: If the system is busy, obtain the time period information. If the time period information is the early morning period, increase the priority of tenants whose tenant tags include those with automatic scheduling tasks.

[0122] If the current time period is a working period, the priority of tenants whose tenant tags include manually scheduled tasks can be increased;

[0123] If the current time period information is from another time period, the tenant's priority will not be adjusted and can be set to the default priority.

[0124] Step 3: Obtain the priority information of the tenant; if it is a high-priority tenant, increase the resource quota of the high-priority tenant and decrease the resource quota of the low-priority tenant while keeping the resource quota within the resource quota range. Do not adjust the resource quota of the medium-priority tenant.

[0125] Step 4: If the tenant is in an idle state, obtain the resource status of the tenant. While keeping the resource quota within the resource quota range, reduce the resource quota of the tenant in the idle state and increase the resource quota of the tenant in the busy state.

[0126] In this embodiment of the invention, the current cluster resource utilization rate, tenant tags, tenant resource utilization rate, and time period information of the big data platform are obtained. The tenant tags include the default priority tags corresponding to the tenants. The resource load status of the big data platform is determined based on the cluster resource utilization rate. If the resource load status of the big data platform is busy, the priority of the tenants is adjusted according to the time period information. The resource quotas corresponding to each tenant are adjusted according to the tenant priority. If the resource load status of the big data platform is idle, the resource quotas corresponding to the tenants are adjusted according to the tenant resource utilization rate. Compared with the prior art, which can only statically provide resource quotas to tenants and cannot coordinate the resource quotas of tenants, this invention determines the resource load status of the big data platform by obtaining the cluster resource utilization rate. In the busy state, the priority of tenants is adjusted according to the time period information, and the resource quotas corresponding to tenants are adjusted according to the priority, realizing dynamic adjustment of the resource quotas of tenants and ensuring the execution of tasks of important tenants. In the idle state, the resource quotas of tenants are adjusted according to the tenant resource utilization rate, avoiding the waste of big data platform resources and enabling each tenant to flexibly use the resources of the big data platform.

[0127] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0128] Reference Figure 4 The diagram illustrates a structural block diagram of a resource quota adjustment device provided in an embodiment of the present invention, applied to a big data platform. The big data platform has multiple tenants and may specifically include the following modules:

[0129] The information acquisition module 401 is used to acquire the current cluster resource utilization rate, tenant tags, tenant resource utilization rate, and time period information of the big data platform; the tenant tags include the default priority tags corresponding to the tenants;

[0130] The load status determination module 402 is used to determine the resource load status of the big data platform based on the cluster resource utilization rate.

[0131] The priority adjustment module 403 is used to adjust the priority of the tenant according to the time period information if the resource load status of the big data platform is busy.

[0132] The first quota adjustment module 404 is used to adjust the resource quotas corresponding to each tenant according to the tenant's priority.

[0133] The second quota adjustment module 405 is used to adjust the resource quota corresponding to the tenant based on the tenant's resource utilization rate if the resource load status of the big data platform is idle.

[0134] In one embodiment, the load status determination module includes:

[0135] The first load status determination submodule is used to determine that the resource load status of the big data platform is busy if the cluster resource utilization rate is greater than a first value.

[0136] The second load status determination submodule is used to determine that the resource load status of the big data platform is idle if the cluster resource utilization rate is less than a second value; wherein the first value is greater than the second value.

[0137] In one embodiment, the time period information includes the early morning period, working hours, and other time periods; the tenant tag includes automatically scheduled tasks and manually scheduled tasks; and the priority adjustment module includes:

[0138] The first priority adjustment submodule is used to increase the priority of tenants whose tenant tags include automatic scheduling tasks if the current time period information is the early morning period.

[0139] The second priority adjustment submodule is used to increase the priority of tenants whose tenant tags include manually scheduled tasks if the current time period information is a working time period.

[0140] The third priority adjustment submodule is used to set the tenant's priority to the default priority if the current time period information is from another time period.

[0141] In one embodiment, the first quota adjustment module includes:

[0142] The quota range acquisition submodule is used to obtain the resource quota range corresponding to each tenant;

[0143] The first quota reduction submodule is used to reduce the resource quota of low-priority tenants while keeping the resource quota within the resource quota range.

[0144] The first quota increase submodule is used to increase the resource quota of high-priority tenants after reducing the resource quota of low-priority tenants, while keeping the resource quota within the resource quota range.

[0145] In one embodiment, the second quota adjustment module includes:

[0146] The tenant status determination submodule is used to determine the resource status of a tenant based on the tenant's resource utilization rate.

[0147] The resource quota adjustment submodule is used to adjust the tenant's resource quota based on the tenant's resource status, while keeping the resource quota within the resource quota range.

[0148] In one embodiment, the tenant status determination submodule includes:

[0149] The first tenant status determination subunit is used to determine that the resource status of the tenant is idle if the tenant's resource utilization rate is less than a third value.

[0150] The second tenant status determination subunit is used to determine that the resource status of the tenant is busy if the tenant resource utilization rate is greater than the fourth value; wherein the fourth value is greater than the third value.

[0151] In one embodiment, the resource quota adjustment submodule includes:

[0152] The first resource quota reduction unit is used to reduce the resource quota of tenants in an idle state while keeping the resource quota within the resource quota range.

[0153] The second resource quota increase unit is used to increase the resource quota of tenants in a busy state after reducing the resource quota of tenants in an idle state, while keeping the resource quota within the resource quota range.

[0154] In this embodiment of the invention, the current cluster resource utilization rate, tenant tags, tenant resource utilization rate, and time period information of the big data platform are obtained. The tenant tags include the default priority tags corresponding to the tenants. The resource load status of the big data platform is determined based on the cluster resource utilization rate. If the resource load status of the big data platform is busy, the priority of the tenants is adjusted according to the time period information. The resource quotas corresponding to each tenant are adjusted according to the tenant priority. If the resource load status of the big data platform is idle, the resource quotas corresponding to the tenants are adjusted according to the tenant resource utilization rate. Compared with the prior art, which can only statically provide resource quotas to tenants and cannot coordinate the resource quotas of tenants, this invention determines the resource load status of the big data platform by obtaining the cluster resource utilization rate. In the busy state, the priority of tenants is adjusted according to the time period information, and the resource quotas corresponding to tenants are adjusted according to the priority, realizing dynamic adjustment of the resource quotas of tenants and ensuring the execution of tasks of important tenants. In the idle state, the resource quotas of tenants are adjusted according to the tenant resource utilization rate, avoiding the waste of big data platform resources and enabling each tenant to flexibly use the resources of the big data platform.

[0155] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0156] This invention also provides an electronic device, comprising:

[0157] It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described resource quota adjustment method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0158] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described resource quota adjustment method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0159] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0160] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0161] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0162] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0164] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0165] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0166] The above provides a detailed description of a resource quota adjustment method, a resource quota adjustment device, an electronic device, and a computer-readable storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for adjusting resource quota, characterized in that, The method is applied to a big data platform, the big data platform has multiple tenants, and the method comprises the following steps: obtaining current cluster resource utilization, tenant labels, tenant resource usage and time period information of the big data platform; wherein the tenant labels comprise default priority labels corresponding to the tenants; determining a resource load state of the big data platform according to the cluster resource utilization; if the resource load state of the big data platform is a busy state, adjusting the priority of the tenants according to the time period information; adjusting the resource quota of each tenant according to the priority of the tenant; if the resource load state of the big data platform is an idle state, adjusting the resource quota of the tenant according to the tenant resource usage; wherein the time period information comprises a midnight time period, a working time period and other time periods, the tenant labels comprise automatic scheduling tasks and manual scheduling tasks, and the adjusting of the priority of the tenants according to the time period information comprises the following steps: if the current time period information is the midnight time period, increasing the priority of the tenants whose tenant labels comprise automatic scheduling tasks; if the current time period information is the working time period, increasing the priority of the tenants whose tenant labels comprise manual scheduling tasks; if the current time period information is the other time period, the priority of the tenants is the default priority.

2. The method of claim 1, wherein, The determining of the resource load state of the big data platform according to the cluster resource utilization comprises the following steps: if the cluster resource utilization is greater than a first value, determining that the resource load state of the big data platform is a busy state; if the cluster resource utilization is less than a second value, determining that the resource load state of the big data platform is an idle state; wherein the first value is greater than the second value.

3. The method of claim 1, wherein, The adjusting of the resource quota of each tenant according to the priority of the tenant comprises the following steps: obtaining the resource quota range of each tenant; under the premise of keeping the resource quota in the resource quota range, reducing the resource quota of the tenants with low priority; after reducing the resource quota of the tenants with low priority, under the premise of keeping the resource quota in the resource quota range, increasing the resource quota of the tenants with high priority.

4. The method of claim 1, wherein, The adjusting of the resource quota of the tenant according to the tenant resource usage comprises the following steps: determining the resource state of the tenant according to the tenant resource usage; under the premise of keeping the resource quota in the resource quota range, adjusting the resource quota of the tenant according to the resource state of the tenant.

5. The method of claim 4, wherein, The determining of the resource state of the tenant according to the tenant resource usage comprises the following steps: if the tenant resource usage is less than a third value, determining that the resource state of the tenant is an idle state; if the tenant resource usage is greater than a fourth value, determining that the resource state of the tenant is a busy state; wherein the fourth value is greater than the third value.

6. The method of claim 5, wherein, The adjusting of the resource quota of the tenant according to the resource state of the tenant under the premise of keeping the resource quota in the resource quota range comprises the following steps: under the premise of keeping the resource quota in the resource quota range, reducing the resource quota of the tenant in the idle state; After reducing the resource quota of the tenant in the idle state, the resource quota of the tenant in the busy state is increased while keeping the resource quota in the range of the resource quota.

7. An apparatus for adjusting resource quota, characterized by, The device is applied to a big data platform, and the big data platform has multiple tenants. An information acquisition module is configured to acquire current cluster resource utilization, tenant labels, tenant resource usage, and period information of the big data platform, wherein the tenant labels include default priority labels corresponding to the tenants. A load state determination module is configured to determine a resource load state of the big data platform according to the cluster resource utilization. A priority adjustment module is configured to adjust the priority of the tenants according to the period information if the resource load state of the big data platform is the busy state. A first quota adjustment module is configured to adjust the resource quota corresponding to each tenant according to the priority of the tenant. A second quota adjustment module is configured to adjust the resource quota corresponding to each tenant according to the tenant resource usage if the resource load state of the big data platform is the idle state. The period information includes an early morning period, a working period, and other periods, the tenant labels include automatic scheduling tasks and manual scheduling tasks, and the priority adjustment module includes: A first priority adjustment submodule is configured to increase the priority of the tenant whose tenant label includes the automatic scheduling task if the current period information is the early morning period. A second priority adjustment submodule is configured to increase the priority of the tenant whose tenant label includes the manual scheduling task if the current period information is the working period. A third priority adjustment submodule is configured to set the priority of the tenant as the default priority if the current period information is the other period.

8. An electronic device, comprising: The device includes: A processor, a memory, and a computer program stored on the memory and capable of running on the processor, wherein the computer program is executed by the processor to implement the steps of the resource quota adjustment method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer readable storage medium, and the computer program is executed by the processor to implement the steps of the resource quota adjustment method according to any one of claims 1 to 6.

Images