Elastic resource quota method and device for generative large model and related product

Abstract

The invention relates to an elastic resource quota method and device for a generative large model and a related product, and the method comprises the steps: calculating a resource remaining quota and a preset overuse quota mode of a user according to a unit time processing task load in a current preset time period; determining the computing resource overuse quota of the unit time processing task load of the user at the next moment; according to the quota proportion of the target service in the overuse quota service set in the overuse quota service set and the computing resource overuse quota of the unit time processing task quantity of the user at the next moment, determining the computing resource overuse quota of the unit time processing task quantity of the target service at the next moment; and according to the computing resource overuse quota of the unit time processing task quantity of the target service at the next moment and the computing resource average use quota of the unit time processing task quantity of the target service in the current preset time period, determining the computing resource quota of the unit time processing task quantity of the target service at the next moment.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device and related products for elastic resource quota for generative large models. Background Art

[0002] Generative large models (such as large language models, image generation models, etc.) are artificial intelligence models based on deep learning technology, with powerful language understanding and text generation capabilities. Generative large models usually require a large amount of computing resources, including CPU (Central Processing Unit) resources, GPU (Graphics Processing Unit) resources and storage resources, especially GPU resources, which are not only expensive but also scarce. As a result, when the computing resources of generative large models cannot be fully utilized, it will cause a huge waste of resources. In actual applications, multiple businesses under the same user may face different traffic peaks and traffic troughs at different times. In this regard, traditional resource management methods often lack flexibility and are difficult to adapt to such dynamically changing computing resource allocation needs; for example, some businesses may require a large amount of computing resources during peak traffic hours to handle sudden requests. If the computing resources are insufficient, it will make it difficult for the business to run stably, and a large amount of computing resources will be idle during the traffic trough period, resulting in low computing resource utilization. Summary of the invention

[0003] Based on this, it is necessary to provide a method, device and related products for elastic resource quota for generative large models that ensure stable business operation while improving computing resource utilization.

[0004] In a first aspect, an embodiment of the present application further provides a method for elastic resource quota for a generative large model, the method comprising:

[0005] Obtain the remaining amount of computing resources for the user to process tasks per unit time within the current preset period;

[0006] Include all services that meet the over-quota conditions of the user within the current preset period into the over-quota service set;

[0007] Determine the overuse quota of computing resources for the user's processing task volume per unit time at the next moment according to the remaining amount of computing resources for the user's processing task volume per unit time in the current preset period and the preset overuse quota method, wherein the next moment is the moment immediately after the current preset period and adjacent to the current preset period;

[0008] Determine the computing resource overuse quota for the first target business to process tasks per unit time at the next moment according to the quota proportion of the first target business in the overuse quota business set and the computing resource overuse quota for the user to process tasks per unit time at the next moment;

[0009] The computing resource quota for processing tasks per unit time of the first target business at the next moment is determined based on the computing resource overuse quota for processing tasks per unit time of the first target business at the next moment and the average computing resource usage quota for processing tasks per unit time of the first target business in the current preset time period.

[0010] In one embodiment, the amount of tasks processed per unit time includes the number of requests processed per minute and / or the number of tokens processed per minute;

[0011] When the amount of processing tasks per unit time is the number of requests processed per minute, the over-quota business set is the RPM over-quota business set, the first target business is any business in the RPM over-quota business set, and the over-quota condition is the RPM over-quota condition; wherein the RPM over-quota condition is: the average usage amount of computing resources of RPM is less than or equal to the first preset threshold, and the maximum usage amount of computing resources of RPM is less than the computing resource application quota of RPM;

[0012] When the amount of tasks processed per unit time is the number of tokens processed per minute, the over-quota business set is the TPM over-quota business set, the first target business is any business in the TPM over-quota business set, and the over-quota condition is the TPM over-quota condition; wherein the TPM over-quota condition is: the average usage amount of TPM's computing resources is less than or equal to the second preset threshold, and the maximum usage amount of TPM's computing resources is less than the TPM's computing resource application quota.

[0013] In one embodiment, according to the remaining amount of computing resources for processing tasks per unit time of the user in the current preset period and the preset over-usage quota method, determining the over-usage quota of computing resources for processing tasks per unit time of the user at the next moment includes:

[0014] The overuse quota of computing resources for processing tasks per unit time of the user at the next moment is determined based on the product of the remaining amount of computing resources for processing tasks per unit time of the user in the current preset period and the preset overuse quota percentage.

[0015] In one embodiment, the method further comprises:

[0016] Determine a first difference between a remaining quota of computing resources for processing tasks per unit time of the user in the current preset time period and an over-usage quota of computing resources for processing tasks per unit time of the user at the next moment;

[0017] Include all services of the user that meet the preemption quota conditions within the current preset time period into the preemption quota service set;

[0018] Determine the computing resource preemption quota of the user's task processing amount per unit time at the next moment according to the first difference and the preset preemption quota method;

[0019] Determine the computing resource preemption quota for the processing task volume per unit time of the second target business at the next moment according to the quota proportion of the second target business in the preemption quota business set and the computing resource preemption quota for the processing task volume per unit time of the user at the next moment;

[0020] The computing resource quota for processing tasks per unit time of the second target business at the next moment is determined based on the computing resource preemption quota for processing tasks per unit time of the second target business at the next moment and the computing resource application quota for processing tasks per unit time of the second target business.

[0021] In one embodiment, the amount of tasks processed per unit time includes the number of requests processed per minute and / or the number of tokens processed per minute;

[0022] When the processing task volume per unit time is the number of requests processed per minute, the preemption quota service set is the RPM preemption quota service set, the second target service is any service in the RPM preemption quota service set, and the preemption quota condition is the RPM preemption quota condition; wherein the RPM preemption quota condition is: the maximum usage amount of the computing resources of the RPM is equal to or greater than the computing resource application quota of the RPM;

[0023] When the amount of tasks processed per unit time is the number of tokens processed per minute, the preemption quota business set is the TPM preemption quota business set, the second target business is any business in the TPM preemption quota business set, and the preemption quota condition is the TPM preemption quota condition; among which, the TPM preemption quota condition is: the maximum usage amount of TPM's computing resources is equal to or greater than the TPM's computing resource application quota.

[0024] In one embodiment, determining the computing resource preemption quota of the user's task processing amount per unit time at the next moment according to the first difference and the preset preemption quota method includes:

[0025] The computing resource preemption quota of the user's task processing amount per unit time at the next moment is determined according to the product of the first difference and the preset preemption quota percentage.

[0026] In a second aspect, an embodiment of the present application further provides a flexible resource quota device for a generative large model, the device comprising:

[0027] The remaining quota determination module is used to obtain the remaining quota of computing resources for the user's task processing per unit time in the current preset period;

[0028] The service set composition module is used to include the services of the user that meet the over-quota conditions within the current preset period into the over-quota service set;

[0029] A user over-usage quota module is used to determine the over-usage quota of computing resources for processing tasks per unit time of the user at the next moment according to the remaining amount of computing resources for processing tasks per unit time of the user in the current preset period and the preset over-usage quota method, wherein the next moment is the moment immediately after the current preset period and adjacent to the current preset period;

[0030] The service overuse quota module is used to determine the computing resource overuse quota of the first target service for processing tasks per unit time at the next moment according to the quota proportion of the first target service in the overuse quota service set and the computing resource overuse quota of the user for processing tasks per unit time at the next moment;

[0031] The business resource quota module is used to determine the computing resource quota for the first target business to process tasks per unit time at the next moment based on the computing resource overuse quota for the first target business to process tasks per unit time at the next moment and the average computing resource usage for the first target business to process tasks per unit time within the current preset time period.

[0032] In one embodiment, the device further comprises:

[0033] The remaining quota determination module is further used to determine the first difference between the remaining quota of computing resources for processing tasks per unit time of the user in the current preset period and the over-use quota of computing resources for processing tasks per unit time of the user at the next moment;

[0034] The service set composition module is also used to include each service of the user that meets the preemption quota condition within the current preset time period into the preemption quota service set;

[0035] A user preemption quota module, used to determine the computing resource preemption quota of the user's task processing amount per unit time at the next moment according to the first difference and a preset preemption quota method;

[0036] The business preemption quota module is used to determine the computing resource preemption quota of the second target business for processing the task volume per unit time at the next moment according to the quota proportion of the second target business in the preemption quota business set and the computing resource preemption quota of the user for processing the task volume per unit time at the next moment;

[0037] The business resource quota module is also used to determine the computing resource quota for the second target business to process tasks per unit time at the next moment based on the computing resource preemption quota for the second target business to process tasks per unit time at the next moment and the computing resource application quota for the second target business to process tasks per unit time.

[0038] In a third aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0039] In a fourth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, performs the steps of the method described in the first aspect.

[0040] The above-mentioned elastic resource quota method, device and related products for generative large models obtain the remaining amount of computing resources for the user's unit time processing task volume in the current preset time period; include each business of the user that meets the overuse quota condition in the current preset time period into the overuse quota business set; determine the computing resource overuse quota of the user's unit time processing task volume at the next moment according to the remaining amount of computing resources for the user's unit time processing task volume in the current preset time period and the preset overuse quota method, wherein the next moment is the moment immediately adjacent to the current preset time period after the current preset time period; determine the computing resource overuse quota of the first target business in the overuse quota business set according to the quota proportion of the first target business in the overuse quota business set and the computing resource overuse quota of the user's unit time processing task volume at the next moment; determine the computing resource overuse quota of the first target business's unit time processing task volume at the next moment according to the computing resource overuse quota of the first target business's unit time processing task volume at the next moment and the average usage amount of computing resources for the first target business's unit time processing task volume in the current preset time period.

[0041] In this way, according to the remaining amount of computing resources for processing tasks per unit time of the user in the current preset time period, and according to the overuse quota of computing resources for processing tasks per unit time of the user at the next moment, the computing resource quota for processing tasks per unit time of the first target business at the next moment is dynamically updated, thereby making full use of the inconsistent peak and low peak characteristics of different business traffic of users, ensuring stable business operation, and at the same time improving the computing resource utilization rate of user-generated large model resources, ensuring full utilization of user-generated large model computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 One of the flowcharts of the elastic resource quota method for a generative large model according to an embodiment;

[0044] Figure 2 The second flowchart of the elastic resource quota method for a generative large model according to an embodiment of the present invention;

[0045] Figure 3 The third flowchart of the elastic resource quota method for a generative large model according to an embodiment of the present invention;

[0046] Figure 4 One of the flowcharts of the elastic resource quota method for a generative large model according to an embodiment is also included;

[0047] Figure 5 The flexible resource quota method for a generative large model according to an embodiment also includes a second flow chart;

[0048] Figure 6 The elastic resource quota method for a generative large model according to an embodiment also includes a flowchart diagram 3;

[0049] Figure 7 The figure is a schematic diagram of the structure of a flexible resource quota device for a generative large model according to an embodiment. DETAILED DESCRIPTION

[0050] In order to facilitate understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. Embodiments of the present application are provided in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0052] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first resistor may be referred to as a second resistor, and similarly, a second resistor may be referred to as a first resistor. Both the first resistor and the second resistor are resistors, but they are not the same resistor.

[0053] It can be understood that the “connection” in the following embodiments should be understood as “electrical connection”, “communication connection”, etc. if the connected circuits, modules, units, etc. have electrical signals or data transmission between each other.

[0054] It can be understood that “at least one” means one or more, “plurality” means two or more, and “at least a portion of an element” means a part or all of an element.

[0055] When used herein, the singular forms "a", "an", and "said / the" may also include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "include / comprise" or "have" and the like specify the presence of stated features, wholes, steps, operations, components, parts, or combinations thereof, but do not exclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts, or combinations thereof. At the same time, the term "and / or" used in this specification includes any and all combinations of the relevant listed items.

[0056] In an exemplary embodiment, referring to Figure 1 , a flexible resource quota method for a generative large model is provided, which may include the following steps S102~S110.

[0057] S102, obtaining the remaining amount of computing resources for processing tasks per unit time of the user in the current preset time period.

[0058] The preset time period is, for example, 1 hour, and the embodiment of the present application does not specifically limit the size of the preset time period. The current preset time period is, for example, the 1 hour that has just passed. The remaining amount of computing resources for the amount of tasks processed per unit time by the user in the current preset time period can be understood as: the remaining amount of computing resources for the amount of tasks processed per unit time after the user has just passed 1 hour.

[0059] For example, the amount of tasks processed per unit time is the number of tasks processed per unit time, where the tasks are, for example, requests, data, text, etc.

[0060] Optionally, obtain the computing resource usage quota Used for the user's task processing per unit time in the current preset time period; then, use the total quota of computing resources TotalQuota for the user's task processing per unit time available (available to the user, that is, purchased or leased by the user), subtract the computing resource usage quota Used for the user's task processing per unit time in the current preset time period, to obtain the remaining computing resource quota CurrentBuffer for the user's task processing per unit time in the current preset time period.

[0061] For example, CurrentBuffer=TotalQuota-Used.

[0062] S104: Add all services of the user that meet the over-quota conditions within the current preset time period into the over-quota service set.

[0063] Among them, the over-usage quota is a flexible quota allocated to the business to support a certain over-usage of computing resources. Users can carry out multiple businesses. After the current preset period, the over-usage quota business set ReservedSet is composed of the businesses that meet the over-usage quota conditions. For example, after 1 hour, the over-usage quota business set ReservedSet is composed of business A that meets the over-usage quota conditions, business B that meets the over-usage quota conditions, business C that meets the over-usage quota conditions, and business D that meets the over-usage quota conditions.

[0064] S106, based on the remaining amount of computing resources for the user's task processing per unit time in the current preset time period and the preset overuse quota method, determine the user's overuse quota for computing resources for the task processing per unit time at the next moment, where the next moment is the moment immediately after the preset time period.

[0065] For example, the preset over-quota method includes a preset over-quota ratio, a preset over-quota percentage, or a preset over-quota calculation method.

[0066] For example, if the next moment is the 61st minute, the current preset time period is the 60th minute to the 1st minute immediately before the 61st minute; if the next moment is the 62nd minute, the current preset time period is the 61st minute to the 2nd minute immediately before the 62nd minute; if the next moment is the 63rd minute, the current preset time period is the 62nd minute to the 3rd minute immediately before the 63rd minute; in this way, the current preset time period and the next moment immediately after the current preset time period and immediately before the current preset time period are determined in a sliding window manner.

[0067] S108, determining the computing resource overuse quota for processing tasks per unit time of the first target business at the next moment according to the quota proportion of the first target business in the overuse quota business set and the computing resource overuse quota for processing tasks per unit time of the user at the next moment.

[0068] Among them, the first target business is any business in the over-quota business set ReservedSet. For example, when the first target business is business A in the over-quota business set ReservedSet, the over-quota quota of computing resources for processing tasks per unit time of business A at the next moment is determined according to the quota ratio of business A in the over-quota business set ReservedSet and the over-quota of computing resources for processing tasks per unit time of the user at the next moment Reserved. Among them, the quota ratio of business A in the over-quota business set ReservedSet is: the ratio of the computing resource application quota Quota_A for processing tasks per unit time of business A to the sum of the computing resource application quotas for processing tasks per unit time of each business in the over-quota business set ReservedSet ReservedSet.

[0069] For example, ReservedSetQuota = (Quota_A) + (Quota_B) + (Quota_C) + (Quota_D).

[0070] Optionally, the quota ratio of the first target business in the over-quota business set ReservedSet is multiplied by the computing resource over-quota quota Reserved of the user's task processing volume per unit time at the next moment to determine the computing resource over-quota quota of the first target business for processing task volume per unit time at the next moment.

[0071] For example, Reserved_A=Reserved×[(Quota_A) / ReservedSetQuota].

[0072] S110, determine the computing resource quota for processing tasks per unit time of the first target business at the next moment based on the computing resource overuse quota for processing tasks per unit time of the first target business at the next moment, and the average computing resource usage quota for processing tasks per unit time of the first target business in the current preset time period.

[0073] Among them, the average usage of computing resources for the first target business to process tasks per unit time within the preset time period can be understood as: the average value of the usage of computing resources for the first target business to process tasks per unit time within the preset time period, for example, the average value of the usage of computing resources for the business A to process tasks per unit time within the just 1 hour.

[0074] Optionally, the computing resource overuse quota of the first target business's unit time task processing volume at the next moment is determined as the computing resource quota of the first target business's unit time task processing volume at the next moment by summing up the computing resource overuse quota of the first target business's unit time task processing volume in the current preset time period, and the first target business is calculated with this computing resource quota at the next moment.

[0075] In this way, in this embodiment of the present application, the computing resource quota for processing tasks per unit time of the first target business at the next moment is dynamically updated according to the remaining amount of computing resources for processing tasks per unit time of the user within the current preset time period and the computing resource overuse quota for processing tasks per unit time of the user at the next moment, thereby making full use of the inconsistent peak and low peak characteristics of different business traffic of users, ensuring stable business operation, and at the same time improving the computing resource utilization rate of user-generated large model resources, ensuring full utilization of user-generated large model computing resources.

[0076] Optionally, the amount of tasks processed per unit time includes the number of requests processed per minute and / or the number of tokens processed per minute.

[0077] In an exemplary embodiment, when the processing task volume per unit time is the number of requests processed per minute (Request Per Minutes, RPM), refer to Figure 2 , a flexible resource quota method for generating a large model may include the following steps S202~S210.

[0078] S202, obtaining the remaining amount of computing resources CurrentRPMBuffer of the RPM of the user in the current preset time period.

[0079] The total RPM computing resource quota TotalRPMQuota available to the user can be used, minus the RPM computing resource usage quota RPMUsed of the user in the current preset time period, to obtain the RPM computing resource remaining quota CurrentRPMBuffer of the user in the current preset time period.

[0080] That is, CurrentRPMBuffer=TotalRPMQuota-RPMUsed.

[0081] S204: Add all services of the user that meet the RPM over-quota conditions within the current preset time period into the RPM over-quota service set ReservedRPMSet.

[0082] Among them, for any business, the RPM over-quota condition is: the average computing resource usage quota of the business RPM in the current preset period is less than or equal to the first preset threshold, and the maximum computing resource usage quota of the business RPM in the current preset period is less than the computing resource application quota of the business RPM. The maximum computing resource usage quota of the business RPM in the current preset period can be understood as: the maximum value of the computing resource usage quota of the business RPM in the current preset period.

[0083] For example, for business A1, satisfying the RPM over-quota condition means that within the last hour, the average usage of computing resources of RPM of business A1 is less than or equal to the first preset threshold, and the maximum usage of computing resources of RPM of business A1 is less than the computing resource application quota RPMQuota_A1 of RPM of business A1. For business B1, satisfying the RPM over-quota condition means that within the last hour, the average usage of computing resources of RPM of business B1 is less than or equal to another first preset threshold, and the maximum usage of computing resources of RPM of business B1 is less than the computing resource application quota RPMQuota_B1 of RPM of business B1.

[0084] S206, determining the user's RPM computing resource overuse quota ReservedRPM at the next moment according to the user's RPM computing resource remaining quota CurrentRPMBuffer in the current preset time period and the preset overuse quota method.

[0085] S208, determining the computing resource overuse quota of the RPM of the first target business at the next moment according to the quota proportion of the first target business in the RPM overuse quota business set ReservedRPMSet and the computing resource overuse quota ReservedRPM of the user's RPM at the next moment.

[0086] For example, when the first target business is business A1 in the RPM over-quota business set ReservedRPMSet, the product of the quota ratio of business A1 in the RPM over-quota business set ReservedRPMSet and the user's RPM computing resource over-quota ReservedRPM at the next moment is determined as the RPM computing resource over-quota ReservedRPM_A1 of business A1 at the next moment. Among them, the quota ratio of business A1 in the RPM over-quota business set ReservedRPMSet is: the ratio of the RPM computing resource application quota RPMQuota_A1 of business A1 to the sum of the RPM computing resource application quotas of each business in the RPM over-quota business set ReservedRPMSet ReservedRPMSetQuota.

[0087] ReservedRPMA1=ReservedRPM×[(RPMQuota_A1) / ReservedRPMSetQuota].

[0088] For example, services A1, B1, C1 and D1 all meet the RPM over-usage quota conditions. Services A1, B1, C1 and D1 form the RPM over-usage quota service set ReservedRPMSet, then ReservedRPMSetQuota=(RPMQuota_A1)+(RPMQuota_B1)+(RPMQuota_C1)+(RPMQuota_D1).

[0089] S210, determining the RPM computing resource quota of the first target business at the next moment according to the RPM computing resource overuse quota of the first target business at the next moment and the RPM computing resource average usage amount of the first target business in the current preset time period.

[0090] For example, for business A1, the computing resource overuse quota ReservedRPM_A1 of the RPM of business A1 in the 61st minute and the average computing resource usage amount of the RPM of business A1 from the 60th minute to the 1st minute are summed to determine the computing resource quota of the RPM of business A1 in the 61st minute, and business A1 performs RPM calculation with this computing resource quota in the 61st minute.

[0091] In an exemplary embodiment, when the amount of processing tasks per unit time is the number of tokens processed per minute (Tokens Per Minute, TPM), refer to Figure 3, a flexible resource quota method for generating a large model may include the following steps S302~S310.

[0092] S302, obtaining the remaining amount of computing resources CurrentTPMBuffer of the user's TPM in the current preset time period.

[0093] The total quota of TPM computing resources available to the user, TotalTPMQuota, can be used to deduct the TPM computing resource usage quota TPMUsed of the user in the current preset period to obtain the remaining quota of TPM computing resources CurrentTPMBuffer of the user in the current preset period.

[0094] That is, CurrentTPMBuffer=TotalTPMQuota-TPMUsed.

[0095] S304: Add all services of the user that meet the TPM over-quota conditions within the current preset time period into the TPM over-quota service set ReservedTPMSet.

[0096] Among them, for any business, the TPM over-usage quota condition is: the average usage of computing resources of the business TPM in the current preset period is less than or equal to the second preset threshold, and the maximum usage of computing resources of the business TPM in the current preset period is less than the computing resource application quota of the business TPM. The maximum usage of computing resources of the business TPM in the current preset period can be understood as: the maximum value of the computing resource usage of the business TPM in the current preset period.

[0097] For example, for business A2, satisfying the TPM over-quota condition means that within the last hour, the average usage of computing resources of the TPM of business A2 is less than or equal to the second preset threshold, and the maximum usage of computing resources of the TPM of business A2 is less than the computing resource application quota TPMQuota_ A2 of the TPM of business A2. For business B2, satisfying the TPM over-quota condition means that within the last hour, the average usage of computing resources of the TPM of business B2 is less than or equal to another second preset threshold, and the maximum usage of computing resources of the TPM of business B2 is less than the computing resource application quota TPMQuota_ B2 of the TPM of business B2.

[0098] S306 , determining the user's TPM computing resource overuse quota ReservedTPM at the next moment according to the user's TPM computing resource remaining quota CurrentTPMBuffer in the current preset time period and the preset overuse quota method.

[0099] S308 , determining the computing resource overuse quota of the TPM of the first target business at the next moment according to the quota ratio of the first target business in the TPM overuse quota business set ReservedTPMSet and the computing resource overuse quota ReservedTPM of the user at the next moment.

[0100] For example, when the first target business is business A2 in the TPM over-quota business set ReservedTPMSet, the product of the quota ratio of business A2 in the TPM over-quota business set ReservedTPMSet and the computing resource over-quota ReservedTPM of the user's TPM at the next moment is determined as the computing resource over-quota ReservedTPM_ A2 of the TPM of business A2 at the next moment. Among them, the quota ratio of business A2 in the TPM over-quota business set ReservedTPMSet is: the ratio of the computing resource application quota TPMQuota_A2 of the TPM of business A2 to the sum of the computing resource application quotas of the TPM of each business in the TPM over-quota business set ReservedTPMSet ReservedTPMSetQuota.

[0101] ReservedTPMA2=ReservedTPM×[(TPMQuota_ A2) / ReservedTPMSetQuota].

[0102] For example, business A2, business B2, business C2 and business D2 all meet the TPM overuse quota conditions, and business A2, business B2, business C2 and business D2 constitute the TPM overuse quota business set ReservedTPMSet, then ReservedTPMSetQuota=(TPMQuota_ A2)+(TPMQuota_ B2)+(TPMQuota_ C2)+(TPMQuota_ D2).

[0103] S310, determining a TPM computing resource quota of the first target business at the next moment according to the TPM computing resource overuse quota of the first target business at the next moment and the average TPM computing resource usage amount of the first target business in the current preset time period.

[0104] For example, for business A2, the computing resource excess quota ReservedTPM_ A2 of TPM of business A2 at the 61st minute is summed with the average computing resource usage amount of TPM of business A2 from the 60th minute to the 1st minute to determine the computing resource quota of TPM of business A2 at the 61st minute, and business A2 performs TPM calculation with this computing resource quota at the 61st minute.

[0105] In an exemplary embodiment, in step S106, based on the remaining amount of computing resources for the user's unit time processing tasks in the current preset time period and the preset overuse quota method, the overuse quota of computing resources for the user's unit time processing tasks is determined, including: based on the remaining amount of computing resources for the user's unit time processing tasks in the current preset time period, multiplied by the preset overuse quota percentage, the overuse quota of computing resources for the user's unit time processing tasks at the next moment is determined.

[0106] For example, when the amount of tasks processed per unit time is RPM, the user's RPM computing resource overuse quota ReservedRPM at the next moment is determined based on the product of the user's RPM computing resource remaining amount CurrentRPMBuffer in the current preset time period and the preset overuse quota percentage RPMReservedPercent, where the size of RPMReservedPercent can be configured according to actual application needs, for example, it can be configured based on past experience and actual application needs.

[0107] That is, ReservedRPM=CurrentRPMBuffer×RPMReservedPercent.

[0108] For example, when the amount of tasks processed per unit time is TPM, the user's TPM computing resource overuse quota ReservedTPM at the next moment is determined based on the product of the user's TPM computing resource remaining amount CurrentTPMBuffer in the current preset time period and the preset overuse quota percentage TPMReservedPercent, where the size of TPMReservedPercent can be configured according to actual application needs, for example, it can be configured based on past experience and actual application needs.

[0109] That is, ReservedTPM=CurrentTPMBuffer×TPMReservedPercent.

[0110] In an exemplary embodiment, referring to Figure 4The elastic resource quota method for generating a large model may also include the following steps S402~S410.

[0111] S402, determining a first difference between a remaining quota of computing resources for processing tasks per unit time of a user in a current preset period and an overused quota of computing resources for processing tasks per unit time of the user at a next moment.

[0112] That is, the difference between CurrentBuffer and Reserved is determined as the first difference Avail.

[0113] S404: Add all services of the user that meet the preemption quota conditions within the current preset time period into the preemption quota service set.

[0114] Among them, the preemption quota is a flexible quota allocated to the business to support the business to preempt computing resources. Users can carry out multiple businesses. After the current preset period, the preemption quota business set GrabSet is composed of the businesses that meet the preemption quota conditions among the multiple businesses. For example, after 1 hour, the preemption quota business set GrabSet is composed of business E that meets the preemption quota conditions, business F that meets the preemption quota conditions, business G that meets the preemption quota conditions, and business H that meets the preemption quota conditions.

[0115] S406: Determine the computing resource preemption quota for the amount of tasks that the user will process per unit time at the next moment according to the first difference and a preset preemption quota method.

[0116] For example, the preset preemption quota method includes a preset preemption quota ratio, a preset preemption quota percentage, or a preset preemption quota calculation method.

[0117] S408, determining the computing resource preemption quota for the second target business to process the task volume per unit time at the next moment according to the quota proportion of the second target business in the preemption quota business set and the computing resource preemption quota for the user to process the task volume per unit time at the next moment.

[0118] Among them, the second target business is any business in the preemption quota business set GrabSet. For example, when the second target business is business E in the preemption quota business set GrabSet, the computing resource preemption quota Grab_E for the unit time processing task volume of business E at the next moment is determined according to the quota ratio of business E in the preemption quota business set GrabSet and the computing resource preemption quota Grab for the unit time processing task volume of the user at the next moment. Among them, the quota ratio of business E in the preemption quota business set GrabSet is: the ratio of the computing resource application quota Quota_E for the unit time processing task volume of business E to the sum of the computing resource application quotas GrabSetQuota for the unit time processing task volume of each business in the preemption quota business set GrabSet.

[0119] For example, GrabSetQuota = (Quota_E) + (Quota_F) + (Quota_G) + (Quota_H).

[0120] Optionally, the quota ratio of the second target business in the preemptive quota business set GrabSet is multiplied by the computing resource overuse quota Grab of the user's unit time task processing volume at the next moment to determine the computing resource overuse quota of the second target business for unit time task processing volume at the next moment.

[0121] For example, Grab_E=Grab×[(Quota_E) / GrabSetQuota].

[0122] S410, determining the computing resource quota for the second target business to process tasks per unit time at the next moment based on the computing resource preemption quota for the second target business to process tasks per unit time at the next moment and the computing resource application quota for the second target business to process tasks per unit time.

[0123] Optionally, the computing resource preemption quota of the second target business's unit time processing task volume at the next moment is summed with the computing resource application quota of the second target business's unit time processing task volume to determine the computing resource quota of the second target business's unit time processing task volume at the next moment, and the second target business is calculated with the computing resource quota at the next moment. For example, the computing resource quota of the unit time processing task volume of business E at the next moment is Grab_E+ Quota_E.

[0124] In this way, in this embodiment of the present application, based on the remaining amount of computing resources for the user's unit time processing task volume within the current preset time period, the computing resource quota for the user's unit time processing task volume is preempted according to the computing resource quota for the user's unit time processing task volume at the next moment, and the computing resource quota for the second target business unit time processing task volume at the next moment is dynamically updated, thereby making full use of the inconsistent peak and low peak characteristics of different business traffic of users, ensuring stable business operation, and at the same time improving the computing resource utilization rate of user-generated large model resources, ensuring full utilization of user-generated large model computing resources.

[0125] In an exemplary embodiment, when the amount of processing tasks per unit time is RPM, refer to Figure 5 The elastic resource quota method for generating a large model may also include the following steps S502~S510.

[0126] S502, determining a first difference between the user's RPM computing resource remaining quota CurrentRPMBuffer in the current preset time period and the user's RPM computing resource excess quota ReservedRPMSet at the next moment.

[0127] That is, the difference between CurrentRPMBuffer and ReservedRPM is determined as the first difference AvailRPM.

[0128] S504: Add all services of the user that meet the RPM preemption quota conditions within the current preset time period into the RPM preemption quota service set GrabRPMSet.

[0129] Among them, for any business, the RPM preemption quota condition is: the maximum usage amount of computing resources of the business RPM in the current preset time period is equal to or greater than the computing resource application quota of the business RPM.

[0130] For example, for service E1, satisfying the RPM preemption quota condition means that within the last hour, the maximum usage amount of computing resources of the RPM of service E1 is equal to or greater than the computing resource application quota RPMQuota_ E1 of the RPM of service E1.

[0131] S506: Determine the computing resource preemption quota GrabRPM of the RPM of the user at the next moment according to the first difference AvailRPM and the preset preemption quota method.

[0132] S508, determining the computing resource preemption quota of the RPM of the second target business at the next moment according to the quota proportion of the second target business in the RPM preemption quota business set GrabRPMSet and the computing resource preemption quota GrabRPM of the RPM of the user at the next moment.

[0133] For example, when the second target business is business E1 in the RPM preemption quota business set GrabRPMSet, the quota ratio of business E1 in the RPM preemption quota business set GrabRPMSet is multiplied by the user's RPM computing resource preemption quota GrabRPM at the next moment, and is determined as the RPM computing resource preemption quota GrabRPM_E1 of business E1 at the next moment. Among them, the quota ratio of business E1 in the RPM preemption quota business set GrabRPMSet is: the ratio of the RPM computing resource application quota RPMQuota_ E1 of business E1 to the sum of the RPM computing resource application quotas of each business in the RPM preemption quota business set GrabRPMSet, GrabRPMSetQuota.

[0134] GrabRPM_E1=GrabRPM×[(RPMQuota_E1) / GrabRPMSetQuota].

[0135] For example, services E1, F1, G1 and H1 all meet the RPM preemption quota conditions. Services E1, F1, G1 and H1 form the RPM preemption quota service set GrabRPMSet, then GrabRPMSetQuota=(RPMQuota_E1)+(RPMQuota_F1)+(RPMQuota_G1)+(RPMQuota_H1).

[0136] S510 , determining the computing resource quota of the RPM of the second target service at the next moment according to the computing resource preemption quota of the RPM of the second target service at the next moment and the computing resource application quota of the RPM of the second target service.

[0137] For example, the RPM computing resource quota of service E1 at the next moment is GrabRPM_E1+ RPMQuota_E1.

[0138] In an exemplary embodiment, when the amount of processing tasks per unit time is TPM, refer to Figure 6 The elastic resource quota method for generating a large model may also include the following steps S602~S610.

[0139] S602, determining a first difference between the remaining amount of computing resources of the TPM of the user in the current preset time period, CurrentTPMBuffer, and the computing resources excess quota ReservedTPMSet of the TPM of the user at the next moment.

[0140] That is, the difference between CurrentTPMBuffer and ReservedTPM is determined as the first difference AvailTPM.

[0141] S604, adding each service of the user that meets the TPM preemption quota condition within the current preset time period into the AvailRPM PM preemption quota service set GrabTPMSet.

[0142] Among them, for any business, the TPM preemption quota condition is: the maximum usage amount of computing resources of the business TPM in the current preset time period is equal to or greater than the computing resource application quota of the business TPM.

[0143] For example, for business E2, meeting the TPM preemption quota condition means that within the last 2 hours, the maximum usage amount of computing resources of the TPM of business E2 is equal to or greater than the computing resource application quota TPMQuota_ E2 of the TPM of business E2.

[0144] S606: Determine GrabTPM, a computing resource preemption quota of the user's TPM at the next moment, according to the first difference AvailTPM and a preset preemption quota method.

[0145] S608, determining the TPM computing resource preemption quota of the second target business at the next moment according to the quota proportion of the second target business in the TPM preemption quota business set GrabTPMSet and the TPM computing resource preemption quota GrabTPM of the user at the next moment.

[0146] For example, when the second target business is business E2 in the TPM preemption quota business set GrabTPMSet, the quota ratio of business E2 in the TPM preemption quota business set GrabTPMSet is multiplied by the user's TPM computing resource preemption quota GrabTPM at the next moment to determine the TPM computing resource preemption quota GrabTPM_E2 of business E2 at the next moment. Among them, the quota ratio of business E2 in the TPM preemption quota business set GrabTPMSet is: the ratio of the TPM computing resource application quota TPMQuota_ E2 of business E2 to the sum of the TPM computing resource application quotas of each business in the TPM preemption quota business set GrabTPMSet, GrabTPMSetQuota.

[0147] GrabTPM_E2=GrabTPM×[(TPMQuota_E2) / GrabTPMSetQuota].

[0148] For example, services E2, F2, G2 and H2 all meet the TPM preemption quota conditions, and services E2, F2, G2 and H2 constitute the TPM preemption quota service set GrabTPMSet, then GrabTPMSetQuota=(TPMQuota_E2)+(TPMQuota_F2)+(TPMQuota_G2)+(TPMQuota_H2).

[0149] S610 , determining a TPM computing resource quota of the second target business at the next moment according to the TPM computing resource preemption quota of the second target business at the next moment and the TPM computing resource application quota of the second target business.

[0150] For example, the TPM computing resource quota of service E2 at the next moment is GrabTPM_E2+ TPMQuota_E2.

[0151] In an exemplary embodiment, in step S406, determining the computing resource preemption quota of the user's task processing amount per unit time at the next moment according to the first difference and the preset preemption quota method includes:

[0152] The computing resource preemption quota of the user's task processing amount per unit time at the next moment is determined according to the product of the first difference and the preset preemption quota percentage.

[0153] For example, when the amount of tasks processed per unit time is RPM, the user's RPM computing resource preemption quota GrabRPM at the next moment is determined based on the product of the user's RPM computing resource remaining amount CurrentRPMBuffer in the current preset time period and the preset preemption quota percentage RPMGrabPercent, where the size of RPMGrabPercent can be configured according to actual application needs, for example, it can be configured based on past experience and actual application needs.

[0154] That is, GrabRPM = AvailRPM × RPMGrabPercent;

[0155] AvailRPM=CurrentRPMBuffer-ReservedRPM.

[0156] For example, when the amount of tasks processed per unit time is TPM, the user's TPM computing resource preemption quota GrabTPM at the next moment is determined based on the product of the user's TPM computing resource remaining amount CurrentTPMBuffer in the current preset time period and the preset preemption quota percentage TPMGrabPercent, where the size of TPMGrabPercent can be configured according to actual application needs, for example, it can be configured based on past experience and actual application needs.

[0157] That is, GrabTPM = AvailTPM × TPMGrabPercent;

[0158] AvailTPM =CurrentTPMBuffer-ReservedTPM.

[0159] It should also be noted that the computing resource application quota Quota of each business can be applied for by each business before the first preset time period in the entire model operation process, and during the first preset time period in the entire model operation process, each business will calculate with the computing resource application quota Quota.

[0160] RPM is a key indicator for measuring the processing power of a large generative model, reflecting the number of requests that a large generative model can process per minute. In AI model inference pricing and performance evaluation, RPM is an important consideration, which is directly related to the service capability and concurrent processing capability of the AI ​​model, that is, how many requests the AI ​​model can process simultaneously without performance bottlenecks. In the field of AI and machine learning, Tokens are the basic units for model processing data. TPM is an important indicator for measuring model processing capabilities (especially text processing capabilities), which reflects the amount of data or text that the model can process per minute, and is crucial for evaluating the performance and efficiency of the model.

[0161] Traditional resource management methods are expensive due to the large cost of generating large models. The fixed quota management method cannot maximize the utilization of computing resources for different businesses under the same user in different traffic scenarios, resulting in a large waste of costs and increased user usage costs.

[0162] In this regard, the technical solution of the embodiment of the present application determines the remaining amount of computing resources based on the different traffic trends under different services of the same user, and can flexibly supply other demand services in the same time period, ensuring that the computing resources of the generative large model under the same user have a high utilization rate throughout the entire period. At the same time, on the basis of the elastic allocation quota, it also ensures that the computing resources of the generative large model of the business in different time periods are met, ensuring that the business is in a lossless state under the elastic allocation.

[0163] The technical problems solved by the technical solutions of the embodiments of the present application are as follows: a) Traditional resource management solutions cannot flexibly respond to real-time changes in business traffic. b) During peak business traffic, insufficient technical resources may cause service delays or unavailability. c) During low business traffic, waste of technical resources leads to increased costs. The technical effects achieved by the technical solutions of the embodiments of the present application in solving technical problems are as follows: a) Improving resource utilization efficiency: ensuring that resources are fully utilized by adjusting resource quotas according to the real-time high and low peak states of business traffic. b) Reducing operating costs: reducing resource allocation during low traffic to effectively save costs. c) Ensuring business continuity: ensuring stable business operation by increasing resource quotas during peak traffic. d) Improving system response speed: quickly responding to changes in business traffic, adjusting resources in a timely manner, and reducing system response delays. e) Enhancing user experience: improving user satisfaction and loyalty by maintaining high performance in business scenarios.

[0164] To sum up, the technical solution of the embodiment of the present application achieves efficient utilization of resources and effective control of costs through a resource elasticity adjustment strategy based on peak and low flow rates of business traffic, while ensuring business stability and optimization of user experience, that is, ensuring that the total TPM quota and RPM total quota purchased by the user can be maximized.

[0165] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0166] Based on the same inventive concept, the embodiment of the present application also provides an elastic resource quota device for a generative large model for implementing the elastic resource quota method for a generative large model involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the elastic resource quota device for a generative large model provided below can refer to the limitations of the elastic resource quota method for a generative large model above, and will not be repeated here.

[0167] In an exemplary embodiment, referring to Figure 7 , provides an elastic resource quota device for a generative large model, the device comprising:

[0168] The remaining quota determination module 710 is used to obtain the remaining quota of computing resources for processing tasks per unit time of the user in the current preset period;

[0169] The service set composition module 720 is used to include the services of the user that meet the over-usage quota conditions within the current preset period into the over-usage quota service set;

[0170] The user over-usage quota module 730 is used to determine the over-usage quota of computing resources for processing tasks per unit time of the user at the next moment according to the remaining amount of computing resources for processing tasks per unit time of the user in the current preset period and the preset over-usage quota method, wherein the next moment is the moment immediately after the current preset period and adjacent to the current preset period;

[0171] The service overuse quota module 740 is used to determine the computing resource overuse quota of the first target service for processing the task volume per unit time at the next moment according to the quota proportion of the first target service in the overuse quota service set and the computing resource overuse quota of the user for processing the task volume per unit time at the next moment;

[0172] The business resource quota module 750 is used to determine the computing resource quota for the first target business to process tasks per unit time at the next moment based on the computing resource overuse quota for the first target business to process tasks per unit time at the next moment and the average computing resource usage for the first target business to process tasks per unit time within the current preset time period.

[0173] In an exemplary embodiment, the apparatus further comprises:

[0174] The remaining quota determination module is further used to determine the first difference between the remaining quota of computing resources for processing tasks per unit time of the user in the current preset period and the over-use quota of computing resources for processing tasks per unit time of the user at the next moment;

[0175] The service set composition module is also used to include each service of the user that meets the preemption quota condition within the current preset time period into the preemption quota service set;

[0176] A user preemption quota module, used to determine the computing resource preemption quota of the user's task processing amount per unit time at the next moment according to the first difference and a preset preemption quota method;

[0177] The business preemption quota module is used to determine the computing resource preemption quota of the second target business for processing the task volume per unit time at the next moment according to the quota proportion of the second target business in the preemption quota business set and the computing resource preemption quota of the user for processing the task volume per unit time at the next moment;

[0178] The business resource quota module is also used to determine the computing resource quota for the second target business to process tasks per unit time at the next moment based on the computing resource preemption quota for the second target business to process tasks per unit time at the next moment and the computing resource application quota for the second target business to process tasks per unit time.

[0179] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the elastic resource quota method for generating a large model provided in any of the above embodiments is implemented.

[0180] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for elastic resource quota for generative large models provided in any of the above embodiments is implemented.

[0181] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the elastic resource quota method for generating large models provided in any of the above embodiments.

[0182] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0183] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0184] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for elastic resource quota for generative large models, characterized in that: The method comprises: Obtain the remaining amount of computing resources for the user to process tasks per unit time within the current preset period; Include all services that meet the over-quota conditions of the user within the current preset period into the over-quota service set; Determine the overuse quota of computing resources for the user's processing task volume per unit time at the next moment according to the remaining amount of computing resources for the user's processing task volume per unit time in the current preset period and the preset overuse quota method, wherein the next moment is the moment immediately after the current preset period and adjacent to the current preset period; Determine the computing resource overuse quota for the first target business to process tasks per unit time at the next moment according to the quota proportion of the first target business in the overuse quota business set and the computing resource overuse quota for the user to process tasks per unit time at the next moment; The computing resource quota for processing tasks per unit time of the first target business at the next moment is determined based on the computing resource overuse quota for processing tasks per unit time of the first target business at the next moment and the average computing resource usage quota for processing tasks per unit time of the first target business in the current preset time period.

2. The method according to claim 1, characterized in that The amount of tasks processed per unit time includes the number of requests processed per minute and / or the number of tokens processed per minute; When the amount of processing tasks per unit time is the number of requests processed per minute, the over-quota business set is the RPM over-quota business set, the first target business is any business in the RPM over-quota business set, and the over-quota condition is the RPM over-quota condition; wherein the RPM over-quota condition is: the average usage amount of computing resources of RPM is less than or equal to the first preset threshold, and the maximum usage amount of computing resources of RPM is less than the computing resource application quota of RPM; When the amount of tasks processed per unit time is the number of tokens processed per minute, the over-quota business set is the TPM over-quota business set, the first target business is any business in the TPM over-quota business set, and the over-quota condition is the TPM over-quota condition; wherein the TPM over-quota condition is: the average usage amount of TPM's computing resources is less than or equal to the second preset threshold, and the maximum usage amount of TPM's computing resources is less than the TPM's computing resource application quota.

3. The method according to claim 1, characterized in that: According to the remaining amount of computing resources for the user's unit time processing task volume in the current preset period and the preset over-use quota method, the user's unit time processing task volume over-use quota for the next moment is determined, including: The overuse quota of computing resources for processing tasks per unit time of the user at the next moment is determined based on the product of the remaining amount of computing resources for processing tasks per unit time of the user in the current preset period and the preset overuse quota percentage.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Determine a first difference between a remaining quota of computing resources for processing tasks per unit time of the user in the current preset time period and an over-usage quota of computing resources for processing tasks per unit time of the user at the next moment; Include all services of the user that meet the preemption quota conditions within the current preset time period into the preemption quota service set; Determine the computing resource preemption quota of the user's task processing amount per unit time at the next moment according to the first difference and the preset preemption quota method; Determine the computing resource preemption quota for the processing task volume per unit time of the second target business at the next moment according to the quota proportion of the second target business in the preemption quota business set and the computing resource preemption quota for the processing task volume per unit time of the user at the next moment; The computing resource quota for processing tasks per unit time of the second target business at the next moment is determined based on the computing resource preemption quota for processing tasks per unit time of the second target business at the next moment and the computing resource application quota for processing tasks per unit time of the second target business.

5. The method according to claim 4, characterized in that The amount of tasks processed per unit time includes the number of requests processed per minute and / or the number of tokens processed per minute; When the processing task volume per unit time is the number of requests processed per minute, the preemption quota service set is the RPM preemption quota service set, the second target service is any service in the RPM preemption quota service set, and the preemption quota condition is the RPM preemption quota condition; wherein the RPM preemption quota condition is: the maximum usage amount of the computing resources of the RPM is equal to or greater than the computing resource application quota of the RPM; When the amount of tasks processed per unit time is the number of tokens processed per minute, the preemption quota business set is the TPM preemption quota business set, the second target business is any business in the TPM preemption quota business set, and the preemption quota condition is the TPM preemption quota condition; among which, the TPM preemption quota condition is: the maximum usage amount of TPM's computing resources is equal to or greater than the TPM's computing resource application quota.

6. The method according to claim 5, characterized in that Determining the computing resource preemption quota of the user's task processing amount per unit time at the next moment according to the first difference and the preset preemption quota method includes: The computing resource preemption quota of the user's task processing amount per unit time at the next moment is determined according to the product of the first difference and the preset preemption quota percentage.

7. A flexible resource quota device for a generative large model, characterized in that: The device comprises: The remaining quota determination module is used to obtain the remaining quota of computing resources for the user's task processing per unit time in the current preset period; The service set composition module is used to include the services of the user that meet the over-quota conditions within the current preset period into the over-quota service set; A user over-usage quota module is used to determine the over-usage quota of computing resources for processing tasks per unit time of the user at the next moment according to the remaining amount of computing resources for processing tasks per unit time of the user in the current preset period and the preset over-usage quota method, wherein the next moment is the moment immediately after the current preset period and adjacent to the current preset period; The service overuse quota module is used to determine the computing resource overuse quota of the first target service for processing tasks per unit time at the next moment according to the quota proportion of the first target service in the overuse quota service set and the computing resource overuse quota of the user for processing tasks per unit time at the next moment; The business resource quota module is used to determine the computing resource quota for the first target business to process tasks per unit time at the next moment based on the computing resource overuse quota for the first target business to process tasks per unit time at the next moment and the average computing resource usage for the first target business to process tasks per unit time within the current preset time period.

8. The device according to claim 7, characterized in that The device also includes: The remaining quota determination module is further used to determine the first difference between the remaining quota of computing resources for processing tasks per unit time of the user in the current preset period and the over-use quota of computing resources for processing tasks per unit time of the user at the next moment; The service set composition module is also used to include each service of the user that meets the preemption quota condition within the current preset time period into the preemption quota service set; A user preemption quota module, used to determine the computing resource preemption quota of the user's task processing amount per unit time at the next moment according to the first difference and a preset preemption quota method; The business preemption quota module is used to determine the computing resource preemption quota of the second target business for processing the task volume per unit time at the next moment according to the quota proportion of the second target business in the preemption quota business set and the computing resource preemption quota of the user for processing the task volume per unit time at the next moment; The business resource quota module is also used to determine the computing resource quota for the second target business to process tasks per unit time at the next moment based on the computing resource preemption quota for the second target business to process tasks per unit time at the next moment and the computing resource application quota for the second target business to process tasks per unit time.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images