Method, system, device and program product for adjusting user computing resource quota

Abstract

The present disclosure provides a user computing resource quota adjustment method, system, device and program product. The adjustment method comprises: in response to receiving a calling request of a target user, obtaining an initial computing resource quota of the target user, an estimated token amount of the target large model executing the calling request processing, and a first mapping relationship between tokens and computing resource quotas at the current time; in response to the target large model executing the calling request, obtaining an actual token amount of the target large model executing the calling request processing; based on the first mapping relationship, calculating the estimated computing resource quota and the actual computing resource quota consumed by the target user; after freezing the estimated computing resource quota in the initial computing resource quota, adjusting the initial computing resource quota after the freezing processing based on the estimated computing resource quota and the actual computing resource quota. In this way, the quota adjustment of the user is more reasonable, and the experience of both the model service party and the user can be improved at the same time.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a method, system, device, and program product for adjusting user computing resource quotas. Background Technology

[0002] In the operation of large model services, in order to avoid the situation where users' computing resource quotas are insufficient only after the model service is delivered, the large model gateway will usually freeze a portion of the user's computing resource quotas directly based on the user's call requests, and then adjust the frozen quotas after the model service is delivered.

[0003] However, in this process, the rules for calculating computing resource quotas may be affected by various factors, leading to an unreasonable adjustment of the frozen quota. This results in a poor user experience for both the model service provider and the user. Therefore, the method for adjusting user computing resource quotas still needs improvement. Summary of the Invention

[0004] The technical problem to be solved by this disclosure is to overcome the defect in the prior art that the adjustment of the frozen quota after the model service is delivered will lead to the lack of rationality in the adjustment of the frozen quota, and to provide a method, system, device and program product for adjusting the user computing resource quota.

[0005] This disclosure solves the above-mentioned technical problems through the following technical solution:

[0006] In a first aspect, this disclosure provides a method for adjusting user computing resource quotas, which is applied to a large model gateway. The adjustment method includes:

[0007] In response to receiving a call request from a target user, the system obtains the initial computing resource quota of the target user and estimates the estimated number of lexical units that the target large model needs to process to execute the call request; and obtains the first mapping relationship between lexical units and computing resource quota at the current moment.

[0008] Based on the estimated number of lexical units and the first mapping relationship, the estimated computing resource quota required by the target user is calculated, and the estimated computing resource quota is frozen in the initial computing resource quota.

[0009] In response to the completion of the call request by the target large model, obtain the actual number of lexical units actually processed by the target large model in executing the call request;

[0010] Based on the actual number of lexical units and the first mapping relationship, calculate the actual amount of computing resources consumed by the target user;

[0011] Based on the estimated computing resource limit and the actual computing resource limit, the initial computing resource limit after the freeze process is adjusted.

[0012] Optionally, the step of obtaining the actual number of lexical units actually processed by the target large model in response to the completion of the call request includes:

[0013] Obtain the field information returned by the target large model;

[0014] Based on the field information, obtain the word calculation rules that match the field information;

[0015] Based on the lexical calculation rules, the actual number of lexical units is calculated.

[0016] Optionally, the initial computing resource quota is stored in different storage locations; the step of freezing the estimated computing resource quota within the initial computing resource quota includes:

[0017] Obtain the freeze priority of the storage location;

[0018] Based on the freezing priority, the initial computing resource quota on the storage location is frozen sequentially until the estimated computing resource quota is frozen;

[0019] or,

[0020] Obtain the freeze ratio corresponding to the storage location;

[0021] After dividing the estimated computing resource quota based on the freezing ratio, the estimated computing resource quota corresponding to the storage location is frozen respectively.

[0022] Optionally, the step of adjusting the initial computing resource quota after freezing based on the estimated computing resource quota and the actual computing resource quota includes:

[0023] Calculate the difference between the estimated resource amount and the actual calculated resource amount;

[0024] In response to the difference being a positive number, the difference is unfrozen from the frozen estimated resource amount;

[0025] In response to the difference amount being negative, the difference amount is frozen within the unfrozen initial computing resource amount.

[0026] Optionally, in response to the difference being positive, the step of unfreezing the difference from the frozen estimated resource limit includes:

[0027] Obtain the thawing priority of the storage location;

[0028] Based on the unfreezing priority, the initial computing resource quota in the storage location is unfrozen sequentially until the difference quota is unfrozen;

[0029] or,

[0030] Obtain the thawing ratio corresponding to the storage location;

[0031] After dividing the difference amount based on the unfreezing ratio, the difference amount corresponding to the storage location is unfrozen respectively.

[0032] Optionally, the invocation request includes input data, output data, and tool data required to process the input data;

[0033] The steps of responding to receiving a call request from a target user, obtaining the target user's initial computing resource quota, estimating the estimated number of lexical units required for the target large model to execute the call request, and obtaining the first mapping relationship between lexical units and computing resource quota at the current moment include:

[0034] Based on the input data, the output data, and the tool data, the estimated number of lexical units required for the target large model to execute the invocation request is estimated.

[0035] Optionally, the invocation request includes a preset large model specified by the target user; the step of estimating the estimated number of lexical units required for the target large model to execute the invocation request includes:

[0036] If the working state of the preset large model does not match the preset working state, then all large models are traversed, and the large model that matches the preset working state is taken as the target large model.

[0037] Secondly, this disclosure provides a system for adjusting user computing resource quotas, wherein the method for adjusting user computing resource quotas is applied to a large model gateway, and the adjustment system includes:

[0038] The response module, in response to receiving a call request from a target user, obtains the initial computing resource quota of the target user, estimates the estimated number of lexical units that the target large model needs to process to execute the call request, and obtains the first mapping relationship between lexical units and computing resource quota at the current moment;

[0039] The calculation module is used to calculate the estimated computing resource quota required by the target user based on the estimated number of lexical elements and the first mapping relationship, and freeze the estimated computing resource quota in the initial computing resource quota;

[0040] The response module also responds to the completion of the call request by the target large model by obtaining the actual number of lexical units actually processed by the target large model in executing the call request.

[0041] The calculation module is also used to calculate the actual amount of computing resources actually consumed by the target user based on the actual number of lexical units and the first mapping relationship;

[0042] The adjustment module is used to adjust the initial computing resource quota after freezing based on the estimated computing resource quota and the actual computing resource quota.

[0043] Optionally, the adjustment system includes:

[0044] The acquisition module is used to acquire the field information returned by the target large model;

[0045] The acquisition module is also used to acquire word calculation rules that match the field information based on the field information;

[0046] The calculation module is also used to calculate the actual number of lexical units based on the lexical calculation rules.

[0047] Optionally, the initial computing resource quota is stored in different storage locations; the computing module includes:

[0048] The first acquisition unit is used to acquire the freezing priority of the storage location;

[0049] The first freezing unit is used to freeze the initial computing resource quota on the storage location sequentially based on the freezing priority, until the estimated computing resource quota is frozen;

[0050] or,

[0051] The second acquisition unit is used to acquire the freeze ratio corresponding to the storage location;

[0052] The second freezing unit is used to divide the estimated computing resource quota based on the freezing ratio, and then freeze the estimated computing resource quota corresponding to the storage location respectively.

[0053] Optionally, the adjustment module includes:

[0054] The calculation unit is used to calculate the difference between the estimated calculation resource amount and the actual calculation resource amount;

[0055] The response unit, in response to the difference amount being a positive number, unfreezes the difference amount from the frozen estimated resource amount;

[0056] The response unit also freezes the difference amount in the unfrozen initial computing resource amount if the difference amount is negative.

[0057] Optionally, the response unit includes:

[0058] The first acquisition subunit is used to acquire the unfreezing priority of the storage location;

[0059] The first unfreeze subunit is used to unfreeze the initial computing resource quota on the storage location sequentially based on the unfreeze priority, until the difference quota is unfrozen;

[0060] or,

[0061] The second acquisition subunit is used to acquire the thawing ratio corresponding to the storage location;

[0062] The second unfreezing subunit is used to divide the difference amount based on the unfreezing ratio and then unfreeze the corresponding difference amount at the storage location respectively.

[0063] Optionally, the invocation request includes input data, output data, and tool data required to process the input data;

[0064] The response module includes:

[0065] The estimation unit is used to estimate the number of lexical units that the target large model needs to process to execute the call request, based on the input data, the output data, and the tool data.

[0066] Optionally, the call request includes a preset large model specified by the target user; the response module also responds if the working status of the preset large model does not match the preset working status, then iterates through all large models and takes the large model that matches the preset working status as the target large model.

[0067] Thirdly, this disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and used to run on the processor, wherein the processor executes the computer program to implement the above-described method for adjusting user computing resource quotas.

[0068] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for adjusting user computing resource quotas.

[0069] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method for adjusting user computing resource quotas.

[0070] Based on common knowledge in the field, the above-mentioned preferred conditions can be combined arbitrarily to obtain various preferred embodiments of this disclosure.

[0071] The positive and progressive effects of this disclosure are as follows: Based on the first mapping relationship at the moment the target user's call request is received, the estimated amount of computing resources required by the target user and the actual amount of computing resources actually consumed by the user are calculated separately. This ensures that the estimated amount of computing resources and the actual amount of computing resources are calculated based on the same mapping relationship, thereby avoiding the phenomenon that the calculation basis of the estimated amount of computing resources and the actual amount of computing resources are different, which leads to the lack of rationality in adjusting the user's quota. This improves the experience of both the model service provider and the user. Attached Figure Description

[0072] Figure 1 A first flowchart of a method for adjusting user computing resource quotas provided in Embodiment 1 of this disclosure;

[0073] Figure 2 A second flowchart of a method for adjusting user computing resource quotas provided in Embodiment 1 of this disclosure;

[0074] Figure 3 A third flowchart of a method for adjusting user computing resource quotas provided in Embodiment 1 of this disclosure;

[0075] Figure 4 A fourth flowchart of a method for adjusting user computing resource quotas provided in Embodiment 1 of this disclosure;

[0076] Figure 5 A schematic diagram of a user computing resource quota adjustment system provided in Embodiment 2 of this disclosure;

[0077] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of this disclosure. Detailed Implementation

[0078] The present disclosure is further illustrated below by way of embodiments, but the present disclosure is not limited to the scope of the embodiments described herein.

[0079] The prefixes such as "first" and "second" used in this disclosure are merely for distinguishing different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes to distinguish descriptive objects in this disclosure does not restrict the described objects; the description of the described objects is given in the context of the embodiments, and the use of such prefixes should not constitute unnecessary restrictions. Furthermore, in the description of this embodiment, unless otherwise stated, "multiple" means two or more. It should also be noted that the order of the step numbers in this disclosure is for enhancing readability, and the actual order can be adjusted according to actual circumstances.

[0080] Example 1

[0081] In order to improve the experience of both the model service provider and the user after using a large model, Embodiment 1 of this disclosure provides a method for adjusting the user's computing resource quota. Figure 1 This is a first flowchart of a method for adjusting user computing resource quotas according to Embodiment 1 of this disclosure. This method is applied to a large model gateway, which connects multiple large models. After receiving a user's call request, the large model gateway calls the large model corresponding to the requesting router. The method for adjusting user computing resource quotas includes the following steps:

[0082] Step 101: In response to receiving the call request from the target user, obtain the target user's initial computing resource quota and estimate the estimated number of lexical units required for the target large model to execute the call request. Also, obtain the first mapping relationship between the lexical units and the computing resource quota at the current moment.

[0083] It should be noted that the mapping parameters in this first mapping relationship include at least one of the following: the version identifier of the target large model, the version number of the target large model, the time multiplier, the model multiplier, the user multiplier, the storage location of the initial computing resource quota, and the required tool information.

[0084] Step 102: Based on the estimated number of word elements and the first mapping relationship, calculate the estimated computing resource amount required by the target user, and freeze the estimated computing resource amount in the initial computing resource amount.

[0085] Furthermore, based on the relationship between the estimated computing resource quota calculated based on the estimated number of lexical units and the first mapping relationship, the estimated computing resource quota, the fixed frozen computing resource quota, and the minimum computing resource quota to be frozen, the estimated computing resource quota to be frozen in the initial computing resource quota can be determined.

[0086] For example: Suppose the calculated estimated resource amount is... The fixed amount of computing resources frozen is The minimum amount of computing resources that need to be frozen is The estimated calculated resource quota that needs to be frozen in the initial calculated resource quota is: .

[0087] when When greater than 0, ;

[0088] when When less than or equal to 0, .

[0089] Step 103: In response to the completion of the call request by the target large model, obtain the actual number of lexical units actually processed by the target large model in executing the call request.

[0090] Step 104: Based on the actual number of lexical units and the first mapping relationship, calculate the actual amount of computing resources consumed by the target user.

[0091] Specifically, based on the actual number of lexical units and the first mapping relationship, the formula for calculating the actual computing resource consumption of the target user is as follows:

[0092] Assume the actual amount of lexical units consumed comes from several dimensions. These dimensions can include at least one of the following: the amount of lexical units consumed in processing input data, the amount of lexical units consumed in processing output data, the amount of lexical units consumed in reading cached data, the amount of lexical units consumed in processing inference data, and the amount of lexical units consumed in processing other data. Cached data includes explicit cached data and implicit cached data.

[0093] The first mapping relationship includes the mapping relationship between the actual number of lexical units in each dimension and the actual amount of computing resources.

[0094] Let the total number of dimensions be Then the sum of the actual computing resource amounts for each dimension for:

[0095] ;

[0096] in, This represents the actual number of lexical units corresponding to the d-th dimension. Calculate the resource quota per unit of the actual word count corresponding to the d-th dimension; this value is greater than 0. Calculate the basic computing quota corresponding to the resource quota for each unit corresponding to the d-th dimension. This value is a non-negative number. This represents the time multiplier corresponding to the t-th time in the d-th dimension. This value is non-negative and has a default value of 1. This represents the model scaling factor corresponding to the d-th dimension. This value is non-negative and has a default value of 1. This represents the corresponding user multiplier. This value is non-negative and has a default value of 1.

[0097] The actual computing resource quota also includes tool surcharges. During the calculation process, the tool surcharge can be added to the tool multiplier. and user ratio The tool surcharge refers to the tools used by the target model when executing the request. For example, if a user's request includes "What's the weather like today?", the target needs to invoke a weather tool when processing the request.

[0098] Therefore, the actual computational resource quota for:

[0099] ;

[0100] in, The representation retains nine decimal places according to the preset rounding method. The maximum value is taken as the representation. The fixed mode is a special case where the unit of measurement for both the number of requested dimensions and the actual amount of computing resources is 1; the expression mode first executes the restricted function, then multiplies it by the model multiplier and the user multiplier, where each usage variable obtains its corresponding time multiplier based on the variable-dimension mapping.

[0101] Step 105: Adjust the initial computing resource quota after freezing based on the estimated computing resource quota and the actual computing resource quota.

[0102] In this embodiment, based on the first mapping relationship at the moment the target user's call request is received, the estimated computing resource amount required by the target user and the actual computing resource amount actually consumed by the user are calculated respectively. This ensures that the estimated computing resource amount and the actual computing resource amount are calculated based on the same mapping relationship, thereby avoiding the phenomenon that the calculation basis of the estimated computing resource amount and the actual computing resource amount are different, which leads to the lack of rationality in adjusting the user's quota, thus improving the experience of both the model service provider and the user.

[0103] In one embodiment, the step of obtaining the actual number of lexical units actually processed by the target large model in response to the completion of the call request includes:

[0104] S2-1: Obtain the field information returned by the target large model.

[0105] S2-2: Based on field information, obtain the word calculation rules that match the field information.

[0106] S2-3: Calculate the actual number of lexical units based on the lexical unit calculation rules.

[0107] The actual lexical quantity includes at least one of the following: the lexical quantity consumed by the target large model in processing user input data, the lexical quantity consumed in processing output data, and the lexical quantity consumed in processing images, audio, reasoning, and tool calls.

[0108] Based on the lexical calculation rules, determine the amount of lexicals consumed by the target large model when processing user input data.

[0109] Assumption The total number of input words returned by the target large model. The number of terms consumed by the target large model when reading from the cache. The number of terms consumed by the cache creation for the target large model. This indicates whether the number of lexical units consumed by the target large model for cache creation needs to be calculated independently. The number of lexical units consumed by the target large model for processing user input data is... :

[0110] ;

[0111] in, The number of lexical units consumed by the large model representing the target cache needs to be calculated independently; otherwise, This allows the number of terms consumed by cache creation to be less than the number of terms consumed by the target large model when processing user input data. The calculations are completed in advance, thus avoiding omissions and duplicate calculations in the future.

[0112] In this embodiment, different large models return different field information, and the field names, hierarchical structures, and semantic definitions within this information may vary. For example, some large models return different field names; some large models' returned field information already includes the number of terms consumed by cache reading or cache creation in the input data, while others report the number of terms consumed by cache reading and writing separately; some large models also distinguish between five-minute cache creation and one-hour cache creation, or add indicators for image, audio, inference, tool calls, and dedicated terminology calculations. Therefore, by matching the terminology calculation rules with the field information of the target large model, the actual number of terms processed by the target large model can be calculated more accurately. This, in turn, allows for more reasonable subsequent adjustments to user quotas.

[0113] In one embodiment, the initial computing resource quota is stored in a different storage location. The step of freezing the estimated computing resource quota within the initial computing resource quota includes:

[0114] S3-1: Get the freeze priority of the storage location.

[0115] S3-2: Based on the freezing priority, freeze the initial computing resource quota at the storage location in sequence until the estimated computing resource quota is frozen.

[0116] In this embodiment, since a user's initial computing resource quota may be located in different storage locations, the desired freezing order will also differ. Therefore, freezing the estimated computing resource quota based on the freezing priority of storage locations can improve the user experience. Furthermore, freezing the estimated computing resource quota based on freezing priority can effectively prevent insufficient computing resources in a single storage location from causing freezing failure, thereby improving the freezing success rate.

[0117] In one embodiment, the initial computing resource quota is stored in a different storage location. The step of freezing the estimated computing resource quota within the initial computing resource quota includes:

[0118] S4-1: Get the freeze ratio corresponding to the storage location.

[0119] S4-2: After dividing the estimated computing resource quota based on the freezing ratio, freeze the corresponding estimated computing resource quota at the storage location respectively.

[0120] In this embodiment, the estimated computing resource quota is frozen according to the freeze ratio corresponding to the storage location. This allows the estimated resources to be directly allocated to each storage location according to the preset ratio during the freezing phase, thus enabling a one-time freezing operation instead of attempting to freeze each storage location sequentially. This ensures that the computing resource quota for each storage location is accurately controllable, while also reducing the complexity of needing to refreeze due to partial success or failure in freezing some storage locations, thereby improving the success rate of the freezing operation.

[0121] In one embodiment, the step of adjusting the initial computing resource quota after freezing, based on the estimated computing resource quota and the actual computing resource quota, includes:

[0122] S5-1: Calculate the difference between the estimated resource amount and the actual calculated resource amount.

[0123] S5-2: In response to a positive difference amount, the difference amount is unfrozen from the frozen estimated resource amount.

[0124] S5-3: In response to a negative difference amount, freeze the difference amount in the unfrozen initial computing resource amount.

[0125] When the difference amount is positive, the difference amount is unfrozen from the frozen estimated resource amount. The calculation method is as follows:

[0126] ;

[0127] When the difference amount is negative, the difference amount is frozen within the unfrozen initial computing resource amount. The calculation method is as follows:

[0128] ;

[0129] When the user's unfrozen estimated resource quota is less than In this case, the difference in computing resource quota is actually frozen within the unfrozen initial computing resource quota. The calculation method is as follows:

[0130] ;

[0131] in, The estimated amount of resources not frozen for the user is less than At that time, the amount of computing resources undertaken by the large model gateway.

[0132] Assume the first The initial computing resource allocation for each storage location is Let C be the amount of computing resources to be processed (frozen or unfrozen). The required computing resource allocation for each storage location is :

[0133] ;

[0134] in, This represents the amount of computational resources required for the operation at the j-th storage location.

[0135] The formula for calculating the unsuccessful computing resource quota R among the pending computing resource quotas is as follows:

[0136] ;

[0137] If, during the stage of freezing estimated computing resources, a failure to complete the operation is detected to have a greater than 0 amount of computing resources, the user's call request is directly rejected, and the frozen computing resources are unfrozen. During the stage of calculating the actual computing resources, if a failure to complete the operation is detected to have a greater than 0 amount of computing resources, the overdrawn computing resources are recorded according to the policy, either directly or added to the computing resources allocated to the large model gateway. In subsequent unfreezing operations, it is not necessary to recalculate the storage location of the frozen computing resources; instead, the resources can be unfrozen directly in reverse.

[0138] In this embodiment, based on the difference between the estimated computing resource quota and the actual computing resource quota, a corresponding adjustment operation is performed on the initial computing resource quota after the freeze process. This improves the accuracy of the initial computing resource quota adjustment, thereby enhancing the user experience for both the model service provider and the user.

[0139] In one embodiment, in response to a positive difference amount, the step of unfreezing the difference amount from the frozen estimated resource amount includes:

[0140] S6-1: Get the thawing priority of the storage location.

[0141] Preferably, the thawing priority can be the opposite of the freezing priority described above.

[0142] S6-2: Based on the unfreezing priority, unfreeze the initial computing resource quota in the storage location in sequence until the difference quota is unfrozen.

[0143] In this embodiment, by setting the unfreezing priority to the same as the freezing priority, the amount is deducted in the forward direction when freezing and released in the reverse direction when unfreezing, that is, starting from the last frozen pool and moving backward, the unfreezing operation of the initial calculation of resource quota is turned into the inverse operation of the freezing operation, thereby fundamentally eliminating the risk of moving to the wrong storage location.

[0144] In one embodiment, in response to a positive difference amount, the step of unfreezing the difference amount from the frozen estimated resource amount includes:

[0145] S7-1: Obtain the thawing ratio corresponding to the storage location.

[0146] Preferably, the thawing ratio can be the same as the freezing ratio described above.

[0147] S7-2: After dividing the difference amount based on the unfreezing ratio, unfreeze the corresponding difference amount in the storage location respectively.

[0148] In this embodiment, by setting the thawing ratio and the freezing ratio to be the same, the difference amount is divided into several parts according to the freezing ratio before thawing, and each storage location thaws its own share simultaneously. This can not only fundamentally eliminate the problem of thawing the difference amount incorrectly, but also fundamentally eliminate the risk of thawing the wrong storage location.

[0149] In one embodiment, the invocation request includes input data, output data, and tool data required to process the input data.

[0150] In response to receiving a call request from the target user, the steps of obtaining the target user's initial computing resource quota and estimating the estimated number of lexical units required for the target large model to execute the call request, as well as obtaining the first mapping relationship between lexical units and computing resource quota at the current moment, include:

[0151] Based on input data, output data, and tool data, the estimated number of lexical units required to process the target large model to execute the call request is estimated.

[0152] For example, we can estimate the number of tokens in the input data of the target large model processing call request based on the following method.

[0153] Assume the number of characters in the input data is The estimated character for each lexical unit is the conversion factor. Then the estimated number of tokens consumed by the input data. :

[0154] .

[0155] Similarly, the estimated number of lexical units in the target large model's processing call request and the estimated number of lexical units in the tool data can be predicted.

[0156] In this embodiment, based on input data, output data, and tool data, the estimated number of terms that need to be processed to execute a call request by the target large model can be more accurately estimated. This, in turn, makes subsequent adjustments to the user's quota more accurate.

[0157] In one embodiment, the invocation request includes a preset large model specified by the target user. Prior to the step of estimating the estimated number of lexical units required to process the target large model in executing the invocation request, the following steps are included:

[0158] If the working state of the preset large model does not match the preset working state, then all large models are traversed, and the large model that matches the preset working state is taken as the target large model.

[0159] It should be noted that the preset working state can be set according to the actual situation. For example, the following can indicate that the working state of the preset large model is inconsistent with the preset working state: network connection to the preset large model is lost, DNS (Domain Name System) resolution of the preset large model fails, connection to the preset large model is rejected, connection request to the preset large model times out, authentication fails, key is invalid, or the number of consecutive failures reaches a threshold.

[0160] In this embodiment, when the working state of the user-specified preset large model does not match the preset working state, the large model gateway does not directly report an error, but automatically selects an alternative model to execute the request. This ensures that the user's call is not interrupted and automatically switches to other large models without the user's awareness, thereby greatly improving the availability and fault tolerance of the large model gateway.

[0161] In one embodiment, constraints can also be established based on the message identifier and stage identifier of each stage to constrain each stage and prevent duplicate processing. For example, when in the freeze stage: the message identifier can be "Freezing" and the stage identifier can be "Freeze Stage".

[0162] In one embodiment, an example is given to illustrate how to adjust user computing resource quotas when cached data is included.

[0163] Assume the target large model returns a total input vocabulary of 12,000 words and a total output vocabulary of 2,000 words. The target large model consumes 3,000 words for cache reading and 5,000 words for cache creation. Furthermore, the vocabulary consumed by the target large model for cache creation needs to be calculated independently. The vocabulary consumed by the target large model for processing user input data is... :

[0164] .

[0165] Furthermore, the mapping parameters in the first mapping relationship of the target large model include: model scaling factor. User group multiplier .

[0166] Assuming the estimated resource amount is This requires freezing 0.015 units in the first storage location and 0.01 units in the second storage location. This will reduce the API key's limit by 0.025 units. The API key is used to authenticate the user's identity, and if the API key limit is insufficient, subsequent calls cannot be performed.

[0167] Assume the unit price per million units of computing resources is as follows: user input data 1.2, output 4, cache read 0.3, cache creation 1.5; the time multiplier for input, output, and cache-related dimensions is 0.8. The computing resource allocation for text processing and caching is as follows: In one embodiment...

[0168] After adding the model multiplier and user multiplier, the actual calculated resource amount is:

[0169]

[0170] The required unfrozen computing resource quota is 0.0066832. The system first unfroze 0.0066832 from the second storage location. The remaining frozen quota of 0.0033168 in the second storage location, together with the 0.015 frozen in the first storage location, constitutes 0.0183168. The API key quota is simultaneously restored to 0.0066832. Even if the large model version is updated or the multiplier is adjusted during request execution, the actual computing resource quota will still use the mapping parameters in the first mapping relationship, and the unfreezing will be limited to the original allocation record as the upper limit of restoration.

[0171] In one embodiment, how to adjust the user's computing resource quota when the user's unfrozen computing resource quota is less than the difference quota.

[0172] Assuming the estimated frozen computing resource quota is 1.5, and the actual computing resource quota is 2.1, the difference is 0.6. If further freezing is attempted, and it's discovered that only 0.3 of the unfrozen computing resource quota remains, the large model gateway will freeze another 0.3, with the remaining 0.3 absorbed by the platform. The freezing process will be recorded to ensure interpretability of each step and prevent unexplained negative computing resource quotas from being created when users haven't authorized on-demand freezing.

[0173] In one embodiment, the streaming response consists of multiple shards, and the large-model gateway aggregates usage after the stream ends or an interruption is acknowledged. When the same message identifier is repeatedly submitted for actual computing resource quotas, the first submission writes a "message identifier + actual computing resource quota stage" transaction and attempts to change the request state from frozen estimated computing resource quota to frozen actual computing resource quota for unique constraint. When frozen actual computing resource quota and unfrozen computing resource quota arrive concurrently, they can be prepared separately, but only one process can update the request state from frozen estimated computing resource quota to the final state. The winning process continues to process subsequent processes, while the failing process marks ineffective transactions as failed and reads the current final state and returns it. When the compensation task scans the request again, it will also stop processing repeatedly because the final state is no longer frozen estimated computing resource quota or the stage transaction already exists.

[0174] In one embodiment, when the original measurement standard of the target large model requires additional dedicated computational resources, it is only necessary to configure the mapping relationship with JSONPath (a query language specifically used to find and extract specific information in JSON data), default values, units of measurement, variable names, and unit credit limits, etc., without modifying the system's lifecycle code; simply add an enabled measurement item to the new mapping relationship. Before publishing a new mapping relationship, the system performs normalization and validation on the configuration: dimension identifiers and units of measurement are unified to a conventional format; the size of the unit of measurement must be greater than 0, otherwise it will fall back to 1; the default value must not be less than 0; variable names must begin with a letter or underscore and contain only letters, numbers, or underscores. After the price version is published, the above price items, expressions, and time multipliers corresponding to each dimension together constitute the version data that can be requested to be frozen.

[0175] During the stage of freezing the estimated computational resource quota, since the target large model has not yet returned the actual usage, the system uses a default value of 0 for estimation, and limits the risk by a fixed pre-frozen computational resource quota, a minimum pre-frozen computational resource quota, or other predictable standard dimensions. After the request is completed, the actual stage of freezing the computational resource quota starts resolving the path from the root node of the original usage object of the target large model: the dot in the path represents the object field, the square brackets in quotation marks represent the field name, and the numeric square brackets represent the array subscript. If the target node is a number, or a string that can be converted to a number, its non-negative value is obtained; if the path does not exist, the node is empty, the node type is not supported, or the text cannot be converted to a number, the default value is used; if the read value is less than 0, it is treated as 0. In this example, the original quantity is 5,000.

[0176] After obtaining the original quantity, the system performs unit normalization according to the unit of measurement size, that is, divides 5,000 by 1,000 to obtain the expression variable value of 5. This value is simultaneously written into the custom indicator set for this usage, with both the variable name and dimension identifier. Thus, the measurement bin can be used for fractal dimension calculations or as a restricted expression variable, while the original field names of the target large model will not be included in any process or idempotent control code.

[0177] In the fractal dimension model, the basic computational resource limit for each item in the mapping relationship is 5 multiplied by 0.24, which is 1.2 credit units. Assuming that the time multiplier for this dimension in the first mapping relationship is 1.1, the model multiplier is 1.2, and the user group multiplier is 0.9, then the final actual computational resource limit for this item is 1.425600000 credit units. The measurement details simultaneously record each identifier, dimension, original quantity (5,000), measurement unit size (1,000), unit credit limit (0.24), time multiplier (1.1), model multiplier (1.2), user group multiplier (0.9), and final unit credit limit, allowing the value and calculation process of this extended indicator to be independently traceable.

[0178] In expression mode, administrators can reference corresponding dimensions in restricted expressions, such as multiplying the variable by 0.24 to form the basic computing resource quota. The expression evaluation module matches the variable with the corresponding time multiplier in the mapping relationship based on the "variable name - dimension identifier" mapping, and then adds the frozen model multiplier and user group multiplier. Regardless of whether a new version containing another path or another unit price is released during the request execution, all process calculations of this request use the same mapping relationship and do not read the currently effective configuration.

[0179] Once the actual computing resource quota is formed, if the actual computing resource quota is higher than the estimated computing resource quota, it is deducted from the source of the available, unfrozen initial computing resource quota. If it is lower than the estimated computing resource quota, it is restored according to the original path of the frozen path allocation record saved in the pre-failure stage. The repeated actual computing resource quota continues to be uniquely constrained by "message identifier + actual computing resource quota stage". When the actual computing resource quota and the unfrozen difference quota are concurrent, they continue to be constrained by the request final state condition update. Therefore, the new indicator only changes the standard usage set and the item-by-item computing resource quota, and does not change other processes or the final state competition mechanism.

[0180] The "no-code extension" in this embodiment has clear boundaries: when the new metric already exists in the original usage object of the target large model, can be obtained through supported object fields or array subscript paths, and can be directly converted into non-negative values, it can be accessed simply through configuration; when the target large model changes the authentication protocol or response transmission format at the same time, or when the new metric requires cross-field aggregation, decryption, signature verification, or complex semantic derivation, the adapter of the target large model still needs to complete the protocol parsing or derivation calculation first, and then hand the result over to the pricing module as a numerical field that can be read through a configurable path.

[0181] In one embodiment, if the original usage field is missing, negative, or the type cannot be converted, it is treated as zero or the default amount for the metering item, and the reason for the exception is recorded in the usage event. When the total cache creation amount is less than the sum of the five-minute and one-hour sub-dimensions, the total amount is corrected by the sum of the sub-dimensions to prevent the sub-dimensions from being truncated. When the metering version cannot be recovered, the model call fails and there is no valid usage, or the request is terminated before being sent to the supplier, the actual computing resource quota can be set to zero and the original path can be unfrozen. When some sources are successfully updated and subsequent sources fail during the allocation of computing resource quotas, the allocation records that have already been written are restored item by item in reverse to avoid only restoring errors. When the expression calculation result is negative, non-numeric, or exceeds the configuration limit, it is treated as zero, the limit value, or the billing failure strategy, and the abnormal computing resource quota is not directly written. When a late callback arrives after the final state has been completed, only an audit event is appended, and the computing resource quota is no longer changed; when manual adjustment is required, a new adjustment business identifier and an independent accounting stage are used.

[0182] In one embodiment, Figure 2 This is a second flowchart of a method for adjusting user computing resource quotas provided in Embodiment 1 of this disclosure, combined with... Figure 2 To further explain how to adjust user computing resource quotas:

[0183] Client or upper-layer application requests first enter the large model gateway for authentication and routing. Based on the estimated number of lexical units and the first mapping relationship, a preliminary estimated computational resource limit is calculated, and the required amount of this limit is frozen in the storage location. Then, based on the actual number of lexical units consumed by the target large model and the first mapping relationship, the actual computational resource limit is calculated. Reverse recovery is performed based on the difference between the preliminary and actual computational resource limits. Each stage has corresponding hierarchical idempotency (i.e., the constraints constituted by message identifiers and stage identifiers as described above) and terminal control. Furthermore, each process and time is recorded for subsequent auditing.

[0184] In one embodiment, Figure 3 This is a fourth flowchart of a method for adjusting user computing resource quotas provided in Embodiment 1 of this disclosure, combined with... Figure 3 To further explain how to adjust user computing resource quotas:

[0185] Determine the target large model and the corresponding first mapping relationship. Determine the estimated computing resource quota based on the first mapping relationship. If it is the first time to write the estimated computing resource quota, freeze the computing resource quota in the storage location. Otherwise, do not perform the freezing operation.

[0186] In one embodiment, Figure 4 This is a fourth flowchart of a method for adjusting user computing resource quotas provided in Embodiment 1 of this disclosure, combined with... Figure 4 To further explain how to adjust user computing resource quotas:

[0187] If the target large model is invoked and the metering response is successful, the normalized usage (and the lexical matching rules mentioned above) that matches it is used to eliminate cache duplication, and the first mapping relationship is reused to calculate the computing resource quota. If the metering response is successful, it indicates that stage j has been executed, and the subsequent unfreezing operation is triggered.

[0188] In one embodiment, the modules involved in the method for adjusting user computing resource quotas are specifically described.

[0189] The gateway routing and dual-model tracing module receives the preset large model identifier carried in the request and calls the routing strategy to obtain the target large model identifier. It then retrieves the first mapping relationship matching the target large model, and the request log simultaneously saves both the preset large model and the target large model. Subsequent processes carry both identifiers to explain the model mapping and metering source.

[0190] The heterogeneous usage normalization module stores the field information and semantic recognition rules returned by the target large model and extracts standard usage from the original response or streaming aggregation results. For target large models that "report cache usage separately," the system reconstructs ordinary input, cache reads, and cache creation into a unified total input caliber; for target large models where "the total input already includes cache usage," the system maintains its total. During metering, depending on whether a separate metering method is configured for cache creation, the system deducts the cache read usage and the cache creation usage that needs to be metered independently from the total input to obtain mutually exclusive ordinary input usage.

[0191] The cache creation sub-dimensions are corrected according to the following rules: when there are only total and hourly quantities, the five-minute quantity is taken as the non-negative value of "total minus hourly quantity"; when the sum of the sub-dimensions exceeds the total quantity, the total quantity is corrected by the sum of the sub-dimensions; when all extracted quantities are less than zero or cannot be parsed, they fall back to zero or the configured default quantity.

[0192] The first mapping relationship fixing module is used to fix the first mapping relationship corresponding to the target large model read in the estimation stage, so that the subsequent measurement methods are all based on the first preset large model.

[0193] The Fractal Dimension Measurement and Expression module is used for measurement according to different dimensions and expression methods.

[0194] The estimated usage and freezing module divides the number of input characters by the character token conversion factor and rounds up to obtain the estimated input token quantity. It then uses the maximum allowed output token quantity as the estimated output token quantity and adds the number of tools known before the request. If the fixed pre-frozen computing resource quota is greater than zero, the fixed pre-frozen computing resource quota is used; otherwise, the estimated computing resource quota corresponding to the estimated token quantity is used for freezing, and is constrained by the minimum pre-frozen computing resource quota.

[0195] The storage location allocation module supports basic source preferences for each storage location. API key limits are synchronously frozen or unfrozen as credential-level boundaries.

[0196] In the differential unfreezing and original path recovery module, when the actual computing resource quota is greater than the estimated actual computing resource quota, the module first continues to freeze from the corresponding storage location; the portion still not covered is written to the overdraft computing resource quota or the actual computing resource quota absorbed by the platform according to the strategy. When the actual computing resource quota is less than the estimated actual computing resource quota, the module first unfreezes from the corresponding storage location. If the request fails, the usage is invalid, or the frozen version cannot be recovered, the actual computing resource quota can be set to zero and the original path unfreezing can be performed.

[0197] The phase idempotency, final state control, and compensation modules separate the estimated freeze phase, actual freeze phase, and unfreeze phase into independent phases. A unique constraint of "message identifier + phase" prevents repeated changes to computing resource allocation within the same phase. Request logs use conditional updates to compare and exchange final states; only when the current state is estimated freeze complete can it be updated to actual freeze or unfreeze. Concurrency failures read the latest final state and return valid transactions. Compensation tasks scan requests that have been stuck in the actual freeze state for an extended period or have incomplete details, replaying the actual freeze or unfreeze based on the upstream call results, using the same phase key and final state conditions during replay.

[0198] In one embodiment, the above is a lexical calculation rule for the target large model determined by the field name, and can be stored as a vendor adaptation configuration, mapping script, or declarative rule; as long as the outputs of different large models can generate the same mutually exclusive standard usage, they are all equivalent implementations.

[0199] The first mapping relationship mentioned above can be persisted with the request log, or it can be written to an independent context table, reliable messages, immutable event streams, or signed tokens; as long as subsequent stages can recover according to the request identifier and the content cannot be overwritten by the current configuration.

[0200] Final state control can be achieved through relational database condition updates, version number locks, comparison and exchange in distributed consistent storage, or transactions with equivalent atomic semantics.

[0201] In addition to "message identifier + stage", the stage idempotent key can also be formed by combining tenant identifier, request identifier, business type and stage, as long as the same accounting stage can only correspond to one valid transaction.

[0202] Compensation can be triggered by timed scans, delayed messages, transaction message lookups, or manual replay, but the phase idempotent key, the first mapping relationship, and the final state condition must continue to be reused.

[0203] Time-sharing rules can be configured by week and time interval, or by holidays, regions, capacity levels or external cost signals to pre-generate multiplier snapshots; actual settlement still uses the snapshot frozen during pre-deduction.

[0204] Extended metrics can be extracted via JSONPath, or output via structured queries, protocol field mapping, or vendor adapters; as long as the metrics undergo unit normalization and dimension mapping before entering the pricing function.

[0205] In one embodiment, an implementation method for adjusting user computing resource quotas is specifically described.

[0206] The large model gateway generates globally unique or tenant-unique message identifiers to complete user, API key, and model parameter verification.

[0207] The routing strategy selects the target large model; if no model is available, it does not enter the freezing state and directly returns a routing failure.

[0208] Query the published first mapping relationship of the target large model. If no valid first mapping relationship is found, reject the request according to the configuration, and do not temporarily select other mapping relationships to calculate the actual computing resource quota after the response.

[0209] The first mapping relationship records time-sharing rules. Matching conditions include weekday, start and end times, and priority; rules spanning midnight can be split into two intervals or indicate a day spanning when the start time is greater than the end time.

[0210] Calculate the estimated resource limit. Before freezing, check idempotent keys; if unique key conflicts occur, read existing freeze information and context.

[0211] Perform a conditional atomic update on the source of computing resource quota for each storage location. If successful, immediately record the allocated and frozen computing resource quota. If any subsequent step fails, perform compensation recovery based on the successfully recorded allocation item.

[0212] The target large model is invoked. Synchronous responses directly obtain usage; streaming responses aggregate fragmented usage after the stream ends, is actively interrupted, or a timeout is acknowledged. Duplicate fragments within the same stream are deduplicated using fragment identifiers or cumulative sequence numbers.

[0213] Based on the type of the target large model and the original field structure, the input caliber is determined, cache sub-dimension supplementation, total amount correction and non-negativity are performed, and then the standard usage object is generated.

[0214] Read the metering items in the first mapping relationship and calculate the actual computing resource quota for each item; the expression can only access allowed usage variables and mathematical functions, and is not allowed to execute arbitrary code or access external resources.

[0215] The actual computing resource quota or unfreezing phase transaction is written, followed by a competition for the final state. The winner of the final state update processes the subsequent procedures, while the loser marks the ineffective state as failed or canceled and does not execute any subsequent actions.

[0216] After processing is complete, verify the conservation relationship between each process from each source; if the verification fails, record the exception and hand it over to the compensation task for processing.

[0217] In this embodiment, this disclosure can calculate the corresponding lexical quantity using matching lexical calculation rules for large models with different field information, thereby avoiding duplicate or missed lexical quantity measurement between ordinary input, cache reading, and cache creation. Before the large model estimates the estimated lexical quantity required to process a user's call request, the mapping relationship between the lexical quantity and the user's computing resource quota is fixed, so that the calculation basis of the estimated lexical quantity and the actual lexical quantity is the same, making the subsequent unfreezing process more reasonable. Furthermore, each step is recorded with item-by-item interpretable details for better traceability. The difference quota for the same call request is allocated to various storage locations according to source preference, the source-by-source allocation mapping is saved, and the API key quota is maintained synchronously, so that the difference quota can be unfrozen along the original frozen path. In addition, through a two-layer control of stage idempotent keys and final state condition updates, when various processes changing the user's initial computing resource heat are concurrent, the actual number of processes changing the initial computing resource quota does not exceed one. This establishes an audit chain that runs through the pre-defined model, the actual large model, the estimated computing resource quota, the actual computing resource quota, and the unfreezing of the difference quota. This allows for subsequent tracing of routes, versions, usage, multipliers, and quota sources based on records from each process, facilitating dispute resolution and cost analysis. It allows access to the target large model's metrics via declarative JSONPath (a query language specifically designed for finding and extracting specific information from JSON data), enabling the integration of extended metrics for various large models without modifying the main lifecycle code. Each stage of the same call request generates at most one valid computing resource quota transaction; the same request can only migrate from the "frozen" atomic state to the "unfrozen" final state; the difference quota does not exceed the per-source allocation quota recorded in the pre-deducted context; when the mapping relationship between lexical units and computing resource quota changes during request execution, the actual result is still determined by the mapping relationship established at the time of the request.

[0218] In this embodiment, a lifecycle control layer for computing resource quotas can be set between the request entry and response exit of the large model gateway. The request entry determines the target large model after routing and selects a first mapping relationship between published terms and computing resource quotas. This first mapping relationship is related to time-sharing rules, time multipliers for each dimension, model multipliers, user group multipliers, and user quota storage location preferences. Based on information available before the request, the system allocates frozen quotas for different storage locations according to the freezing priority / freezing ratio of the initial computing resource quota storage location, while simultaneously updating the API key to calculate the computing resource quota. The response exit does not remap the relationship but extracts and corrects the actual number of terms based on the field information returned by the target large model to generate mutually exclusive standard dimensions, reusing the aforementioned first mapping relationship to calculate the actual computing resource quota. The system compares the actual computing resource quota with the estimated computing resource quota and allocates the difference in quota for different storage locations for unfreezing according to the unfreezing priority / unfreezing ratio of the initial computing resource quota storage location. Furthermore, each stage of the actual operation on the user's initial computing resource quota (such as the freezing stage and the unfreezing stage) uses "message identifier + stage" as the idempotent key. The request log enters the unique final state from the frozen state through conditional updates, thus forming a closed loop of "routing - freezing - calling - normalization - unfreezing - recovery - auditing".

[0219] Example 2

[0220] Corresponding to the aforementioned embodiments of the method for adjusting user computing resource quotas, this disclosure also provides embodiments of a system for adjusting user computing resource quotas. Figure 5 This is a schematic diagram of a user computing resource quota adjustment system provided in Embodiment 2 of this disclosure. The user computing resource quota adjustment system 50 includes:

[0221] The response module 51, in response to receiving the call request from the target user, obtains the initial computing resource quota of the target user, estimates the estimated number of lexical units that the target large model needs to process to execute the call request; and obtains the first mapping relationship between the lexical units and the computing resource quota at the current moment.

[0222] The calculation module 52 is used to calculate the estimated computing resource quota required by the target user based on the estimated number of word elements and the first mapping relationship, and freeze the estimated computing resource quota in the initial computing resource quota;

[0223] The response module 51 also responds to the target large model completing the call request by obtaining the actual number of lexical units actually processed by the target large model in executing the call request;

[0224] The calculation module 52 is also used to calculate the actual amount of computing resources actually consumed by the target user based on the actual number of lexical units and the first mapping relationship;

[0225] The adjustment module 53 is used to adjust the initial computing resource quota after freezing based on the estimated computing resource quota and the actual computing resource quota.

[0226] In one embodiment, the adjustment system includes:

[0227] The acquisition module is used to retrieve field information returned by the target large model;

[0228] The acquisition module is also used to obtain word calculation rules that match the field information based on the field information;

[0229] The calculation module 52 is also used to calculate the actual number of lexical units based on the lexical calculation rules.

[0230] In one embodiment, the initial computing resource quota is stored in different storage locations; the computing module 52 includes:

[0231] The first acquisition unit is used to acquire the freezing priority of the storage location;

[0232] The first freezing unit is used to freeze the initial computing resource quota in the storage location sequentially based on the freezing priority, until the estimated computing resource quota is frozen.

[0233] or,

[0234] The second acquisition unit is used to acquire the freeze ratio corresponding to the storage location;

[0235] The second freezing unit is used to divide the estimated computing resource quota based on the freezing ratio, and then freeze the corresponding estimated computing resource quota at the storage location.

[0236] In one embodiment, the adjustment module 53 includes:

[0237] The calculation unit is used to calculate the difference between the estimated calculation resource amount and the actual calculation resource amount.

[0238] The response unit, in response to a positive difference amount, unfreezes the difference amount from the frozen estimated resource amount;

[0239] The response unit also responds to a negative difference amount by freezing the difference amount within the unfrozen initial computing resource amount.

[0240] In one embodiment, the response unit includes:

[0241] The first acquisition subunit is used to acquire the unfreezing priority of the storage location;

[0242] The first unfreeze subunit is used to unfreeze the initial computing resource quota in the storage location sequentially based on the unfreeze priority, until the difference quota is unfrozen.

[0243] or,

[0244] The second acquisition subunit is used to acquire the thawing ratio corresponding to the storage location;

[0245] The second thawing subunit is used to divide the difference amount based on the thawing ratio and then thaw the corresponding difference amount at the storage location.

[0246] In one embodiment, the invocation request includes input data, output data, and tool data required to process the input data;

[0247] Response module 51 includes:

[0248] The estimation unit is used to estimate the number of tokens that the target large model needs to process to execute the call request, based on the input data, output data, and tool data.

[0249] In one embodiment, the call request includes a preset large model specified by the target user; the response module 51 also responds that if the working state of the preset large model does not match the preset working state, then it traverses all large models and takes the large model that matches the preset working state as the target large model.

[0250] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs.

[0251] Example 3

[0252] Figure 6 This is a schematic diagram of the structure of an electronic device according to an example embodiment of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, it implements the user computing resource quota adjustment method described in any of the above embodiments. Figure 6 The electronic device 60 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0253] like Figure 6As shown, the electronic device 60 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 60 may include, but are not limited to: at least one processor 61, at least one memory 62, and a bus 63 connecting different system components (including memory 62 and processor 61).

[0254] Bus 63 includes a data bus, an address bus, and a control bus.

[0255] The memory 62 may include volatile memory, such as random access memory (RAM) 621 and / or cache memory 622, and may further include read-only memory (ROM) 623.

[0256] The memory 62 may also include a program tool 625 (or utility) having a set (at least one) program module 624, such program module 624 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0257] The processor 61 executes various functional applications and data processing by running computer programs stored in the memory 62, such as the user computing resource quota adjustment method provided in any of the above embodiments.

[0258] Electronic device 60 can also communicate with one or more external devices 64 (e.g., keyboard, pointing device, etc.). This communication can be performed through input / output (I / O) interface 65. Furthermore, electronic device 60 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 66. As shown, network adapter 66 communicates with other modules of electronic device 60 via bus 63. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 60, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0259] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0260] Example 4

[0261] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the user computing resource quota adjustment method provided in any of the above embodiments.

[0262] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0263] Example 5

[0264] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the user computing resource quota adjustment method described in any of the preceding embodiments.

[0265] The program code for executing the computer program product of this disclosure can be written in any combination of one or more programming languages, and the program code can be executed entirely on a user device, partially on a user device, as a stand-alone software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0266] While specific embodiments of this disclosure have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of this disclosure is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of this disclosure, but all such changes and modifications fall within the scope of protection of this disclosure.

Claims

1. A method for adjusting user computing resource quotas, characterized in that, The method for adjusting user computing resource quotas is applied to the large model gateway, and the adjustment method includes: In response to receiving a call request from a target user, the system obtains the initial computing resource quota of the target user, estimates the estimated number of lexical units that the target large model needs to process to execute the call request, and obtains the first mapping relationship between lexical units and computing resource quota at the current moment. Based on the estimated number of lexical units and the first mapping relationship, the estimated computing resource quota required by the target user is calculated, and the estimated computing resource quota is frozen in the initial computing resource quota. In response to the completion of the call request by the target large model, obtain the actual number of lexical units actually processed by the target large model in executing the call request; Based on the actual number of lexical units and the first mapping relationship, calculate the actual amount of computing resources consumed by the target user; Based on the estimated computing resource limit and the actual computing resource limit, the initial computing resource limit after the freeze process is adjusted.

2. The adjustment method as described in claim 1, characterized in that, The step of obtaining the actual number of lexical units actually processed by the target large model in response to the completion of the call request includes: Obtain the field information returned by the target large model; Based on the field information, obtain the word calculation rules that match the field information; Based on the lexical calculation rules, the actual number of lexical units is calculated.

3. The adjustment method as described in claim 1, characterized in that, The initial computing resource quota is stored in different storage locations; the step of freezing the estimated computing resource quota in the initial computing resource quota includes: Obtain the freeze priority of the storage location; Based on the freezing priority, the initial computing resource quota on the storage location is frozen sequentially until the estimated computing resource quota is frozen; or, Obtain the freeze ratio corresponding to the storage location; After dividing the estimated computing resource quota based on the freezing ratio, the estimated computing resource quota corresponding to the storage location is frozen respectively.

4. The adjustment method as described in claim 3, characterized in that, The step of adjusting the initial computing resource quota after freezing based on the estimated computing resource quota and the actual computing resource quota includes: Calculate the difference between the estimated resource amount and the actual calculated resource amount; In response to the difference being a positive number, the difference is unfrozen from the frozen estimated resource amount; In response to the difference amount being negative, the difference amount is frozen within the unfrozen initial computing resource amount.

5. The adjustment method as described in claim 4, characterized in that, The step of unfreezing the difference amount from the frozen estimated resource amount in response to the difference amount being positive includes: Obtain the thawing priority of the storage location; Based on the unfreezing priority, the initial computing resource quota in the storage location is unfrozen sequentially until the difference quota is unfrozen; or, Obtain the thawing ratio corresponding to the storage location; After dividing the difference amount based on the unfreezing ratio, the difference amount corresponding to the storage location is unfrozen respectively.

6. The adjustment method as described in claim 1, characterized in that, The invocation request includes input data, output data, and tool data required to process the input data; In response to receiving a call request from a target user, the system obtains the target user's initial computing resource quota and estimates the estimated number of lexical units that the target large model needs to process to execute the call request. The steps for obtaining the first mapping relationship between the lexical units at the current moment and the computing resource quota include: Based on the input data, the output data, and the tool data, the estimated number of lexical units required for the target large model to execute the invocation request is estimated.

7. The adjustment method as described in claim 1, characterized in that, The invocation request includes a preset large model specified by the target user; Before the step of the large model for estimating the target target quantity to process the required number of estimated lexical units for the invocation request is executed, the following is included: If the working state of the preset large model does not match the preset working state, then all large models are traversed, and the large model that matches the preset working state is taken as the target large model.

8. A system for adjusting user computing resource quotas, characterized in that, The method for adjusting user computing resource quotas is applied to the large model gateway, and the adjustment system includes: The response module, in response to receiving a call request from a target user, obtains the initial computing resource quota of the target user, estimates the estimated number of lexical units that the target large model needs to process to execute the call request, and obtains the first mapping relationship between lexical units and computing resource quota at the current moment; The calculation module is used to calculate the estimated computing resource quota required by the target user based on the estimated number of lexical elements and the first mapping relationship, and freeze the estimated computing resource quota in the initial computing resource quota; The response module also responds to the completion of the call request by the target large model by obtaining the actual number of lexical units actually processed by the target large model in executing the call request. The calculation module is also used to calculate the actual amount of computing resources actually consumed by the target user based on the actual number of lexical units and the first mapping relationship; The adjustment module is used to adjust the initial computing resource quota after freezing based on the estimated computing resource quota and the actual computing resource quota.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and for running on the processor, characterized in that, When the processor executes the computer program, it implements the method for adjusting the user computing resource quota as described in any one of claims 1 to 8.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method for adjusting the user computing resource quota as described in any one of claims 1 to 8.

Images