Resource processing method and device for large model, and electronic device

Abstract

The present disclosure provides a resource processing method and device for a large model, electronic equipment and readable storage medium, relating to the technical field of artificial intelligence such as large model and deep learning. The resource processing method for the large model comprises: determining a target traffic corresponding to a current time of the large model; obtaining a traffic threshold and a traffic load value according to the current time; in response to determining that the target traffic is greater than the traffic threshold, obtaining a target second type resource according to the target traffic, the traffic load value, a first type resource and an already effective second type resource; and performing a resource acquisition operation or a resource release operation according to the target second type resource. The present disclosure can achieve the purpose of self-adaptive adjustment of the resources used by the large model while ensuring stable operation of the large model, thereby reducing the resource use cost of the large model.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to the fields of artificial intelligence technology such as large models and deep learning. It provides a method, apparatus, electronic device, and readable storage medium for resource processing of large models. Background Technology

[0002] With the advent of the era of large-scale models, native AI (Artificial Intelligence) applications have surged. This explosive growth in AI-native applications inevitably leads to increased traffic, which in turn increases the resource demands of large-scale models. Therefore, given the rising inference costs of large-scale models and the continuous growth in traffic, how to rationally manage the resources used by large-scale models is a pressing issue that needs to be addressed. Summary of the Invention

[0003] According to a first aspect of this disclosure, a resource processing method for a large model is provided, comprising: determining a target traffic volume corresponding to the current time of the large model; obtaining a traffic threshold and a traffic load value based on the current time; in response to determining that the target traffic volume is greater than the traffic threshold, obtaining a target second type of resource based on the target traffic volume, the traffic load value, a first type of resource, and an effective second type of resource; and performing a resource acquisition operation or a resource release operation based on the target second type of resource.

[0004] According to a second aspect of this disclosure, a resource processing apparatus for a large model is provided, comprising: a determining unit for determining a target traffic flow corresponding to the current time of the large model; an acquiring unit for acquiring a traffic threshold and a traffic load value based on the current time; a responding unit for obtaining a target second type resource based on the target traffic flow, the traffic load value, a first type of resource, and an active second type of resource in response to determining that the target traffic flow is greater than the traffic threshold; and a processing unit for performing a resource acquisition operation or a resource release operation based on the target second type resource.

[0005] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0006] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described above.

[0007] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described above.

[0008] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0010] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0011] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure;

[0012] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure;

[0013] Figure 4 This is a schematic diagram according to the fourth embodiment of the present disclosure;

[0014] Figure 5 This is a schematic diagram according to the fifth embodiment of the present disclosure;

[0015] Figure 6 This is a block diagram of an electronic device used to implement the resource processing method for large models according to embodiments of the present disclosure. Detailed Implementation

[0016] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and mechanisms are omitted in the following description.

[0017] Figure 1 This is a schematic diagram based on the first embodiment of this disclosure. (See diagram below.) Figure 1 As shown, the resource processing method for large models in this embodiment specifically includes the following steps:

[0018] S101. Determine the target flow rate corresponding to the current moment in the large model;

[0019] S102. Based on the current time, obtain the traffic threshold and traffic load value;

[0020] S103. In response to determining that the target traffic is greater than the traffic threshold, a target second type of resource is obtained based on the target traffic, the traffic load value, the first type of resource, and the effective second type of resource;

[0021] S104. Based on the target second type of resource, perform a resource acquisition operation or a resource release operation.

[0022] The resource processing method for large models in this embodiment first obtains the traffic threshold and traffic load value based on the current time after determining the target traffic of the large model. Then, in response to determining that the target traffic is greater than the traffic threshold, the target second type of resource is obtained based on the target traffic, traffic load value, first type of resource, and the effective second type of resource. Finally, a resource acquisition operation or a resource release operation is performed based on the target second type of resource. In addition to using the target traffic, first type of resource, and effective second type of resource, this embodiment also uses the traffic load value obtained from the current time to obtain the target second type of resource. This fully considers the traffic change pattern of the large model at different times, which can improve the accuracy of obtaining the target second type of resource and make the execution of resource processing operations more accurate. Under the premise of ensuring the stable operation of the large model, it achieves the purpose of adaptive adjustment of the resources used by the large model, thereby improving the resource utilization rate of the large model and reducing the resource usage cost of the large model.

[0023] In this embodiment, the large model is a machine learning model with a very large number of parameters, such as a multimodal large model or a large language model. The large model generates an output token based on the input token. The token (i.e., the sequence unit) can be a character, word, image patch or audio frame.

[0024] In this embodiment, the traffic is the TPM (Token Per Minute) of the large model, i.e., minute-level traffic; where TPM includes the number of tokens input into the large model per minute and the number of tokens output by the large model per minute.

[0025] In this embodiment, resources refer to the token usage of the large model (i.e., the number of tokens that the large model can use). Token usage includes the number of token inputs and the number of token outputs. When the token usage is less than the TPM of the large model (i.e., the large model has insufficient resources), resources need to be acquired (i.e., additional token usage is acquired). When the token usage is greater than the TPM of the large model (i.e., the large model has surplus resources), resources need to be released (i.e., excess token usage is released).

[0026] In this embodiment, the first type of resource is the token usage corresponding to the first type, and the second type of resource is the token usage corresponding to the second type; wherein, the first type is a prepaid type (e.g., a monthly package type with a monthly prepaid model), and the second type is a postpaid type (e.g., an hourly package type with an hourly pay-per-hour model), that is, the first type of resource is the token usage obtained through a prepaid method, and the second type of resource is the token usage obtained through a postpaid method.

[0027] Specifically, in this embodiment, when executing S101 to determine the target traffic corresponding to the current time of the large model, the implementation method can be as follows: obtain the current traffic corresponding to the current time of the large model. In this embodiment, the current traffic obtained is the TPM of the large model corresponding to the current time; determine the first historical time according to the current time, obtain the first historical traffic corresponding to the first historical time of the large model. In this embodiment, one or more first historical times located before the current time can be determined according to a first preset number. In this embodiment, the first historical time is a minute-level time, and the obtained first historical traffic is the TPM of the large model corresponding to the first historical time; obtain the target traffic according to the current traffic and the first historical traffic. In this embodiment, the target traffic determined is the average TPM or the maximum TPM of the large model within a certain period corresponding to the current time.

[0028] In other words, in addition to using the current traffic corresponding to the current moment of the large model, this embodiment also uses the first historical traffic corresponding to the first historical moment of the large model to jointly obtain the target traffic, so that the obtained target traffic can reflect the processing of tokens by the large model within a certain period of time, thereby improving the accuracy of the obtained target traffic.

[0029] For example, if the current time is 10:12 on July 23, 2025, and the first preset number is 3, then the first historical time determined by S101 in this embodiment is 10:11, 10:10, and 10:09. If the current flow corresponding to the current time is flow 1, and the first historical flow corresponding to the first historical time is flow 2, flow 3, and flow 4 respectively, then when executing S101 in this embodiment, the average value between flow 1, flow 2, flow 3, and flow 4 can be used as the target flow, or the maximum flow selected from flow 1, flow 2, flow 3, and flow 4 can be used as the target flow.

[0030] In this embodiment, when executing S101, the current traffic corresponding to the large model obtained at the current moment can also be directly determined as the target traffic.

[0031] In this embodiment, after executing S101 to determine the target traffic corresponding to the current time of the large model, S102 is executed to obtain the traffic threshold and traffic load value according to the current time.

[0032] In this embodiment, when executing S102, different methods can be used to obtain the traffic threshold and traffic load value based on the judgment result of whether the current time is an adjustment time. In this embodiment, the traffic threshold obtained is the prepaid lined traffic, which is used to compare with the target traffic of the large model at the current time, so as to determine whether the prepaid lined traffic can meet the processing needs of the large model at the current time based on the comparison result. The traffic load value obtained in this embodiment is used to combine with the target traffic at the current time to obtain the predicted traffic, so as to further obtain the target second type of resource for determining what kind of resource processing operation to perform based on the obtained predicted traffic.

[0033] Specifically, in this embodiment, when executing S102 to obtain the traffic threshold and traffic load value based on the current time, the implementation method can be as follows: in response to determining that the current time is the adjustment time, a historical traffic set is obtained based on the current time. The historical traffic set obtained in this embodiment includes multiple second historical traffic sets, which are hourly traffic sets. Each second historical traffic set corresponds to the maximum TPM in each hour of a second historical time set in the large model. Based on the obtained historical traffic set, the traffic threshold and traffic load value are obtained.

[0034] In other words, in this embodiment, when the current time is determined to be the adjustment time, the traffic threshold and traffic load value are obtained in real time by acquiring the historical traffic set corresponding to the current time, so that the obtained traffic threshold and traffic load value have high timeliness.

[0035] In addition, when executing S102 in this embodiment, besides obtaining the traffic threshold and traffic load value based on the historical traffic set, redundant traffic can also be obtained at the same time; wherein, the redundant traffic obtained in this embodiment is used to combine the target traffic and traffic load value corresponding to the current moment to obtain the predicted traffic.

[0036] It is understandable that if the current time is the adjustment time, after executing S102 to obtain the traffic threshold, traffic load value and redundant traffic based on the historical traffic set, this embodiment can also use the latest obtained traffic threshold, traffic load value and redundant traffic to update the traffic threshold, traffic load value and redundant traffic obtained in the previous adjustment time.

[0037] In this embodiment, when executing S102 to determine whether the current time is an adjustment time, the previous adjustment time (i.e., the historical adjustment time) can be obtained first. Then, if the time interval (e.g., number of days) between the current time and the previous adjustment time is greater than or equal to a preset time interval, the current time is determined to be an adjustment time. The preset time interval is set in advance according to actual needs and can be 1 day, 2 days, etc.

[0038] Specifically, in this embodiment, when executing S102 to obtain the historical traffic set based on the current time, the implementation method can be as follows: determine the second historical time based on the current time. In this embodiment, one or more second historical times located before the current time can be determined according to the second preset number. The second historical time is a day-level time, and the second preset number can be 1 (i.e., one day), 7 (i.e., one week), or 30 (i.e., one month); for each hour in the second historical time, select the largest first historical traffic from all the first historical traffic (the first historical traffic is the TPM corresponding to each minute of the large model) for that hour, and use it as the second historical traffic for that hour. That is, the obtained second historical traffic is hour-level historical traffic; obtain the historical traffic set based on the multiple second historical traffic.

[0039] In other words, since the traffic obtained in this embodiment is minute-level traffic (i.e., TPM), when obtaining the historical traffic set based on the current time, this embodiment converts the minute-level traffic into hour-level traffic, thereby obtaining the historical traffic set based on the second historical traffic corresponding to different hours. This can greatly reduce the number of historical traffic included in the historical traffic set, thereby improving the efficiency of obtaining the traffic threshold and traffic load value based on the historical traffic set.

[0040] For example, if the current time is 10:12 on July 23, 2025, and the determined second historical time is July 22, 2025, for one hour of that second historical time, such as the hour from 10:00 to 11:00 on July 22, 2025, in this embodiment, when executing S102, the largest first historical flow can be selected from the 60 first historical flows (i.e., 60 TPMs) included in the hour from "10:00 on July 22, 2025" to "11:00 on July 22, 2025". The first historical traffic (the largest TPM out of 60 TPMs) is selected as the hourly traffic (i.e., the second historical traffic) corresponding to the hour "10:00 to 11:00". This allows for the acquisition of 24 second historical traffic for each day's second historical moment, thus achieving the conversion between the first and second historical traffic. This ensures that each second historical moment includes only 24 second historical traffic, rather than including the first historical traffic for every minute of each hour, thereby greatly reducing the number of historical traffic included in the acquired historical traffic set.

[0041] It is understood that when this embodiment executes S102, if it is determined that the current time is not an adjustment time, the flow threshold and flow load value obtained at the previous adjustment time are obtained. Furthermore, the redundant flow obtained at the previous adjustment time can also be obtained.

[0042] In other words, this embodiment stores the traffic threshold, traffic load value, and redundant traffic obtained at each adjustment time, so that when the current time is not an adjustment time, the stored traffic threshold, traffic load value, and redundant traffic can be used for subsequent processing.

[0043] In this embodiment, when executing S102 to obtain the traffic threshold based on the historical traffic set, multiple resource costs can be obtained for each second historical traffic in the historical traffic set when each second historical traffic is used as a preset traffic threshold. Then, the second historical traffic corresponding to the minimum resource cost among the multiple resource costs is used as the traffic threshold.

[0044] In this embodiment, when executing S102, the resource cost can be obtained using the following calculation formula:

[0045]

[0046] In the above calculation formula: COS is the resource cost; x represents the second historical traffic selected from the historical traffic set; M1 represents the unit price of the first type of resource corresponding to 10000 TPM; H1 represents the unit price of the second type of resource corresponding to 10000 TPM; T i The second historical flow that is greater than x in the historical flow concentration.

[0047] In other words, this embodiment can obtain the resource costs corresponding to different second historical flows in the historical traffic set, and then use the second historical flow corresponding to the minimum resource cost as the traffic threshold to achieve the purpose of obtaining the traffic threshold based on the historical traffic set.

[0048] In this embodiment, after executing S102 to obtain the traffic threshold and traffic load value based on the current time, S103 is executed in response to determining that the target traffic is greater than the traffic threshold, and the target second type resource is obtained based on the target traffic, traffic load value, first type resource and effective second type resource.

[0049] In this embodiment, the effective second type of resource is the token usage of the corresponding second type that has been acquired at the current time. The effective second type of resource can be 0 or a value greater than 0; the first type of resource can be the token usage of the corresponding traffic threshold.

[0050] Specifically, in this embodiment, when executing S103 to obtain the target second type resource based on the target traffic, traffic load value, first type resource and effective second type resource, the implementation method can be as follows: obtain the predicted traffic based on the multiplication result between the target traffic and the traffic load value; obtain the target second type resource based on the subtraction result between the obtained predicted traffic and the first type resource and the effective second type resource.

[0051] In other words, since the traffic load value is obtained based on the historical traffic set, it can reflect the changing pattern of historical traffic at different times. Therefore, this embodiment obtains the predicted traffic for determining the target second type of resource based on the two pieces of information: target traffic and traffic load value. Compared with the prior art, which does not consider traffic changes and directly uses the target traffic as the predicted traffic, this embodiment can effectively avoid the risk of traffic limiting caused by uncertain factors, thereby improving the accuracy of the obtained predicted traffic and the target second type of resource, and further improving the accuracy of subsequent adaptive adjustment of resources.

[0052] In addition, if redundant traffic is obtained when S102 is executed in this embodiment, when S103 is executed to obtain the predicted traffic based on the multiplication result between the target traffic and the traffic load value, this embodiment can first obtain the multiplication result between the target traffic and the traffic load value, and then obtain the predicted traffic based on the addition result between the obtained multiplication result and the redundant traffic.

[0053] In other words, in addition to the target traffic and traffic load value, this embodiment also combines redundant traffic to obtain the predicted traffic, thereby ensuring that the obtained predicted traffic can adapt to more scenarios (such as traffic surge scenarios, traffic drop scenarios, traffic continuous growth scenarios, etc.), thereby further improving the accuracy of the obtained predicted traffic and the accuracy of adaptive adjustment of resources.

[0054] In this embodiment, when executing S103 to obtain the target second type resource based on the target traffic, traffic load value, first type resource, and effective second type resource, the following calculation formula can be used:

[0055] H need =(load×x+c)-MH

[0056] In the above calculation formula: H need The target is the second type of resource; x is the target traffic; load is the traffic load value; c is the redundant traffic; M is the first type of resource; H is the second type of resource that has been activated.

[0057] In this embodiment, after obtaining the target second type of resource by executing S103, S104 is executed to perform a resource acquisition operation or a resource release operation based on the target second type of resource.

[0058] In this embodiment, when executing S104, resource processing is achieved by adopting a corresponding resource adaptive adjustment method based on the comparison result between the obtained target second type resource and 0.

[0059] Specifically, in this embodiment, when executing S104, if in response to determining that the target second type of resource is greater than 0, a resource acquisition operation is performed to obtain the token usage of the corresponding target second type of resource, so that even when the TPM of the large model is high at the current moment, there is sufficient token usage for inference, thereby improving the running stability of the large model.

[0060] In this embodiment, when executing S104, if it is determined that the target second type of resource is less than 0, a resource release operation is performed to release the token usage of the absolute value of the target second type of resource, thereby avoiding resource waste caused by the large model occupying the second type of resource and reducing the resource usage cost of the large model.

[0061] In other words, this embodiment obtains the target second type of resources based on the traffic threshold, traffic load value, and redundant traffic obtained from the historical traffic prediction analysis of the large model, as well as the target traffic of the large model determined at the current moment. Then, it adaptively adjusts the resources based on the obtained target second type of resources, so as to achieve the purpose of elastic scaling of the resources (especially the second type of resources) used by the large model through real-time feedback. This embodiment can naturally solve the problem of insufficient acquisition of second type of resources caused by inaccurate traffic prediction, and it also has strong robustness to complex scenarios such as sudden increase or decrease in traffic and continuous increase in traffic, thereby helping the business to reduce costs while ensuring business availability.

[0062] Figure 2 This is a schematic diagram according to the second embodiment of this disclosure. (See diagram below.) Figure 2 As shown in the figure, when executing S102 "obtain the traffic threshold based on the acquired historical traffic set", this embodiment can be implemented in the following way:

[0063] S201. Select a second historical traffic from the historical traffic set as a candidate traffic;

[0064] S202. In response to determining that the candidate flow is greater than the initial flow, the initial flow is set as the candidate flow;

[0065] S203. Based on the initial traffic and all the second historical traffic in the historical traffic set, obtain the target candidate resources;

[0066] S204. In response to determining that the target candidate resource is greater than the initial resource, proceed to the step of selecting a second historical traffic from the historical traffic set, and repeat this process until it is determined that the target candidate resource is less than the initial resource, and use the candidate traffic at this time as the traffic threshold.

[0067] In other words, this embodiment obtains target candidate resources based on the second historical traffic that meets the corresponding conditions selected from the historical traffic set. Then, the second historical traffic selected when the obtained target candidate resources are less than the initial resources is determined as the traffic threshold, thereby improving the accuracy of the determined traffic threshold and the efficiency of the traffic threshold determination.

[0068] It is understood that the candidate traffic obtained by executing S201 in this embodiment is a second historical traffic selected from the historical traffic set each time. For example, different second historical traffic can be selected from the historical traffic set in chronological order. The initial traffic used by executing S202 in this embodiment is a pre-set traffic, such as 0, during the first processing. Otherwise, the initial traffic is a candidate traffic selected from the historical traffic set that is greater than the initial traffic during the previous processing. The initial resource in this embodiment is the pre-set optimal resource.

[0069] In this embodiment, when S202 is executed to set the initial traffic as candidate traffic, the candidate traffic selected this time is used to update the current initial traffic.

[0070] In this embodiment, when executing S203, a first type of candidate resource can be obtained firstly based on the initial traffic (for example, the product of the initial traffic and the number of second historical traffic in the historical traffic set can be used as the first type of candidate resource, and the initial traffic can be regarded as prepaid line traffic). Then, a second historical traffic greater than the initial traffic is obtained from the historical traffic set. Then, a second type of candidate resource is obtained based on the traffic difference between each obtained second historical traffic and the initial traffic (the second type of candidate resource can be regarded as the corresponding postpaid resource, and the traffic difference is the first type of candidate resource). Finally, the target candidate resource is obtained based on the sum of the first type of candidate resource and the second type of candidate resource.

[0071] In other words, this embodiment obtains a first type of candidate resource (the first type of candidate resource is the prepaid token usage of the large model at multiple second historical moments) based on the determined number of initial traffic and second historical traffic. On the other hand, it obtains a second type of candidate resource (the second type of candidate resource is the postpaid token usage of the large model at multiple second historical moments) based on the traffic difference between the determined initial traffic and a specific second historical traffic (the specific second historical traffic is the second historical traffic that is greater than the initial traffic). This achieves the goal of using the initial traffic as prepaid lined traffic to obtain the target candidate resources corresponding to multiple second historical moments of the large model, thereby improving the accuracy of the obtained target candidate resources.

[0072] It is understandable that, when the initial resource corresponds to the cost of token usage, this embodiment can also obtain the unit price of token usage corresponding to the first type and the unit price of token usage corresponding to the second type when executing S203. Then, based on the obtained unit price and the candidate resources of the corresponding type, the cost corresponding to the target candidate resource can be obtained. Then, based on the comparison result between the costs, it can be determined whether the target candidate resource is greater than the initial resource.

[0073] Figure 3 This is a schematic diagram based on a third embodiment of the present disclosure. (See diagram below.) Figure 3 As shown in the figure, this embodiment illustrates a specific flowchart for obtaining a traffic threshold based on a historical traffic set: S301, Obtain the historical traffic set; S302, Obtain the initial traffic and initial resources, with the initial traffic being 0 during the first processing; S303, Select a second historical traffic from the historical traffic set as a candidate traffic; S304, Determine whether the candidate traffic is greater than the initial traffic. If so, set the initial traffic as the candidate traffic; otherwise, proceed to S303 to re-obtain the second historical traffic; S305, Obtain a first type of candidate resource based on the initial traffic, i.e., treat the initial traffic as prepaid (e.g., monthly package) lined traffic, and then obtain the first type of candidate resource based on the prepaid lined traffic and the number of second historical traffic in the historical traffic set; S30 6. Select a second historical traffic volume greater than the initial traffic volume from the historical traffic set. Obtain a second type of candidate resource based on the traffic difference between the selected second historical traffic volume and the initial traffic volume. That is, obtain the second type of candidate resource based on the traffic difference between the second historical traffic volume exceeding the prepaid limit traffic volume and the prepaid limit traffic volume. S307. Obtain the target candidate resource based on the first type of candidate resource and the second type of candidate resource. S308. Determine whether the target candidate resource is greater than the initial resource. If so, proceed to S303 to re-acquire the second historical traffic volume for the next processing. Otherwise, proceed to S309. S309. Output the current candidate traffic volume as the traffic threshold. In this embodiment, the current first type of candidate resource and the second type of candidate resource can also be output simultaneously.

[0074] In other words, this embodiment determines target candidate resources based on a peak elasticity strategy (i.e., traffic during peak periods is acquired using a postpaid method, while traffic during normal periods is acquired using a prepaid method). This can improve the accuracy of the determined target candidate resources and further reduce resource usage costs compared to other strategies (such as peak strategies or postpaid strategies).

[0075] Figure 4 This is a schematic diagram according to the fourth embodiment of this disclosure. (See diagram below.) Figure 4As shown in the figure, when executing S102 "obtain the traffic load value based on the acquired historical traffic set", this embodiment can be implemented in the following way:

[0076] S401. Obtain a set of flow difference values ​​based on the flow difference values ​​between two adjacent second historical flows in the historical flow set;

[0077] S402. Based on the set of flow difference values, a flow change curve is obtained by fitting.

[0078] S403. Obtain the flow load value based on the slope of the flow rate change curve.

[0079] In other words, this embodiment fits the flow change curve based on the flow difference set obtained from the historical flow set, and then obtains the flow load value based on the slope of the fitted flow change curve. This allows the obtained flow load value to reflect the historical flow change pattern, thereby improving the accuracy of the obtained flow load value and thus improving the accuracy of the predicted flow obtained based on the flow load value.

[0080] In this embodiment, when executing S401, two adjacent second historical flows can be obtained first, and then the flow difference between the two second historical flows can be obtained based on the subtraction result between the latter second historical flow and the former second historical flow.

[0081] In this embodiment, when executing S402, a linear straight line can be fitted based on multiple flow differences in the flow difference set, and the resulting fitting result can be used as the flow change curve.

[0082] In addition, after obtaining the flow difference set by executing S401, this embodiment may also include the following: determining the 99th percentile value of the flow difference based on the multiple flow differences in the obtained flow difference set. In this embodiment, the 99th percentile value of the flow difference indicates that 99% of the flow differences in the flow difference set are less than or equal to this value; and using the determined 99th percentile value of the flow difference as redundant flow, which can be used as the Y-axis intercept in the flow change curve.

[0083] In other words, based on the obtained traffic difference set, this embodiment can not only obtain the traffic load value, but also use the 99th percentile value of the traffic difference as redundant traffic, which can improve the accuracy of the obtained redundant traffic.

[0084] Figure 5 This is a schematic diagram according to the fifth embodiment of this disclosure. (See diagram below.) Figure 5 As shown, the resource processing device 500 for large models in this embodiment includes:

[0085] Unit 501 is used to determine the target flow rate of the large model at the current moment;

[0086] The acquisition unit 502 is used to acquire the traffic threshold and the traffic load value based on the current time.

[0087] Response unit 503 is configured to, in response to determining that the target traffic is greater than the traffic threshold, obtain the target second type resource based on the target traffic, the traffic load value, the first type resource, and the effective second type resource;

[0088] Processing unit 504 is used to perform a resource acquisition operation or a resource release operation based on the target second type of resource.

[0089] Specifically, when determining the target traffic corresponding to the current moment of the large model, the determining unit 501 can adopt the following implementation method: obtain the current traffic corresponding to the current moment of the large model; determine the first historical moment based on the current moment, and obtain the first historical traffic corresponding to the first historical moment of the large model; obtain the target traffic based on the obtained current traffic and the first historical traffic.

[0090] In other words, in addition to using the current traffic corresponding to the current moment of the large model, the determining unit 501 will also use the first historical traffic corresponding to the first historical moment of the large model to jointly obtain the target traffic, so that the obtained target traffic can reflect the processing of tokens by the large model within a certain period of time, thereby improving the accuracy of the obtained target traffic.

[0091] The determination unit 501 can also directly determine the target flow by corresponding the large model to the current flow at the current moment.

[0092] In this embodiment, after the determining unit 501 determines the target traffic corresponding to the current time of the large model, the obtaining unit 502 obtains the traffic threshold and traffic load value according to the current time.

[0093] The acquisition unit 502 can obtain the traffic threshold and traffic load value in different ways based on the judgment result of whether the current time is the adjustment time.

[0094] Specifically, when the acquisition unit 502 acquires the traffic threshold and traffic load value based on the current time, it can adopt the following implementation method: in response to determining that the current time is the adjustment time, it acquires the historical traffic set based on the current time; and obtains the traffic threshold and traffic load value based on the acquired historical traffic set.

[0095] In other words, when the current time is determined to be the adjustment time, the acquisition unit 502 obtains the traffic threshold and traffic load value in real time by acquiring the historical traffic set corresponding to the current time, so that the obtained traffic threshold and traffic load value have high timeliness.

[0096] In addition to obtaining the traffic threshold and traffic load value based on the historical traffic set, the acquisition unit 502 can also obtain the redundant traffic at the same time.

[0097] It is understandable that if the current time is the adjustment time, after obtaining the traffic threshold, traffic load value and redundant traffic based on the obtained historical traffic set, the acquisition unit 502 can also use the latest obtained traffic threshold, traffic load value and redundant traffic to update the traffic threshold, traffic load value and redundant traffic obtained in the previous adjustment time.

[0098] When determining whether the current time is an adjustment time, the acquisition unit 502 can first acquire the previous adjustment time, and then determine the current time as an adjustment time if the time interval (e.g., number of days) between the current time and the previous adjustment time is greater than or equal to a preset time interval. The preset time interval is set in advance according to actual needs and can be 1 day, 2 days, etc.

[0099] When the acquisition unit 502 acquires the historical traffic set based on the current time, it can adopt the following implementation method: determine the second historical time based on the current time; for each hour in the second historical time, select the largest first historical traffic from all the first historical traffic corresponding to that hour as the second historical traffic corresponding to that hour; acquire the historical traffic set based on multiple second historical traffic.

[0100] In other words, since the traffic obtained in this embodiment is minute-level traffic (i.e., TPM), when the acquisition unit 502 acquires the historical traffic set according to the current time, it converts the minute-level traffic into hour-level traffic, thereby obtaining the historical traffic set according to the second historical traffic corresponding to different hours. This can greatly reduce the number of historical traffic included in the historical traffic set, thereby improving the efficiency of obtaining the traffic threshold and traffic load value according to the historical traffic set.

[0101] Understandably, if in response to determining that the current time is not an adjustment time, the acquisition unit 502 acquires the flow threshold and flow load value obtained at the previous adjustment time. Furthermore, it can also acquire the redundant flow obtained at the previous adjustment time.

[0102] In other words, this embodiment stores the traffic threshold, traffic load value, and redundant traffic obtained at each adjustment time, so that when the current time is not an adjustment time, the stored traffic threshold, traffic load value, and redundant traffic can be used for subsequent processing.

[0103] When the acquisition unit 502 obtains the traffic threshold based on the acquired historical traffic set, it can adopt the following implementation method: select a second historical traffic from the historical traffic set as a candidate traffic; in response to determining that the candidate traffic is greater than the initial traffic, set the initial traffic as the candidate traffic; obtain the target candidate resource based on the initial traffic and all the second historical traffic in the historical traffic set; in response to determining that the target candidate resource is greater than the initial resource, proceed to the step of selecting a second historical traffic from the historical traffic set, and repeat this process until it is determined that the target candidate resource is less than the initial resource, and use the candidate traffic at this time as the traffic threshold.

[0104] In other words, the acquisition unit 502 acquires target candidate resources based on the second historical traffic that meets the corresponding conditions selected from the historical traffic set, and then determines the second historical traffic selected when the obtained target candidate resources are less than the initial resources as the traffic threshold, thereby improving the accuracy of the determined traffic threshold and the efficiency of the traffic threshold determination.

[0105] The acquisition unit 502 can first obtain a first type of candidate resource based on the initial traffic, then select a second historical traffic that is greater than the initial traffic from the historical traffic set, and then obtain a second type of candidate resource based on the traffic difference between each second historical traffic and the initial traffic. Finally, the target candidate resource is obtained based on the sum of the first type of candidate resource and the second type of candidate resource.

[0106] When the acquisition unit 502 obtains the flow load value based on the acquired historical flow set, it can do so in the following ways: obtain a flow difference set based on the flow difference between two adjacent second historical flows in the historical flow set; fit a flow change curve based on the flow difference set; and obtain the flow load value based on the slope of the flow change curve.

[0107] In other words, the acquisition unit 502 fits the flow change curve based on the flow difference set obtained from the historical flow set, and then obtains the flow load value based on the slope of the fitted flow change curve. This allows the acquired flow load value to reflect the historical flow change pattern, thereby improving the accuracy of the acquired flow load value and thus improving the accuracy of the predicted flow obtained based on the flow load value.

[0108] In addition, after obtaining the flow difference set, the acquisition unit 502 may also include the following: determining the 99th percentile value of the flow difference based on the multiple flow differences in the obtained flow difference set; taking the determined 99th percentile value of the flow difference as redundant flow, which can be used as the Y-axis intercept in the flow change curve.

[0109] In other words, based on the obtained traffic difference set, the acquisition unit 502 can not only obtain the traffic load value, but also use the 99th percentile value of the traffic difference as redundant traffic, which can improve the accuracy of the obtained redundant traffic.

[0110] In this embodiment, after the acquisition unit 502 acquires the traffic threshold and traffic load value based on the current time, the response unit 503 responds to the determination that the target traffic is greater than the traffic threshold, and obtains the target second type resource based on the target traffic, traffic load value, first type resource and effective second type resource.

[0111] In this embodiment, the effective second type of resource is the token usage of the corresponding second type that has been acquired at the current time. The effective second type of resource can be 0 or a value greater than 0; the first type of resource can be the token usage of the corresponding traffic threshold.

[0112] Specifically, when the response unit 503 obtains the target second type resource based on the target traffic, traffic load value, first type resource and effective second type resource, the implementation method can be as follows: obtain the predicted traffic based on the multiplication result between the target traffic and the traffic load value; obtain the target second type resource based on the subtraction result between the obtained predicted traffic and the first type resource and the effective second type resource.

[0113] In other words, since the traffic load value is obtained based on the historical traffic set, it can reflect the changing pattern of historical traffic at different times. Therefore, the response unit 503 obtains the predicted traffic for determining the target second type of resource based on the two pieces of information: the target traffic and the traffic load value. Compared with the prior art, which does not consider the change of traffic and directly uses the target traffic as the predicted traffic, this embodiment can effectively avoid the risk of traffic limiting caused by uncertain factors, thereby improving the accuracy of the obtained predicted traffic and the target second type of resource, and thus improving the accuracy of subsequent adaptive adjustment of resources.

[0114] In addition, if redundant traffic is also obtained, when the response unit 503 obtains the predicted traffic based on the multiplication result between the target traffic and the traffic load value, it can first obtain the multiplication result between the target traffic and the traffic load value, and then obtain the predicted traffic based on the sum of the obtained multiplication result and the redundant traffic.

[0115] In other words, in addition to the target traffic and traffic load value, the response unit 503 also combines redundant traffic to obtain the predicted traffic, thereby ensuring that the obtained predicted traffic can adapt to more scenarios (such as traffic surge scenarios, traffic drop scenarios, traffic continuous growth scenarios, etc.), and can further improve the accuracy of the obtained predicted traffic and the accuracy of adaptive adjustment of resources.

[0116] When the response unit 503 obtains the target second type resource based on the target traffic, traffic load value, first type resource, and the effective second type resource, it can use the following calculation formula:

[0117] H need =(load×x+c)-MH

[0118] In the above calculation formula: H need The target is the second type of resource; x is the target traffic; load is the traffic load value; c is the redundant traffic; M is the first type of resource; H is the second type of resource that has been activated.

[0119] In this embodiment, after the response unit 503 obtains the target second type of resource, the processing unit 504 performs a resource acquisition operation or a resource release operation based on the target second type of resource.

[0120] Specifically, if the target second type of resource is determined to be greater than 0, the processing unit 504 performs a resource acquisition operation to obtain the token usage of the corresponding target second type of resource, so that the large model has sufficient token usage for inference even when the TPM is high at the current moment, thereby improving the running stability of the large model.

[0121] If the target second type resource is determined to be less than 0, the processing unit 504 performs a resource release operation to release the token usage of the absolute value of the target second type resource, thereby avoiding resource waste caused by the large model occupying the second type resource and reducing the resource usage cost of the large model.

[0122] In other words, this embodiment obtains the target second type of resources based on the traffic threshold, traffic load value, and redundant traffic obtained from the historical traffic prediction analysis of the large model, as well as the target traffic of the large model determined at the current moment. Then, it adaptively adjusts the resources based on the obtained target second type of resources, so as to achieve the purpose of elastic scaling of the resources (especially the second type of resources) used by the large model through real-time feedback. This embodiment can naturally solve the problem of insufficient acquisition of second type of resources caused by inaccurate traffic prediction, and it also has strong robustness to complex scenarios such as sudden increase or decrease in traffic and continuous increase in traffic, thereby helping the business to reduce costs while ensuring business availability.

[0123] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0124] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0125] like Figure 6 The diagram shown is a block diagram of an electronic device for a resource processing method for large models according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0126] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0127] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0128] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as resource processing methods for large models. For example, in some embodiments, resource processing methods for large models may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 608.

[0129] In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by computing unit 601, one or more steps of the resource processing method for large models described above may be performed. Alternatively, in other embodiments, computing unit 601 may be configured to perform the resource processing method for large models by any other suitable means (e.g., by means of firmware).

[0130] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0131] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable resource processing apparatus for large models, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0132] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0133] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for showing information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0134] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0135] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the management difficulties and weak business scalability inherent in traditional physical hosts and VPS (Virtual Private Server) services. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0136] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0137] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A resource processing method for large models, comprising: Determine the target flow rate for the current moment corresponding to the large model; Based on the current time, obtain the traffic threshold and traffic load value; In response to determining that the target traffic is greater than the traffic threshold, a target second type of resource is obtained based on the target traffic, the traffic load value, the first type of resource, and the effective second type of resource. The first type of resource is the prepaid type of token usage, and the second type of resource is the postpaid type of token usage. Based on the target second type of resource, perform a resource acquisition operation or a resource release operation at the current time; The determination of the target flow corresponding to the current moment for the large model includes: Obtain the current traffic corresponding to the current moment in the large model; Based on the current time, a first historical time is determined, and the first historical traffic corresponding to the first historical time of the large model is obtained. The first historical time is a first preset number of historical times located before the current time. The target traffic is obtained based on the current traffic and the first historical traffic; The step of obtaining the traffic threshold based on the current time includes: In response to determining that the current time is an adjustment time, a historical traffic set is obtained based on the current time; Select a second historical traffic flow from the aforementioned historical traffic set as a candidate traffic flow; In response to determining that the candidate traffic is greater than the initial traffic, the initial traffic is set as the candidate traffic; Obtaining target candidate resources based on the initial traffic and all second historical traffic in the historical traffic set includes: obtaining a first type of candidate resource based on the initial traffic; obtaining second historical traffic greater than the initial traffic from the historical traffic set; obtaining a second type of candidate resource based on the traffic difference between each obtained second historical traffic and the initial traffic; and obtaining the target candidate resource based on the sum of the first type of candidate resource and the second type of candidate resource. In response to determining that the target candidate resource is greater than the initial resource, the process proceeds to selecting a second historical traffic from the historical traffic set, and this process is repeated until it is determined that the target candidate resource is less than the initial resource. The candidate traffic at this point is then used as the traffic threshold.

2. The method according to claim 1, wherein, The step of obtaining the traffic load value based on the current time includes: In response to determining that the current time is an adjustment time, a historical traffic set is obtained based on the current time; The traffic load value is obtained based on the historical traffic set.

3. The method according to claim 2, wherein, The step of obtaining the historical traffic set based on the current time includes: Determine the second historical moment based on the current moment; For each hour in the second historical moment, the largest first historical flow is selected from all the first historical flows corresponding to that hour, and used as the second historical flow corresponding to that hour; The historical traffic set is obtained based on multiple second historical traffic sets.

4. The method according to claim 2, wherein, The step of obtaining the traffic load value based on the historical traffic set includes: Based on the flow difference between two adjacent second historical flows in the historical flow set, a flow difference set is obtained; Based on the set of flow difference values, a flow change curve is obtained by fitting the curve. The flow load value is obtained based on the slope of the flow rate change curve.

5. The method according to claim 4, further comprising: After obtaining the set of flow difference values, the 99th percentile value of the flow difference value is determined based on multiple flow difference values ​​in the set of flow difference values. The 99th percentile of the traffic difference is taken as the redundant traffic.

6. The method according to claim 1, wherein, The step of obtaining the target second type of resource based on the target traffic, the traffic load value, the first type of resource, and the already activated second type of resource includes: The predicted flow rate is obtained by multiplying the target flow rate by the flow load value. The target second type of resource is obtained by subtracting the predicted traffic from the first type of resource and the effective second type of resource.

7. The method according to claim 6, wherein, The step of obtaining the predicted flow based on the product of the target flow and the flow load value includes: Obtain redundant traffic; The predicted flow is obtained by multiplying the target flow by the flow load value and adding it to the redundant flow.

8. The method according to claim 1, wherein, The step of performing the resource acquisition operation based on the target second type of resource includes: In response to determining that the target second type of resource is greater than 0, the resource acquisition operation is performed.

9. The method according to claim 1, wherein, The step of performing the resource release operation based on the target second type of resource includes: In response to determining that the target second type resource is less than 0, the resource release operation is performed.

10. A resource processing apparatus for large models, comprising: The determination unit is used to determine the target flow rate of the large model at the current moment; The acquisition unit is used to acquire the traffic threshold and traffic load value based on the current time. A response unit is configured to respond to determining that the target traffic is greater than the traffic threshold, and to obtain a target second type resource based on the target traffic, the traffic load value, the first type resource and the effective second type resource, wherein the first type resource is the prepaid type of token usage and the second type of token usage is the postpaid type. The processing unit is configured to perform a resource acquisition operation or a resource release operation at the current time, based on the target second type of resource. Specifically, when determining the target flow corresponding to the current moment for the large model, the determining unit performs the following: Obtain the current traffic corresponding to the current moment in the large model; Based on the current time, a first historical time is determined, and the first historical traffic corresponding to the first historical time of the large model is obtained. The first historical time is a first preset number of historical times located before the current time. The target traffic is obtained based on the current traffic and the first historical traffic; When the acquisition unit obtains the traffic threshold based on the current time, it specifically performs the following: In response to determining that the current time is an adjustment time, a historical traffic set is obtained based on the current time; Select a second historical traffic flow from the aforementioned historical traffic set as a candidate traffic flow; In response to determining that the candidate traffic is greater than the initial traffic, the initial traffic is set as the candidate traffic; Obtaining target candidate resources based on the initial traffic and all second historical traffic in the historical traffic set includes: obtaining a first type of candidate resource based on the initial traffic; obtaining second historical traffic greater than the initial traffic from the historical traffic set; obtaining a second type of candidate resource based on the traffic difference between each obtained second historical traffic and the initial traffic; and obtaining the target candidate resource based on the sum of the first type of candidate resource and the second type of candidate resource. In response to determining that the target candidate resource is greater than the initial resource, the process proceeds to selecting a second historical traffic from the historical traffic set, and this process is repeated until it is determined that the target candidate resource is less than the initial resource. The candidate traffic at this point is then used as the traffic threshold.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-9.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-9.

Images