A method and system for evaluating computing power requirements in business scenarios of large language models

Abstract

The present invention provides a method and system for evaluating computing power requirements of business scenarios of large language models, which belongs to the field of large model technology, and specifically includes: determining the call estimation amounts of different business requirements, taking the business requirements of all basic processing units involved as target business requirements, and determining that the call estimation amount of the target business requirement meets the requirements based on the deviation of the call estimation amount between the target business requirement and other business requirements, and determining the interference business requirements of the target business requirement based on the overlap of the call processing sequence of the basic processing units of the target business requirement with other business requirements, and determining the estimation processing method of the computing power requirements of the business scenario based on the call estimation amount of the interfering business requirements and the overlap data with the basic processing units of the target business requirements, and determining the allocation processing of the computing power requirements of the business scenario based on the estimation processing method, thereby improving the utilization and processing efficiency of the computing power requirements.

Description

Technical Field

[0001] The present invention belongs to the field of large model technology, and in particular relates to a method and system for evaluating computing power requirements for business scenarios of large language models. Background Art

[0002] Large language models are applied in various business scenarios. The business logic of different scenarios varies significantly, and the business volume in different scenarios also varies greatly. How to accurately assess the computing power requirements of business scenarios for large language models, further optimize resource allocation, and save computing power costs has become a technical problem that needs to be solved urgently.

[0003] To evaluate the computing power requirements of large models in different business scenarios, existing technical solutions often evaluate computing power requirements based on the large model's configuration scheme. However, this has the following technical problems:

[0004] Large models often contain multiple basic processing units such as modal conversion units, description and vectorization units, deep reasoning units, complex execution units, and simple execution units. The demand for different basic processing units in different business scenarios varies, and the calling data also varies. Therefore, if the computing power requirements cannot be evaluated and processed in combination with the calling data and the demand data, the accuracy of the evaluation results of the computing power requirements cannot be guaranteed.

[0005] To solve the above technical problems, this application provides a method and system for evaluating computing power requirements in business scenarios of large language models. Summary of the Invention

[0006] To achieve the purpose of the present invention, the present invention adopts the following technical solutions:

[0007] Specifically, this application provides a method for evaluating computing power requirements for business scenarios of large language models, specifically including:

[0008] S1 divides the large language model into multiple basic processing units. Based on the requirements of the basic processing units for different business needs in the business scenario, it determines the matching processing units for different business needs. Based on the overlapping data of the matching processing units for different business needs, it determines that the degree of correlation between the business needs meets the requirements, and then proceeds to the next step;

[0009] S2 determines the call estimation amounts of different business requirements, taking the business requirements of all the basic processing units involved as the target business requirements, and determining whether the call estimation amount of the target business requirements meets the requirements based on the deviation of the call estimation amount between the target business requirements and other business requirements. Then, the interference business requirements of the target business requirements are determined based on the overlap of the call processing sequence of the basic processing units of the target business requirements with other business requirements;

[0010] S3 determines the estimation and processing method of the computing power requirements of the business scenario based on the call estimate of the interfering business demand and the overlapping data with the basic processing unit of the target business demand, and determines the allocation and processing of the computing power requirements of the business scenario based on the estimation and processing method.

[0011] The beneficial effects of the present invention are:

[0012] The interference business demand of the target business demand is determined based on the overlap of other business demands and the calling processing order of the basic processing units of the target business demand. Not only the number of basic processing units with simultaneous calling demands is taken into consideration, but also the impact of the difference in the calling processing order of the basic processing units on the computing power demand of the basic processing units of the target business demand during simultaneous calling processing is taken into consideration, thereby laying the foundation for allocating and processing computing power demands according to simultaneous calling demands under the interfering business demands.

[0013] The estimation and processing method of the computing power requirements of the business scenario is determined based on the estimated call amount of the interfering business needs and the overlapping data of the basic processing units with the target business needs, thereby fully taking into account the differences in the degree of simultaneous call requirements of other interfering business needs and target business needs in different basic processing units, and then estimating the computing power requirements of the business scenario based on the differences in the degree of simultaneous call requirements. This not only meets the computing power requirements in the business scenario, but also avoids the occurrence of idle computing power resources due to the low degree of simultaneous call requirements of different basic processing units in the business scenario and the allocation of too much computing power resources.

[0014] A further technical solution is that the basic processing unit includes a modal conversion unit, a description and vectorization unit, a deep reasoning unit, a complex execution unit, and a simple execution unit.

[0015] A further technical solution is that the matching processing unit is a basic processing unit required by the business needs.

[0016] A further technical solution is to determine whether the degree of correlation between business requirements meets the requirements, including:

[0017] Using the overlapping data of the matching processing units with different business requirements, determine the business requirements with the same matching processing units and treat them as the same business requirements;

[0018] Classify the same business requirements into the same requirement group;

[0019] Based on the number of business requirements in different requirement groups, determine whether the degree of correlation between business requirements meets the requirements.

[0020] A further technical solution is that when the number of business demands is greater than a preset business demand number threshold and the number of demand groups is greater than a preset group number, it is determined that the degree of association between the business demands does not meet the requirement.

[0021] A further technical solution is that when the degree of correlation between business needs does not meet the requirements, the first estimation method is used to measure the computing power requirements of the business scenario.

[0022] A further technical solution is that the first estimation method is to determine the comprehensive estimation of different basic processing units by the sum of the call estimation amounts of the business demands corresponding to different basic processing units, to determine the computing power demands of different basic processing units by multiplying the comprehensive estimation amount by the preset computing power demand amount corresponding to the basic processing unit, and to use the sum of the computing power demands of different basic processing units as the allocation processing of the computing power demands of the business scenario.

[0023] A further technical solution is that the method for determining the estimation processing method of the computing power requirement of the business scenario is:

[0024] Based on the overlap of the calling processing sequence of the basic processing units of the interfering business demand and the target business demand, the basic processing units with the consistent calling processing sequence are determined, and the basic processing units are used as the consistent sequence units; based on the overlap of the basic processing units of the interfering business demand and the target business demand, the overlapping basic processing units are determined, and the overlapping units are used as the overlapping units; the number of overlaps is determined according to the number of the consistent sequence units and the average of the number of the overlapping units;

[0025] Based on the estimated call amount of the interfering service demand, determining the interfering service demand whose estimated call amount is greater than a preset estimated call amount threshold, and treating it as a frequently called demand;

[0026] Based on the number of overlaps of different frequent call requirements, determine the estimation and processing method for the computing power requirements of the business scenario.

[0027] A further technical solution is to determine an estimation method for computing power requirements of the business scenario based on the number of overlaps of different frequent call requirements, specifically including:

[0028] When the sum of the overlapping numbers of different frequently called demands is greater than a preset overlapping number threshold, the sum of the estimated call amounts of different interfering business demands in different basic processing units and the basic estimated amounts of the target business demands in different basic processing units is used as the target estimated amount, and the computing power requirements of the business scenario are estimated based on the target estimated amount.

[0029] When the sum of the overlapping numbers of different frequent call requirements is not greater than the preset overlapping number threshold, the basic estimated amount of different basic processing units according to the target business requirements is used as the target estimated amount, and the computing power requirements of the business scenario are estimated based on the target estimated amount.

[0030] A further technical solution is that, when the sum of the overlapping numbers of different frequently called demands is not greater than a preset overlapping number threshold, it is necessary to further determine the overlap between the frequently called demands and other interfering service demands in different basic processing units, specifically including:

[0031] Other interfering business demands whose number of overlaps with the basic processing unit that calls the frequent demands is greater than the preset value of the overlap number are regarded as associated business demands;

[0032] The sum of the associated business requirements and the frequent calling requirements in different basic processing units, and the sum of the basic estimated quantities of the target business requirements in different basic processing units are used as the target estimated quantity, and the computing power requirements of the business scenario are estimated based on the target estimated quantity.

[0033] A further technical solution is to determine the allocation process of the computing power requirements of the business scenario based on the estimation processing method, specifically including:

[0034] Determine the computing power requirement for the business scenario using the estimation processing method;

[0035] Based on the computing power demand in the business scenario, the computing power demand of the business scenario is allocated and processed.

[0036] In a second aspect, the present invention provides a computer system comprising: a memory and a processor that are communicatively connected, and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the above-mentioned method for evaluating computing power requirements for business scenarios of a large language model when running the computer program.

[0037] Other features and advantages will be described in the following description. The objectives and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description and drawings.

[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and other features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the accompanying drawings.

[0040] Figure 1This is a flowchart of a method for evaluating computing power requirements for business scenarios with large language models.

[0041] Figure 2 It is a flow chart to determine the degree of correlation between business requirements to meet the requirements;

[0042] Figure 3 It is a flow chart to determine the estimated number of calls for the target business needs to meet the requirements;

[0043] Figure 4 It is a flow chart of a method for determining an interference business requirement of a target business requirement;

[0044] Figure 5 It is a framework diagram of a computer system. DETAILED DESCRIPTION

[0045] To help those skilled in the art better understand the technical solutions in this specification, the following will provide a clear and complete description of the technical solutions in the embodiments of this specification, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this specification without creative work should fall within the scope of protection of this specification.

[0046] In this application, by dividing the large language model into multiple basic processing units, and based on the different business needs in the business scenarios, the demand for different basic processing units, and combining the call estimation data of different business needs, the computing power requirements of different business scenarios are evaluated, thereby improving the utilization rate of computing power requirements.

[0047] The same business requirements of the matching processing units will be treated as the same business requirements. When the number of business requirements with the same business requirements in the business scenario accounts for more than 0.6, it is determined that the degree of correlation between the business requirements does not meet the requirements.

[0048] When the estimated call amounts of the target business demand in different basic processing units are all greater than the estimated call amounts of other business demands, it is determined that the estimated call amount of the target business demand meets the requirements.

[0049] The interfering service demand is other service demand that has the same number of basic processing units as the target service demand, which is four or more, and has the same calling processing order, which has the same number of basic processing units, which is two or more.

[0050] An interfering business demand with a call estimate greater than a preset call estimate threshold is regarded as a frequent call demand. When the sum of the overlapping numbers of different frequent call demands is greater than the preset overlapping number threshold, the sum of the call estimates of the different interfering business demands in different basic processing units and the basic estimates of the target business demands in different basic processing units is regarded as the target estimate. The computing power requirements of the business scenario are estimated based on the target estimate.

[0051] When the sum of the overlapping numbers of different frequent call demands is not greater than the preset overlapping number threshold, the basic estimated amounts of different basic processing units are used as target estimated amounts according to the target business needs, and the computing power requirements of the business scenario are estimated based on the target estimated amounts, thereby reducing the reliability of the allocation processing of computing power requirements when parallel processing demands are low, and avoiding the waste of computing power resources when parallel processing demands are low.

[0052] Example 1

[0053] like Figure 1 As shown, this application provides a method for evaluating computing power requirements for business scenarios of large language models, specifically including:

[0054] S1 divides the large language model into multiple basic processing units. Based on the requirements of the basic processing units for different business needs in the business scenario, it determines the matching processing units for different business needs. Based on the overlapping data of the matching processing units for different business needs, it determines that the degree of correlation between the business needs meets the requirements, and then proceeds to the next step;

[0055] Furthermore, the basic processing unit includes a modality conversion unit, a description and vectorization unit, a deep reasoning unit, a complex execution unit, and a simple execution unit.

[0056] Specifically, the matching processing unit is a basic processing unit required by the business demand.

[0057] Specifically, such as Figure 2 As shown, determine the degree of correlation between business needs to meet the requirements, including:

[0058] Using the overlapping data of the matching processing units with different business requirements, determine the business requirements with the same matching processing units and treat them as the same business requirements;

[0059] Classify the same business requirements into the same requirement group;

[0060] Based on the number of business requirements in different requirement groups, determine whether the degree of correlation between business requirements meets the requirements.

[0061] It can be understood that when the number of business requirements is greater than a preset business requirement number threshold and the number of requirement groups is greater than a preset group number, it is determined that the degree of association between the business requirements does not meet the requirement.

[0062] It should be noted that when the degree of correlation between business needs does not meet the requirements, the first estimation method is used to measure the computing power requirements of the business scenario.

[0063] Specifically, the first estimation method is to determine the comprehensive estimation of different basic processing units by the sum of the call estimation amounts of the business demands corresponding to different basic processing units, determine the computing power demands of different basic processing units by multiplying the comprehensive estimation amount by the preset computing power demands corresponding to the basic processing units, and use the sum of the computing power demands of different basic processing units as the allocation processing of the computing power demands of the business scenario.

[0064] In another possible embodiment, determining that the degree of correlation between business requirements meets the requirements specifically includes:

[0065] Using the overlapping data of the matching processing units with different business requirements, determine the business requirements with the same matching processing units and treat them as the same business requirements;

[0066] Based on the number of identical business requirements, determine whether the degree of correlation between business requirements meets the requirements.

[0067] Furthermore, when the number of the same business requirements is greater than a preset threshold of the number of the same requirements, it is determined that the degree of association between the business requirements does not meet the requirement.

[0068] S2 determines the call estimation amounts of different business requirements, taking the business requirements of all the basic processing units involved as the target business requirements, and determining whether the call estimation amount of the target business requirements meets the requirements based on the deviation of the call estimation amount between the target business requirements and other business requirements. Then, the interference business requirements of the target business requirements are determined based on the overlap of the call processing sequence of the basic processing units of the target business requirements with other business requirements;

[0069] Furthermore, the estimated call amount of the business demand is determined according to the number of personnel who have the call authority for the business demand, specifically according to the product of the number of personnel and a preset proportional factor.

[0070] Specifically, such as Figure 3 As shown, determining that the estimated call amount of the target business demand meets the requirements specifically includes:

[0071] According to the calling processing order of the basic processing units of the business requirements, the basic processing unit to be called first is determined and used as the target processing unit. The sum of the calling estimated quantities of the business requirements that are consistent with the target processing unit is used as the estimated quantity of the target processing unit.

[0072] Determine the deviation amount in different target processing units based on the deviation between the estimated amount of calls of target business requirements in different target processing units and the inferred estimated amount;

[0073] When the deviations from different target processing units are all within a preset range, it is determined that the estimated call quantity of the target business demand meets the requirement.

[0074] It should be noted that when the call estimate of the target business demand does not meet the requirements, the inferred estimate of the target processing unit is directly used as the basic estimate of the target business demand in the target processing unit. If not, the call estimate of the target business demand in the target processing unit is used as the basic estimate of different basic processing units.

[0075] It is understandable that determining whether the estimated call volume of the target business demand meets the requirements specifically includes:

[0076] According to the calling processing order of the basic processing units of the business requirements, the basic processing unit to be called first is determined and used as the target processing unit. The sum of the calling estimated quantities of the business requirements that are consistent with the target processing unit is used as the estimated quantity of the target processing unit.

[0077] Determine the deviation amount in different target processing units based on the deviation between the estimated amount of calls of target business requirements in different target processing units and the inferred estimated amount;

[0078] It is determined whether the estimated call amount of the target business demand meets the requirement according to the maximum value of the deviation amounts of different target processing units.

[0079] Furthermore, when the maximum value of the deviations among different target processing units is smaller than a preset deviation threshold, it is determined that the estimated call amount of the target business demand meets the requirement.

[0080] Specifically, such as Figure 4 As shown, the method for determining the interfering service demand of the target service demand is:

[0081] Based on the overlap of the calling processing sequence of the basic processing units of other business requirements and the target business requirement, determine the basic processing units with the same calling processing sequence, and use them as the sequence-consistent units;

[0082] Based on the overlap between other business requirements and the basic processing units of the target business requirement, the overlapping basic processing units are determined and used as the overlapping units;

[0083] Whether the target service demand is an interfering service demand is determined according to the number of the overlapping units and the number of the sequence-consistent units.

[0084] Further, determining whether the target service demand is an interfering service demand based on the number of the overlapping units and the number of the units with consistent order specifically includes:

[0085] When the first-called basic processing unit of the other service requirement is consistent with the target service requirement, it is determined that it is an interfering service requirement;

[0086] When the basic processing unit first called by the other business needs is inconsistent with the target business needs, the number of overlaps is determined based on the number of overlapping units and the average number of sequence-consistent units. When the number of overlaps is greater than the preset number threshold, the other business needs are determined to be interfering business needs. When the number of overlaps is greater than the preset number threshold, it is determined that it is not an interfering business need.

[0087] S3 determines the estimation and processing method of the computing power requirements of the business scenario based on the call estimate of the interfering business demand and the overlapping data with the basic processing unit of the target business demand, and determines the allocation and processing of the computing power requirements of the business scenario based on the estimation and processing method.

[0088] It should be noted that the method for estimating the computing power requirements of the business scenario is as follows:

[0089] Based on the overlap of the calling processing sequence of the basic processing units of the interfering business demand and the target business demand, the basic processing units with the consistent calling processing sequence are determined, and the basic processing units are used as the consistent sequence units; based on the overlap of the basic processing units of the interfering business demand and the target business demand, the overlapping basic processing units are determined, and the overlapping units are used as the overlapping units; the number of overlaps is determined according to the number of the consistent sequence units and the average of the number of the overlapping units;

[0090] Based on the estimated call amount of the interfering service demand, determining the interfering service demand whose estimated call amount is greater than a preset estimated call amount threshold, and treating it as a frequently called demand;

[0091] Based on the number of overlaps of different frequent call requirements, determine the estimation and processing method for the computing power requirements of the business scenario.

[0092] Furthermore, based on the number of overlaps of different frequent call demands, an estimation method for computing power requirements of the business scenario is determined, specifically including:

[0093] When the sum of the overlapping numbers of different frequently called demands is greater than a preset overlapping number threshold, the sum of the estimated call amounts of different interfering business demands in different basic processing units and the basic estimated amounts of the target business demands in different basic processing units is used as the target estimated amount, and the computing power requirements of the business scenario are estimated based on the target estimated amount.

[0094] When the sum of the overlapping numbers of different frequent call requirements is not greater than the preset overlapping number threshold, the basic estimated amount of different basic processing units according to the target business requirements is used as the target estimated amount, and the computing power requirements of the business scenario are estimated based on the target estimated amount.

[0095] It is understandable that when the sum of the overlapping numbers of different frequently called demands is not greater than the preset overlapping number threshold, it is necessary to further determine the overlap between the frequently called demands and other interfering business demands in different basic processing units, specifically including:

[0096] Other interfering business demands whose number of overlaps with the basic processing unit that calls the frequent demands is greater than the preset value of the overlap number are regarded as associated business demands;

[0097] The sum of the associated business requirements and the frequent calling requirements in different basic processing units, and the sum of the basic estimated quantities of the target business requirements in different basic processing units are used as the target estimated quantity, and the computing power requirements of the business scenario are estimated based on the target estimated quantity.

[0098] In another possible embodiment, the method for determining the estimation method of the computing power requirement of the business scenario is:

[0099] Determining interference values of different interfering service demands based on the estimated call amounts of the different interfering service demands and in combination with the estimated call amounts of other interfering service demands and the overlap with the interfering service demands in different basic processing units;

[0100] It should be further explained that in the above steps, if the number of interfering business demands does not meet the requirements, that is, the number is too large, specifically greater than the threshold, then the sum of the call estimates of different interfering business demands in different basic processing units and the basic estimates of the target business demands in different basic processing units will be used as the target estimate, and the computing power requirements of the business scenario will be estimated based on the target estimate.

[0101] In addition, even if the number of interfering business demands is not large, it is necessary to consider the call estimation amounts of different interfering business demands. When the sum of the call estimation amounts of different interfering business demands does not meet the requirements, that is, the call estimation amount is too large, it can be specifically determined by setting a threshold. In this case, it is also necessary to take the sum of the call estimation amounts of different interfering business demands in different basic processing units and the basic estimation amounts of the target business demands in different basic processing units as the target estimation amount, and perform estimation processing of the computing power requirements of the business scenario based on the target estimation amount.

[0102] It should also be noted that if all the above conditions are met, and at the same time, the interference values of different interfering business demands are less than the preset interference threshold, it means that the computing power requirements of other interfering business demands are relatively small. Therefore, the basic estimated amounts of different basic processing units can be used as target estimated amounts according to the target business needs, and the computing power requirements of the business scenario can be estimated based on the target estimated amounts.

[0103] However, at the same time, if there are multiple interfering business demands with relatively large interference values, it also means that the interference degree of the interfering business demands is relatively large. Therefore, it is also necessary to take the sum of the call estimates of different interfering business demands in different basic processing units and the basic estimates of the target business demands in different basic processing units as the target estimates, and perform the estimation and processing of the computing power requirements of the business scenario based on the target estimates.

[0104] Through the above-mentioned estimation and processing method of generating differentiated computing power requirements based on the interference situation of business needs, it is possible to fully consider the difference in the degree of interference between other business needs and target business needs on the basis of meeting the computing power requirements in the business scenario, that is, the difference in the demand for calling multiple basic processing units at the same time between other business needs and target business needs. This not only ensures that a larger computing power space is allocated when the demand level is large, but also avoids the waste of computing power space when the demand for concurrent processing is low.

[0105] Based on the overlap of the calling processing sequence of the basic processing units of the interfering business demand and the target business demand, the basic processing units with the consistent calling processing sequence are determined, and the basic processing units are used as the consistent sequence units; based on the overlap of the basic processing units of the interfering business demand and the target business demand, the overlapping basic processing units are determined, and the overlapping units are used as the overlapping units; the number of overlaps is determined according to the number of the consistent sequence units and the average of the number of the overlapping units;

[0106] It is understandable that when the sum of the overlapping numbers of different frequently called demands is greater than a preset overlapping number threshold, the sum of the estimated call quantities of different interfering business demands in different basic processing units and the basic estimated quantities of the target business demands in different basic processing units is used as the target estimated quantity, and the computing power demand of the business scenario is estimated based on the target estimated quantity.

[0107] It should be further explained that even if the sum of the numbers of overlaps of different frequently called requirements is not greater than the preset overlap threshold, it is necessary to proceed to the next step.

[0108] According to the number of overlaps and interference values of different interfering business demands, the interference assessment amount of the target business demand of the business scenario is determined, and based on the interference assessment amount, an estimation processing method for the computing power demand of the business scenario is determined.

[0109] In one embodiment, the interference assessment amount of the target business demand can be determined according to a mathematical model based on the overlapping number of different interfering business demands and the interference value, specifically determined according to the output of a neural network prediction model with the overlapping number of different interfering business demands and the interference value as input.

[0110] Specifically, when the interference assessment value is greater than 0.35, it means that the interference degree of the interference business demand is large, that is, the demand for calling multiple basic units at the same time is large. Therefore, it is also necessary to take the sum of the call estimation value of different interference business demands in different basic processing units and the basic estimation value of the target business demand in different basic processing units as the target estimation value, and perform the estimation of the computing power demand of the business scenario based on the target estimation value. When the interference assessment value is less than 0.1, the basic estimation value of the target business demand in different basic processing units is used as the target estimation value, and the computing power demand of the business scenario is estimated based on the target estimation value.

[0111] If it is between 0.1 and 0.35, other interfering business demands whose number of overlaps with the basic processing units that call frequent demands is greater than the preset value of the overlap number will be regarded as associated business demands. The sum of the associated business demands and the frequently called demands in different basic processing units and the sum of the basic estimated quantities of the target business demands in different basic processing units will be used as the target estimated quantity, and the computing power requirements of the business scenario will be estimated based on the target estimated quantity.

[0112] Through the above-mentioned estimation processing, the differences in the degree of concurrent call processing requirements of different basic processing units during actual operation are fully taken into account, and then differentiated estimation processing strategies are determined, thereby ensuring the computing power requirements in business scenarios while reducing the application cost of computing power.

[0113] Example 2

[0114] Second, as Figure 5 As shown, the present invention provides a computer system, comprising: a memory and a processor that are communicatively connected, and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the above-mentioned method for evaluating computing power requirements for business scenarios of a large language model when running the computer program.

[0115] Optionally, determine whether the degree of correlation between business requirements meets the requirements, including:

[0116] S11 determines the same business requirements of the matching processing units based on the overlapping data of the matching processing units with different business requirements, and regards them as the same business requirements;

[0117] Specifically, when the same business requirements do not exist, it can be directly determined whether the degree of correlation between the business requirements meets the requirements.

[0118] It is understandable that when the number of identical business requirements is too large, that is, greater than a fixed threshold, it is directly determined that the degree of correlation between the business requirements does not meet the requirements without considering other data.

[0119] It should be further explained that when the number of identical business requirements is between 1 and 2, it can also be determined that the degree of correlation between the business requirements meets the requirements. If and only if the number of identical business requirements is within a certain number range, it is necessary to proceed to step S12.

[0120] S12: grouping the same business requirements into the same requirement group, and determining the processing unit overlap values of the different requirement groups based on the number of the same business requirements in the different requirement groups and the deviation of the matching processing units with other business requirements;

[0121] Specifically, when the number of demand groups is too large, and at the same time the number of the same business demands is within a certain range, it can be directly determined that the degree of correlation between the business demands does not meet the requirements.

[0122] In addition, if the number of business demands is greater than the preset business demand threshold and the number of demand groups is greater than the preset group number, it can also be determined that the degree of correlation between the business demands does not meet the requirements. If and only if the number of business demands is greater than the preset business demand threshold and the number of demand groups is not greater than the preset group number, there is no need to further evaluate the processing unit overlap value of the demand group.

[0123] Specifically, when there are multiple demand groups with processing unit overlap values greater than a threshold, it can be directly determined that the degree of correlation between business needs does not meet the requirements. When there are no multiple demand groups with processing unit overlap values greater than the threshold, proceed to the next step.

[0124] It is understandable that the processing unit overlap value and the demand overlap value of the demand group are fixed in terms of input and output. Therefore, those skilled in the art can determine them through neural networks and other methods on this basis.

[0125] S13 determines the demand overlap value between the business demands according to the number of demand groups and the processing unit overlap values of different demand groups, and determines whether the association between the business demands meets the requirements based on the demand overlap value.

[0126] It can be understood that the demand overlap value ranges from 0 to 1, wherein when the demand overlap value is greater than a preset demand overlap threshold, it is determined that the association between the business demands meets the requirements.

[0127] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the device, apparatus, and non-volatile computer storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simplified. For relevant details, refer to the descriptions of the method embodiments.

[0128] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0129] The foregoing description is merely one or more embodiments of this specification and is not intended to limit this specification. It will be apparent to those skilled in the art that various modifications and variations may be made to one or more embodiments of this specification. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of one or more embodiments of this specification are intended to be within the scope of the claims of this specification.

Claims

1. A method for evaluating computing power requirements for business scenarios of large language models, characterized by: Specifically include: Divide the large language model into multiple basic processing units. Based on the requirements of the basic processing units for different business needs in the business scenario, determine the matching processing units for different business needs. Based on the overlapping data of the matching processing units for different business needs, determine the degree of correlation between the business needs. If it meets the requirements, proceed to the next step. Determine the call estimates for different business requirements, taking the business requirements of all involved basic processing units as target business requirements, and determining whether the call estimate of the target business requirement meets the requirements based on the deviation between the call estimates of the target business requirement and other business requirements. Then, determine the interfering business requirements of the target business requirement based on the overlap between the call processing sequence of the basic processing units of the target business requirement and other business requirements. Determining an estimation method for computing power requirements of the business scenario based on the estimated call amount of the interfering business demand and the overlap data with the basic processing units of the target business demand, and determining allocation processing for computing power requirements of the business scenario based on the estimation method; The method for estimating the computing power requirements of the business scenario is as follows: Based on the overlap of the calling processing sequence of the basic processing units of the interfering business demand and the target business demand, the basic processing units with the consistent calling processing sequence are determined, and the basic processing units are used as the consistent sequence units; based on the overlap of the basic processing units of the interfering business demand and the target business demand, the overlapping basic processing units are determined, and the overlapping units are used as the overlapping units; the number of overlaps is determined according to the number of the consistent sequence units and the average of the number of the overlapping units; Based on the estimated call amount of the interfering service demand, determining the interfering service demand whose estimated call amount is greater than a preset estimated call amount threshold, and treating it as a frequently called demand; Determine the estimated processing method for computing power requirements of the business scenario based on the number of overlaps of different frequent call requirements; When the sum of the overlapping numbers of different frequently called demands is greater than a preset overlapping number threshold, the sum of the estimated call amounts of different interfering business demands in different basic processing units and the basic estimated amounts of the target business demands in different basic processing units is used as the target estimated amount, and the computing power requirements of the business scenario are estimated based on the target estimated amount. When the sum of the overlapping numbers of different frequent call requirements is not greater than the preset overlapping number threshold, the basic estimated amount of different basic processing units according to the target business requirements is used as the target estimated amount, and the computing power requirements of the business scenario are estimated based on the target estimated amount.

2. The method for evaluating computing power requirements for business scenarios of a large language model according to claim 1, characterized in that: The basic processing unit includes a modality conversion unit, a description and vectorization unit, a deep reasoning unit, a complex execution unit, and a simple execution unit.

3. The method for evaluating computing power requirements for business scenarios of a large language model according to claim 1, characterized in that: The matching processing unit is a basic processing unit required by the business needs.

4. The method for evaluating computing power requirements for business scenarios of a large language model according to claim 1, wherein: Determine whether the degree of correlation between business needs meets the requirements, including: Using the overlapping data of the matching processing units with different business requirements, determine the business requirements with the same matching processing units and treat them as the same business requirements; Classify the same business requirements into the same requirement group; Based on the number of business requirements in different requirement groups, determine whether the degree of correlation between business requirements meets the requirements.

5. The method for evaluating computing power requirements for business scenarios of large language models according to claim 1, characterized in that: When the degree of correlation between business needs does not meet the requirements, the first estimation method is used to measure the computing power requirements of the business scenario.

6. The method for evaluating computing power requirements for business scenarios of a large language model according to claim 5, characterized in that: The first estimation method is to determine the comprehensive estimation of different basic processing units by the sum of the call estimation amounts of the business demands corresponding to different basic processing units, determine the computing power demands of different basic processing units by multiplying the comprehensive estimation amount by the preset computing power demands corresponding to the basic processing units, and use the sum of the computing power demands of different basic processing units as the allocation processing of the computing power demands of the business scenario.

7. The method for evaluating computing power requirements for business scenarios of a large language model according to claim 1, characterized in that: When the sum of the overlapping numbers of different frequently called demands is not greater than the preset overlapping number threshold, it is necessary to further determine the overlap between the frequently called demands and other interfering business demands in different basic processing units, including: Other interfering business demands whose number of overlaps with the basic processing unit that calls the frequent demands is greater than the preset value of the overlap number are regarded as associated business demands; The sum of the basic estimated quantities of the associated business needs and the frequently called needs in different basic processing units, and the sum of the basic estimated quantities of the target business needs in different basic processing units are used as the target estimated quantity, and the computing power requirements of the business scenario are estimated based on the target estimated quantity.

8. A computer system comprising: A memory and a processor in communication connection, and a computer program stored in the memory and capable of running on the processor, characterized in that when the processor runs the computer program, it executes the business scenario computing power requirement assessment method of a large language model as described in any one of claims 1-7.

Images