Business scene computing power demand evaluation method and system of large language model

Abstract

The invention provides a business scene computing power demand evaluation method and system for a large language model, and belongs to the technical field of large models, and the method specifically comprises the steps: determining the call estimation amount of different business demands, taking the business demands of all related basic processing units as target business demands, and carrying out the calculation of the business scene computing power demand. Determining an interference service demand of the target service demand according to a coincidence condition of calling processing sequences of other service demands and basic processing units of the target service demand when determining that the calling estimated amount of the target service demand meets the requirement according to a deviation condition of the calling estimated amount between the target service demand and the other service demands, and determining the interference service demand of the target service demand according to the coincidence condition of the calling processing sequences of the other service demands and the basic processing units of the target service demand. According to the method, the estimation processing method of the computing power demand of the service scene is determined according to the call estimation amount interfering with the service demand and the coincident data of the basic processing unit of the target service demand, and the allocation processing of the computing power demand of the service scene is determined based on the estimation processing method, so that the utilization processing efficiency of the computing power demand is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of large models, and particularly relates to a method and system for evaluating the computing power requirements of business scenarios of large language models. Background Art

[0002] Large language models are applied in various business scenarios. The business logics of different scenarios are significantly different, and the business volumes of different scenarios are also very different. How to accurately evaluate the computing power requirements of business scenarios for large language models, further optimize resource allocation, and save computing power costs has become a technical problem to be solved urgently.

[0003] In order to evaluate the computing power requirements of large models applied in different business scenarios, existing technical solutions often evaluate the computing power requirements according to the setting schemes of large models, etc., but there are the following technical problems: Large models often include multiple basic processing units such as a modality conversion unit, a description and vectorization unit, a deep reasoning unit, a complex execution unit, and a simple execution unit. The demand degrees for different basic processing units in different business scenarios deviate, and there are also deviations in the called data. Therefore, if the evaluation and processing of computing power requirements cannot be combined with the called data and demand data, the accuracy of the evaluation and processing results of computing power requirements cannot be guaranteed.

[0004] To solve the above technical problems, the present application provides a method and system for evaluating the computing power requirements of business scenarios of large language models. Summary of the Invention

[0005] To achieve the purpose of the present invention, the present invention adopts the following technical solutions: Specifically, the present application provides a method for evaluating the computing power requirements of business scenarios of large language models, which specifically includes: S1 Divide the large language model into multiple basic processing units. Based on the demand situations of the basic processing units with different business requirements in the business scenario, determine the matching processing units for different business requirements. When the coincidence data of the matching processing units for different business requirements determines that the correlation degree between business requirements meets the requirements, proceed to the next step; S2 Determine the call estimation amounts for different business requirements. Regard the business requirements of all the basic processing units involved as the target business requirements. Based on the deviation situation of the call estimation amounts between the target business requirements and other business requirements, when the call estimation amounts of the target business requirements meet the requirements, determine the interfering business requirements of the target business requirements according to the coincidence situation of the call processing sequences of the basic processing units of other business requirements and the target business requirements; S3 determines the estimation processing method of the computing power requirement of the service scenario based on the estimated call volume of the interfering service requirement and the overlapping data of the basic processing units of the target service requirement, and determines the allocation processing of the computing power requirement of the service scenario based on the estimation processing method.

[0006] The beneficial effects of the present invention are as follows: Determine the interfering service requirements of the target service requirement based on the overlapping situation of the call processing order of other service requirements and the basic processing units of the target service requirement. This not only takes into account the number of basic processing units with simultaneous call requirements, but also considers the impact on the computing power requirements of the basic processing units of the target service requirement during simultaneous call processing due to the difference in the call processing order of the basic processing units. Furthermore, it lays a foundation for the allocation processing of computing power requirements according to the simultaneous call requirements under the interfering service requirements.

[0007] Determine the estimation processing method of the computing power requirement of the service scenario based on the estimated call volume of the interfering service requirement and the overlapping data of the basic processing units of the target service requirement, so as to fully consider the difference in the degree of simultaneous call requirements between other interfering service requirements and the target service requirement in different basic processing units. Then, estimate the computing power requirement of the service scenario according to the difference in the degree of simultaneous call requirements. This not only meets the computing power requirements in the service scenario, but also avoids the situation of idle computing power resources caused by excessive allocation of computing power resources due to the low degree of simultaneous call requirements of different basic processing units in the service scenario.

[0008] A further technical solution is that the basic processing units include a modality conversion unit, a description and vectorization unit, a deep reasoning unit, a complex execution unit, and a simple execution unit.

[0009] A further technical solution is that the matching processing unit is the basic processing unit required by the service requirement.

[0010] A further technical solution is that determining that the correlation degree between service requirements meets the requirements specifically includes: Determine the service requirements with the same matching processing unit based on the overlapping data of the matching processing units of different service requirements, and regard them as the same service requirements; Divide the same service requirements into the same requirement group; Determine whether the correlation degree between service requirements meets the requirements according to the number of service requirements in different requirement groups.

[0011] A further technical solution is that when the number of requirement groups with the number of service requirements greater than the preset service requirement quantity threshold is greater than the preset group quantity, it is determined that the correlation degree between service requirements does not meet the requirements.

[0012] A further technical solution is that when the degree of association between business requirements does not meet the requirements, the first estimation method is used to measure the computing power requirements of the business scenario.

[0013] A further technical solution is that the first estimation method is to determine the comprehensive estimation amount of different basic processing units by the sum of the call estimation amounts of the business requirements corresponding to different basic processing units, and use the product of the comprehensive estimation amount and the preset computing power requirement amount corresponding to the basic processing unit to determine the computing power requirement amounts of different basic processing units, and use the sum of the computing power requirement amounts of different basic processing units as the allocation processing of the computing power requirements of the business scenario.

[0014] A further technical solution is that the method for determining the estimation processing method of the computing power requirements of the business scenario is as follows: Based on the coincidence of the call processing order of the basic processing units of the interfering business requirements and the target business requirements, determine the basic processing units with the same call processing order and use them as the units with the same order. Based on the coincidence of the basic processing units of the interfering business requirements and the target business requirements, determine the coincident basic processing units and use them as the coincident units. According to the average value of the number of units with the same order and the number of coincident units, determine the number of coincidences; Based on the call estimation amount of the interfering business requirements, determine the interfering business requirements whose call estimation amount is greater than the preset call estimation amount threshold and use them as the frequently called requirements; According to the number of coincidences of different frequently called requirements, determine the estimation processing method of the computing power requirements of the business scenario.

[0015] A further technical solution is that according to the number of coincidences of different frequently called requirements, determining the estimation processing method of the computing power requirements of the business scenario specifically includes: When the sum of the number of coincidences of different frequently called requirements is greater than the preset number of coincidences threshold, then use the sum of the call estimation amounts of different interfering business requirements in different basic processing units and the basic estimation amounts of the target business requirements in different basic processing units as the target estimation amount, and based on the target estimation amount, perform the estimation processing of the computing power requirements of the business scenario; When the sum of the number of coincidences of different frequently called requirements is not greater than the preset number of coincidences threshold, then use the basic estimation amounts of the target business requirements in different basic processing units as the target estimation amount, and based on the target estimation amount, perform the estimation processing of the computing power requirements of the business scenario.

[0016] A further technical solution is that when the sum of the overlapping quantities of different frequently called requirements is not greater than a preset overlapping quantity threshold, it is also necessary to further determine the overlapping situation between the frequently called requirements and other interfering service requirements in different basic processing units, specifically including: Regarding other interfering service requirements with the overlapping quantity in the basic processing unit of the frequently called requirements being greater than the preset overlapping quantity value as associated service requirements; Based on the sum of the associated service requirements and the frequently called requirements in different basic processing units and the sum of the basic estimated quantities of the target service requirements in different basic processing units as the target estimated quantity, perform the estimation process of the computing power requirements for the service scenario based on the target estimated quantity.

[0017] A further technical solution is that based on the above estimation processing method, determine the allocation process of the computing power requirements for the service scenario, specifically including: Determine the computing power demand quantity in the service scenario by using the above estimation processing method; According to the computing power demand quantity in the service scenario, perform the allocation process of the computing power requirements for the service scenario.

[0018] In a second aspect, the present invention provides a computer system, including: a memory and a processor connected by communication, and a computer program stored on the memory and capable of running on the processor, wherein when the processor runs the computer program, it executes the above-mentioned method for evaluating the computing power requirements of the service scenario of a large language model.

[0019] Other features and advantages will be described in the subsequent specification. The objectives and other advantages of the present invention are achieved and obtained by the structures specifically pointed out in the specification and the drawings.

[0020] To make the above objectives, features, and advantages of the present invention more obvious and understandable, the following specifically gives preferred embodiments and, in conjunction with the accompanying drawings, makes a detailed description as follows. Description of the Drawings

[0021] By referring to the accompanying drawings and describing its exemplary embodiments in detail, the above and other features and advantages of the present invention will become more obvious.

[0022] Figure 1 is a flowchart of a method for evaluating the computing power requirements of the service scenario of a large language model; Figure 2 is a flowchart for determining that the degree of association between service requirements meets the requirements; Figure 3 is a flowchart for determining that the call estimation quantity of the target service requirement meets the requirements; Figure 4 is a flowchart of a method for determining the interfering service requirements of the target service requirement; Figure 5 It is a framework diagram of a computer system. Detailed implementation manners

[0023] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all the embodiments. Based on the embodiments of this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this specification.

[0024] In this application, by dividing the large language model into multiple basic processing units, and according to the demand situations of different basic processing units in different business requirements in the business scenario, and combining the call estimation data of different business requirements, the computing power demand of different business scenarios is evaluated, thereby improving the utilization rate of the computing power demand.

[0025] The business requirements with the same matching processing unit will be regarded as the same business requirements. When the proportion of the number of business requirements of the same business requirements in the business scenario is above 0.6, it is determined that the degree of association between the business requirements does not meet the requirements.

[0026] When the call estimation amounts of the target business requirements in different basic processing units are all greater than the call estimation amounts of other business requirements, it is determined that the call estimation amounts of the target business requirements meet the requirements.

[0027] The interfering business requirements are other business requirements with the same number of basic processing units as the target business requirements being more than 4 and the number of basic processing units with the same call processing order being more than 2.

[0028] The interfering business requirements with call estimation amounts greater than the preset call estimation amount threshold are regarded as frequently called requirements. When the sum of the coincidence numbers of different frequently called requirements is greater than the preset coincidence number threshold, the sum of the call estimation amounts of different interfering business requirements in different basic processing units and the basic estimation amounts of the target business requirements in different basic processing units is used as the target estimation amount, and the computing power demand of the business scenario is estimated based on the target estimation amount; When the sum of the coincidence numbers of different frequently called requirements is not greater than the preset coincidence number threshold, the basic estimation amounts of the target business requirements in different basic processing units are used as the target estimation amount, and the computing power demand of the business scenario is estimated based on the target estimation amount, thereby reducing the reliability of the computing power demand allocation processing in the case of less parallel processing requirements and avoiding the waste of computing power resources in the case of less parallel processing requirements.

[0029] Embodiment 1 As Figure 1 shown, the present application provides a method for evaluating the computing power requirements of business scenarios of large language models, specifically including: S1 Divide the large language model into multiple basic processing units, determine the matching processing units for different business requirements based on the requirements of the basic processing units for different business requirements in the business scenario, and when the correlation degree between different business requirements meets the requirements based on the overlapping data of the matching processing units for different business requirements, proceed to the next step; Furthermore, the basic processing units include a modality conversion unit, a description and vectorization unit, a deep reasoning unit, a complex execution unit, and a simple execution unit.

[0030] Specifically, the matching processing unit is the basic processing unit required by the business requirement.

[0031] Specifically, as Figure 2 shown, determining that the correlation degree between business requirements meets the requirements specifically includes: Based on the overlapping data of the matching processing units for different business requirements, determine the business requirements with the same matching processing units and use them as the same business requirements; Divide the same business requirements into the same requirement group; Based on the number of business requirements in different requirement groups, determine whether the correlation degree between business requirements meets the requirements.

[0032] It can be understood that when the number of requirement groups with the number of business requirements greater than the preset business requirement quantity threshold is greater than the preset group quantity, it is determined that the correlation degree between business requirements does not meet the requirements.

[0033] It should be noted that when the correlation degree between business requirements does not meet the requirements, the first estimation method is used for the measurement processing of the computing power requirements of the business scenario.

[0034] Specifically, the first estimation method is to determine the comprehensive estimation quantity of different basic processing units based on the sum of the call estimation quantities of the business requirements corresponding to different basic processing units, determine the computing power requirements of different basic processing units based on the product of the comprehensive estimation quantity and the preset computing power requirement corresponding to the basic processing unit, and use the sum of the computing power requirements of different basic processing units as the allocation processing of the computing power requirements of the business scenario.

[0035] In another possible embodiment, determining that the correlation degree between business requirements meets the requirements specifically includes: Determine the same business requirements of the matching processing units for different business requirements, and use them as the same business requirements; Determine whether the degree of association between business requirements meets the requirements according to the number of the same business requirements.

[0036] Furthermore, when the number of the same business requirements is greater than the preset same requirement quantity threshold, it is determined that the degree of association between business requirements does not meet the requirements.

[0037] S2 Determine the call estimation quantity of different business requirements. Use the business requirements of all basic processing units involved as the target business requirements. Based on the deviation of the call estimation quantity between the target business requirements and other business requirements, when it is determined that the call estimation quantity of the target business requirements meets the requirements, determine the interfering business requirements of the target business requirements according to the coincidence of the call processing sequence between other business requirements and the basic processing units of the target business requirements; Furthermore, the call estimation quantity of the business requirements is determined according to the number of personnel with call permissions for the business requirements, specifically determined according to the product of the number of personnel and the preset proportional factor.

[0038] Specifically, as Figure 3 shown, determining that the call estimation quantity of the target business requirements meets the requirements specifically includes: According to the call processing sequence of the basic processing units of the business requirements, determine the first-called basic processing unit, and use it as the target processing unit. Use the sum of the call estimation quantities of the business requirements with the same target processing unit as the speculation estimation quantity of the target processing unit; Based on the deviation of the call estimation quantity of the target business requirements from the speculation estimation quantity in different target processing units, determine the deviation quantity in different target processing units; When the deviation quantities in different target processing units are all within the preset range, it is determined that the call estimation quantity of the target business requirements meets the requirements.

[0039] It should be noted that when the call estimation quantity of the target business requirements does not meet the requirements, directly use the speculation estimation quantity of the target processing unit as the basic estimation quantity of the target business requirements in the target processing unit. Otherwise, use the call estimation quantity of the target business requirements in the target processing unit as the basic estimation quantity of different basic processing units.

[0040] It can be understood that determining that the call estimation quantity of the target business requirements meets the requirements specifically includes: Determine the first-called basic processing unit according to the call processing order of the basic processing units required by the business needs, and use it as the target processing unit. Take the sum of the call estimation amounts of the business needs that are consistent with the target processing unit as the speculative estimation amount of the target processing unit; Determine the deviation amounts for different target processing units based on the deviation between the call estimation amounts and the speculative estimation amounts of the target business needs in different target processing units; Determine whether the call estimation amount of the target business needs meets the requirements according to the maximum value of the deviation amounts for different target processing units.

[0041] Furthermore, when the maximum value of the deviation amounts for different target processing units is less than the preset deviation threshold, it is determined that the call estimation amount of the target business needs meets the requirements.

[0042] Specifically, as Figure 4 shown, the method for determining the interfering business needs of the target business needs is: Determine the basic processing units with the same call processing order based on the coincidence of the call processing order of other business needs and the basic processing units of the target business needs, and use them as the units with the same order; Determine the coincident basic processing units based on the coincidence of other business needs and the basic processing units of the target business needs, and use them as the coincident units; Determine whether the target business needs are interfering business needs based on the number of the coincident units and the number of the units with the same order.

[0043] Furthermore, determining whether the target business needs are interfering business needs based on the number of the coincident units and the number of the units with the same order specifically includes: When the first-called basic processing unit of the other business needs is the same as that of the target business needs, it is determined that it belongs to the interfering business needs; When the first-called basic processing unit of the other business needs is different from that of the target business needs, determine the coincidence number according to the average value of the number of the coincident units and the number of the units with the same order. When the coincidence number is greater than the preset number threshold, it is determined that the other business needs are interfering business needs. When the coincidence number is greater than the preset number threshold, it is determined that it does not belong to the interfering business needs.

[0044] S3 Determine the estimation processing method for the computing power requirements of the business scenario based on the call estimation amounts of the interfering business needs and the coincidence data with the basic processing units of the target business needs, and determine the allocation processing of the computing power requirements of the business scenario based on the estimation processing method.

[0045] It should be noted that the method for determining the method for estimating the computing power requirements of the business scenario is as follows: Based on the coincidence of the call processing order of the basic processing units of the interfering service demand and the target service demand, determine the basic processing units with the same call processing order, and use them as the units with the same order. Based on the coincidence of the basic processing units of the interfering service demand and the target service demand, determine the coincident basic processing units, and use them as the coincident units. According to the average value of the number of units with the same order and the number of coincident units, determine the number of coincidences; Based on the call estimation quantity of the interfering service demand, determine the interfering service demands with a call estimation quantity greater than the preset call estimation quantity threshold, and use them as the frequently called demands; According to the number of coincidences of different frequently called demands, determine the method for estimating the computing power requirements of the business scenario.

[0046] Furthermore, according to the number of coincidences of different frequently called demands, determining the method for estimating the computing power requirements of the business scenario specifically includes: When the sum of the number of coincidences of different frequently called demands is greater than the preset number of coincidences threshold, then use the sum of the call estimation quantities of different interfering service demands in different basic processing units and the basic estimation quantities of the target service demand in different basic processing units as the target estimation quantity, and perform the estimation processing of the computing power requirements of the business scenario based on the target estimation quantity; When the sum of the number of coincidences of different frequently called demands is not greater than the preset number of coincidences threshold, then use the basic estimation quantities of the target service demand in different basic processing units as the target estimation quantity, and perform the estimation processing of the computing power requirements of the business scenario based on the target estimation quantity.

[0047] It can be understood that when the sum of the number of coincidences of different frequently called demands is not greater than the preset number of coincidences threshold, it is also necessary to further judge the coincidence situation between the frequently called demands and other interfering service demands in different basic processing units, specifically including: Use the other interfering service demands with the number of coincidences of the basic processing units with the frequently called demands greater than the preset value of the number of coincidences as the associated service demands; According to the sum of the associated service demands and the frequently called demands in different basic processing units and the sum of the basic estimation quantities of the target service demand in different basic processing units, use them as the target estimation quantity, and perform the estimation processing of the computing power requirements of the business scenario based on the target estimation quantity.

[0048] In another possible embodiment, the method for determining the method for estimating the computing power requirements of the business scenario is as follows: Estimate the call volume of different interfering service requirements, and combine the call volume estimates of other interfering service requirements and the overlapping situation of the interfering service requirements in different basic processing units to determine the interference values of different interfering service requirements; It should be further noted that in the above steps, when the number of interfering service requirements does not meet the requirements, that is, the number is too large, specifically greater than the threshold, then the sum of the call volume estimates of different interfering service requirements in different basic processing units and the basic estimates of the target service requirement in different basic processing units is used as the target estimate, and the computing power requirement estimation process of the service scenario is carried out based on the target estimate.

[0049] In addition, even when the number of interfering service requirements is small, it is necessary to consider the call volume estimates of different interfering service requirements. When the sum of the call volume estimates of different interfering service requirements does not meet the requirements, that is, the call volume estimate is too large, specifically, it can be determined by setting a threshold, then the sum of the call volume estimates of different interfering service requirements in different basic processing units and the basic estimates of the target service requirement in different basic processing units is also used as the target estimate, and the computing power requirement estimation process of the service scenario is carried out based on the target estimate.

[0050] It should also be noted that if the above conditions are all met, and at the same time, the interference values of different interfering service requirements are all less than the preset interference threshold, it means that the computing power requirements of other interfering service requirements are relatively small. Therefore, the basic estimates of the target service requirement in different basic processing units can be used as the target estimate, and the computing power requirement estimation process of the service scenario is carried out based on the target estimate.

[0051] However, at the same time, if there are multiple interfering service requirements with large interference values, it also means that the interference degree of the interfering service requirements is relatively large. Therefore, the sum of the call volume estimates of different interfering service requirements in different basic processing units and the basic estimates of the target service requirement in different basic processing units is also used as the target estimate, and the computing power requirement estimation process of the service scenario is carried out based on the target estimate.

[0052] Through the above method of generating differential computing power requirement estimates according to the interference situation of service requirements, it is possible to fully consider the difference in the interference degree between other service requirements and the target service requirement on the basis of meeting the computing power requirements in the service scenario, that is, the difference in the requirements for multiple basic processing units to be called simultaneously by other service requirements and the target service requirement. This not only ensures that a larger computing power space is allocated when the demand degree is large, but also avoids the waste of computing power space when the concurrent processing demand is low.

[0053] Based on the coincidence of the call processing order of the interfering service requirements and the basic processing units of the target service requirements, determine the basic processing units with the same call processing order and use them as the units with the same order. Based on the coincidence of the basic processing units of the interfering service requirements and the target service requirements, determine the coincident basic processing units and use them as the coincident units. According to the average value of the number of units with the same order and the number of coincident units, determine the number of coincidences; It can be understood that when the sum of the number of coincidences of different call frequency requirements is greater than the preset coincidence number threshold, then the sum of the call estimation amounts of different interfering service requirements in different basic processing units and the basic estimation amounts of the target service requirements in different basic processing units is used as the target estimation amount, and the estimation processing of the computing power requirements of the service scenario is performed based on the target estimation amount; It should be further noted that even when the sum of the number of coincidences of different call frequency requirements is not greater than the preset coincidence number threshold, it is necessary to further proceed to the next step.

[0054] Based on the number of coincidences and the interference value of different interfering service requirements, determine the interference evaluation amount of the target service requirements of the service scenario, and determine the estimation processing method of the computing power requirements of the service scenario based on the interference evaluation amount.

[0055] In one of the embodiments, the interference evaluation amount of the target service requirements can be determined according to a mathematical model based on the number of coincidences and the interference value of different interfering service requirements, specifically determined according to the output of a neural network prediction model with the number of coincidences and the interference value of different interfering service requirements as input quantities.

[0056] Specifically, when the interference evaluation amount is greater than 0.35, it indicates that the interference degree of the interfering service requirements is relatively large, that is, the requirement degree of simultaneously calling multiple basic units is relatively large. Therefore, it is also necessary to use the sum of the call estimation amounts of different interfering service requirements in different basic processing units and the basic estimation amounts of the target service requirements in different basic processing units as the target estimation amount, and perform the estimation processing of the computing power requirements of the service scenario based on the target estimation amount. When the interference evaluation amount is less than 0.1, the basic estimation amounts of the target service requirements in different basic processing units are used as the target estimation amount, and the estimation processing of the computing power requirements of the service scenario is performed based on the target estimation amount.

[0057] If it is between 0.1 and 0.35, other interfering service requirements with the number of coincidences greater than the preset value of the number of coincidences of the basic processing unit with frequent calls are regarded as associated service requirements. Based on the sum of the associated service requirements and the frequent call requirements in different basic processing units, and the sum of the basic estimation amounts of the target service requirements in different basic processing units, as the target estimation amount, the estimation processing of the computing power requirements of the service scenario is performed based on the target estimation amount.

[0058] Through the above estimation processing, the differences in the demand degrees of concurrent call processing of different basic processing units in the actual operation process are fully considered, and then the strategy of differential estimation processing is determined, so as to ensure the computing power requirements in the service scenario on the basis of reducing the application cost of computing power.

[0059] Embodiment 2 In the second aspect, as Figure 5 shown, the present invention provides a computer system, including: a memory and a processor connected by communication, and a computer program stored on the memory and capable of running on the processor. When the processor runs the computer program, it executes the above-mentioned method for evaluating the computing power requirements of the service scenario of a large language model.

[0060] Optionally, determining that the degree of association between service requirements meets the requirements specifically includes: S11 Use the coincidence data of the matching processing units of different service requirements to determine the service requirements with the same matching processing unit, and regard them as the same service requirements; Specifically, when there are no same service requirements, it can be directly determined that the degree of association between service requirements meets the requirements.

[0061] It can be understood that when the number of same service requirements is relatively large, that is, greater than a fixed threshold, it is directly determined that the degree of association between service requirements does not meet the requirements, and without considering other data, it can be determined that the degree of association between service requirements does not meet the requirements at this time.

[0062] It should be further noted that when the number of same service requirements is between 1 and 2, it can also be determined that the degree of association between service requirements meets the requirements. Only when the number of same service requirements is within a certain number range, it is necessary to go to step S12.

[0063] S12 Divide the same service requirements into the same requirement group, and determine the processing unit coincidence value of different requirement groups according to the number of same service requirements in different requirement groups and the deviation situation of the matching processing units with other service requirements; Specifically, when the number of demand groups is relatively large, and at the same time the number of the same business demands is within a certain range, it can be directly determined that the degree of association between business demands does not meet the requirements.

[0064] In addition, if the number of demand groups whose number of business demands is greater than the preset business demand quantity threshold is greater than the preset group quantity, it can also be determined that the degree of association between business demands does not meet the requirements. When and only when the number of demand groups whose number of business demands is greater than the preset business demand quantity threshold is not greater than the preset group quantity, there is no need to further evaluate the coincidence value of the processing units of the demand groups at this time.

[0065] Specifically, when there are multiple demand groups whose coincidence value of processing units is greater than the threshold, it can be directly determined that the degree of association between business demands does not meet the requirements. When there are no multiple demand groups whose coincidence value of processing units is greater than the threshold, proceed to the next step.

[0066] It can be understood that since the coincidence value of the processing units and the demand coincidence value of the demand groups have fixed input and output quantities, those skilled in the art can determine them through neural networks and other means on this basis.

[0067] S13 Determine the demand coincidence value between business demands according to the number of demand groups and the coincidence value of the processing units of different demand groups, and determine whether the association situation between business demands meets the requirements based on the demand coincidence value.

[0068] It can be understood that the value range of the demand coincidence value is between 0 and 1. When the demand coincidence value is greater than the preset demand coincidence threshold, it is determined that the association situation between business demands meets the requirements.

[0069] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the device, equipment, and non-volatile computer storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0070] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the specific order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0071] The above are only one or more embodiments of this specification and are not intended to limit this specification. For those skilled in the art, various changes and modifications can be made to one or more embodiments of this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included within the scope of the claims of this specification.

Claims

1. A method for evaluating the computing power requirements of business scenarios of large language models, characterized in that, Specifically include: Divide the large language model into multiple basic processing units. Based on the requirements of the basic processing units for different business needs in the business scenario, determine the matching processing units for different business needs. When the degree of association between business needs meets the requirements based on the overlapping data of the matching processing units for different business needs, proceed to the next step; Determine the estimated call volume for different business needs. Consider all the business needs of the basic processing units involved as target business needs. Based on the deviation of the estimated call volume between the target business needs and other business needs, when the estimated call volume of the target business needs meets the requirements, determine the interfering business needs of the target business needs based on the overlapping situation of the call processing order between other business needs and the basic processing units of the target business needs; Based on the estimated call volume of the interfering business needs and the overlapping data with the basic processing units of the target business needs, determine the estimation processing method for the computing power requirements of the business scenario, and based on the estimation processing method, determine the allocation processing of the computing power requirements of the business scenario.

2. The method for evaluating the computing power requirements of the business scenarios of the large language model according to claim 1, wherein, The basic processing units include a modality conversion unit, a description and vectorization unit, a deep reasoning unit, a complex execution unit, and a simple execution unit.

3. The method for evaluating the computing power requirements of the business scenarios of the large language model according to claim 1, wherein The matching processing unit is the basic processing unit required by the business need.

4. The method for evaluating the computing power requirements of the business scenarios of the large language model according to claim 1, wherein, Determining that the degree of association between business needs meets the requirements specifically includes: Based on the overlapping data of the matching processing units for different business needs, determine the business needs with the same matching processing unit and regard them as the same business needs; Divide the same business needs into the same requirement group; Based on the number of business needs in different requirement groups, determine whether the degree of association between business needs meets the requirements.

5. The method for evaluating the computing power requirements of the business scenarios of the large language model according to claim 1, wherein, When the degree of association between business needs does not meet the requirements, then use the first estimation method to measure the computing power requirements of the business scenario.

6. The method for evaluating the computing power requirements of the business scenarios of the large language model according to claim 5, wherein, The first estimation method is to determine the comprehensive estimation volume of different basic processing units by summing the estimated call volumes of the business needs corresponding to different basic processing units, determine the computing power requirements of different basic processing units by multiplying the comprehensive estimation volume by the preset computing power requirements corresponding to the basic processing units, and take the sum of the computing power requirements of different basic processing units as the allocation processing of the computing power requirements of the business scenario.

7. The method for evaluating the computing power requirements of the business scenarios of the large language model according to claim 1, wherein The method for determining the estimation processing method for the computing power requirements of the business scenario is: Based on the overlapping situation of the call processing order between the interfering business needs and the basic processing units of the target business needs, determine the basic processing units with the same call processing order and regard them as the order-consistent units. Based on the overlapping situation of the interfering business needs and the basic processing units of the target business needs, determine the overlapping basic processing units and regard them as the overlapping units. Determine the overlapping quantity based on the average value of the number of order-consistent units and the number of overlapping units; Based on the estimated call volume of the interfering business needs, determine the interfering business needs with an estimated call volume greater than the preset estimated call volume threshold and regard them as the frequently called requirements; Determine the estimation processing method for the computing power requirement of the business scenario according to the overlapping quantity of different call frequency requirements.

8. The method for evaluating the computing power requirements of the business scenarios of the large language model according to claim 7, wherein, Determine the estimation processing method for the computing power requirement of the business scenario according to the overlapping quantity of different call frequency requirements, specifically including: When the sum of the overlapping quantities of different call frequency requirements is greater than the preset overlapping quantity threshold, then the sum of the call estimation quantities of different interference service requirements in different basic processing units and the basic estimation quantities of the target service requirement in different basic processing units is used as the target estimation quantity, and the estimation processing of the computing power requirement of the business scenario is carried out based on the target estimation quantity; When the sum of the overlapping quantities of different call frequency requirements is not greater than the preset overlapping quantity threshold, then the basic estimation quantities of the target service requirement in different basic processing units are used as the target estimation quantity, and the estimation processing of the computing power requirement of the business scenario is carried out based on the target estimation quantity.

9. The method for evaluating the computing power requirements of the business scenarios of the large language model according to claim 8, wherein, When the sum of the overlapping quantities of different call frequency requirements is not greater than the preset overlapping quantity threshold, it is also necessary to further judge the overlapping situation between the call frequency requirement and other interference service requirements in different basic processing units, specifically including: Take other interference service requirements with the overlapping quantity greater than the preset overlapping quantity value in the basic processing unit of the call frequency requirement as associated service requirements; According to the sum of the associated service requirements and the call frequency requirement in different basic processing units and the sum of the basic estimation quantities of the target service requirement in different basic processing units, as the target estimation quantity, the estimation processing of the computing power requirement of the business scenario is carried out based on the target estimation quantity.

10. A computer system, comprising: A memory and a processor connected by communication, and a computer program stored on the memory and capable of running on the processor, wherein when the processor runs the computer program, it executes the method for evaluating the computing power requirement of the business scenario of a large language model according to any one of claims 1-9.

Images