A method and device for measuring computing power based on throughput as a metric

Abstract

The embodiment of the present invention provides a computing power measurement method and device based on throughput as a measurement indicator, the method comprising: obtaining computing power card data tables of multiple resource pools; the computing power card data table comprises multiple data entries, one data entry comprises parameters collected by calling the computing power card to perform intelligent computing services; the intelligent computing services are to call models to perform reasoning tasks or training tasks; obtaining evaluation requirement information; according to the computing power card data table, querying the evaluation parameters corresponding to calling the computing power card to perform intelligent computing services according to the evaluation requirement information; the evaluation parameters at least include unit throughput. By obtaining the computing power card data tables of multiple resource pools and aggregating different training models and business parameters, the corresponding indicator parameters can be quickly obtained according to the computing power card data table, and the computing power card or business performance can be uniformly evaluated and predicted based on the obtained indicator parameters.

Description

Technical Field

[0001] The present invention relates to the field of computing power measurement technology, and in particular to a computing power measurement method based on throughput as a measurement indicator, a computing power measurement device based on throughput as a measurement indicator, an electronic device and a computer-readable storage medium. Background Art

[0002] There are more and more heterogeneous computing cards (Compute Cards) replacing GPU cards in the computing industry. Different heterogeneous computing card manufacturers have different integration solutions. Each computing card manufacturer only specifies hardware indicators such as the number of floating-point calculations per unit time, the size of the computing card HBM memory, the HBM memory bandwidth, and the communication bandwidth. Intelligent computing services are mainly based on training and reasoning. This type of business is not a simple floating-point calculation, but a comprehensive application of operators, hard card storage, communication, and hardware acceleration units. Therefore, the differences in indicators among manufacturers are also large. Heterogeneous computing cards with similar indicator values ​​have large performance differences in the same business scenarios, and there is no unified evaluation standard.

[0003] Although there is a mapping relationship between hardware indicators and business performance, hardware indicators alone cannot effectively reflect business performance, and currently computing power manufacturers only specify indicators for a single computing power element, not comprehensive indicators. When there are many business parameters in different training and reasoning scenarios, it is impossible to effectively predict computing power card resource usage and evaluate computing power cards or business performance without unified measurement and evaluation standards. Summary of the invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a computing power measurement method based on throughput as a measurement indicator, a computing power measurement device based on throughput as a measurement indicator, an electronic device and a computer storage medium to overcome the above problems or at least partially solve the above problems.

[0005] In order to solve the above problem, a first aspect of an embodiment of the present invention provides a computing power measurement method based on throughput as a metric, the method comprising:

[0006] Obtain computing power card data tables of multiple resource pools; the computing power card data table includes multiple data entries, one of the data entries includes parameters collected by calling the computing power card to execute intelligent computing services, the parameters include: service type, model name, model key parameters, computing power hardware indicators, minimum hardware resources, average power consumption, unit throughput; the intelligent computing service is calling the model to execute inference tasks or training tasks;

[0007] Obtain assessment needs information;

[0008] According to the computing power card data table, the evaluation parameters corresponding to calling the computing power card to execute the intelligent computing business according to the evaluation requirement information are queried; the evaluation parameters at least include the unit throughput.

[0009] Optionally, the computing power card data tables of the multiple resource pools are generated in the following manner:

[0010] Measure the computing power hardware indicators of computing power cards in multiple resource pools;

[0011] Determine whether the computing power hardware index of the computing power card is greater than a test threshold;

[0012] If the computing power hardware index of the computing power card is greater than the test threshold, then obtaining the parameters of the computing power card for executing the intelligent computing service;

[0013] A computing power card data table is generated according to the parameters of the computing power card for executing the intelligent computing business and the computing power hardware indicators.

[0014] Optionally, obtaining the assessment requirement information includes:

[0015] Obtaining first evaluation requirement information; the first evaluation requirement information includes business type, model name, parameter quantity, accuracy, number of cycles, business data size and business completion time;

[0016] According to the computing power card data table, querying the evaluation parameters corresponding to calling the computing power card to execute the intelligent computing business according to the evaluation requirement information; the evaluation parameters at least include the unit throughput, including:

[0017] For multiple resource pools that need to be predicted, query the computing power card data table respectively according to the first evaluation demand information, and call the computing power cards in each resource pool to execute the unit throughput corresponding to the intelligent computing business;

[0018] According to the unit throughput corresponding to the computing power cards in each resource pool being called to execute the intelligent computing business, the number of computing power cards and power consumption required for the multiple resource pools that need to be predicted to execute the intelligent computing business are determined.

[0019] Optionally, determining the number and power consumption of computing power cards required for the plurality of resource pools to be predicted to execute the intelligent computing service according to the unit throughput corresponding to the calling of computing power cards in each resource pool to execute the intelligent computing service includes:

[0020] Determine the number of computing power cards required for the multiple resource pools to be predicted to execute the intelligent computing service according to the service data size, the number of cycles, the service completion time, and the unit throughput corresponding to the calling of computing power cards in each resource pool to execute the intelligent computing service;

[0021] According to the computing power card data table, query the average power consumption corresponding to calling the computing power cards in each resource pool to execute the intelligent computing business according to the first evaluation requirement information;

[0022] The power consumption required for executing intelligent computing services by the multiple resource pools that need to be predicted is determined according to the number of computing power cards required by each resource pool and the average power consumption.

[0023] Optionally, obtaining the assessment requirement information includes:

[0024] Obtaining second evaluation requirement information; the second evaluation requirement information includes a business type, multiple model names, and a computing power card name;

[0025] According to the computing power card data table, querying the evaluation parameters corresponding to calling the computing power card to execute the intelligent computing business according to the evaluation requirement information; the evaluation parameters at least include the unit throughput, including:

[0026] According to the computing power card data table, query the unit throughput corresponding to calling a computing power card to use multiple models to execute intelligent computing services according to the second evaluation requirement information;

[0027] Sorting according to the unit throughput corresponding to calling a computing power card to use multiple models to execute intelligent computing services;

[0028] According to the sorting results, the performance rankings of the multiple models are determined.

[0029] Optionally, obtaining the assessment requirement information includes:

[0030] Obtaining third evaluation requirement information; the third evaluation requirement information includes names of multiple computing power cards;

[0031] According to the computing power card data table, querying the evaluation parameters corresponding to calling the computing power card to execute the intelligent computing business according to the evaluation requirement information; the evaluation parameters at least include the unit throughput, including:

[0032] According to the computing power card data table, query the unit throughput corresponding to calling multiple computing power cards to execute intelligent computing services according to the third evaluation requirement information;

[0033] Sorting according to the unit throughput corresponding to calling the multiple computing power cards to execute the intelligent computing business;

[0034] According to the sorting results, the performance rankings of the multiple computing power cards are determined.

[0035] Optionally, the sorting according to the unit throughput corresponding to the calling of multiple computing power cards to execute the intelligent computing business includes:

[0036] Performing a weighted average calculation on the unit throughputs corresponding to calling the multiple computing power cards to execute the intelligent computing services, to obtain a weighted average of the unit throughputs;

[0037] The weighted average values ​​of the unit throughputs are sorted to obtain a sorting result.

[0038] According to a second aspect of an embodiment of the present invention, there is provided a computing power measurement device based on throughput as a metric, the device comprising:

[0039] A data table acquisition module is used to obtain the computing power card data tables of multiple resource pools; the computing power card data table includes multiple data entries, one of which includes parameters collected by calling the computing power card to execute intelligent computing services, and the parameters include: service type, model name, model key parameters, computing power hardware indicators, minimum hardware resources, average power consumption, and unit throughput; the intelligent computing service is to call the model to execute an inference task or a training task;

[0040] A demand information acquisition module is used to acquire evaluation demand information;

[0041] An evaluation parameter acquisition module is used to query the evaluation parameters corresponding to calling the computing power card to execute the intelligent computing business according to the evaluation requirement information according to the computing power card data table; the evaluation parameters at least include the unit throughput.

[0042] The data table acquisition module includes:

[0043] The indicator measurement submodule is used to measure the computing power hardware indicators of the computing power cards of multiple resource pools;

[0044] The indicator judgment submodule is used to judge whether the computing power hardware indicator of the computing power card is greater than the test threshold;

[0045] A parameter acquisition submodule, used for acquiring parameters for the computing power card to execute intelligent computing services if the computing power hardware index of the computing power card is greater than a test threshold;

[0046] The data table generation submodule is used to generate a computing power card data table according to the parameters of the computing power card for executing intelligent computing services and computing power hardware indicators.

[0047] Optionally, the demand information acquisition module includes:

[0048] A first requirement information acquisition submodule is used to acquire first evaluation requirement information; the first evaluation requirement information includes business type, model name, parameter quantity, accuracy, number of cycles, business data size and business completion time;

[0049] The evaluation parameter acquisition module includes:

[0050] A first query submodule is used to query, for multiple resource pools that need to be predicted, the unit throughput corresponding to the intelligent computing business executed by the computing power cards in each resource pool according to the first evaluation requirement information according to the computing power card data table;

[0051] The first determination submodule is used to determine the number and power consumption of computing power cards required for the multiple resource pools that need to be predicted to execute the intelligent computing business based on the unit throughput corresponding to the computing power cards in each resource pool being called to execute the intelligent computing business.

[0052] Optionally, the first determining submodule includes:

[0053] A quantity determination subunit is used to determine the number of computing power cards required for the multiple resource pools to be predicted to perform the intelligent computing business according to the business data size, the number of cycles, the business completion time and the unit throughput corresponding to the calling of the computing power cards in each resource pool to perform the intelligent computing business;

[0054] A power consumption query subunit, configured to query, according to the computing power card data table, the average power consumption corresponding to calling the computing power cards in each resource pool to execute the intelligent computing service according to the first evaluation requirement information;

[0055] The power consumption determination subunit is used to determine the power consumption required for the multiple resource pools that need to be predicted to execute intelligent computing services based on the number of computing power cards required by each resource pool and the average power consumption.

[0056] Optionally, the demand information acquisition module includes:

[0057] A second requirement information acquisition submodule is used to acquire second evaluation requirement information; the second evaluation requirement information includes a business type, multiple model names, and a computing power card name;

[0058] The evaluation parameter acquisition module includes:

[0059] A second query submodule is used to query, according to the computing power card data table, the unit throughput corresponding to calling a computing power card according to the second evaluation requirement information to use multiple models to execute intelligent computing services;

[0060] A first sorting submodule is used to sort according to the unit throughput corresponding to calling a computing power card to use multiple models to execute intelligent computing services;

[0061] The model performance determination submodule is used to determine the performance ranking of the multiple models according to the sorting results.

[0062] Optionally, the demand information acquisition module includes:

[0063] A third requirement information acquisition submodule is used to acquire third evaluation requirement information; the third evaluation requirement information includes the names of multiple computing power cards;

[0064] The evaluation parameter acquisition module includes:

[0065] A third query submodule is used to query the unit throughput corresponding to calling multiple computing power cards to execute intelligent computing services according to the computing power card data table according to the third evaluation requirement information;

[0066] The second sorting submodule is used to sort according to the unit throughput corresponding to calling multiple computing power cards to execute intelligent computing services;

[0067] The computing power card performance determination submodule is used to determine the performance rankings of the multiple computing power cards according to the sorting results.

[0068] Optionally, the second sorting submodule includes:

[0069] The average value acquisition subunit is used to perform a weighted average operation on the unit throughput corresponding to calling the multiple computing power cards to execute the intelligent computing business, so as to obtain the weighted average value of the unit throughput;

[0070] The average value sorting subunit is used to sort the weighted average values ​​of the unit throughputs to obtain a sorting result.

[0071] According to a third aspect of an embodiment of the present invention, an electronic device is provided, comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of a computing power measurement method based on throughput as a metric as described above are implemented.

[0072] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a computing power measurement method based on throughput as a metric as described above are implemented.

[0073] The technical solution provided by the embodiments of the present invention may have the following beneficial effects:

[0074] The embodiment of the present invention provides a computing power measurement method based on throughput as a measurement indicator, the method comprising: obtaining computing power card data tables of multiple resource pools; the computing power card data table comprises multiple data entries, one of the data entries comprises parameters collected by calling the computing power card to execute intelligent computing services, the parameters comprising: service type, model name, model key parameters, computing power hardware indicators, minimum hardware resources, average power consumption, unit throughput; the intelligent computing service is to call the model to execute reasoning tasks or training tasks; obtain evaluation requirement information; according to the computing power card data table, query the evaluation parameters corresponding to calling the computing power card to execute the intelligent computing service according to the evaluation requirement information; the evaluation parameters at least include the unit throughput. By obtaining the computing power card data tables of multiple resource pools and aggregating different training models and business parameters, the corresponding indicator parameters can be quickly obtained according to the computing power card data table, and the computing power card or business performance can be uniformly evaluated and predicted according to the obtained indicator parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 It is a flowchart of a method for measuring computing power based on throughput as a metric provided by an embodiment of the present invention;

[0076] Figure 2 It is a logic block diagram of a computing power measurement method based on throughput as a measurement indicator provided by an embodiment of the present invention;

[0077] Figure 3 It is a structural block diagram of a computing power measurement device based on throughput as a measurement indicator provided by an embodiment of the present invention;

[0078] Figure 4 is a structural block diagram of an electronic device provided by an embodiment of the present invention;

[0079] Figure 5 It is a structural block diagram of a computer storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0080] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0081] One of the core concepts of the embodiment of the present invention is that when there are many business parameters in different training and reasoning scenarios, it is impossible to effectively predict the use of computing power card resources and evaluate the performance of computing power cards or services without unified measurement and evaluation standards. Therefore, the embodiment of the present invention provides a computing power measurement method based on throughput as a measurement indicator, the method comprising: obtaining computing power card data tables of multiple resource pools; the computing power card data table includes multiple data entries, one of the data entries includes parameters collected by calling the computing power card to perform intelligent computing services, the parameters include: business type, model name, model key parameters, computing power hardware indicators, minimum hardware resources, average power consumption, unit throughput; the intelligent computing service is to call the model to perform reasoning tasks or training tasks; obtain evaluation requirement information; according to the computing power card data table, query the evaluation parameters corresponding to calling the computing power card to perform intelligent computing services according to the evaluation requirement information; the evaluation parameters at least include the unit throughput. By obtaining the computing power card data tables of multiple resource pools to aggregate different training models and business parameters, the corresponding indicator parameters can be quickly obtained according to the computing power card data table, and the computing power card or service performance can be uniformly evaluated and predicted according to the acquired indicator parameters.

[0082] Reference Figure 1 , shows a flowchart of a method for measuring computing power based on throughput as a metric provided by an embodiment of the present invention, wherein the method may specifically include the following steps:

[0083] Step 101, obtaining a computing power card data table of multiple resource pools; the computing power card data table includes multiple data entries, one of the data entries includes parameters collected by calling a computing power card to execute an intelligent computing service, the parameters include: service type, model name, model key parameters, computing power hardware indicators, minimum hardware resources, average power consumption, unit throughput; the intelligent computing service is calling a model to execute an inference task or a training task;

[0084] A computing power card is a hardware device used for computing and processing tasks, and is typically used in high-performance computing, data analysis, artificial intelligence and other fields.

[0085] Resource Pool refers to the centralized management of a group of available resources so that they can be efficiently allocated and used when needed.

[0086] The business types include large language model training, visual model training, natural language model training, and audio model training, which are not limited in the embodiments of the present invention. Models of the same category are comparable and have similar performance.

[0087] The model name refers to the specific test model name such as but not limited to: LLAMA1, Qwen, resnet50, yoyo, bert, bert_hf and other models.

[0088] The key parameters of the model refer to the parameters that can significantly affect the computing power performance of the computing power card, including but not limited to the training framework, weight model size, parameter quantity, floating point precision or variable precision, etc. For training models, there are also convergence speeds, etc. The embodiments of the present invention include but are not limited to the parameters mentioned above.

[0089] The computing power hardware indicators refer to the actual test indicators of various heterogeneous computing powers collected by the collector, mainly including computing power card hardware model, hardware topology, f16 / f32 / f64, variable precision floating point operation rate, computing power card HBM memory size, computing power card HBM memory bandwidth, computing power card communication bandwidth, storage size, storage bandwidth, power consumption, etc. The measurement and collection of computing power hardware indicators mainly measure the deviation from the manufacturer's rated value and establish a mapping relationship with the final business throughput. The test values ​​of computing power resource pools with the same hardware configuration are similar.

[0090] Minimum hardware resources refer to the minimum type of heterogeneous computing power cards required for a model or the number of nodes and heterogeneous computing power cards required to complete the training task.

[0091] Unit power consumption refers to the average power consumption of each computing power card for different computing power cards during the measurement process.

[0092] Throughput is the main measurement indicator for intelligent computing services such as training and inference. It is the amount of business data processed per unit time. The larger the business data processed, the better the performance. Unit throughput refers to the throughput of a single computing power card per second.

[0093] In this embodiment, the computing power measurement unit sends hardware indicator measurement tasks to the same heterogeneous computing power resource pool to obtain parameters. The collector collects relevant parameters according to the hardware measurement tasks. The data aggregation unit tests the collected parameters of each resource pool to obtain test results, and aggregates and stores the test result data to obtain computing power card data tables of multiple resource pools.

[0094] The computing power card data table includes multiple data entries. One data entry includes the parameters collected by calling the computing power card to execute the intelligent computing business. That is, each computing power card has corresponding parameters for executing the intelligent computing business. For example, one business type can correspond to multiple training models and obtain multiple data entries.

[0095] In some embodiments, step 101 includes the following sub-steps:

[0096] Sub-step S11, measuring the computing power hardware indicators of the computing power cards of multiple resource pools;

[0097] Computing power hardware indicators are key parameters for evaluating and comparing the performance of different computing devices, and are used to understand the processing power, efficiency, and applicable scenarios of the hardware.

[0098] In this embodiment, the computing power measurement unit first sends a hardware indicator measurement task to the same heterogeneous computing power resource pool to measure the computing power hardware indicators of the computing power cards of multiple resource pools, mainly including the computing power card hardware model, hardware topology, f16 / f32 / f64, variable precision floating point operation rate, computing power card HBM memory size, computing power card HBM memory bandwidth, communication bandwidth between computing power cards, storage size, storage bandwidth, power consumption, etc. The collector collects the computing power hardware indicator information of the above multiple resource pools to obtain the computing power hardware indicators of the computing power cards of multiple resource pools.

[0099] Sub-step S12, determining whether the computing power hardware index of the computing power card is greater than a test threshold;

[0100] The test threshold is the nominal value of the computing power manufacturer multiplied by the threshold coefficient, and the threshold coefficient is in the range of 0-1.

[0101] In this embodiment, the computing power hardware indicators of the computing power cards of multiple resource pools collected by the collector are compared and verified with the nominal values ​​of the computing power manufacturers to determine whether the collected computing power hardware indicators of the computing power cards are greater than the test threshold, that is, to determine whether the computing power hardware indicators of the computing power cards are greater than the nominal value of the computing power manufacturer multiplied by the threshold coefficient.

[0102] Sub-step S13, if the computing power hardware index of the computing power card is greater than the test threshold, obtaining the parameters of the computing power card for executing the intelligent computing service;

[0103] After the collected computing power hardware indicators of the computing power card are judged, if the computing power hardware indicators of the computing power card are greater than or equal to the test threshold, it means that it meets the test range, and the parameters of the computing power card for executing the intelligent computing business are obtained to continue the test. If the computing power hardware indicators of the computing power card are less than the test threshold, it means that there is a problem with its hardware integration or the resources have been occupied, and the test is stopped.

[0104] In this embodiment, after the computing power hardware index of the computing power card is judged, if the computing power hardware index of the computing power card is greater than the test threshold, it means that it meets the test range, and the parameters of the computing power card for executing the intelligent computing business will be obtained to continue the test.

[0105] Sub-step S14, generating a computing power card data table according to the parameters of the computing power card for executing the intelligent computing service and the computing power hardware indicators.

[0106] The computing power measurement unit sends training tasks to multiple computing power resource pools one by one. The training tasks can include large language model training, visual model training, natural language model training, audio model training, multimodal model training, and the corresponding reasoning models of these models, but are not limited to the listed models. The collector collects the key parameters of training or reasoning tasks, such as throughput. If there are multiple cards or multiple machines and multiple cards, the total throughput needs to be divided by the number of computing power cards to obtain the unit computing power card throughput; since some training and reasoning tasks require a minimum number of computing power cards, the collector needs to collect the minimum number of computing power cards. Further obtain the parameters of the computing power card to execute the intelligent computing business.

[0107] In this embodiment, the collected computing power hardware indicators and the acquired parameters of the computing power card for executing the intelligent computing business are summarized, the relevant parameters are classified and stored to obtain different data entries, and the data entries are summarized to generate a computing power card data table.

[0108] Step 102, obtaining evaluation requirement information;

[0109] In some embodiments, step 102 includes the following sub-steps:

[0110] Sub-step S21, obtaining first evaluation requirement information; the first evaluation requirement information includes business type, model name, parameter quantity, accuracy, number of cycles, business data size and business completion time;

[0111] Parameters refer to the total number of weights and biases that need to be learned in a machine learning model. The size of the parameters directly affects the complexity, training time, and performance of the model.

[0112] In the field of machine learning and data analysis, precision is an important performance indicator used to evaluate the performance of classification models.

[0113] The number of cycles refers to the number of processing epochs. The number of epochs refers to the number of times the entire training dataset is completely traversed by the model. Each traversal of the entire dataset is called an epoch. The choice of the number of processing epochs has an important impact on the training effect, convergence speed and overfitting risk of the model.

[0114] Business data size refers to the amount of data generated in the process of executing business, including structured data (such as table data in the database) and unstructured data (such as text, images, videos, etc.).

[0115] Business Completion Time refers to the total time from the start to the end of a business.

[0116] In this embodiment, when it is necessary to predict the hardware configuration required for the business, such as the number of computing power cards and power consumption, it is necessary to obtain evaluation requirement information, and obtain corresponding results by inputting the business type, model name, parameter quantity, accuracy, number of cycles, business data size and business completion time to predict the hardware configuration required for the business, such as the number of computing power cards and power consumption.

[0117] Sub-step S22, obtaining second evaluation requirement information; the second evaluation requirement information includes a business type, multiple model names, and a computing power card name;

[0118] Compute Card is a hardware device used for high-performance computing and data processing, usually used in artificial intelligence, deep learning, scientific computing and other fields. Different compute cards have different names and models.

[0119] In this embodiment, when it is necessary to evaluate the processing performance of different models of the same type of business, it is necessary to obtain the corresponding evaluation requirement information. By inputting the business type, multiple model names, and computing power card name, the corresponding results can be output to evaluate the processing performance of different models of the same type of business.

[0120] Sub-step S23, obtaining third evaluation requirement information; the third evaluation requirement information includes the names of multiple computing power cards;

[0121] In this embodiment, when it is necessary to comprehensively evaluate the performance differences of heterogeneous computing power cards, it is necessary to obtain the names of multiple computing power cards, input the names of multiple computing power cards to output corresponding results, and comprehensively evaluate the performance differences of heterogeneous computing power cards based on the results.

[0122] In this embodiment, the computing power card data table is queried through the first evaluation requirement information, the second evaluation requirement information, and the third evaluation requirement information, and the evaluation requirement information is obtained and the evaluation parameters corresponding to the computing power card for executing the intelligent computing business are called simultaneously and in parallel.

[0123] Step 103: According to the computing power card data table, query the evaluation parameters corresponding to calling the computing power card to execute the intelligent computing business according to the evaluation requirement information; the evaluation parameters at least include the unit throughput.

[0124] Intelligent computing business refers to calling the model to perform corresponding reasoning tasks or training tasks.

[0125] Evaluation parameters refer to indicators used to evaluate model performance and effects during machine learning and data analysis. The corresponding evaluation parameters obtained based on the evaluation requirements can be used to predict the number of computing power cards and power consumption, evaluate model performance, and comprehensively evaluate computing power card performance.

[0126] In this embodiment, the generated computing power card data table can be queried according to the evaluation requirement information to obtain the evaluation parameters corresponding to the computing power card executing the intelligent computing business. The evaluation parameters include unit throughput. Based on the unit throughput, the number of computing power cards and power consumption are predicted, the model performance is evaluated, and the computing power card performance is comprehensively evaluated.

[0127] In some embodiments, step 103 includes the following sub-steps:

[0128] Sub-step S31, for multiple resource pools that need to be predicted, query the computing power card data table to query the unit throughput corresponding to the computing power card in each resource pool executed according to the first evaluation requirement information to execute the intelligent computing business;

[0129] The first evaluation requirement information includes business type, model name, parameter quantity, precision, number of cycles, business data size and business completion time, and the evaluation parameters of multiple resource pools are obtained according to the first evaluation requirement information.

[0130] In this embodiment, for different resource pools that need to be predicted, the first evaluation requirement information corresponding to the resource pool that needs to be predicted is input: business type, model name, parameter quantity, accuracy, number of cycles, business data size and business completion time. According to this information data, the computing power card data table is queried separately to obtain the unit throughput of each resource pool in the business, and the unit throughput corresponding to each resource pool is obtained.

[0131] Sub-step S32, based on the unit throughput corresponding to the computing power cards in each resource pool being called to execute the intelligent computing business, determine the number of computing power cards and power consumption required for the multiple resource pools that need to be predicted to execute the intelligent computing business.

[0132] In this embodiment, according to the unit throughput corresponding to the computing power cards in each resource pool that are obtained to execute the intelligent computing business, the number of computing power cards and power consumption required for multiple resource pools to execute the intelligent computing business are calculated according to the parameters in the evaluation requirement information, and the corresponding resource pool name and the number of computing power cards and power consumption required for the corresponding resource pool are output to predict the number of computing power cards and power consumption required for each resource pool.

[0133] In some embodiments, step S32 includes the following sub-steps:

[0134] Determine the number of computing power cards required for the multiple resource pools to be predicted to perform the intelligent computing service according to the business data size, the number of cycles, the business completion time, and the unit throughput corresponding to the calling of computing power cards in the multiple resource pools to perform the intelligent computing service;

[0135] The calculation method for the number of computing power cards is: number of computing power cards = business data size * number of cycles / unit throughput / business completion time.

[0136] In this embodiment, based on the business data size, number of cycles, business completion time in the evaluation demand information and the unit throughput corresponding to the computing power cards in each resource pool executing the intelligent computing business obtained by querying the computing power card data table, the calculation result is obtained according to the calculation method of the number of computing power cards, and the number of computing power cards required for the multiple resource pools to execute the intelligent computing business that need to be predicted is determined.

[0137] According to the computing power card data table, query the average power consumption corresponding to calling the computing power cards in each resource pool to execute the intelligent computing business according to the first evaluation requirement information;

[0138] The unit power consumption is the average power consumption of each computing power card during the computing power measurement process.

[0139] In this embodiment, the computing power card data table is queried according to the evaluation demand information to obtain the average power consumption corresponding to the computing power cards of each resource pool executing the intelligent computing business, and the average power consumption is used to subsequently obtain the predicted power consumption.

[0140] The power consumption required for executing intelligent computing services by the multiple resource pools that need to be predicted is determined according to the number of computing power cards required by each resource pool and the average power consumption.

[0141] The power consumption is calculated as follows: power consumption = number of computing power cards * average power consumption.

[0142] In this embodiment, the predicted power consumption of multiple resource pools that need to be predicted to execute intelligent computing services is calculated according to the calculated number of computing power cards and the average power consumption obtained by querying the computing power card data table in accordance with the power consumption calculation method.

[0143] Sub-step S33, querying, according to the computing power card data table, the unit throughput corresponding to calling a computing power card to use multiple models to execute intelligent computing services according to the second evaluation requirement information;

[0144] In this embodiment, the business type, multiple model names, and computing power card names in the evaluation requirement information are input to query the computing power card data table to obtain the unit throughput corresponding to the execution of intelligent computing business by multiple models.

[0145] Sub-step S34, sorting according to the unit throughput corresponding to calling a computing power card to use multiple models to execute intelligent computing services;

[0146] In this embodiment, the unit throughputs corresponding to the intelligent computing services executed by the multiple models queried are sorted from large to small, and the sorting results are output.

[0147] Sub-step S35, determining the performance rankings of the multiple models according to the sorting results.

[0148] In this embodiment, a corresponding model for unit throughput sorting is obtained according to the output sorting results, so as to determine the performance ranking of multiple models. The higher the ranking, the better the performance of the model.

[0149] Sub-step S36, querying, according to the computing power card data table, the unit throughput corresponding to calling multiple computing power cards to execute the intelligent computing business according to the third evaluation requirement information;

[0150] In this embodiment, according to the evaluation requirement information, multiple computing power card names in the evaluation requirement information are input to query the computing power card data table to obtain the unit throughput corresponding to the intelligent computing services executed by the multiple computing power cards.

[0151] Sub-step S37, sorting according to the unit throughput corresponding to the calling of the multiple computing power cards to execute the intelligent computing business;

[0152] In this embodiment, the computing power card data table is queried according to the names of different computing power cards to obtain a set of unit throughputs corresponding to different services. For the same and heterogeneous computing power cards in different resource pools, the value of the maximum unit throughput is taken to obtain the unit throughputs corresponding to the execution of intelligent computing services by multiple computing power cards, and they are sorted from large to small.

[0153] In some embodiments, step S37 includes the following sub-steps:

[0154] Performing a weighted average calculation on the unit throughputs corresponding to calling the multiple computing power cards to execute the intelligent computing services, to obtain a weighted average of the unit throughputs;

[0155] Weighted Average Calculation is a statistical method used to calculate the average of a set of data, where each data point has a weight that indicates the relative importance of the data point in calculating the average.

[0156] The weighted average calculation method for unit throughput is:

[0157]

[0158] BytesPS_Aavg is the weighted average value, ThroughputPS_Ai is the maximum unit throughput, is the number of computing power card names, and Bytes / Throughput is the number of unit throughputs queried.

[0159] In this embodiment, the corresponding unit throughput obtained by querying the computing power card data table according to the names of different computing power cards is calculated according to the weighted average operation method to obtain the weighted average value of the unit throughput.

[0160] The weighted average values ​​of the unit throughputs are sorted to obtain a sorting result.

[0161] In this embodiment, the calculated weighted average values ​​of the unit throughputs are sorted from large to small and the sorting result is output.

[0162] Sub-step S38, determining the performance rankings of the multiple computing power cards according to the sorting results.

[0163] In this embodiment, the computing power card name corresponding to the weighted average value of the unit throughput is output according to the obtained sorting result, so as to determine the performance ranking of multiple computing power cards. The higher the ranking, the higher the computing power value of the computing power card.

[0164] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0165] Reference Figure 2 , shows a logic block diagram of a computing power measurement method based on throughput as a metric provided by an embodiment of the present invention, which specifically includes the following processes:

[0166] The computing power measurement unit first sends the hardware indicator measurement task to the resource pool, and then the collector will collect the computing power hardware indicators and judge the computing power hardware indicators to determine whether the computing power hardware indicators are greater than the test threshold. If the computing power hardware indicators are less than the test threshold, the test is stopped. If the computing power hardware indicators are greater than or equal to the test threshold, the parameters are obtained to continue the test. After the parameter acquisition is completed, it is determined whether the parameters are obtained completely. If the parameters are not obtained completely, it is necessary to repeat the operation to obtain all the parameters. If the parameters are obtained completely, the acquired parameters will be used to generate a computing power card data table. According to the computing power card data table, subsequent evaluation applications are used to predict the number and power consumption of computing power cards in the resource pool, evaluate the performance of multiple models, and evaluate the performance of different computing power cards.

[0167] Reference Figure 3 , shows a structural block diagram of a computing power measurement device based on throughput as a metric provided by an embodiment of the present invention, which may specifically include the following modules:

[0168] A data table acquisition module is used to obtain the computing power card data tables of multiple resource pools; the computing power card data table includes multiple data entries, one of which includes parameters collected by calling the computing power card to execute intelligent computing services, and the parameters include: service type, model name, model key parameters, computing power hardware indicators, minimum hardware resources, average power consumption, and unit throughput; the intelligent computing service is to call the model to execute an inference task or a training task;

[0169] A demand information acquisition module is used to acquire evaluation demand information;

[0170] An evaluation parameter acquisition module is used to query the evaluation parameters corresponding to calling the computing power card to execute the intelligent computing business according to the evaluation requirement information according to the computing power card data table; the evaluation parameters at least include the unit throughput.

[0171] The data table acquisition module includes:

[0172] The indicator measurement submodule is used to measure the computing power hardware indicators of the computing power cards of multiple resource pools;

[0173] The indicator judgment submodule is used to judge whether the computing power hardware indicator of the computing power card is greater than the test threshold;

[0174] A parameter acquisition submodule, used for acquiring parameters for the computing power card to execute intelligent computing services if the computing power hardware index of the computing power card is greater than a test threshold;

[0175] The data table generation submodule is used to generate a computing power card data table according to the parameters of the computing power card for executing intelligent computing services and computing power hardware indicators.

[0176] Optionally, the demand information acquisition module includes:

[0177] A first requirement information acquisition submodule is used to acquire first evaluation requirement information; the first evaluation requirement information includes business type, model name, parameter quantity, accuracy, number of cycles, business data size and business completion time;

[0178] The evaluation parameter acquisition module includes:

[0179] A first query submodule is used to query, for multiple resource pools that need to be predicted, the unit throughput corresponding to the intelligent computing business executed by the computing power cards in each resource pool according to the first evaluation requirement information according to the computing power card data table;

[0180] The first determination submodule is used to determine the number and power consumption of computing power cards required for the multiple resource pools that need to be predicted to execute the intelligent computing business based on the unit throughput corresponding to the computing power cards in each resource pool being called to execute the intelligent computing business.

[0181] Optionally, the first determining submodule includes:

[0182] A quantity determination subunit is used to determine the number of computing power cards required for the multiple resource pools to be predicted to perform the intelligent computing business according to the business data size, the number of cycles, the business completion time and the unit throughput corresponding to the calling of the computing power cards in each resource pool to perform the intelligent computing business;

[0183] A power consumption query subunit, configured to query, according to the computing power card data table, the average power consumption corresponding to calling the computing power cards in each resource pool to execute the intelligent computing service according to the first evaluation requirement information;

[0184] The power consumption determination subunit is used to determine the power consumption required for the multiple resource pools that need to be predicted to execute intelligent computing services based on the number of computing power cards required by each resource pool and the average power consumption.

[0185] Optionally, the demand information acquisition module includes:

[0186] A second requirement information acquisition submodule is used to acquire second evaluation requirement information; the second evaluation requirement information includes a business type, multiple model names, and a computing power card name;

[0187] The evaluation parameter acquisition module includes:

[0188] A second query submodule is used to query, according to the computing power card data table, the unit throughput corresponding to calling a computing power card according to the second evaluation requirement information to use multiple models to execute intelligent computing services;

[0189] A first sorting submodule is used to sort according to the unit throughput corresponding to calling a computing power card to use multiple models to execute intelligent computing services;

[0190] The model performance determination submodule is used to determine the performance ranking of the multiple models according to the sorting results.

[0191] Optionally, the demand information acquisition module includes:

[0192] A third requirement information acquisition submodule is used to acquire third evaluation requirement information; the third evaluation requirement information includes the names of multiple computing power cards;

[0193] The evaluation parameter acquisition module includes:

[0194] A third query submodule is used to query the unit throughput corresponding to calling multiple computing power cards to execute intelligent computing services according to the computing power card data table according to the third evaluation requirement information;

[0195] The second sorting submodule is used to sort according to the unit throughput corresponding to calling multiple computing power cards to execute intelligent computing services;

[0196] The computing power card performance determination submodule is used to determine the performance rankings of the multiple computing power cards according to the sorting results.

[0197] Optionally, the second sorting submodule includes:

[0198] The average value acquisition subunit is used to perform a weighted average operation on the unit throughput corresponding to the calling of multiple computing power cards to execute the intelligent computing business, so as to obtain a weighted average value of the unit throughput;

[0199] The average value sorting subunit is used to sort the weighted average values ​​of the unit throughputs to obtain a sorting result.

[0200] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0201] An embodiment of the present invention further provides an electronic device, including:

[0202] It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, each process of an embodiment of the above-mentioned computing power measurement method based on throughput as a measurement indicator is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0203] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned embodiment of a computing power measurement method based on throughput as a measurement indicator are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0204] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0205] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0206] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0207] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0208] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0209] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0210] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0211] The above is a detailed introduction to a computing power measurement method based on throughput as a measurement indicator and a computing power measurement device based on throughput as a measurement indicator provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A computing power measurement method based on throughput as a metric, characterized in that: The method comprises: Obtain computing power card data tables of multiple resource pools; the computing power card data table includes multiple data entries, one of the data entries includes parameters collected by calling the computing power card to execute intelligent computing services, the parameters include: service type, model name, model key parameters, computing power hardware indicators, minimum hardware resources, average power consumption, unit throughput; the intelligent computing service is calling the model to execute inference tasks or training tasks; Obtain assessment needs information; According to the computing power card data table, the evaluation parameters corresponding to calling the computing power card to execute the intelligent computing business according to the evaluation requirement information are queried; the evaluation parameters at least include the unit throughput.

2. The method according to claim 1, characterized in that The computing power card data tables of the multiple resource pools are generated in the following manner: Measure the computing power hardware indicators of computing power cards in multiple resource pools; Determine whether the computing power hardware index of the computing power card is greater than a test threshold; If the computing power hardware index of the computing power card is greater than the test threshold, then obtaining the parameters of the computing power card for executing the intelligent computing service; A computing power card data table is generated according to the parameters of the computing power card for executing the intelligent computing business and the computing power hardware indicators.

3. The method according to claim 1, characterized in that The obtaining of assessment requirement information includes: Obtaining first evaluation requirement information; the first evaluation requirement information includes business type, model name, parameter quantity, accuracy, number of cycles, business data size and business completion time; According to the computing power card data table, querying the evaluation parameters corresponding to calling the computing power card to execute the intelligent computing business according to the evaluation requirement information; the evaluation parameters at least include the unit throughput, including: For multiple resource pools that need to be predicted, query the computing power card data table respectively according to the first evaluation demand information, and call the computing power cards in each resource pool to execute the unit throughput corresponding to the intelligent computing business; According to the unit throughput corresponding to the computing power cards in each resource pool being called to execute the intelligent computing business, the number of computing power cards and power consumption required for the multiple resource pools that need to be predicted to execute the intelligent computing business are determined.

4. The method according to claim 3, characterized in that: The determining, according to the unit throughput corresponding to the calling of the computing power cards in each resource pool to execute the intelligent computing service, the number and power consumption of the computing power cards required for the multiple resource pools to be predicted to execute the intelligent computing service, includes: Determine the number of computing power cards required for the multiple resource pools to be predicted to execute the intelligent computing service according to the service data size, the number of cycles, the service completion time, and the unit throughput corresponding to the calling of computing power cards in each resource pool to execute the intelligent computing service; According to the computing power card data table, query the average power consumption corresponding to calling the computing power cards in each resource pool to execute the intelligent computing business according to the first evaluation requirement information; The power consumption required for executing intelligent computing services by the multiple resource pools that need to be predicted is determined according to the number of computing power cards required by each resource pool and the average power consumption.

5. The method according to claim 1, characterized in that The obtaining of assessment requirement information includes: Obtaining second evaluation requirement information; the second evaluation requirement information includes a business type, multiple model names, and a computing power card name; According to the computing power card data table, querying the evaluation parameters corresponding to calling the computing power card to execute the intelligent computing business according to the evaluation requirement information; the evaluation parameters at least include the unit throughput, including: According to the computing power card data table, query the unit throughput corresponding to calling a computing power card to use multiple models to execute intelligent computing services according to the second evaluation requirement information; Sorting according to the unit throughput corresponding to calling a computing power card to use multiple models to execute intelligent computing services; According to the sorting results, the performance rankings of the multiple models are determined.

6. The method according to claim 1, characterized in that The obtaining of assessment requirement information includes: Obtaining third evaluation requirement information; the third evaluation requirement information includes names of multiple computing power cards; According to the computing power card data table, querying the evaluation parameters corresponding to calling the computing power card to execute the intelligent computing business according to the evaluation requirement information; the evaluation parameters at least include the unit throughput, including: According to the computing power card data table, query the unit throughput corresponding to calling multiple computing power cards to execute intelligent computing services according to the third evaluation requirement information; Sorting according to the unit throughput corresponding to calling the multiple computing power cards to execute the intelligent computing business; According to the sorting results, the performance rankings of the multiple computing power cards are determined.

7. The method according to claim 6, characterized in that The sorting according to the unit throughput corresponding to the calling of multiple computing power cards to execute the intelligent computing business includes: Performing a weighted average calculation on the unit throughputs corresponding to calling the multiple computing power cards to execute the intelligent computing services, to obtain a weighted average of the unit throughputs; The weighted average values ​​of the unit throughputs are sorted to obtain a sorting result.

8. A computing power measurement device based on throughput as a metric, characterized in that: The device comprises: A data table acquisition module is used to obtain the computing power card data tables of multiple resource pools; the computing power card data table includes multiple data entries, one of which includes parameters collected by calling the computing power card to execute intelligent computing services, and the parameters include: service type, model name, model key parameters, computing power hardware indicators, minimum hardware resources, average power consumption, and unit throughput; the intelligent computing service is to call the model to execute an inference task or a training task; A demand information acquisition module is used to acquire evaluation demand information; An evaluation parameter acquisition module is used to query the evaluation parameters corresponding to calling the computing power card to execute the intelligent computing business according to the evaluation requirement information according to the computing power card data table; the evaluation parameters at least include the unit throughput.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of a computing power measurement method based on throughput as a metric are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of a computing power measurement method based on throughput as a metric are implemented as described in any one of claims 1-7.

Images