Method for determining energy efficiency of server and related device

Abstract

The invention discloses an energy efficiency determination method for a server, and is applied to the server in the technical field of artificial intelligence. According to the method, a plurality of neural network models capable of covering mainstream application scenarios of the AI server are designed to serve as AI loads, and the operation condition of the AI server during operation of each neural network model is tested, so that the energy efficiency of the AI server is obtained through comprehensive calculation, the blank of the AI server energy efficiency benchmark test is filled, and the test efficiency of the AI server is improved. And the follow-up energy efficiency optimization of the AI server can be conveniently supported.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a method for determining energy efficiency of a server and related devices. Background Art

[0002] With the advancement of science and technology, social and environmental issues have become more prominent. Therefore, in order to ensure the sustainable development of society, energy conservation and emission reduction have become the main theme of current social development. In the field of Information Technology (IT), servers consume a lot of electricity, so the energy efficiency of servers is particularly important.

[0003] At present, in the related art, server energy efficiency is evaluated through server energy efficiency evaluation tools, so as to optimize the energy efficiency of the server in a targeted manner based on the energy efficiency of the server, thereby reducing the energy consumption of the server. However, the server energy efficiency evaluation tools in the related art only support the energy efficiency evaluation of the central processing unit (CPU), memory and hard disk in general servers, and do not support the energy efficiency evaluation of AI servers.

[0004] With the rapid development of the AI ​​field, AI servers are being used more and more widely. Therefore, there is an urgent need for a method to determine the energy efficiency of AI servers. Summary of the invention

[0005] The present application provides a method for determining the energy efficiency of a server, which can accurately determine the energy efficiency of an AI server.

[0006] The first aspect of the present application provides a method for determining the energy efficiency of a server, which is applied to determine the energy efficiency of an AI server. The method comprises: first, obtaining the operating status of the AI ​​server when running each of the multiple AI loads respectively. Among them, the multiple AI loads run by the AI ​​server include multiple neural network models, and the multiple neural network models are used to perform different AI tasks respectively, that is, one neural network model serves as an AI load. The operating status of the AI ​​server when running each AI load includes the number and power of the AI ​​server running the AI ​​load within a set time. That is, the AI ​​server runs each AI load separately, and during the period when the AI ​​server runs each AI load, the number of times the AI ​​server runs the load and the power during the operation can be obtained.

[0007] In addition, AI servers are equipped with hardware for performing AI calculations, such as graphics processing units (GPUs), neural-network processing units (NPUs), and tensor processing units (TPUs).

[0008] Then, based on the operation status of the AI ​​server when running each AI load, the energy efficiency value of the AI ​​server when running each AI load is determined. For example, based on the number of times the AI ​​server runs each AI load within a set time, the number of times the AI ​​server runs each AI load within a unit time (i.e., per second) can be determined, and then the number of times the AI ​​server runs the AI ​​load per second is divided by the power of the AI ​​server when running the AI ​​load, thereby obtaining the energy efficiency value.

[0009] Finally, based on the energy efficiency value of the AI ​​server when running each of the multiple AI loads, a target energy efficiency value of the AI ​​server is determined, and the target energy efficiency value is used to indicate the energy efficiency of the AI ​​server when running the AI ​​load. For example, by performing a comprehensive weighted calculation (such as calculating a weighted geometric mean) on the energy efficiency value of the AI ​​server when running each AI load, the target energy efficiency value of the AI ​​server can be obtained.

[0010] In this solution, multiple neural network models for executing different AI tasks are designed as AI loads based on the mainstream application scenarios of current AI servers. In addition, each AI load is run on the AI ​​server separately to obtain the number of times and average power when the AI ​​server runs each AI load within the set time, and then determine the energy efficiency value of the AI ​​server when running each AI load; finally, by combining the energy efficiency values ​​corresponding to multiple AI loads, the energy efficiency of the AI ​​server when running the AI ​​load can be obtained, realizing the energy efficiency evaluation of the AI ​​server.

[0011] In general, this solution designs multiple neural network models that can cover the mainstream application scenarios of AI servers as AI loads, and tests the operation of the AI ​​server when running each neural network model, so as to comprehensively calculate the energy efficiency of the AI ​​server, fill the gap in the AI ​​server energy efficiency benchmark test, and facilitate the subsequent energy efficiency tuning of the AI ​​server.

[0012] In a possible implementation, the multiple AI loads further include multiple AI operators, and the multiple AI operators are all operators used in the neural network model. In other words, the multiple AI loads include multiple neural network models and multiple AI operators at the same time.

[0013] In this solution, by designing the AI ​​operator as the smallest basic unit of AI computing as the AI ​​load, the AI ​​load can be constructed based on the essence of AI computing, ensuring that the designed AI load can cope with various uncertain scenarios in the future and that the designed AI load can adapt to the actual application scenarios of the AI ​​server.

[0014] In a possible implementation, in the process of determining the target energy efficiency value of the AI ​​server, the weight value corresponding to each AI load can be obtained based on the type of each AI load in the multiple AI loads. The type of the AI ​​load can be a neural network model or an AI operator, and the AI ​​load of the type of the neural network model corresponds to different weight values ​​than the AI ​​load of the type of the operator.

[0015] Then, based on the weight value corresponding to each AI load, the energy efficiency value of the AI ​​server when running each AI load is weighted to obtain the geometric mean, so as to obtain the target energy efficiency value of the AI ​​server.

[0016] Since AI operators are the smallest basic units of AI computing, and neural network models are composed of a large number of AI operators, the load levels of neural network models and AI operators in multiple AI loads vary greatly. Based on this, in this solution, when the energy efficiency values ​​corresponding to multiple AI loads are combined to obtain the energy efficiency value of the AI ​​server, different weight values ​​can be assigned to different types of AI loads, thereby ensuring the accuracy of the target energy efficiency value obtained by the final comprehensive evaluation.

[0017] In one possible implementation, each of the multiple AI operators is used to perform any of the following types of operations: activation function operation, convolution operation, pooling operation, normalization operation, matrix multiplication operation, data discarding operation and full connection operation.

[0018] In a possible implementation, the operation status of each AI load when the AI ​​server runs each AI load under multiple different load states can be obtained to obtain the operation status of each AI load under different load states. In other words, for the same AI load, the AI ​​server runs the AI ​​load independently under different load states to obtain the operation status of the same AI load under different load states.

[0019] Then, based on the operation status of the AI ​​server when running each AI load under each load state, the energy efficiency value of the AI ​​server when running each AI load under each load state is determined, that is, the energy efficiency values ​​of the same AI load under different load states can be obtained. Furthermore, the geometric mean of the energy efficiency values ​​of the AI ​​server when running the same AI load under multiple different load states is calculated to obtain the energy efficiency value of the AI ​​server when running each AI load.

[0020] In this solution, by obtaining the operating conditions of the AI ​​server when it runs the same AI load under different load states, and determining the energy efficiency of the AI ​​server when running the AI ​​load based on the operating conditions of the AI ​​server under different load states, the state of the AI ​​server when running the AI ​​load in the actual application scenario can be truly restored, ensuring that the calculated energy efficiency value is consistent with the actual application scenario, and improving the accuracy of the calculated energy efficiency value.

[0021] In one possible implementation, in the process of determining the energy efficiency value of the AI ​​server when running the AI ​​load, the preset number of operations corresponding to each AI load can be first obtained, wherein the preset number of operations is used to indicate the number of times the preset server runs the AI ​​load within a set time. In other words, each AI load is independently run within a set time by a preset server in advance, and then the number of times the preset server runs each AI load within the set time is obtained. The preset server can be a pre-specified server, which can also be used to perform AI calculations.

[0022] Then, by calculating the ratio between the target ratio corresponding to each AI load and the power of the AI ​​server running each AI load, the energy efficiency value of the AI ​​server when running each AI load is obtained. The target ratio corresponding to each AI load is the ratio between the number of times the AI ​​server runs each AI load within a set time and the preset number of operations corresponding to each AI load. In other words, after calculating the ratio of the number of times the AI ​​server runs the AI ​​load within a set time and the number of times the preset server runs the AI ​​load within a set time, the ratio is divided by the power of the AI ​​server when running the AI ​​load to obtain the energy efficiency value.

[0023] In this solution, by introducing the number of times a preset server runs each AI load as a reference, the magnitude difference between different AI loads can be effectively balanced, ensuring that the energy efficiency value of the AI ​​server running different AI loads can be comprehensively calculated in the future.

[0024] In one possible implementation, each of the multiple neural network models is used to perform any of the following types of AI tasks: computer vision processing tasks, natural language processing tasks, ranking learning processing tasks, content recommendation processing tasks, and data mining tasks.

[0025] In a second aspect, the present application provides a device for determining energy efficiency of a server, including: an acquisition module, used to acquire the operating status of an AI server when it runs each of a plurality of AI loads respectively, the operating status including the number of times and power at which the AI ​​server runs the AI ​​load within a set time period, the plurality of AI loads including a plurality of neural network models, the plurality of neural network models being used to perform different AI tasks respectively, and hardware for performing AI calculations being deployed on the AI ​​server; a processing module, used to determine the energy efficiency value of the AI ​​server when it runs each AI load based on the operating status of the AI ​​server when it runs each AI load; the processing module is also used to determine a target energy efficiency value of the AI ​​server based on the energy efficiency value of the AI ​​server when it runs each of the plurality of AI loads, the target energy efficiency value being used to indicate the energy efficiency of the AI ​​server when it runs the AI ​​load.

[0026] In one possible implementation, the multiple AI loads also include multiple AI operators, and the multiple AI operators are all operators used in the neural network model.

[0027] In one possible implementation, the processing module is further used to: obtain a weight value corresponding to each AI load based on the type of each AI load among multiple AI loads, wherein an AI load of a neural network model type and an AI load of an operator type correspond to different weight values; based on the weight value corresponding to each AI load, weightedly calculate the geometric mean of the energy efficiency value of the AI ​​server when running each AI load to obtain the target energy efficiency value of the AI ​​server.

[0028] In one possible implementation, each of the multiple AI operators is used to perform any of the following types of operations: activation function operation, convolution operation, pooling operation, normalization operation, matrix multiplication operation, data discarding operation and full connection operation.

[0029] In one possible implementation, the acquisition module is specifically used to obtain the operating conditions of the AI ​​server when running each AI load under multiple different load states, and obtain the operating conditions under different load states corresponding to each AI load; the processing module is also used to: determine the energy efficiency value of the AI ​​server when running each AI load under each load state based on the operating conditions of the AI ​​server when running each AI load under each load state; calculate the geometric mean of the energy efficiency values ​​when the AI ​​server runs the same AI load under multiple different load states, and obtain the energy efficiency value of the AI ​​server when running each AI load.

[0030] In one possible implementation, the acquisition module is further used to obtain a preset number of operations corresponding to each AI load, where the preset number of operations is used to indicate the number of times a preset server runs the AI ​​load within a set duration; the processing module is further used to obtain an energy efficiency value of the AI ​​server when running each AI load by obtaining the ratio between a target ratio corresponding to each AI load and the power of the AI ​​server running each AI load, where the target ratio corresponding to each AI load is the ratio between the number of times the AI ​​server runs each AI load within a set duration and the preset number of operations corresponding to each AI load.

[0031] In one possible implementation, each of the multiple neural network models is used to perform any of the following types of AI tasks: computer vision processing tasks, natural language processing tasks, ranking learning processing tasks, content recommendation processing tasks, and data mining tasks.

[0032] In one possible implementation, any one of the following hardware is deployed on the AI ​​server: GPU, NPU, and TPU.

[0033] A third aspect of the present application provides a computer-readable storage medium, which stores a computer program. When the computer-readable storage medium is run on a computer, the computer executes a method as implemented in any one of the first or second aspects.

[0034] A fourth aspect of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute a method as in any one of the implementations of the first aspect or the second aspect.

[0035] In a fifth aspect, the present application provides a chip, comprising one or more processors, wherein some or all of the processors are used to read and execute a computer program stored in a memory to execute the method in any one of the implementations of the first or second aspect.

[0036] Optionally, the chip includes a memory, and the memory is connected to the processor through a circuit or wire. Optionally, the chip also includes a communication interface, and the processor is connected to the communication interface. The communication interface is used to receive data and / or information to be processed, and the processor obtains the data and / or information from the communication interface, processes the data and / or information, and outputs the processing results through the communication interface. The communication interface can be an input and output interface. The method provided in the present application can be implemented by one chip, or by multiple chips in collaboration.

[0037] Among them, the technical effects brought about by any design method in the second to fifth aspects can refer to the technical effects brought about by different implementation methods in the above-mentioned first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of the structure of an AI server provided in an embodiment of the present application;

[0039] Figure 2 A schematic diagram of a system architecture provided for an embodiment of the present application;

[0040] Figure 3 A flow chart of a method for determining energy efficiency of a server provided in an embodiment of the present application;

[0041] Figure 4 A schematic diagram of a process for calculating a target energy efficiency value of an AI server by running multiple AI loads on the AI ​​server provided in an embodiment of the present application;

[0042] Figure 5 A schematic diagram of a process for calculating energy efficiency values ​​by running the same AI load under different load conditions on an AI server provided in an embodiment of the present application;

[0043] Figure 6 A schematic diagram of determining the energy efficiency value of an AI server by running a neural network model and an AI operator on the AI ​​server provided in an embodiment of the present application;

[0044] Figure 7 A schematic diagram of a system architecture for determining energy efficiency of an AI server provided in an embodiment of the present application;

[0045] Figure 8 A schematic diagram of calculating the overall energy efficiency score of an AI server provided in an embodiment of the present application;

[0046] Fig. 9 A schematic diagram of the structure of a device for determining energy efficiency of a server provided in an embodiment of the present application;

[0047] Fig.10 A schematic diagram of the structure of an execution device provided in an embodiment of the present application;

[0048] Fig.11 A schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application are described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only embodiments of a part of the present application, rather than all embodiments. It is known to those of ordinary skill in the art that with the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0050] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the descriptions used in this way can be interchanged where appropriate, so that the embodiments can be implemented in a sequence other than that illustrated or described in the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of the steps that appear in the present application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved. The division of units in this application is a logical division. There may be other division methods when it is implemented in actual applications. For example, multiple units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection between units can be electrical or other similar forms, which are not limited in this application. In addition, the units or sub-units described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed in multiple circuit units, and some or all of the units may be selected according to actual needs to achieve the purpose of the present application.

[0051] To facilitate understanding, some terminology concepts involved in the embodiments of the present application are explained below.

[0052] (1) Neural network model

[0053] The neural network model is described based on the mathematical model of neurons. In essence, it is a complex network system formed by a large number of simple processing units (called neurons) that are widely interconnected. The neural network model reflects many basic characteristics of human brain functions and is a highly complex nonlinear dynamic learning system. In addition, the neural network model has large-scale parallel, distributed storage and processing, self-organization, self-adaptation and self-learning capabilities, and is particularly suitable for processing imprecise and fuzzy information processing problems that require considering many factors and conditions at the same time.

[0054] Specifically, the neural network model is composed of neurons. A neuron can refer to an operation unit that takes xs (i.e., input data) and intercept 1 as input, and the output of the operation unit can be:

[0055]

[0056] Where s=1, 2, ...n, n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolution layer, and the activation function can be a sigmoid function. The neural network model is a network formed by connecting multiple single neural units mentioned above, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the characteristics of the local receptive field. The local receptive field can be an area composed of several neurons.

[0057] (2) Convolutional Neural Network (CNN)

[0058] A convolutional neural network is a deep neural network with a convolutional structure, which is mainly used in the field of computer vision. A convolutional neural network includes a feature extractor consisting of a convolutional layer and a subsampling layer. The feature extractor can be regarded as a filter, and the convolution process can be regarded as using a trainable filter to convolve with an input image or feature map. A convolutional layer refers to a neural unit layer in a convolutional neural network that performs convolution processing on the input signal (for example, the first convolutional layer and the second convolutional layer in this embodiment). In the convolutional layer of a convolutional neural network, a neural unit can be connected only to some neural units in adjacent layers. A convolutional layer usually contains several feature planes, and each feature plane can be composed of some neural units arranged in a rectangular shape. The neural units in the same feature plane share weights, and the shared weights here are convolution kernels. Shared weights can be understood as the way to extract image information is independent of position. The implicit principle is that the statistical information of a part of the image is the same as that of other parts. That means that the image information learned in a certain part can also be used in another part. So for all positions on the image, we can use the same learning to obtain image information. In the same convolution layer, multiple convolution kernels can be used to extract different image information. Generally speaking, the more convolution kernels there are, the richer the image information reflected by the convolution operation.

[0059] The convolution kernel can be initialized in the form of a matrix of random size, and the convolution kernel can obtain reasonable weights through learning during the training process of the convolutional neural network. In addition, the direct benefit of shared weights is to reduce the connections between the layers of the convolutional neural network, while reducing the risk of overfitting.

[0060] (3) Recurrent Neural Network (RNN)

[0061] A recurrent neural network is a type of recurrent neural network that takes sequence data as input, recurses in the direction of the sequence evolution, and all nodes (recurrent units) are connected in a chain. Among them, bidirectional recurrent neural networks (Bi-RNN) and long short-term memory networks (LSTM) are common recurrent neural networks.

[0062] Recurrent neural networks have memory, parameter sharing, and Turing completeness, so they have certain advantages in learning the nonlinear features of sequences. Recurrent neural networks are widely used in natural language processing (NLP), such as speech recognition, language modeling, machine translation, and other fields, and are also used in various time series forecasting.

[0063] (4) Attention Network

[0064] Attention network is a network model that uses attention mechanism to improve the model training speed. It is widely used in computer vision and natural language processing. At present, typical attention networks include Transformer network. Models that apply attention mechanism can assign different weights to each part of the input sequence, thereby extracting more important feature information from the input sequence, so that the model can finally obtain more accurate output.

[0065] In deep learning, the attention mechanism can be implemented by a weight vector that describes importance: when predicting or inferring an element, the weight vector is used to determine the correlation between the element and other elements. For example, for a pixel in an image or a word in a sentence, the attention vector can be used to quantitatively estimate the correlation between the target element and other elements, and the weighted sum of the attention vectors can be used as an approximation of the target value.

[0066] The attention mechanism in deep learning simulates the attention mechanism of the human brain. For example, when humans look at a painting, although the human eye can see the whole picture of the whole painting, when humans observe it in depth and carefully, the eyes actually focus on only a part of the pattern in the whole painting. At this time, the human brain mainly focuses on this small pattern. In other words, when humans observe an image carefully, the human brain does not pay equal attention to the whole image, but has a certain weight distinction. This is the core idea of ​​the attention mechanism.

[0067] Simply put, the human visual processing system tends to selectively focus on certain parts of the image and ignore other irrelevant information, which helps the human brain perceive. Similarly, in the attention mechanism of deep learning, in some problems involving language, speech or vision, some parts of the input may be more relevant than other parts. Therefore, through the attention mechanism in the attention model, the attention model can perform different processing on different parts of the input data, so that the attention model only dynamically focuses on the data related to the task.

[0068] At present, the server energy efficiency evaluation tools in related technologies are used to evaluate the energy efficiency of general servers, that is, they only support the energy efficiency evaluation of the CPU, memory and hard disk in general servers, and do not support the energy efficiency evaluation of AI servers.

[0069] For example, see Figure 1 , Figure 1 A schematic diagram of the structure of an AI server provided in an embodiment of the present application. Figure 1 As shown, the AI ​​server includes a CPU, memory, hard disk, and AI computing hardware. The AI ​​computing hardware may include any one or more of a graphics processing unit (GPU), a neural network processing unit (NPU), and a tensor processing unit (TPU).

[0070] Unlike general servers that use CPUs as computing power, AI servers mainly use AI computing hardware as computing power, and AI computing hardware accounts for a larger proportion of the energy efficiency of all hardware in the entire AI server. The current server energy efficiency evaluation tools mainly evaluate the energy efficiency of hardware such as CPUs, memory, and hard disks. For example, they evaluate the energy efficiency of servers based on encryption and decryption, compression and decompression, data caching, and data reading. They do not provide evaluation methods suitable for AI servers, so related technologies are difficult to accurately measure the energy efficiency of AI servers.

[0071] In view of this, the embodiment of the present application provides a method for determining the energy efficiency of a server. By targeting the mainstream application scenarios of current AI servers, multiple neural network models for executing different AI tasks are designed as AI loads. In addition, each AI load is run on the AI ​​server separately to obtain the number of times and average power when the AI ​​server runs each AI load within a set time, and then determine the energy efficiency value when the AI ​​server runs each AI load; finally, by combining the energy efficiency values ​​corresponding to multiple AI loads, the energy efficiency of the AI ​​server when running the AI ​​load can be obtained, and the energy efficiency evaluation of the AI ​​server can be realized.

[0072] See also Figure 2 , Figure 2 A schematic diagram of a system architecture provided in an embodiment of the present application. Figure 2 As shown, the system architecture includes an energy efficiency calculation device and an AI server. The energy efficiency calculation device is connected to the AI ​​server to control the AI ​​server to run the AI ​​load, and obtains the running status of the AI ​​server when running each AI load (that is, the number of times and power of the AI ​​server running the AI ​​load within a set time) to calculate the energy efficiency value of the AI ​​server. Exemplarily, the energy efficiency calculation device can be a server, a personal computer, a laptop, a smart phone, a tablet computer and other devices, which can obtain the status of the AI ​​server running the AI ​​load and calculate the energy efficiency.

[0073] See also Figure 3 , Figure 3 A flow chart of a method for determining energy efficiency of a server provided in an embodiment of the present application. Figure 3 As shown, the process of the method for determining the energy efficiency of the server includes the following steps 301-303.

[0074] Step 301, obtaining the operating status of each of the multiple AI loads when the AI ​​server runs each AI load, the operating status including the number of times and power at which the AI ​​server runs the AI ​​load within a set time period, the multiple AI loads including multiple neural network models, the multiple neural network models are respectively used to perform different AI tasks, and the AI ​​server is deployed with hardware for performing AI calculations.

[0075] In this embodiment, in order to determine the energy efficiency of the AI ​​server, multiple AI loads are designed in advance for the AI ​​server, and the multiple AI loads include multiple neural network models, that is, one neural network model is used as an AI load. In addition, the multiple neural network models as AI are used to perform different AI tasks, so that the AI ​​load can cover the mainstream application scenarios of the AI ​​server. Among them, the AI ​​server may be deployed with any of the following hardware for performing AI calculations: GPU, NPU and TPU. In addition, the AI ​​server implements the operation of the AI ​​load through the deployed hardware for performing AI calculations.

[0076] In the process of determining the energy efficiency of the AI ​​server, the AI ​​server runs each of the multiple AI loads respectively. Furthermore, during the period when the AI ​​server runs each AI load, the number of times and the power of the AI ​​server running the AI ​​load within a set time period are determined respectively, that is, the operation status of the AI ​​server when running each AI load is determined. That is, the AI ​​server runs each AI load separately, and during the period when the AI ​​server runs each AI load, the number of times the AI ​​server runs the load and the power during the operation can be obtained.

[0077] The number of times the AI ​​server runs the AI ​​load within the set time is used to indicate the size of the load processed by the AI ​​server. The more times the AI ​​server runs the AI ​​load within the set time, the greater the load processed by the AI ​​server; the fewer times the AI ​​server runs the AI ​​load within the set time, the smaller the load processed by the AI ​​server.

[0078] It should be noted that since the AI ​​server usually runs the neural network model by inputting sample data into the neural network model, the sample data is processed by executing the operation in the neural network model, thereby realizing the complete operation of the neural network model. In other words, each time the AI ​​server processes sample data through the neural network model, a complete operation of the neural network model is completed. Therefore, in this embodiment, the number of times the AI ​​server runs the AI ​​load can be determined by determining the number of sample data processed by the AI ​​server. Taking the AI ​​load run by the AI ​​server as an image classification model as an example, the process of the AI ​​server running the image classification model is to input the image into the image classification model and execute the operation in the image classification model to obtain the image classification result. Therefore, the number of times the AI ​​server runs the image classification model within a set time length can be determined by calculating the number of images processed by the AI ​​server through the image classification model within a set time length.

[0079] The power of the AI ​​server running the AI ​​load within the set duration is used to indicate the energy efficiency of the AI ​​server in processing the load. The greater the power of the AI ​​server running the AI ​​load within the set duration, the greater the energy efficiency of the AI ​​server; the smaller the power of the AI ​​server running the AI ​​load within the set duration, the smaller the energy efficiency of the AI ​​server. Specifically, the power of the AI ​​server running the AI ​​load within the set duration can be obtained by connecting a power meter to the AI ​​server. The power value of the AI ​​server during operation is measured in real time by connecting to a power meter connected to the AI ​​server (for example, the power meter measures the power value of the AI ​​server once per second), and then the average of the multiple measured power values ​​is calculated to obtain the power of the AI ​​server running the AI ​​load within the set duration.

[0080] In addition, the set duration for the AI ​​server to run each AI load can be a pre-set duration, such as 1 second, 10 seconds or 1 minute, etc., and this embodiment does not specifically limit this.

[0081] Step 302: Determine the energy efficiency value of the AI ​​server when running each AI load based on the running status of the AI ​​server when running each AI load.

[0082] In the operating status of the AI ​​server when running the AI ​​load, since the number of times the AI ​​server runs the AI ​​load indicates the size of the AI ​​server processing load, and the power of the AI ​​server running the AI ​​load indicates the energy efficiency of the AI ​​server processing load, the energy efficiency value of the AI ​​server when running each AI load can be determined by combining the number and power of the AI ​​server running each AI load within the set time.

[0083] Exemplarily, based on the number of times the AI ​​server runs each AI load within a set duration, the number of times the AI ​​server runs each AI load within a unit time (i.e., per second) can be determined, and then the number of times the AI ​​server runs the AI ​​load per second can be divided by the power of the AI ​​server when running the AI ​​load to obtain the energy efficiency value.

[0084] In this embodiment, the AI ​​server runs each AI load independently and calculates the energy efficiency value when running each AI load. However, there may be a certain deviation in the actual load between different AI loads, which may cause the AI ​​server to run different AI loads in the same time. The number of times the AI ​​load is run may also have a large difference. Therefore, in order to balance the magnitude difference between different AI loads and ensure that the energy efficiency values ​​of the AI ​​server running different AI loads can be integrated later, in this embodiment, a designated server can be introduced as a reference to calculate the energy efficiency value of the AI ​​server running each AI load.

[0085] Optionally, in the process of determining the energy efficiency value when the AI ​​server runs the AI ​​load, the preset number of operations corresponding to each AI load can be obtained first, where the preset number of operations is used to indicate the number of times the preset server runs the AI ​​load within a set time. In other words, each AI load is independently run within a set time by a preset server in advance, and then the number of times the preset server runs each AI load within the set time is obtained. The preset server can be a pre-specified server that can also be used to perform AI calculations.

[0086] Then, by calculating the ratio between the target ratio corresponding to each AI load and the power of the AI ​​server running each AI load, the energy efficiency value of the AI ​​server when running each AI load is obtained. The target ratio corresponding to each AI load is the ratio between the number of times the AI ​​server runs each AI load within a set time and the preset number of operations corresponding to each AI load. In other words, after calculating the ratio of the number of times the AI ​​server runs the AI ​​load within a set time and the number of times the preset server runs the AI ​​load within a set time, the ratio is divided by the power of the AI ​​server when running the AI ​​load to obtain the energy efficiency value.

[0087] For example, assuming that the number of times the AI ​​server runs AI load 1 within the set time is T, the power of the AI ​​server running AI load 1 within the set time is W, and the number of times the designated server runs AI load 1 within the set time is F, then the energy efficiency value of the AI ​​server when running AI load 1 is S = T / (W*F).

[0088] In this way, by introducing the number of times a preset server runs each AI load as a reference, the magnitude difference between different AI loads can be effectively balanced, ensuring that the energy efficiency value of the AI ​​server running different AI loads can be comprehensively calculated later.

[0089] Step 303: Based on the energy efficiency value of the AI ​​server when running each of the multiple AI loads, determine a target energy efficiency value of the AI ​​server, where the target energy efficiency value is used to indicate the energy efficiency of the AI ​​server when running the AI ​​load.

[0090] In this embodiment, for multiple AI loads, when the energy efficiency value of the AI ​​server when running each AI load is obtained, the target energy efficiency value of the AI ​​server can be obtained by performing a comprehensive weighted calculation (for example, obtaining a weighted geometric mean) on the energy efficiency value of the AI ​​server when running each AI load. The target energy efficiency value can indicate the energy efficiency of the AI ​​server when running the AI ​​load.

[0091] In general, this solution designs multiple neural network models that can cover the mainstream application scenarios of AI servers as AI loads, and tests the operation of the AI ​​server when running each neural network model, so as to comprehensively calculate the energy efficiency of the AI ​​server, fill the gap in the AI ​​server energy efficiency benchmark test, and facilitate the subsequent energy efficiency tuning of the AI ​​server.

[0092] Optionally, for the multiple neural network models run by the AI ​​server, each of the multiple neural network models is used to perform any of the following types of AI tasks: computer vision processing tasks, natural language processing tasks, sorting learning processing tasks, content recommendation processing tasks, and data mining tasks. Among them, the computer vision processing task is actually an image processing task, which uses computers and other devices to simulate the human visual system to perform various processing on images or videos, so that the computer can observe and understand the world through vision like humans, and has the ability to adapt to the environment autonomously.

[0093] The natural language processing task is actually a task to achieve effective communication between people and computers using natural language, by using computers to imitate humans to identify, classify, translate or answer questions about natural language.

[0094] The learning to rank task (also commonly referred to as the learning to rank task) refers to a task applied in the field of information retrieval, which is mainly to rank the search results by comprehensively considering multiple ranking features.

[0095] The content recommendation task is a task that recommends content of interest to users by recording their preferences, such as recommending products, articles, applications, or music that the users may be interested in.

[0096] Data mining task refers to the task of searching for information hidden in a large amount of data. It is widely used in business, medicine, finance, telecommunications and other fields.

[0097] Specifically, each of the above-mentioned types of AI tasks can be divided into a variety of different processing tasks. For example, computer vision processing tasks can include image classification tasks, image segmentation tasks, target detection tasks, image generation tasks, and image enhancement tasks. Natural language processing tasks can include text translation tasks, text classification tasks, text segmentation tasks, speech recognition tasks, text generation tasks, and other tasks. Therefore, the neural network model used to perform the same type of tasks can actually process different AI tasks, thereby ensuring the universality of AI loads.

[0098] In general, the multiple neural network models run by the AI ​​server can be the current mainstream neural network models, and different neural network models can be used to perform different AI tasks, thereby covering the main application scenarios of the AI ​​server.

[0099] It is understandable that in actual application scenarios, due to the long service life of the server, the AI ​​tasks run by the server during use will usually change over time. For example, the server mainly runs an image classification model in a certain period of time to process image classification tasks; the server may mainly run a text translation model in the next period of time to process text translation tasks; the server may mainly run a content recommendation model in the next period of time to recommend content of interest to users. Therefore, in this embodiment, by designing a neural network model for processing different AI tasks as an AI load, it can cover most of the application scenarios faced by the AI ​​server in actual use, thereby accurately and effectively evaluating the energy efficiency of the server.

[0100] For example, see Figure 4 , Figure 4 A schematic diagram of a process for calculating the target energy efficiency value of an AI server by running multiple AI loads on an AI server is provided in an embodiment of the present application. Figure 4 As shown, the AI ​​loads provided in this embodiment include AI load 1-AI load N, where AI load 1 can be specifically an image classification model, AI load 2 can be specifically an object detection model, AI load 3 can be specifically a speech recognition model... AI load N can be specifically a content recommendation model. In the process of determining the energy efficiency of the AI ​​server, the AI ​​server is controlled to independently run each AI load in AI load 1-AI load N within a set time, and the number of times and power (i.e., the number of times the AI ​​server runs each AI load within the set time) is obtained. Figure 4 In this way, based on the number of times and power when the AI ​​server runs AI load 1 within the set time, energy efficiency value 1 can be calculated, which is the energy efficiency value of the AI ​​server when running AI load 1. Similarly, based on the number of times and power when the AI ​​server runs AI load 2-AI load N within the set time, energy efficiency value 2-energy efficiency value N can be calculated. Finally, by taking the weighted geometric mean of energy efficiency value 1-energy efficiency value N, the target energy efficiency value, that is, the energy efficiency of the AI ​​server when running the AI ​​load, is obtained.

[0101] It is understandable that in actual application scenarios, the load status of AI servers when running AI loads in different time periods may often be different. For example, during the day when the business access volume is high, the AI ​​server may run the AI ​​load at full load, that is, all the computing resources of the AI ​​server are used to run the AI ​​load; during the evening when the business access volume is low, the AI ​​server may run the AI ​​load at low load, that is, part of the computing resources of the AI ​​server are used to run the AI ​​load, while the other part of the computing resources are idle.

[0102] Based on this, in order to truly restore the operating status of the AI ​​server in the actual application scenario, in this embodiment, the operating conditions of the AI ​​server when running the same AI load under different load conditions can be obtained separately, and then the energy efficiency of the AI ​​server when running the AI ​​load can be determined based on the operating conditions of the AI ​​server under different load conditions, to ensure that the energy efficiency value calculated is in line with the actual application scenario.

[0103] Exemplarily, in the above step 301, the energy efficiency calculation device may be to obtain the operating conditions of the AI ​​server when running each AI load under a plurality of different load states, and obtain the operating conditions under different load states corresponding to each AI load. That is to say, for the same AI load, the AI ​​server runs the AI ​​load independently under different load states, and then obtains the operating conditions under different load states corresponding to the same AI load, that is, the number and power of the AI ​​server when running each AI load under different load states. For example, the AI ​​server can run the AI ​​load under load states such as 100% load state (i.e., 100% computing power resources are used to run the AI ​​load), 75% load state (i.e., 75% computing power resources are used to run the AI ​​load), 50% load state, and 25% load state. The specific load state value and quantity can be determined according to the actual situation, and this embodiment does not specifically limit this.

[0104] Then, in the above step 302, when determining the energy efficiency value of the AI ​​server when running each AI load, the energy efficiency value of the AI ​​server when running each AI load under each load state can be determined based on the operation status of the AI ​​server when running each AI load under each load state. In addition, the geometric mean value of the energy efficiency values ​​of the AI ​​server when running the same AI load under multiple different load states is calculated to obtain the energy efficiency value of the AI ​​server when running each AI load.

[0105] That is to say, for each AI load, multiple operating conditions corresponding to each AI load can be obtained, and the multiple operating conditions correspond to different load states. Therefore, based on the operating conditions under each load state, the energy efficiency value when running the AI ​​load under each load state can be calculated, thereby obtaining the energy efficiency values ​​under different load states corresponding to the same AI load. In this way, by comprehensively obtaining the energy efficiency values ​​under different load states corresponding to the same AI load, the energy efficiency value of the AI ​​server when running each AI load can be obtained.

[0106] For example, see Figure 5 , Figure 5 A schematic diagram of a process for calculating energy efficiency values ​​by running the same AI load under different load conditions on an AI server is provided in an embodiment of the present application. Figure 5 As shown, the AI ​​server runs the same AI load under different load states, which is essentially AI load 1 of the image classification model. Moreover, through testing, it is found that the AI ​​server can process 1,000 images simultaneously under full load (i.e., 100% load state). Therefore, in this embodiment, the number of images input to the AI ​​server at the same time can be set to enable the AI ​​server to run AI load 1 under different load states.

[0107] Specifically, when 1,000 images are input into the AI ​​server, the AI ​​server processes 1,000 images simultaneously by running the image classification model, and the AI ​​server is in a 100% load state, and then the energy efficiency value A1 of the AI ​​server running AI load 1 in a 100% load state is calculated.

[0108] When 750 images are input into the AI ​​server, the AI ​​server processes 750 images simultaneously by running the image classification model, and the AI ​​server is in a 75% load state, and then the energy efficiency value A2 of the AI ​​server running AI load 1 in a 75% load state is calculated.

[0109] When 500 images are input into the AI ​​server, the AI ​​server processes 500 images simultaneously by running the image classification model, and the AI ​​server is in a 500% load state, and then the energy efficiency value A3 of the AI ​​server running AI load 1 at 50% load state is calculated.

[0110] When 250 images are input into the AI ​​server, the AI ​​server processes the 250 images simultaneously by running the image classification model, and the AI ​​server is in a 25% load state, and then the energy efficiency value A4 of the AI ​​server running AI load 1 in the 25% load state is calculated.

[0111] Finally, by taking the geometric mean of the energy efficiency values ​​of the AI ​​server when running AI load 1 under the above four different load conditions (i.e., energy efficiency value A1-energy efficiency value A4), the energy efficiency value of the AI ​​server when running AI load 1 is obtained.

[0112] In this solution, by obtaining the operating conditions of the AI ​​server when it runs the same AI load under different load states, and determining the energy efficiency of the AI ​​server when running the AI ​​load based on the operating conditions of the AI ​​server under different load states, the state of the AI ​​server when running the AI ​​load in the actual application scenario can be truly restored, ensuring that the calculated energy efficiency value is consistent with the actual application scenario, and improving the accuracy of the calculated energy efficiency value.

[0113] The above introduces that the energy efficiency value of the AI ​​server when running the AI ​​load is calculated by using the neural network model as the AI ​​load and the AI ​​server running the neural network model. Since AI technology is currently developing rapidly, AI servers will face more and more complex application scenarios in the future. Therefore, if only the current mainstream neural network model is designed as the AI ​​load, it may not be possible to fully cover all application scenarios of the AI ​​server. Based on this, in this embodiment, based on the nature of AI computing, the main functions of AI computing are analyzed to design an operator load that can cope with uncertain scenarios, ensuring that the designed AI load can adapt to the actual application scenarios of the AI ​​server.

[0114] Exemplarily, for the multiple AI loads run by the AI ​​server, in addition to multiple neural network models, the multiple AI loads also include multiple AI operators, and the multiple AI operators are all operators used in the neural network model.

[0115] Among them, each AI operator in the multiple AI operators can be used to perform any of the following types of operations: activation function operation, convolution operation, pooling operation, normalization operation, matrix multiplication operation, data discarding operation and full connection operation. In general, the multiple AI operators can include various types of operators used in the neural network model, thereby ensuring that any neural network model can be obtained based on the combination of multiple AI operators.

[0116] In this way, by designing the AI ​​operator as the smallest basic unit of AI computing as the AI ​​load, it is possible to construct the AI ​​load based on the essence of AI computing, ensuring that the designed AI load can cope with various uncertain scenarios in the future and that the designed AI load can adapt to the actual application scenarios of the AI ​​server.

[0117] It should be noted that, when the AI ​​load is an AI operator, the essence of the AI ​​server running the AI ​​operator is to perform operations on the input data through the AI ​​operator and obtain output data. Therefore, when determining the number of times the AI ​​server runs the AI ​​operator, the number of times the AI ​​server processes the input data by running the AI ​​operator can be used as the number of times the AI ​​server runs the AI ​​operator, thereby determining the specific situation of the AI ​​server running the AI ​​load.

[0118] When multiple AI loads include multiple neural network models and multiple AI operators at the same time, since AI operators are the smallest basic units of AI calculations, and neural network models are composed of a large number of AI operators, the load levels of neural network models and AI operators in multiple AI loads vary greatly. Based on this, when the energy efficiency values ​​corresponding to multiple AI loads are combined to obtain the energy efficiency value of the AI ​​server, different weight values ​​can be assigned to different types of AI loads to ensure the accuracy of the target energy efficiency value obtained by the final comprehensive evaluation.

[0119] Exemplarily, in the process of determining the target energy efficiency value of the AI ​​server, the weight value corresponding to each AI load can be obtained based on the type of each AI load in the multiple AI loads. Among them, the type of AI load can be a neural network model or an AI operator, and the AI ​​load of the type of neural network model corresponds to different weight values ​​from the AI ​​load of the type of operator. Then, based on the weight value corresponding to each AI load, the energy efficiency value of the AI ​​server when running each AI load is weighted to obtain the geometric mean, so as to obtain the target energy efficiency value of the AI ​​server. Among them, the weight value of the AI ​​load of the type of neural network model can be greater than the weight value of the AI ​​load of the type of AI operator, and the specific weight value of each AI load can be determined or adjusted according to the actual situation, and this embodiment does not specifically limit this.

[0120] In addition, in this embodiment, the weight values ​​corresponding to the same type of AI loads may be the same. Therefore, in this embodiment, the geometric mean of the same type of AI loads may be first calculated, and then the geometric mean values ​​of the various types of AI loads may be weighted to obtain the geometric mean values, and finally the target energy efficiency value of the AI ​​server may be obtained. That is, the geometric mean of all AI loads of the neural network model type is first calculated to obtain the geometric mean of the AI ​​load; and the geometric mean of all AI loads of the AI ​​operator type is calculated to obtain the geometric mean of the AI ​​operator. Then, based on the weight values ​​corresponding to the neural network model and the weight values ​​corresponding to the AI ​​operator, the geometric mean of the AI ​​load and the geometric mean of the AI ​​operator are weighted to obtain the geometric mean value to obtain the target energy efficiency value of the AI ​​server.

[0121] For example, see Figure 6, Figure 6 A schematic diagram of determining the energy efficiency value of an AI server by running a neural network model and an AI operator on the AI ​​server is provided in an embodiment of the present application. Figure 6 As shown, the AI ​​loads independently run by the AI ​​server include AI load 1 to AI load N of the type of neural network model, and AI load N+1 to AI load N+M of the type of AI operator. In other words, there are N+M AI loads in total, and N AI loads are neural network models, and M AI loads are AI operators. In this way, by independently running each AI load on the AI ​​server, the number and power of each AI load run by the AI ​​server can be obtained, and then the energy efficiency value of each AI load run by the AI ​​server can be obtained.

[0122] Specifically, for AI load 1, which is an image classification model, the energy efficiency value 1 when the AI ​​server runs AI load 1 can be calculated...For AI load N, which is a content recommendation model, the energy efficiency value N when the AI ​​server runs AI load N can be calculated. For AI load N+1, which is essentially a convolution operation, the energy efficiency value N+1 when the AI ​​server runs AI load N+1 can be calculated...For AI load N+M, which is essentially a matrix multiplication operation, the energy efficiency value N+M when the AI ​​server runs AI load N+M can be calculated.

[0123] In addition, since the energy efficiency value 1-energy efficiency value N are all energy efficiency values ​​corresponding to the neural network model, the geometric mean of the energy efficiency value 1-energy efficiency value N can be calculated to obtain the geometric mean 1. Since the energy efficiency value N+1-energy efficiency value N+M are all energy efficiency values ​​corresponding to the AI ​​operator, the geometric mean of the energy efficiency value N+1-energy efficiency value N+M can be calculated to obtain the geometric mean 2. Finally, a weight value 1 is assigned to the geometric mean 1, and a weight value 2 is assigned to the geometric mean 2, so as to achieve the geometric mean of the geometric mean 1 and the geometric mean 2 based on different weight values ​​to obtain the target energy efficiency value.

[0124] The above introduces a method for determining energy efficiency of a server provided in an embodiment of the present application. For ease of understanding, the specific execution process of the method for determining energy efficiency of the server will be described in detail below with reference to specific examples.

[0125] Specifically, the process of determining the energy efficiency of an AI server mainly includes three steps: 1. Designing an AI load; 2. Loading the AI ​​load through an AI server and obtaining the operating status of the AI ​​server when running the AI ​​load; 3. Determining the energy efficiency of the AI ​​server based on the operating status of the AI ​​server when running the AI ​​load. The following will introduce the above three steps in detail.

[0126] (1) Design AI workloads.

[0127] In this embodiment, two types of AI loads are designed: one is an AI component load (i.e., a neural network model) designed for current typical AI application scenarios; the other is an AI operator load designed based on the essence of AI computing by analyzing the main functions of AI computing.

[0128] The AI ​​component load is essentially a neural network model, which may include neural network models applied in various fields to perform different tasks. For example, the specific load included in the AI ​​component load may be as shown in Table 1 below.

[0129] Table 1

[0130]

[0131] It should be noted that the loads shown in Table 1 above are only some examples. In practical applications, more neural network models can be designed as AI loads according to actual application scenarios and requirements, and no specific limitations are made here.

[0132] The AI ​​operator load is essentially various basic AI operations, which may include various operations or functions applied to the neural network model. For example, the specific load included in the AI ​​operator load may be as shown in Table 2 below.

[0133] Table 2

[0134]

[0135] Similarly, the loads shown in Table 2 above are only some examples. In actual applications, more AI operators can be designed as AI loads according to actual application scenarios and requirements, and no specific limitations are made here.

[0136] (2) Load the AI ​​load through the AI ​​server and obtain the running status of the AI ​​server when running the AI ​​load.

[0137] After the AI ​​load is designed, the AI ​​load that the AI ​​server needs to load can be selected according to the actual situation. The AI ​​load loaded by the AI ​​server can be, for example, a subset or full set of the above-mentioned AI component loads, or a subset or full set of the above-mentioned AI operator loads, or a subset or full set of the above-mentioned AI component loads and a subset or full set of the above-mentioned AI operator loads.

[0138] Specifically, see Figure 7 , Figure 7 A schematic diagram of a system architecture for determining the energy efficiency of an AI server provided in an embodiment of the present application. Figure 7As shown in the figure, the system architecture includes an energy efficiency computing device, an AI server, and a power meter. The power meter is connected to the AI ​​server to measure the power of the AI ​​server when it runs the AI ​​load. The energy efficiency computing device is connected to the AI ​​server and the power meter respectively to control the AI ​​server to run the AI ​​load and obtain the power of the AI ​​server when it runs the AI ​​load through the power meter.

[0139] (3) Determine the energy efficiency of the AI ​​server based on the operating conditions of the AI ​​server when running AI loads.

[0140] See also Figure 8 , Figure 8 A schematic diagram of calculating the overall energy efficiency score of an AI server provided in an embodiment of the present application. Figure 8 As shown, in this embodiment, for the operation status of the AI ​​server when running the AI ​​load, the number of times the AI ​​server runs the AI ​​load within a unit time (for example, per second) is called throughput, and is represented by T; the power of the AI ​​server during the operation of the AI ​​load is called average active power, and is represented by W, where the average active power can be obtained by measuring the power of the AI ​​server during the operation of the AI ​​load in real time through a power meter and taking the power average value. In addition, for each AI load, the AI ​​load is pre-run on a preset server, and the number of times the preset server runs the AI ​​load within a unit time is called the normalized performance value, and is represented by F.

[0141] In this way, for any one of the multiple AI loads, the energy efficiency score of the single AI load (ie, the energy efficiency value in the above embodiment) can be calculated as shown in the following formula 1.

[0142]

[0143] Among them, S′ loadLevel It represents the energy efficiency score of a single AI load under a certain load state. T represents the throughput corresponding to the AI ​​load. W represents the average active power corresponding to the AI ​​load. F represents the normalized performance value corresponding to the AI ​​load.

[0144] It should be noted that in the process of the AI ​​server running the AI ​​component load, for each AI component load, the AI ​​server independently runs each AI component load in different load states, thereby obtaining the throughput and average active power of different load states corresponding to each AI component load. Therefore, based on the throughput and average active power of different load states corresponding to each AI component load, the energy efficiency score of each AI component load in different load states can be calculated respectively by the above formula 1. For example, in this embodiment, for a single AI component load, the energy efficiency scores of the AI ​​component load in four load states of 100% load state, 75% load state, 50% load state and 25% load state are calculated respectively.

[0145] In this way, for each AI component load, the geometric mean of the energy efficiency scores of the same AI component load under different load states can be calculated to obtain the energy efficiency score corresponding to each AI component load, as shown in the following formula 2.

[0146]

[0147] Among them, S′ component represents the energy efficiency score of a single AI component load; exp() represents the exponential function; ln() represents the natural logarithm function; S ′ 100% means the energy efficiency score of the AI ​​component under 100% load; S ′ 75% represents the energy efficiency score when the AI ​​component is at 100% load; S ′ 50% represents the energy efficiency score when the AI ​​component is at 100% load; S ′ 25% represents the energy efficiency score when the AI ​​component is at 100% load.

[0148] In addition, for AI operator load, since AI operator load is the basic unit in AI computing, it is no longer run under different load states, but each AI operator load is run three times under 100% load state, thereby improving the strong consistency of the load, as shown in the following formula 3.

[0149]

[0150] Among them, S′ operator Indicates the energy efficiency score of a single AI operator load; three S ′ 100% represents the load score corresponding to the AI ​​server independently running the same AI operator load three times.

[0151] After obtaining the energy efficiency score of each AI load, the average energy efficiency score of all AI component loads and the average energy efficiency score of all AI operator loads can be comprehensively calculated.

[0152] Specifically, for the AI ​​component load, the AI ​​component load may include the neural network model shown in Table 1 above. Specifically, the geometric mean can be calculated by the following formula 4 to obtain the average energy efficiency score of all AI component loads.

[0153]

[0154] Among them, S component represents the average energy efficiency score of AI component load; S′ k They represent the energy efficiency score of each AI component load, and n is the number of AI component loads.

[0155] For the AI ​​operator load, the AI ​​operator load may include the operators shown in Table 2 above. Specifically, the geometric mean may be calculated by the following formula 5 to obtain the average energy efficiency score of all AI operator loads.

[0156]

[0157] Among them, S operator represents the average energy efficiency score of the AI ​​operator load; S′ k They represent the energy efficiency score of each AI operator load, and n is the number of AI operator loads.

[0158] Finally, the overall energy efficiency score of the AI ​​server can be comprehensively calculated based on the weight values ​​assigned to the AI ​​component load and the AI ​​operator load, as shown in the following formula 6.

[0159] S = exp( W1ln (S component ) +W2ln (S operator )) Formula 6

[0160] Wherein, S represents the overall energy efficiency score of the AI ​​server (i.e., the target energy efficiency value of the above embodiment); S component The average energy efficiency score of the AI ​​component load is expressed as; S operator represents the average energy efficiency score of the AI ​​operator load; W1 represents the weight value of the AI ​​component load; W2 represents the weight value of the AI ​​operator load. Since the AI ​​operator load is the basic unit of AI calculation, all AI operator loads can be considered to be combined into one AI component load, and the weight value is set based on the number of AI component loads. For example, in this embodiment, W1 can be 8 / 9, and W2 can be 1 / 9.

[0161] The above embodiment introduces the energy efficiency determination method of the server provided in the embodiment of the present application. The following will introduce the device for executing the energy efficiency determination method of the server.

[0162] See also Fig. 9 , Fig. 9 This is a schematic diagram of the structure of a device for determining energy efficiency of a server provided in an embodiment of the present application. Fig. 9 As shown, the energy efficiency determination device of the server includes: an acquisition module 901, which is used to obtain the operating status of the AI ​​server when it runs each of the multiple AI loads respectively, the operating status includes the number of times and power of the AI ​​server running the AI ​​load within a set time period, the multiple AI loads include multiple neural network models, and the multiple neural network models are used to perform different AI tasks respectively. The AI ​​server is deployed with hardware for performing AI calculations; a processing module 902, which is used to determine the energy efficiency value of the AI ​​server when running each AI load based on the operating status of the AI ​​server when running each AI load; the processing module 902 is also used to determine the target energy efficiency value of the AI ​​server based on the energy efficiency value of the AI ​​server when running each of the multiple AI loads, and the target energy efficiency value is used to indicate the energy efficiency of the AI ​​server when running the AI ​​load.

[0163] In one possible implementation, the multiple AI loads also include multiple AI operators, and the multiple AI operators are all operators used in the neural network model.

[0164] In one possible implementation, the processing module 902 is further used to: obtain a weight value corresponding to each AI load based on the type of each AI load among multiple AI loads, wherein an AI load of a neural network model type and an AI load of an operator type correspond to different weight values; based on the weight value corresponding to each AI load, perform weighted geometric mean on the energy efficiency value of the AI ​​server when running each AI load to obtain a target energy efficiency value of the AI ​​server.

[0165] In one possible implementation, each of the multiple AI operators is used to perform any of the following types of operations: activation function operation, convolution operation, pooling operation, normalization operation, matrix multiplication operation, data discarding operation and full connection operation.

[0166] In one possible implementation, the acquisition module 901 is specifically used to obtain the operating conditions of the AI ​​server when running each AI load under a variety of different load states, and obtain the operating conditions under different load states corresponding to each AI load; the processing module 902 is also used to: determine the energy efficiency value of the AI ​​server when running each AI load under each load state based on the operating conditions of the AI ​​server when running each AI load under each load state; calculate the geometric mean of the energy efficiency values ​​of the AI ​​server when running the same AI load under a variety of different load states, and obtain the energy efficiency value of the AI ​​server when running each AI load.

[0167] In one possible implementation, the acquisition module 901 is further used to obtain a preset number of operations corresponding to each AI load, where the preset number of operations is used to indicate the number of times a preset server runs the AI ​​load within a set duration; the processing module 902 is further used to obtain an energy efficiency value of the AI ​​server when running each AI load by obtaining the ratio between a target ratio corresponding to each AI load and the power of the AI ​​server running each AI load, where the target ratio corresponding to each AI load is the ratio between the number of times the AI ​​server runs each AI load within a set duration and the preset number of operations corresponding to each AI load.

[0168] In one possible implementation, each of the multiple neural network models is used to perform any of the following types of AI tasks: computer vision processing tasks, natural language processing tasks, ranking learning processing tasks, content recommendation processing tasks, and data mining tasks.

[0169] In one possible implementation, any one of the following hardware is deployed on the AI ​​server: GPU, NPU, and TPU.

[0170] Next, an execution device provided by an embodiment of the present application is introduced. Fig.10 , Fig.10 This is a schematic diagram of the structure of an execution device provided in an embodiment of the present application. The execution device 1000 can be specifically embodied as a mobile phone, a tablet, a laptop computer, a server, etc., which is not limited here. Specifically, the execution device 1000 includes: a receiving unit 1001, a sending unit 1002, a processor 1003 and a memory 1004 (wherein the number of processors 1003 in the execution device 1000 can be one or more, Fig.10 In the example of FIG. 1003 , the processor 1003 may include an application processor 10031 and a communication processor 10032. In some embodiments of the present application, the receiving unit 1001, the sending unit 1002, the processor 1003 and the memory 1004 may be connected via a bus or other means.

[0171] The memory 1004 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1003. A portion of the memory 1004 may also include a non-volatile random access memory (NVRAM). The memory 1004 stores processor and operation instructions, executable modules or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0172] The processor 1003 controls the operation of the execution device. In a specific application, the various components of the execution device are coupled together through a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus, and a status signal bus, etc. However, for the sake of clarity, various buses are referred to as bus systems in the figure.

[0173] The method disclosed in the above embodiment of the present application can be applied to the processor 1003, or implemented by the processor 1003. The processor 1003 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 1003. The above processor 1003 can be a general processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and can further include an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The processor 1003 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined to execute. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1004, and the processor 1003 reads the information in the memory 1004 and completes the steps of the above method in combination with its hardware.

[0174] The receiving unit 1001 can be used to receive input digital or character information, and generate signal input related to the relevant settings and function control of the execution device. The sending unit 1002 can be used to output digital or character information through the first interface; the sending unit 1002 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the sending unit 1002 can also include a display device such as a display screen.

[0175] In one embodiment of the present application, the processor 1003 is configured to execute Figure 3 The method in the corresponding embodiment.

[0176] The electronic device provided in the embodiment of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit, wherein the processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin or a circuit, etc. The processing unit may execute computer execution instructions stored in the storage unit so that the chip in the execution device executes the rendering method described in the above embodiment. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0177] See also Fig.11 , Fig.11 This is a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present application. The present application also provides a computer-readable storage medium. In some embodiments, the above Figure 3 The disclosed methods may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of manufacture.

[0178] Fig.11 Schematically illustrates a conceptual partial view of an example computer-readable storage medium including a computer program for executing a computer process on a computing device, arranged in accordance with at least some embodiments presented herein.

[0179] In one embodiment, the computer readable storage medium 1100 is provided using a signal bearing medium 1101. The signal bearing medium 1101 may include one or more program instructions 1102, which when executed by one or more processors may provide the above-mentioned Figure 3 The functions or parts of the functions described. In addition, Fig.11 Program instructions 1102 in also describe example instructions.

[0180] In some examples, signal bearing medium 1101 may include computer readable medium 1103 such as, but not limited to, a hard drive, a compact disk (CD), a digital video disk (DVD), a digital tape, memory, ROM or RAM, and the like.

[0181] In some embodiments, the signal bearing medium 1101 may include a computer recordable medium 1104, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, etc. In some embodiments, the signal bearing medium 1101 may include a communication medium 1105, such as, but not limited to, a digital and / or analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communication link, a wireless communication link, etc.). Thus, for example, the signal bearing medium 1101 may be communicated by a wireless form of the communication medium 1105 (e.g., a wireless communication medium complying with the IEEE 802 standard or other transmission protocol).

[0182] The one or more program instructions 1102 may be, for example, computer executable instructions or logic implementation instructions. In some examples, the computing device of the computing device may be configured to provide various operations, functions, or actions in response to the program instructions 1102 communicated to the computing device via one or more of the computer readable medium 1103, the computer recordable medium 1104, and / or the communication medium 1105.

[0183] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0184] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0185] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0186] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0187] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0188] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0189] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0190] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for determining energy efficiency of a server, It is characterized in that include: Obtaining the operation status of each of the multiple AI loads respectively run by the artificial intelligence (AI) server, wherein the operation status includes the number of times and power of the AI ​​load run by the AI ​​server within a set time period, wherein the multiple AI loads include multiple neural network models, and the multiple neural network models are respectively used to perform different AI tasks, and the AI ​​server is deployed with hardware for performing AI calculations; Determine, based on an operation condition of the AI ​​server when running each AI load, an energy efficiency value of the AI ​​server when running each AI load; Based on the energy efficiency value of the AI ​​server when running each of the multiple AI loads, a target energy efficiency value of the AI ​​server is determined, where the target energy efficiency value is used to indicate the energy efficiency of the AI ​​server when running the AI ​​load.

2. The method according to claim 1, It is characterized in that The multiple AI loads also include multiple AI operators, and the multiple AI operators are all operators used in the neural network model.

3. The method according to claim 2, It is characterized in that The determining, based on the energy efficiency value of the AI ​​server when running each of the multiple AI loads, a target energy efficiency value of the AI ​​server includes: Based on the type of each AI load in the multiple AI loads, obtain a weight value corresponding to each AI load, wherein an AI load of a neural network model type corresponds to a different weight value than an AI load of an operator type; Based on the weight value corresponding to each AI load, the energy efficiency value of the AI ​​server when running each AI load is weighted to obtain a geometric mean to obtain a target energy efficiency value of the AI ​​server.

4. The method according to claim 2 or 3, It is characterized in that Each of the multiple AI operators is used to perform any one of the following types of operations: activation function operation, convolution operation, pooling operation, normalization operation, matrix multiplication operation, data discarding operation and full connection operation.

5. The method according to any one of claims 1 to 4, It is characterized in that The obtaining the running status of each of the multiple AI loads when the AI ​​server runs each of the multiple AI loads includes: Obtaining the operating conditions of the AI ​​server when running each AI load respectively under a plurality of different load conditions, and obtaining the operating conditions of each AI load under different load conditions; The determining, based on the running condition of the AI ​​server when running each AI load, the energy efficiency value of the AI ​​server when running each AI load includes: Determine, based on the operation status of the AI ​​server when running each AI load under each load state, an energy efficiency value when the AI ​​server runs each AI load under each load state; The geometric mean of the energy efficiency values ​​of the AI ​​server when running the same AI load under multiple different load states is calculated to obtain the energy efficiency value of the AI ​​server when running each AI load.

6. The method according to any one of claims 1 to 5, It is characterized in that The determining, based on the running condition of the AI ​​server when running each AI load, the energy efficiency value of the AI ​​server when running each AI load includes: Obtaining a preset number of operations corresponding to each AI load, where the preset number of operations is used to indicate the number of times a preset server runs the AI ​​load within the set time period; The energy efficiency value of the AI ​​server when running each AI load is obtained by calculating the ratio between the target ratio corresponding to each AI load and the power of the AI ​​server running each AI load. The target ratio corresponding to each AI load is the ratio between the number of times the AI ​​server runs each AI load within the set time and the preset number of operations corresponding to each AI load.

7. The method according to any one of claims 1 to 6, It is characterized in that Each of the multiple neural network models is used to perform any one of the following types of AI tasks: computer vision processing tasks, natural language processing tasks, sorting learning processing tasks, content recommendation processing tasks, and data mining tasks.

8. The method according to any one of claims 1 to 7, It is characterized in that The AI ​​server is deployed with any one of the following hardware: a graphics processing unit GPU, a neural network processing unit NPU, and a tensor processing unit TPU.

9. A device for determining energy efficiency of a server, It is characterized in that include: an acquisition module, configured to acquire the operation status of each of a plurality of AI loads respectively run by the artificial intelligence (AI) server, wherein the operation status includes the number of times and power at which the AI ​​server runs the AI ​​load within a set time period, wherein the plurality of AI loads include a plurality of neural network models, and the plurality of neural network models are respectively used to perform different AI tasks, and hardware for performing AI calculations is deployed on the AI ​​server; A processing module, configured to determine an energy efficiency value of the AI ​​server when running each AI load based on an operation condition of the AI ​​server when running each AI load; The processing module is further used to determine a target energy efficiency value of the AI ​​server based on the energy efficiency value of the AI ​​server when running each of the multiple AI loads, wherein the target energy efficiency value is used to indicate the energy efficiency of the AI ​​server when running the AI ​​load.

10. The device according to claim 9, It is characterized in that The multiple AI loads also include multiple AI operators, and the multiple AI operators are all operators used in the neural network model.

11. The device according to claim 10, It is characterized in that The processing module is further used for: Based on the type of each AI load in the multiple AI loads, obtain a weight value corresponding to each AI load, wherein an AI load of a neural network model type corresponds to a different weight value than an AI load of an operator type; Based on the weight value corresponding to each AI load, the energy efficiency value of the AI ​​server when running each AI load is weighted to obtain a geometric mean to obtain a target energy efficiency value of the AI ​​server.

12. The device according to claim 10 or 11, It is characterized in that Each of the multiple AI operators is used to perform any one of the following types of operations: activation function operation, convolution operation, pooling operation, normalization operation, matrix multiplication operation, data discarding operation and full connection operation.

13. The device according to any one of claims 9 to 12, It is characterized in that The acquisition module is specifically used to acquire the operation status of the AI ​​server when running each AI load respectively under multiple different load states, and obtain the operation status of each AI load under different load states; The processing module is further used for: Determine, based on the operation status of the AI ​​server when running each AI load under each load state, an energy efficiency value when the AI ​​server runs each AI load under each load state; The geometric mean of the energy efficiency values ​​of the AI ​​server when running the same AI load under multiple different load states is calculated to obtain the energy efficiency value of the AI ​​server when running each AI load.

14. The device according to any one of claims 9 to 13, It is characterized in that The acquisition module is further used to acquire a preset number of operations corresponding to each AI load, where the preset number of operations is used to indicate the number of times a preset server runs the AI ​​load within the set time period; The processing module is further used to obtain the energy efficiency value of the AI ​​server when running each AI load by obtaining the ratio between the target ratio corresponding to each AI load and the power of the AI ​​server running each AI load, wherein the target ratio corresponding to each AI load is the ratio between the number of times the AI ​​server runs each AI load within the set time period and the preset number of operations corresponding to each AI load.

15. The device according to any one of claims 9 to 14, It is characterized in that Each of the multiple neural network models is used to perform any one of the following types of AI tasks: computer vision processing tasks, natural language processing tasks, sorting learning processing tasks, content recommendation processing tasks, and data mining tasks.

16. The device according to any one of claims 9 to 15, It is characterized in that The AI ​​server is deployed with any one of the following hardware: GPU, NPU and TPU.

17. A device for determining energy efficiency of a server, It is characterized in that The device comprises a memory and a processor; the memory stores codes, the processor is configured to execute the codes, and when the codes are executed, the device executes the method according to any one of claims 1 to 8.

18. A computer storage medium, It is characterized in that The computer storage medium stores instructions, which, when executed by a computer, cause the computer to implement the method of any one of claims 1 to 8.

19. A computer program product, It is characterized in that The computer program product stores instructions, which, when executed by a computer, cause the computer to implement the method according to any one of claims 1 to 8.

Images