Execution hardware determination method

Abstract

To provide a method of selecting neural network model execution hardware.SOLUTION: The method includes: query reception processing S201 of reading a user query describing a use case of a neural network model, a performance condition, and a candidate hardware restriction on execution hardware; retrieval processing S202 of retrieving a standard model being a neural network model satisfying the majority of the candidate hardware restriction and the performance condition from a model database; trial calculation processing S203 of inputting a performance metric of the standard model and the candidate hardware restriction, which are based on the user query, to an energy prediction model to obtain an energy consumption in candidate hardware; and determination processing S204 to 206 of determining execution hardware for executing a workload responding to the user query on the basis of the obtained energy consumption and a ratio of renewable energy to energy supplied to the candidate hardware.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to a method for determining execution hardware. [Background technology]

[0002] It is desirable to optimize a neural network model for the hardware on which it is executed.Patent Document 1 discloses a method for training and optimizing a machine learning model, the method including the steps of selecting a machine learning model for optimization, generating a set of derived variants of the machine learning model, and for each derived variant, quantizing numerical parameters in the derived variant, compiling the derived variants to generate runtime artifacts, evaluating the set of derived variants for latency in a target hardware architecture and identifying one or more derived variants that meet the latency criteria, training only one or more variants, and evaluating the accuracy of the trained one or more variants. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent Application Publication No. 2023 / 0297835 Summary of the Invention [Problem to be solved by the invention]

[0004] In the invention described in Patent Document 1, it is not possible to select hardware suitable for executing a neural network model from the viewpoint of Brownian energy consumption. [Means for solving the problem]

[0005] An execution hardware determination method according to a first aspect of the present invention is an execution hardware determination method in which a computer determines execution hardware, which is hardware that executes a neural network model, and includes: a query reception process that reads a user query that describes the use case of the neural network model, performance conditions, and constraints of candidate hardware that is a candidate for the execution hardware; a search process that searches a model database for a standard model, which is a neural network model that satisfies most of the constraints of the candidate hardware and the performance conditions; an estimation process that inputs performance metrics of the standard model based on the user query and the constraints of the candidate hardware into an energy prediction model to obtain energy consumption in the candidate hardware; and a determination process that determines the execution hardware that will execute the workload corresponding to the user query based on the obtained energy consumption and a green power ratio, which is the proportion of renewable energy in the energy supplied to the candidate hardware. [Effects of the Invention]

[0006] According to the present invention, it is possible to select hardware suitable for executing a neural network model in terms of the amount of Brownian energy consumed. [Brief explanation of the drawings]

[0007] [Figure 1] Overall configuration diagram including the execution hardware determination system [Figure 2] Configuration diagram of the execution hardware determination system [Figure 3] FIG. 10 is a diagram showing an example of a user query. [Figure 4] An example of a model database [Figure 5] A diagram showing an example of a green power ratio table [Figure 6] Diagram showing the correlation between the program and the neural network model [Figure 7] Flowchart showing how to generate a predictive model [Figure 8] Flowchart showing execution hardware determination processing [Figure 9] A diagram showing an example of calculation of the execution hardware determination program [Figure 10] Flowchart showing the optimization process [Figure 11] A diagram showing an example of a user interface displayed on the console [Figure 12] Flowchart showing optimization processing during execution in Modification 1 DETAILED DESCRIPTION OF THE INVENTION

[0008] In this specification, electricity generated using renewable energy is referred to as "green power," and electricity generated using non-renewable energy is referred to as "brown power." The ratio of green power to brown power is referred to as the "green power ratio." The power composition ratio is, for example, a value between 0 and 1, where 0 means all brown power, 1 means all green power, and 0.5 means half brown power and half green power. For example, electricity generated using wind, geothermal, solar power, etc. is green power, and electricity generated using fossil fuels is brown power. In this specification, a neural network model optimized for specific hardware is referred to as an "optimized model," and an unoptimized model is referred to as a "standard model."

[0009] --Embodiment-- Hereinafter, an embodiment of an execution hardware determination system will be described with reference to FIGS.

[0010] FIG. 1 is an overall configuration diagram including an execution hardware determination system 400. The execution hardware determination system 400 is connected to a first inference hardware 100-1, a second inference hardware 100-2, ..., an N-th inference hardware 100-N, and a client 500 via a network 600. Hereinafter, the first inference hardware 100-1, the second inference hardware 100-2, ..., and the N-th inference hardware 100-N will be collectively referred to as inference hardware 100. The inference hardware 100 differs in at least one of hardware configuration, power configuration ratio, and power rating. The inference hardware 100 may be located in the same data center or in different data centers.

[0011] The client 500 communicates via a network 600 and sends a user query 6000 to the execution hardware determination system 400. The user query 6000 includes a use case, performance conditions, and hardware constraints, as described below. The execution hardware determination system 400 determines the inference hardware 100 that will execute the workload and causes the inference hardware 100 to execute the workload. Hereinafter, the term "workload" refers to a computational process using a neural network model that is selected and optimized based on the user query 6000. Hereinafter, the hardware among the inference hardware 100 that is a candidate for executing the workload will be referred to as "candidate hardware," and the hardware that executes the workload will be referred to as "execution hardware." The execution hardware is selected from the candidate hardware.

[0012] 2 is a configuration diagram of an execution hardware determination system 400. The execution hardware determination system 400 includes a processor 401, a memory 402, a local storage 403, a network interface 404, and an input / output device 405. These components can exchange data with each other via a system bus 406. The processor 401 is, for example, a central processing unit. The memory 402 is a high-speed readable / writable storage device, for example, a DRAM. The local storage 403 is a non-volatile storage device, for example, a hard disk drive. The network interface 404 is a network interface card. The input / output device 405 is, for example, a display adapter.

[0013] The network interface 404 handles all communications with the outside of the execution hardware determination system 400 via the network 600. The input / output device 405 provides an interface for inputting and displaying information to the console 407. The processor 401 loads programs stored in the local storage 403 into the memory 402 and executes them. The processor 401 reads data stored in the local storage 403 into the memory 402 as needed.

[0014] The local storage 403 stores an energy prediction model 41, a prediction model generation program 42, an execution hardware determination program 43, a model optimization program 44, a standard model database 45, and a green power ratio table 46. The energy prediction model 41 is a neural network model that predicts in advance the energy consumption when executing a workload on each inference hardware 100.

[0015] The prediction model generation program 42 generates an energy prediction model 41. The execution hardware determination program 43 determines the execution hardware that will execute the workload based on the user query 6000. The model optimization program 44 optimizes the neural network model for the execution hardware determined by the execution hardware determination program 43, and places the optimized model and workload on the execution hardware. The standard model database 45 stores data on various known neural networks. The green power ratio table 46 stores data on the green power ratio for each data center. While FIG. 2 illustrates the execution hardware determination system 400 as being composed of a single computer, the execution hardware determination system 400 may also be realized by multiple computers operating in cooperation.

[0016] FIG. 3 is a diagram showing an example of a user query 6000. The user query 6000 is related to the construction of a neural network model and includes a use case, performance conditions, and hardware constraints. The use case indicates the application of the neural network model to be created. An example of a use case is "I want to build a model to detect product defects on an assembly line." The performance conditions are the accuracy and performance conditions required during inference, such as "F1 score of more than 0.8, latency of less than 5 milliseconds, and inference speed of more than 10 / second."

[0017] The hardware constraints are constraints on the hardware that executes the neural network model to be constructed. The aforementioned candidate hardware is hardware that satisfies the hardware constraints. This "candidate" refers to a candidate hardware that executes the workload based on the user query. The candidate hardware is determined by the execution hardware determination program 43. The hardware constraints can include computing devices, memory, storage, model size, and power rating. However, the hardware constraints do not necessarily have to include these five items; they only need to include at least computing devices. In the example shown in Figure 3, three hardware constraints are listed, but at least one of them is sufficient. In the example shown in Figure 3, the three hardware constraints are OR conditions, and any one of them needs to be satisfied.

[0018] The hardware constraint may be a condition rather than a specific configuration. For example, it may be specified as "a CPU with eight or more cores" or "a GPU with 12 GB or more of VRAM." The candidate hardware may be determined solely from the description in the user query 6000, or may be determined by referring to other information, such as the green power ratio table 46, to determine the specific hardware configuration. In particular, when a condition such as "a CPU with eight or more cores" is used as a hardware constraint, it is useful for the execution hardware determination program 43 to refer to the specific configurations listed in the green power ratio table 46 and determine all applicable hardware as candidate hardware.

[0019] FIG. 4 is a diagram showing an example of the standard model database 45. The standard model database 45 stores data related to two or more neural networks. The standard model database 45 has multiple records, each corresponding to one neural network. The specific configuration of each neural network may be included in the standard model database 45 or may be stored outside the standard model database 45. Each record in the standard model database 45 has fields for model architecture 3001, use case 3002, evaluation metric 3003, latency 3004, inference speed 3005, computation unit 3006, storage 3007, and power rating 3008.

[0020] Model architecture 3001 is the name of the neural network model. Use case 3002 is a typical situation in which the neural network model is used. Computing unit 3006 and storage 3007 are the main specifications of the computing device that executes the neural network model. Evaluation metric 3003, latency 3004, and inference speed 3005 are the performance of the neural network model when using computing unit 3006 and storage 3007.

[0021] The power rating 3008 is the power consumption when the neural network model is executed using the calculation unit 3006 and the storage 3007. The standard model database 45 may be created manually by an operator or generated automatically. The data stored in the standard model database 45 is obtained from the description of each neural network model or from the Internet. Therefore, the variations in the combination of the model architecture 3001 and the calculation unit 3006 are limited.

[0022] FIG. 5 is a diagram showing an example of a green power ratio table 46. The green power ratio table 46 is composed of multiple records, each of which includes a data center ID 461, a green power ratio 462, and a calculation unit 463. The data center ID 461 is an identifier for identifying a data center. The green power ratio 462 is the ratio of green power to the electricity supplied to the data center, and a value of "1" means that all of the electricity is green power. The data for the green power ratio 462 may be collected manually by an operator or automatically using an API. If the green power ratio for each data center cannot be obtained, the green power ratio for the region in which the data center is located may be used. The calculation unit 463 is a list of computing hardware available in the data center. The data for the calculation unit 463 is obtained from each data center. The calculation unit 463 has a storage 464 field.

[0023] 6 is a diagram showing the correlation between the program and the neural network model. A prediction model generation program 42, a standard model database 45, a green power ratio table 46, and a standard model 7, which is an unoptimized neural network model, are prepared in advance. The standard model 7 is a specific example of the model architecture 3001 described in the standard model database 45. The prediction model generation program 42 generates an energy prediction model 41.

[0024] The execution hardware determination program 43 reads the standard model database 45 and the green power ratio table 46. The execution hardware determination program 43 calls the energy prediction model 41 and uses it for calculations. The execution hardware determination program 43 outputs the execution hardware and the name of the standard model, which are the calculation results, to the model optimization program 44. The model optimization program 44 generates an optimized model 7P by optimizing the standard model 7 for the execution hardware.

[0025] FIG. 7 is a flowchart showing a method for generating an energy prediction model 41 by the prediction model generation program 42. The energy prediction model 41 predicts the energy consumed by the inference hardware 100 to execute a workload that uses a neural network model. First, in step S400, the prediction model generation program 42 generates a dummy dataset for each workload category. The workload categories are the same as the use cases 3002 in the standard model database 45. Various known methods can be used to generate the dummy dataset. Hereinafter, a workload that uses a dummy dataset will be referred to as a "dummy workload." There are as many dummy workloads as there are use cases 3002.

[0026] In the next step S401, the prediction model generation program 42 lists various available hardware configurations and executes a dummy workload using each neural network model in each configuration. If one neural network model has multiple use cases 3002, the neural network model executes a dummy workload for each use case 3002. The "various hardware configurations" in this step are not limited to the hardware specifications 3200 described in the standard model database 45, but include various available hardware.

[0027] This hardware may include new hardware that was not publicly available when the standard model database 45 was created, and all hardware provided as virtual computers that can be accessed on demand via the Internet. The various available hardware configurations identified in this step may also be candidates for hardware that executes the workload based on the user query 6000, and therefore these hardware configurations are also candidate hardware. In the following, the process of enumerating candidate hardware in this step will also be referred to as the "enumeration process."

[0028] In the following step S402, the prediction model generation program 42 measures data related to the dummy workload executed in step S401, i.e., energy consumption and performance metrics 3100 such as accuracy, F1 score, latency, and inference speed. Hereinafter, the process of executing the dummy workload using the candidate hardware in step S401 and the process of measuring performance and energy consumption in step S402 will be referred to as the "measurement process."

[0029] In the next step S403, the prediction model generation program 42 creates training data. This training data includes, as input data, the performance metrics 3100 measured in step S402 and the hardware specifications 3200. This training data includes, as output data, the energy consumption measured in step S402.

[0030] The types of data stored in the training data are the same as those in the standard model database 45. However, while the standard model database 45 only has a limited number of combinations of model architecture 3001 and hardware specifications 3200, the training data has a very large number of such combinations. In the following step S404, the prediction model generation program 42 uses the training data created in step S403 to train the energy prediction model 41, in other words, to update the parameters of the energy prediction model 41. This concludes the description of the processing shown in FIG. 7.

[0031] 8 is a flowchart showing the execution hardware determination process by the execution hardware determination program 43. First, in step S201, the execution hardware determination program 43 reads the user query 6000. As described with reference to FIG. 3, the user query 6000 includes a model use case, a model performance criterion, and candidate inference hardware constraints. Hereinafter, the process in step S201 will also be referred to as "query reception process."

[0032] In the next step S202, the execution hardware determination program 43 selects from the standard model database 45 a standard model that generally satisfies the requirements described in the user query 6000. For example, the execution hardware determination program 43 may select a model that does not completely satisfy the performance metric 3100, such as 90% or 80% of the value described in the user query 6000, although it is desirable that the performance metric 3100 satisfy the value described in the user query 6000. Furthermore, it is desirable that the power rating 3008 is equal to or less than the value described in the user query 6000, but it may also be a value that exceeds 10% or 20% of the value described in the user query 6000. The number of standard models selected in this step is one or more. Hereinafter, the processing in step S202 will also be referred to as a "search processing."

[0033] In the following step S203, the execution hardware determination program 43 calculates the power consumption of the standard model selected in step S202 using the energy prediction model 41. Specifically, the execution hardware determination program 43 inputs the performance metric 3100 for the selected standard model stored in the standard model database 45 and the specifications of the candidate hardware determined from the hardware constraints stored in the user query 6000 to the energy prediction model 41 generated by the prediction model generation program 42. If there are multiple candidate hardware, the execution hardware determination program 43 inputs the specifications of each candidate hardware. As described above, the execution hardware determination program 43 may determine the candidate hardware by referring to data describing specific available hardware configurations, such as the green power ratio table 46.

[0034] In this step, the execution hardware determination program 43 repeats the process the number of times equal to the number of standard models selected in step S202. For example, if two standard models are selected in step S202 and three candidate hardware items exist based on the example user query 6000 shown in FIG. 3, the execution hardware determination program 43 performs six inputs. In this case, six power consumption amounts corresponding to the six inputs are calculated. Hereinafter, the process in step S203 will also be referred to as the "estimation process."

[0035] In the next step S204, the execution hardware determination program 43 refers to the green power ratio table 46 and identifies a data center that satisfies the hardware specifications 3200 described in the user query 6000. For example, in the example of the green power ratio table 46 shown in FIG. 5, when the hardware specifications 3200 are "GPU with 4GB VRAM or 8-core CPU," data centers with IDs "D2" and "D3" are identified.

[0036] In the next step S205, the execution hardware determination program 43 identifies the combination of hardware and data center that will result in the least predicted brown energy consumption. For example, if the energy prediction model 41 calculates that the power consumption of a "GPU with 4GB VRAM" is 200 WH and the power consumption of an "8-core CPU" is 300 WH, the following combinations are identified in the example of the green power ratio table 46 shown in Figure 5.

[0037] That is, in the data center with ID "D2," the Brown energy ratio is "0.8" (1 minus "0.2"), so it consumes "160WH" of Brown energy, which is the product of "200WH" and "0.8." In the data center with ID "D3," the Brown energy ratio is "0.5" (1 minus "0.5"), so it consumes "150WH" of Brown energy, which is the product of "300WH" and "0.5." In other words, in the example shown here, the data center with ID "D3" is identified as the data center with the least Brown energy consumption.

[0038] In the following step S206, the execution hardware determination program 43 identifies a configuration that satisfies the hardware specifications in the data center identified in step S205. For example, in the previous example, a "GPU with 4 GB of VRAM" is identified in the data center with ID "D3." Furthermore, the name or identifier of the standard model corresponding to the configuration that is determined to have the least Brownian energy consumption in step S205 is output to the model optimization program 44 as the optimization target. Hereinafter, the processing in steps S204 to S306 will also be referred to as a "determination process." This concludes the explanation of FIG. 8.

[0039] FIG. 9 is a diagram showing an example of calculations by the execution hardware determination program 43. The example shown in this figure is an example in which the standard models determined in step S202 are two, "M1" and "M2," and there are three candidate hardware. Therefore, in this example, input to the energy prediction model 41 and output of power consumption were performed six times, which is the product of 2 and 3. Furthermore, of the three candidate hardware, only one, "GPU with 8GB VRAM," was present in two locations, "D12" and "D21," and the other two were present in only one location each.

[0040] Therefore, even for a standard model with the same candidate hardware (8GB VRAM), there are two records. The right side of this figure shows the Brown energy consumption, and the fifth record from the top is selected as the lowest at 15WH. In this case, the execution hardware is a GPU with 8GB VRAM, the standard model to be optimized is M2, and the optimized model will be placed in data center D12 in later processing.

[0041] FIG. 10 is a flowchart showing the optimization process executed by the model optimization program 44. First, in step S301, the model optimization program 44 loads a specified standard model. In the following step S302, the model optimization program 44 performs optimization processing, i.e., updates parameters and changes the network configuration. The optimization method is not particularly limited, and various known methods can be used. Model compression methods such as quantization, pruning, and knowledge distillation may be used for optimization. In the following step S303, the model optimization program 44 loads the model updated in step S302 into the execution hardware and performs a test run using a dummy workload. Performance is also measured in this test run.

[0042] In the following step S304, the model optimization program 44 determines whether the results of the test execution in step S303 satisfy the performance conditions described in the user query 6000. If the model optimization program 44 determines that the results of the test execution satisfy the performance conditions, it proceeds to step S305. If the model optimization program 44 determines that the results of the test execution do not satisfy the performance conditions, it returns to step S302 and performs optimization again. In other words, the model optimization program 44 repeats optimization until the performance conditions are satisfied.

[0043] In step S305, the model optimization program 44 places the optimized model and workload on the execution hardware, starts execution, and ends the processing shown in Fig. 10. Hereinafter, the processing in step S305 will be referred to as "execution processing."

[0044] 11 is a diagram showing an example of a user interface displayed on the console 407. A workload display window 7000 shows the correlation between energy consumption and Brownian energy consumption for each of a plurality of workloads. A user query 6000 corresponding to a workload ID is input in advance, and when the operator inputs a workload ID 7001, a start time, and an end time and then presses an add workload button 7003, the execution hardware determination program 43 and the model optimization program 44 start operating. Then, the optimized model optimized by the model optimization program 44 is placed in the data center determined by the execution hardware determination program 43, and the workload is executed.

[0045] The result display area 7002 displays a pie chart along with the workload ID. In each pie chart, the percentage of Brownian energy is displayed as a hatched area, and the total energy consumption of the workload is displayed as the size of the circle. The pie chart is positioned further to the right of the illustration if the total energy consumption of the workload is greater, and further up the illustration if the Brownian energy consumption is greater.

[0046] According to the above-described embodiment, the following effects can be obtained. (1) The execution hardware determination method executed by the execution hardware determination system 400 is as follows: The process includes a query reception process (S201 in FIG. 8) that reads a user query 6000 that describes the use case of a neural network model, performance conditions, and constraints of candidate hardware that is a candidate for execution hardware; a search process (S202) that searches a model database for a standard model, which is a neural network model that satisfies most of the constraints and performance conditions of the candidate hardware; a calculation process (S203) that inputs performance metrics of the standard model based on the user query 6000 and constraints of the candidate hardware into the energy prediction model 41 to obtain energy consumption in the candidate hardware; and a determination process (S206) that determines execution hardware that will execute the workload corresponding to the user query 6000 based on the obtained energy consumption and the green power ratio, which is the proportion of renewable energy in the energy supplied to the candidate hardware. Therefore, hardware suitable for executing a neural network model can be selected in terms of Brownian energy consumption.

[0047] (2) The processing executed by the execution hardware determination system 400 includes an optimization process executed by the model optimization program 44 to generate an optimized model by optimizing the standard model for the execution hardware. Therefore, the neural network model can be optimized for hardware that is suitable for executing the neural network model in terms of Brownian energy consumption.

[0048] (3) The standard model database 45 includes use cases, model performance, and hardware specifications for each neural network model.

[0049] (4) The processes executed by the execution hardware determination system 400 include an energy prediction model generation process executed by the prediction model generation program 42 to generate an energy prediction model 41. As shown in Fig. 7, the energy prediction model generation process includes an enumeration process (S401) that enumerates available candidate hardware, a measurement process (S401 and S402) that uses each of the listed candidate hardware to execute test operations using a dummy data set and a standard model to measure performance and energy consumption, and a generation process (S404) that generates an energy prediction model 41 using the hardware configuration of the candidate hardware and the performance measured in the measurement process as input and the energy consumption measured in the measurement process as output. Therefore, the execution hardware determination system 400 can generate the energy prediction model 41 by itself.

[0050] (5) The optimization process includes model compression technology.

[0051] (6) The processes executed by the execution hardware determination system 400 include a model database creation process for collecting data from the Internet and creating a standard model database 45 .

[0052] (Variation 1) In the above-described embodiment, the processing up to the start of a workload has been described, but the processing during the execution of the workload has not been specifically described. A running optimization program may be further stored in the local storage 403 of the execution hardware determination system 400, and the running optimization program may migrate the workload to another hardware while it is being executed. The purpose of this is to further reduce Brownian energy, and a change in the green power ratio triggers the running optimization program to operate. The green power ratio table 46 may be updated manually by an operator. Alternatively, the green power ratio table 46 may be automatically updated by periodically executing an API that acquires the green power ratio of a region or facility. The running optimization program periodically executes the processing shown in FIG. 12, for example, every five minutes or every hour.

[0053] 12 is a flowchart showing the in-progress optimization process, which executes the optimization process again while a workload is being executed. The in-progress optimization program first references the green power ratio table 46 in step S351 to determine whether the green power ratio of the candidate hardware has changed. If the in-progress optimization program determines that the green power ratio of the candidate hardware has changed, it proceeds to step S352, and if it determines that the green power ratio of the candidate hardware has not changed, it ends the process shown in FIG. 12.

[0054] In step S352, the running optimization program causes the execution hardware determination program 43 to execute an execution hardware determination process. In the following step S353, the running optimization program determines whether the execution hardware determined by the execution hardware determination program 43 has changed from the current execution hardware. If the running optimization program determines that the execution hardware has changed, it proceeds to step S354, but if it determines that the execution hardware has not changed, it ends the process shown in Figure 12. In step S354, the running optimization program causes the model optimization program 44 to execute an optimization process, and ends the process shown in Figure 12.

[0055] According to this first modification, the following effects can be obtained. (7) The processes executed by the execution hardware determination system 400 include an execution process (step S305 in FIG. 10) that executes the optimized model on the execution hardware, and an on-the-fly optimization process (FIG. 12) that executes the determination process and optimization process again if there is a change in the green power ratio in the candidate hardware. Therefore, the execution hardware determination system 400 can reduce Brownian energy even if the green power ratio changes after the workload starts.

[0056] (Variation 2) In the above-described embodiment, the execution hardware determination system 400 not only determines the execution hardware but also optimizes the model. However, the execution hardware determination system 400 may not optimize the model, and another device may optimize the model. Furthermore, the generation of the dummy data set executed in step S400 of FIG. 7 is not a required process for the execution hardware determination system 400, and a dummy data set created in advance may be read. Furthermore, the execution hardware determination system 400 may not be equipped with the prediction model generation program 42, and may instead read a created energy prediction model 41.

[0057] In each of the above-described embodiments and modifications, the functional block configurations are merely examples. Some functional configurations shown as separate functional blocks may be configured as an integrated unit, or a configuration shown in a single functional block diagram may be divided into two or more functions. Furthermore, some of the functions of each functional block may be provided by other functional blocks.

[0058] In each of the above-described embodiments and modifications, the execution hardware determination system 400 may include an input / output interface (not shown), and a program may be loaded from another device as needed via the input / output interface and a medium available to the execution hardware determination system 400. Here, the medium refers to, for example, a storage medium detachable from the input / output interface, or a communication medium, i.e., a wired, wireless, or optical network, or a carrier wave or digital signal propagating through the network. Furthermore, some or all of the functions realized by the program may be realized by a hardware circuit or FPGA.

[0059] The above-described embodiments and modifications may be combined with each other. Although various embodiments and modifications have been described above, the present invention is not limited to these. Other embodiments conceivable within the scope of the technical concept of the present invention are also included within the scope of the present invention. [Explanation of symbols]

[0060] 41: Energy forecasting model 42: Prediction model generation program 43: Execution hardware determination program 44: Model optimization program 45: Standard Model Database 46: Green Power Ratio Table 400: Execution hardware determination system 6000: User queries

Claims

1. An execution hardware determination method in which a computer determines execution hardware that executes a neural network model, comprising: a query receiving process that reads a user query describing a use case of a neural network model, performance requirements, and constraints of candidate hardware that are candidates for the execution hardware; a search process for searching a model database for a standard model, which is a neural network model that satisfies most of the constraints of the candidate hardware and the performance conditions; a calculation process for inputting performance metrics of the standard model based on the user query and constraints of the candidate hardware into an energy prediction model to obtain energy consumption of the candidate hardware; and a determination process for determining the execution hardware that will execute the workload corresponding to the user query based on the obtained energy consumption and a green power ratio, which is the proportion of renewable energy in the energy supplied to the candidate hardware.

2. 2. The execution hardware determination method according to claim 1, The execution hardware determination method further includes an optimization process for generating an optimized model by optimizing the standard model for the execution hardware.

3. 2. The execution hardware determination method according to claim 1, The execution hardware determination method includes the model database including use cases, model performance, and hardware specifications for each neural network model.

4. 2. The execution hardware determination method according to claim 1, further comprising an energy prediction model generation process for generating the energy prediction model; The energy prediction model generation process includes: an enumeration process for enumerating the available candidate hardware; a measurement process for measuring performance and energy consumption by performing test operations using a dummy data set and the standard model using each of the candidate hardware items listed; and a generation process for generating the energy prediction model, which uses the hardware configuration of the candidate hardware and the performance measured in the measurement process as inputs and outputs the energy consumption measured in the measurement process.

5. 3. The execution hardware determination method according to claim 2, an execution process for executing the optimized model on the execution hardware; The execution hardware determination method further includes an on-the-fly optimization process that executes the determination process and the optimization process again when there is a change in the green power ratio in the candidate hardware.

6. 3. The execution hardware determination method according to claim 2, The optimization process includes a model compression technique.

7. 2. The execution hardware determination method according to claim 1, The execution hardware determination method further includes a model database creation process for collecting data from the Internet and creating the model database.

Images