Causal analysis driven neural network model low-energy-consumption hyper-parameter recommendation method

Abstract

The invention discloses a causal analysis-driven neural network model low-energy-consumption hyper-parameter recommendation method. The method comprises the steps of extracting modifiable hyper-parameters, performance and energy consumption measurement from a to-be-tested model according to specific requirements; designing a hyper-parameter variation range according to development requirements, and designing a variation operator according to the variation range; introducing a mutation operator into the original model to obtain a hyper-parameter mutation model; running the variation model to collect performance and energy consumption metrics; performing causal analysis on the measured performance and energy consumption metrics; and evaluating a tradeoff relationship between energy consumption and performance according to a causal analysis result, and recommending hyper-parameters. The method solves the problem that which hyper-parameters are more suitable for being used for optimizing the hyper-parameter method to reduce the energy consumption of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of software system energy consumption, and specifically to a low-energy consumption hyperparameter recommendation method for a neural network model driven by causal analysis. Background Art

[0002] Energy consumption generally refers to the electrical energy consumed by software systems in current research. Given that environmental issues are receiving increasing attention, more and more researchers are concerned about the energy consumption of computer programs. High energy consumption of software systems generates more carbon dioxide, which harms the environment and also increases the financial costs of software development and maintenance.

[0003] With the enhancement of computing power and the support of larger amounts of data, artificial intelligence technology has made breakthrough progress in the past few decades. Artificial intelligence technology has been widely applied to software systems themselves and their development processes. With the increasing popularity of artificial intelligence-based software systems, the energy consumption problem of software systems has gradually become prominent. Currently, researchers are more concerned about the energy consumption of intelligent software systems because these software systems tend to consume more energy than traditional software systems. According to the observation of Papineni et al., from 2012 to 2018, the most advanced neural network models consumed $3 million in resources. The extreme growth of the energy consumption of intelligent software systems is a warning indicating that the energy consumption problem of intelligent software systems should receive more attention.

[0004] With the explosive growth of artificial intelligence technology, the importance of the energy consumption problem of intelligent software systems is increasing. Many works focus on the energy consumption problem of artificial intelligence software systems. For example, Georgiou et al. conducted an evaluation to raise people's awareness of the energy costs of different neural network frameworks; Shanbhag et al. proposed an initial classification of 8 energy patterns for developing neural network applications by analyzing 1361 posts on Stack Overflow. In addition to empirical studies on energy consumption patterns, research on reducing the energy consumption of intelligent software systems has also emerged. In these studies, adjusting hyperparameters as an important method for reducing system energy consumption has been studied. For example, Chavannes et al. provided adding energy consumption to the optimization objective and reducing the energy consumption of the model by adjusting hyperparameters. However, adjusting hyperparameters may lead to a loss of model performance, so this type of method still faces difficulties in practice.

[0005] In summary, the method of optimizing hyperparameters to reduce model energy consumption is effective, but this method faces the risk of side effects of performance loss. In order to be able to safely use this type of method to build low-energy consumption models, it is necessary to reduce the possibility of side effects of performance loss. Summary of the Invention

[0006] The object of the present invention is to provide a method for recommending low - energy consumption hyperparameters of a neural network model driven by causal analysis, exploring the hyperparameter space of the neural network model through mutation operators, and obtaining the trade - off relationship between model performance and energy consumption by causal analysis, so as to recommend low - energy consumption hyperparameters.

[0007] To solve the above - mentioned technical problems, the present invention provides the following technical solutions:

[0008] The steps of the method for recommending low - energy consumption hyperparameters of a neural network model driven by causal analysis include:

[0009] Determine the hyperparameters to be measured, performance metrics, and energy consumption metrics in the model, and select the set of hyperparameters that are allowed to be debugged in the model as basis vectors to construct a vector space of hyperparameters; among them, hyperparameters are the parameters manually set in the neural network model, such as the learning rate, etc.; performance metrics are metrics for measuring the quality of the neural network model, and performance metrics specifically refer to quantified performance metrics, such as error rate, precision rate, and F1 - value, etc.; energy consumption metrics are metrics for the electrical energy consumed by the software system, including direct or indirect metrics. The direct metric is a metric that directly reflects energy consumption, such as the energy consumption reported by a certain hardware driver; the indirect metric is a metric of non - hardware energy consumption, such as running time.

[0010] Select a set of hyperparameter ranges D in the hyperparameter space according to the development requirements, determine the hyperparameter mutation range with limited influence on model performance according to the set of hyperparameter ranges D, and determine the mutation operator. Specifically, according to the set of hyperparameter ranges D, a mutation operator can be designed to mutate a certain hyperparameter or hyperparameter combination in the original model. By calling this mutation operator multiple times, several mutant hyperparameter points allowed by the requirements can be obtained. Among them, the set of hyperparameter ranges D is a set of points in the hyperparameter space accepted by the development requirements and is a subspace of the hyperparameter space. For any point z ∈ D, it is an acceptable hyperparameter point, that is, each hyperparameter value corresponding to the coordinates of this point is within the requirements limit.

[0011] Introduce a mutation operator into the original model to mutate the hyperparameters to obtain a mutant model;

[0012] Run the mutant model to measure the performance metrics and energy consumption metrics;

[0013] Perform causal analysis on the energy consumption metrics and performance metrics, and evaluate the trade - off relationship between the specific energy consumption and performance metrics of the model according to the causal analysis, and recommend the set of hyperparameters that can be changed.

[0014] According to the above - mentioned technical solution, the mutation operator inputs a random number seed seed and a certain hyperparameter or hyperparameter combination h * ∈H * , and the mutation operator outputs a point z i =(xi1 , x i2 , …, x in ), for the input hyperparameter or hyperparameter combination h * in the hyperparameters, assign random values within the allowable range to implement this mutation operator.

[0015] The mutation operator O is a mapping from the Cartesian product of the real number field R and the set H of all hyperparameters or hyperparameter combinations participating in the evaluation * to the set D of hyperparameter range:

[0016]

[0017] where R represents the real number, O represents the mutation operator, seed represents the random number seed; seed represents the random number seed; h * represents a certain hyperparameter or hyperparameter combination for this mutation, h * ∈H * ; H * represents the set of all hyperparameters or hyperparameter combinations participating in the evaluation, where P(H) is the power set of the hyperparameter set H; when all possible hyperparameters or hyperparameter combinations are selected, then H * = P(H), z i represents the hyperparameter space point obtained by calling the mutation operator O.

[0018] According to the above technical solution, call the mutation operator O to obtain the hyperparameter point set Z, Z = (z1, z2, …, z i ) and According to the coordinates of each hyperparameter space point z i in the hyperparameter point set, change the hyperparameter values in the original model, so as to obtain the mutated model; traverse all hyperparameter space points in the hyperparameter point set Z and set the model hyperparameters to obtain a set M of mutated models composed of several mutated models.

[0019] The hyperparameter point set Z is a subset of the acceptable hyperparameter range set D generated by using the mutation operator O, that is, a sample sampled from the hyperparameter range set D.

[0020] The mutated model is a model obtained by setting the model hyperparameters with the coordinates of each hyperparameter space point z i = (x i1 , x i2 , …, x in ) in the hyperparameter point set obtained by the mutation operator, and the hyperparameters that are not mutated are the same as those of the initial model.

[0021] For any model m i in the set M of mutated models, the point coordinates of the hyperparameter combination in the hyperparameter space are zi All comply with z i ∈D.

[0022] According to the above technical solution, traverse and run all mutation models in the mutation model set M, and measure the energy consumption metric when running the mutation model; measure the performance metric when the mutation model is running or ended; after the traversal ends, obtain the metric set E of the energy consumption and the performance metric set P of all models in the mutation model set M.

[0023] Use tools such as Perf and Nvidia-smi to measure the energy consumption metric. The energy consumption metric set E = (e1, e2,..., e k ) is the running energy consumption of each mutation model in the mutation model set M, that is, the energy consumption metric e i is measured when the model m i is running.

[0024] The performance metric set P = (p1, p2,..., p k ) is the quantitative performance metric of each mutation model in the mutation model set M.

[0025] According to the above technical solution, construct a causal graph with each hyperparameter in the hyperparameter point set Z, the energy consumption metric set E, and the performance metric set P, and perform causal inference on the causal graph to obtain the causal inference metric Implement the causal analysis; each hyperparameter in the hyperparameter point set Z refers to the set of values of each hyperparameter in the point set Z;

[0026] The points in the causal graph are a certain hyperparameter or hyperparameter group h input by a certain mutation operator O * , the energy consumption metric E, or the performance metric P;

[0027] Among them, the model energy consumption metric set E and the performance metric set P are collectively called the target metric, and the causal inference metric refers to the expected impact on the target metric Y of whether to mutate the hyperparameter or hyperparameter group h * .

[0028] According to the above technical solution, traverse the hyperparameter or hyperparameter group set H * , calculate the optimization metric according to the causal inference metric and evaluate the trade-off relationship under each hyperparameter or hyperparameter group h ; * The trade-off relationship under each hyperparameter or hyperparameter group h

[0029] Determine the target set T and the side effect set according to the trade-off relationship under each hyperparameter or hyperparameter group h * ;

[0030] Take the target set T and the side effect set As a set of hyperparameters that can be changed, construct a new neural network model;

[0031] Among them, the hyperparameter or hyperparameter group h where there is no trade-off relationship between the energy consumption metric E and the performance metric P * Is the target set T that will not bring performance side effects when optimizing energy consumption; when the target set T may not be found, by sorting the performance optimization amount To obtain the side effect set of the hyperparameter or hyperparameter combination with the smallest l before performance loss Set Is the second choice for optimizing hyperparameters or hyperparameter groups that will only bring acceptable performance loss.

[0032] Among them, the trade-off relationship is to adjust a certain hyperparameter or hyperparameter group h * When, whether optimizing one of the energy consumption metric E and the performance metric P will cause the loss of the other. Through such a relationship, the impact on the model performance when adjusting a certain hyperparameter or hyperparameter group h * Can be quantified. When adjusting a certain hyperparameter or hyperparameter group h * When, if the optimization metrics of the model energy consumption metric E and the performance metric P Have different signs, there is a trade-off relationship.

[0033] According to the above technical solution, the optimization metric Is the metric obtained by changing the sign of some causal inference metrics The reason for proposing the optimization metric is that the optimization directions of the model energy consumption metric E and the performance metric P are different (the smaller the energy consumption metric E, the better, while the larger the performance metric P, the better), and the optimization metric

[0034]

[0035] When the target metric Y represents the energy consumption metric E, the optimization metric Takes the negative value of the causal analysis metric ; when the target metric Y represents the energy consumption metric P, the optimization metric Takes the positive value of the causal analysis metric 。

[0036] According to the above technical solution, the performance optimization amount Is the performance loss brought by reducing the unit energy consumption of the metric hyperparameter or hyperparameter group h * ∈H * -T, and the performance optimization amount

[0037]

[0038] In the formula, represents the causal inference metric corresponding to the energy consumption metric set E, represents the causal inference metric corresponding to the performance metric set P. According to the sorting result of the performance optimization quantity , the side effect set of the l smallest hyperparameters or hyperparameter combinations before performance loss can be selected

[0039] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: The present invention explores the hyperparameter space of the neural network model through the mutation operator, and uses causal analysis to obtain the trade-off relationship between the model performance and energy consumption, so as to recommend low-energy-consuming hyperparameters. The present invention extracts modifiable hyperparameters, performance, and energy consumption metrics from the model to be tested according to specific requirements; designs the hyperparameter mutation range according to the development requirements and designs the mutation operator according to the mutation range; introduces the mutation operator into the original model to obtain the model with hyperparameter mutation; runs the mutated model to collect performance and energy consumption metrics; performs causal analysis on the measured performance and energy consumption metrics; evaluates the trade-off relationship between energy consumption and performance according to the results of causal analysis and recommends hyperparameters. The present invention solves which hyperparameters are more suitable for being used in the method of optimizing hyperparameters to reduce the energy consumption of the model; and provides a method framework for exploring the trade-off relationship between performance and energy consumption for development scenarios concerned about energy consumption, reducing the performance risk of the method of optimizing hyperparameters. At the same time, it solves the problem that neural network model developers will face the risk of performance loss when using optimized hyperparameters to reduce the energy consumption of the model, reduces the difficulty for developers to build energy-saving and environment-friendly neural network models, and thus promotes the development of green and environment-friendly neural network models. Description of the Drawings

[0040] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings:

[0041] Figure 1 is the overall architecture diagram of the method for recommending low-energy-consuming hyperparameters of the neural network model driven by causal analysis of the present invention;

[0042] Figure 2 is the step flow chart of the method for recommending low-energy-consuming hyperparameters of the neural network model driven by causal analysis of the present invention;

[0043] Figure 3 is an example of a causal diagram that may appear in a causal analysis in the embodiment. Detailed Embodiment

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0045] The overall architecture diagram of the method for recommending low-energy consumption hyperparameters of the neural network model driven by causal analysis of the present invention is as Figure 1 shown. That is, first, determine the hyperparameter set H of the model to be tested, the performance indicators concerned by the development requirements, and the measurement of hardware energy consumption according to the development requirements; determine the acceptable hyperparameter range D in the hyperparameter space according to requirements, expectations or experimental attempts, and design a mutation operator O to randomly mutate hyperparameters within this hyperparameter range; call the mutation operator and obtain several new hyperparameter values different from the initial model, and set the model with the new hyperparameter values to obtain a set of mutated models M; run the mutated models respectively, and call tools such as hardware drivers during the running to collect the measurement E of the total hardware energy consumption concerned, and obtain the performance measurement P through performance testing; construct a causal graph for hyperparameter changes, performance and energy consumption measurements, and calculate the impact of changing hyperparameters on performance and energy consumption measurements, that is, the causal analysis measurement; provide hyperparameters or groups of hyperparameters that will not damage performance according to the causal analysis measurement, or hyperparameters or groups of hyperparameters that will only damage performance within an acceptable range.

[0046] The specific process of the present invention is summarized into 6 steps ( Figure 2 ), specifically including:

[0047] Step 1: Determine the hyperparameters, performance and energy consumption measurements to be tested in the model. Specifically, select the changeable hyperparameter set H=(h1, h2,..., h n ) according to the development requirements, and construct a hyperparameter space according to the selected hyperparameter set. Each point coordinate z=(x1, x2,..., x n ) in this space represents a set of hyperparameter values, that is, the point z i represents the point where the i-th hyperparameter h i takes the value of x i . It is also necessary to determine the quantifiable model performance indicators required by the requirements, such as the accuracy rate on the test set, etc.; and the measurement of the total energy consumption of the concerned hardware. For example, if the energy consumption of both the CPU and GPU is concerned at the same time, the energy consumption of the two can be extracted by tools and the sum of the two can be used to represent the total energy consumption.

[0048] Step 2: Determine the hyperparameter mutation range with limited impact on performance and determine the mutation operator. Specifically, further define the set D of acceptable hyperparameter ranges in the hyperparameter space. The set D of hyperparameter ranges is a set of points in the acceptable hyperparameter space, so for any point z ∈ D, it is an acceptable hyperparameter point. The mutation operator O is a mapping from the Cartesian product of the real number field R and the set H * of all hyperparameters or hyperparameter combinations involved in the evaluation:

[0049] O: R×H * →D

[0050] z i =O(seed, h * )

[0051] This mutation operator takes as input a random number seed seed ∈ R and a certain hyperparameter or hyperparameter combination h * to be mutated, and outputs a point z in the set D of hyperparameter ranges i and z i ∈ D. Where a certain hyperparameter or hyperparameter combination h * ∈ H * , the hyperparameters or hyperparameter combinations involved in the evaluation Here, P(H) is the power set of the hyperparameter set H; when all possible hyperparameters or hyperparameter combinations are selected, H * =P(H).

[0052] The specific implementation of this mutation operator can be achieved by taking random values for each hyperparameter in the input hyperparameter combination. Through this mutation operator, several mutated hyperparameter points allowed by the development requirements can be obtained.

[0053] Step 3: Introduce the mutation operator to the original model to mutate the hyperparameters to obtain the mutated model. Specifically, by running the mutation operator O multiple times, a set Z of hyperparameter points containing multiple points belonging to the set D can be obtained and set the model hyperparameters with the coordinates of each hyperparameter space point z i in the point set Z obtained by mutation, and the remaining hyperparameters are the same as those of the initial model, to obtain the mutated model set M. For any model m i in the mutated model set M, the point coordinates z i of its hyperparameter combination in the hyperparameter space all satisfy z i ∈ D.

[0054] Step 4: Run the mutated model to measure the performance and energy consumption metrics. Specifically, when running the mutated model m i , call the tools provided by the system or hardware manufacturer to collect the previously determined energy consumption metrics, so as to obtain the energy consumption metric e i of the model mi According to different determined performance metrics, during or at the end of the model operation, the performance metric p of the model m can be obtained through means such as testing. i of the model m i By repeating the above steps for each model in the mutant model set M, a metric set E of the total hardware energy consumption and a metric set P of the performance can be obtained for each model.

[0055] Step 5: Perform causal analysis on the energy consumption and performance metrics. Based on the measured energy consumption metric set E and performance metric set P, and each hyperparameter or hyperparameter group involved in the mutation in the hyperparameter coordinate point set Z corresponding to the mutant model, a causal graph can be constructed for causal analysis. The points in the causal graph are a certain hyperparameter or hyperparameter group h input by a certain mutation operator O * , the energy consumption metric E or the performance metric P. Among them, an example of the causal graph that may appear in the causal analysis is as Figure 3 shown.

[0056] The method for constructing the causal graph can select an existing causal graph construction method according to requirements or convenience. By constructing the causal graph, it can be known the causal relationship between a certain hyperparameter or hyperparameter group h * in the hyperparameter range set D and the energy consumption metric E and the performance metric P, that is, how the hyperparameter or hyperparameter group H * affects the model energy consumption E and performance P in the hyperparameter space.

[0057] Based on the causal graph, further perform causal inference to obtain the causal inference metric Here, the model energy consumption metric set E and the performance metric set P are collectively referred to as the target metric Y. The causal inference metric refers to the expected impact on the target metric Y when mutating the hyperparameter or hyperparameter group H * All common causal inference metrics can be used as causal analysis metrics For the convenience of explanation, the average treatment effect (ATE) is used as this causal inference metric. The expression of the average treatment effect ATE is as follows,

[0058] ATE = E[Y|do(t = 1)] - E[Y|do(t = 0)]

[0059] The average treatment effect ATE represents the difference between the result means Y when the influencing factor t is assumed to be 0 or 1 (here the influencing factor t has been normalized). Among them, the hyperparameter or hyperparameter group H *As the influencing factor t, while the model energy consumption E and performance P are used as the results Y. E[.] in the formula represents the mean; and do(.) represents the assumed intervention on the influencing factor t, that is, the influencing factor t does not need to have data of 0 or 1 in the data.

[0060] Among them, the influencing factor t, that is, a certain hyperparameter or hyperparameter group h * The normalization of can be achieved by projecting the range of each hyperparameter h i ∈h * in the acceptable hyperparameter space onto a line segment of unit length, where the projection is f:D×H * →[0,1]:

[0061] r = f(z,h * )

[0062] This projection inputs a coordinate point z∈D and the hyperparameter or hyperparameter combination h * ∈H * , and outputs a value r∈[0,1] between 0 and 1. Through this projection, the present invention can project the hyperparameter or hyperparameter group h * onto the line segment of [0,1], so as to bring the influencing factor t into the expression of the average treatment effect ATE.

[0063] By using the causal analysis metric The causal impact of the hyperparameter or hyperparameter group set H * on the model energy consumption E and performance P can be quantified. Specifically, when a certain causal analysis metric is positive, it indicates that this hyperparameter or hyperparameter group h * has a positive effect on the target metric Y, that is, when the projection value of the hyperparameter or hyperparameter group h * increases from 0 to 1, the target metric Y will increase as the result of the hyperparameter or hyperparameter group h * ; vice versa when the causal analysis metric is negative. And the magnitude of the causal analysis metric indicates that the change value of the target metric Y as the result of the hyperparameter or hyperparameter group h * is greater.

[0064] Step 6: Evaluate the trade-off relationship between the specific energy consumption metric E and performance metric P of the model according to the causal analysis, and recommend hyperparameters or hyperparameter groups that can be changed. The causal inference metric obtained by causal inference can illustrate the causal relationship between the hyperparameter or hyperparameter group set H * and the model energy consumption metric E and performance metric P, so as to more comprehensively model and optimize a certain hyperparameter or hyperparameter group h *The impact of the

[0065] scheme on the model energy consumption metric E and performance metric P. * Among them, the trade-off relationship means that when the developer optimizes a certain hyperparameter or hyperparameter group h * , whether there will be a situation where optimizing one of the model energy consumption metric E and performance metric P will cause the other to decline. Specifically, when the changes in the energy consumption metric E and performance metric P cannot be * optimized simultaneously or suffer losses by adjusting a certain hyperparameter or hyperparameter group h * , a trade-off relationship will occur. For example, if adjusting a certain hyperparameter or hyperparameter group h

[0066] reduces the energy consumption metric E but damages the performance metric P, it means that there is a trade-off relationship between energy consumption and accuracy. Understanding this trade-off relationship can evaluate whether the reduction of energy consumption E will have an adverse impact on the model performance P when optimizing a certain hyperparameter or hyperparameter group h Calculating the optimization metric The formula is expressed as:

[0067]

[0068] When the metric Y represents the energy consumption metric E, the optimization metric takes the negative value of the causal analysis metric ; when the metric Y represents the performance metric P, the optimization metric takes the positive value of the causal analysis metric .

[0069] When the optimization metric is positive, it means that when the projection value of the hyperparameter or hyperparameter group h * increases from 0 to 1, the metric Y is optimized by this change, regardless of whether the metric Y is the energy consumption metric E or the performance metric P. Based on the optimization metric it can be known whether there is a trade-off relationship between the model energy consumption metric E and the performance metric P when adjusting a certain hyperparameter or hyperparameter group h * . Specifically, when adjusting a certain hyperparameter or hyperparameter group h * , if the optimization metrics of the model energy consumption metric E and the performance metric P have different signs, there is a trade-off relationship.

[0070] To understand the impact of all allowed hyperparameter adjustments, we can traverse the set H * of hyperparameters or hyperparameter groups, calculate and check any hyperparameter or hyperparameter group h *For the optimization metric between the model energy consumption metric E and the performance metric P If in this process, a certain hyperparameter or hyperparameter group h is found where there is no trade-off relationship between the energy consumption metric E and the performance metric P * , it means that adjusting the hyperparameter or hyperparameter group h * to optimize energy consumption will not bring performance side effects; these hyperparameters or hyperparameter groups h * that do not bring trade-off relationships form a set named the set T of hyperparameters or hyperparameter groups without side effects. The set T without side effects is the set of hyperparameters or hyperparameter combinations that can be used more safely for the method of optimizing energy consumption with hyperparameters.

[0071] Given that the set T without side effects may not be found or may not exist in some cases and a small reduction in performance may be acceptable, we can recommend hyperparameters or hyperparameter groups h * in the remaining set of hyperparameters or hyperparameter groups H * -T that bring a small performance impact * ∈H * -T. To compare any different hyperparameters or hyperparameter groups in the set H -T, define the performance optimization amount obtained per unit energy consumption cost

[0072]

[0073] In the formula, represents the causal inference metric corresponding to the energy consumption metric set E, represents the causal inference metric corresponding to the performance metric set P.

[0074] Since adjusting the hyperparameter or hyperparameter group here conforms to h * ∈H * -T, the causal analysis metrics of the energy consumption metric E and the performance metric P in the expression are of the same sign, otherwise there would be no trade-off relationship between the two, that is, the hyperparameter or hyperparameter group h * ∈T. Therefore, the performance optimization amount obtained per unit energy consumption cost is a non-negative value. Then, the developer should traverse the remaining set of hyperparameters or hyperparameter groups H * -T, calculate and sort the performance optimization amount obtained per unit energy consumption cost to obtain the ranking of the hyperparameters or hyperparameter groups in the remaining set of hyperparameters or hyperparameter groups H * -T for the performance optimization amount . In this ranking, the first l hyperparameters or hyperparameter groups h * can be selected according to the requirements to form a side effect set that brings an acceptable performance loss As the second choice after the set T without side effects.

[0075] Finally, the set T without side effects and the set of side effects Is recommended as a method for optimizing hyperparameters. Based on the recommended hyperparameters, a more environmentally friendly neural network model can be constructed with lower performance risks.

[0076] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0077] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A causal analysis-driven method for recommending low-energy hyperparameters for neural network models, comprising the following steps: Determine the hyperparameters, performance metrics, and energy consumption metrics to be tested in the model, and select the set of hyperparameters that can be debugged in the model as basis vectors to construct a hyperparameter vector space; In the hyperparameter space, a hyperparameter range set D is selected based on development requirements. Based on the hyperparameter range set D, a hyperparameter variation range with limited impact on model performance is determined, and a mutation operator is determined. Introduce a mutation operator into the original model to mutate the hyperparameters to obtain a mutant model; Run the variant model to measure performance metrics and energy consumption metrics; Perform causal analysis on energy consumption metrics and performance metrics, evaluate the trade-off between the model's energy consumption and performance metrics based on the causal analysis, and recommend a set of hyperparameters that can be changed.

2. The method for recommending low-energy consumption hyperparameters of a neural network model driven by causal analysis according to claim 1, wherein, The mutation operator O: wherein, R represents a real number, O represents a mutation operator, seed represents a random number seed; h * represents a certain hyperparameter or hyperparameter combination of this mutation, h * ∈H * ; H * represents the set of all hyperparameters or hyperparameter combinations participating in the evaluation, where P(H) is the power set of the hyperparameter set H; when all possible hyperparameters or hyperparameter combinations are selected, then H * = P(H), z i represents the mutated hyperparameter space point obtained by calling the mutation operator O.

3. The causal analysis-driven low-energy consumption hyperparameter recommendation method for a neural network model according to claim 1, characterized in that: Invoking the mutation operator O to obtain the hyperparameter point set Z, Z = (z1, z2, …, z i ), and Changing the hyperparameter values in the original model according to the coordinates of each hyperparameter space point z in the hyperparameter point set Z i to obtain the mutated model; traversing all hyperparameter space points in the hyperparameter point set Z and setting the model hyperparameters to obtain a mutated model set M composed of several mutated models.

4. The method for recommending low-energy consumption hyperparameters of a neural network model driven by causal analysis according to claim 1, characterized in that All variant models in the variant model set M are traversed and run, and energy consumption metrics are measured when the variant models are run; performance metrics are measured when the variant models are running or at the end; after the traversal is completed, the energy consumption metric set E and performance metric set P of all models in the variant model set M are obtained.

5. The method for recommending low-energy consumption hyperparameters of a neural network model driven by causal analysis according to claim 1, wherein Construct each hyperparameter in the hyperparameter point set Z, the energy consumption measurement set E and the performance measurement set P to construct a causal graph, and perform causal inference on the causal graph to obtain the causal inference measurement Implementing the causal analysis; The points in the causal graph are hyperparameters or hyperparameter groups h input by a mutation operator O. * , energy consumption metric E or performance metric P; Among them, the model energy consumption metric set E and the performance metric set P are collectively referred to as target metrics, and the causal inference metric refers to whether to mutate the hyperparameter or hyperparameter group h * for the expected impact on the target metric Y.

6. The causal analysis-driven low-energy consumption hyperparameter recommendation method for a neural network model according to claim 5, characterized in that: Traverse the set H of hyperparameters or hyperparameter groups * , according to the causal inference metric Calculate the optimization metric and evaluate the trade-off relationships under each hyperparameter or hyperparameter group h * ; Determine the target set T and the side effect set according to the trade-off relationship under each hyperparameter or hyperparameter group h * ​ Take the target set T and the side effect set As a set of hyperparameters that can be changed, build a new neural network model; Among them, the hyperparameter or hyperparameter group h for which there is no trade-off relationship between the energy consumption amount E and the performance metric P * is the target set T that will not cause performance side effects when optimizing energy consumption; when the target set T may not be found, by sorting the performance optimization amount to obtain the side effect set of the hyperparameters or hyperparameter combinations with the l smallest performance losses before 7. The causal analysis driven low energy consumption hyperparameter recommendation method for a neural network model according to claim 6, characterized in that: The optimization metric When the target metric Y represents the energy consumption metric E, the optimization metric Take causal analysis metrics Negative value; when the target metric Y represents the energy consumption metric P, the optimization metric Take causal analysis metrics Positive value.

8. The method for recommending low - energy consumption hyperparameters of a neural network model driven by causal analysis according to claim 1, wherein, The performance optimization amount Where, represents the causal inference metric corresponding to the energy consumption metric set E, represents the causal inference metric corresponding to the performance metric set P.

Images