Terminal power consumption management system and method based on DVFS automatic cross-layer

Abstract

A terminal power consumption management system and method based on automatic cross-layer DVFS includes a CPM engine and a CPM controller, wherein the CPM engine searches for the optimal DVFS configuration for the currently running DNN model, and the CPM controller controls the real-time operation of the DNN model in the AIoT system. The present invention models the DVFS overhead and optimizes the search space using techniques such as merging network layers and automatic encoding and decoding, thereby obtaining the optimal DVFS configuration while taking DVFS overhead into account, thereby controlling the dynamic power consumption of the DNN model during operation. The present invention can significantly reduce the energy consumption of DNN model calculations and improve computing efficiency, which is of great significance for promoting the application deployment and implementation of DNN models and the development of smart IoT.

Description

Technical Field

[0001] The present invention relates to a technology in the field of power consumption management of Internet devices, and in particular to a terminal power consumption management (CPM) system and method based on dynamic voltage frequency scaling (DVFS) and automatic cross-layer. Background Art

[0002] Existing artificial intelligence (AI)-based Internet of Things (AIoT) power management technology uses layer-by-layer dynamic voltage-frequency scaling (DVFS). This technology sets the frequency of each DNN layer to the optimal frequency for that layer, thereby maximizing energy efficiency. However, current technology is overly idealistic, overlooking the energy and latency overhead of DVFS adjustments. These overheads are often greater than the execution overhead of the network layer of most DNN models, rendering the current layer-by-layer management approach ineffective. Therefore, a cross-layer power management method based on DVFS is being implemented. Summary of the Invention

[0003] In response to the shortcomings of existing technologies that cannot obtain the globally optimal target decision frequency and cannot maximize the energy efficiency of edge device neural network inference, the present invention proposes a terminal power consumption management system and method based on DVFS automatic cross-layer. Cross-layer power consumption management of neural network calculations on the artificial intelligence Internet of Things (AIoT) system is performed to improve the efficiency of the real-time AIoT system. By modeling the DVFS overhead and using technologies such as merging network layers and automatic encoding and decoding to optimize the search space, the optimal DVFS configuration is obtained while considering the DVFS overhead to control the dynamic power consumption of the DNN model during operation. The present invention can significantly reduce the energy consumption of DNN model calculations and improve computing efficiency, which is of great significance to promoting the application deployment and implementation of DNN models and the development of smart Internet of Things.

[0004] The present invention is achieved through the following technical solutions:

[0005] The present invention relates to a power consumption management system based on automatic cross-layer DVFS, comprising: a CPM engine and a CPM controller, wherein: the CPM engine searches for the optimal DVFS configuration of the currently running DNN model, and the CPM controller controls the real-time operation process of the DNN model in the AIoT system.

[0006] The CPM engine includes: a modeler, a merger, an optimizer and a decision maker, wherein: the modeler models according to the DVFS overhead and the delay and energy consumption of each layer during the operation of the DNN model; the merger merges the network of the DNN model to form a network block; the optimizer projects the discrete contraction space into the latent space; and the decision maker uses the Bayesian optimization algorithm to search for the optimal DVFS configuration.

[0007] The CPM controller includes: an operation monitor, a frequency distributor, and a load controller. The operation monitor collects power consumption and latency data during operation and outputs the information to the CPM engine. The frequency distributor obtains the DVFS configuration from the CPM engine and sets the operating frequency of each layer in real time. The load controller controls the execution of the DNN model.

[0008] The present invention relates to a terminal power consumption management method based on automatic cross-layer DVFS based on the above-mentioned system. When the DNN model is executed layer by layer on the AIoT system, the CPM controller monitors the execution status of each layer and collects the execution delay and energy consumption overhead data required for each layer of the network when executing on the current AIoT system and outputs it to the CPM engine. The CPM engine calculates the frequency required for the execution of the next layer of the DNN network based on the execution delay and energy consumption overhead data and feeds back to the CPM controller. The CPM controller sets the frequency of the GPU through Devfreq according to the obtained frequency, and the AIoT system continues to execute the next layer of the DNN model.

[0009] Technical Effects

[0010] Compared with the existing technology, the present invention introduces the modeling of DVFS overhead on the basis of layer-by-layer DVFS tuning of the DNN model, and solves the model by merging network layers, automatic encoding and decoding network coding search space, and Bayesian optimization algorithm search space, and finally obtains the optimal DVFS configuration in a relatively short time. While ensuring the computing performance of the DNN model, the present invention further considers the DVFS overhead to obtain a practical and feasible optimization method. Ultimately, the present invention balances the computational cost and performance of the DNN model on the basis of energy conservation, and provides a more advanced solution for the deployment of DNN model computing in edge systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 Schematic diagram of the system of the present invention;

[0012] Figure 2 Flowchart of the present invention;

[0013] Figure 3 and Figure 4 Schematic diagram of the embodiment effect. DETAILED DESCRIPTION

[0014] like Figure 1 As shown, this example relates to a DVFS-based automatic cross-layer power consumption management system, including: a CPM engine and a CPM controller, wherein: the CPM engine includes: a modeler, a merger, an optimizer and a decision maker; the CPM controller includes: an operation monitor, a frequency allocator and a load controller.

[0015] The modeler adds the energy cost of DVFS t when modeling the energy consumption and delay of the DNN model during operation. dvfs With delay overhead e dvfs After the DNN model finishes executing the current layer, before executing the next layer, if the frequency required by the next layer is different from that of the current layer, DVFS must be used to adjust the frequency to the next layer before continuing execution, resulting in overhead due to DVFS switching. After adding DVFS overhead, the latency and energy consumption of the entire DNN model are expressed as: Where: T represents the delay of the entire model, E represents the energy consumption of the entire model; n represents the number of layers of the DNN model, l i represents the i-th layer; represents the frequency of the i-th layer setting; and Respectively represent the i-th layer Delay and energy consumption at frequency; t dvfs and e dvfs Respectively represent the delay and energy consumption of adjusting DVFS; c i Indicates whether the frequency change is sent from layer i-1 to layer i, that is, if and Equal, c i is equal to 0, otherwise c i Equal to 1. The goal of the modeler is to set the corresponding frequency for any i-th layer Make the final T and E as small as possible.

[0016] The merger merges the network layers of the DNN model into network blocks to reduce the search space. Specifically, based on the delay overhead of DVFS switching, the neural network layers that are smaller than this benchmark are merged to form network blocks, thereby reducing the search space while ensuring that DVFS switching can bring benefits. Specifically, D = {l1, l2, ..., l n}={b1,b2,...,b m}, each network block Where: D represents the DNN network model, {l1, l2, ..., l n} represents the network before merging, with a total of n layers, {b1, b2, ..., b m} represents the merged network, which has a total of m network blocks; represents the delay of layer j; each network block b i It includes at least one network layer. When it is a single network layer, the delay of this layer is greater than the delay of the DVFS switching frequency; when it is multiple consecutive network layers, the delay of each network layer is less than the delay of the DVFS switching frequency.

[0017] The optimizer projects the discrete space into a latent space to further optimize the search space, and uses a variational autoencoder / decoder network to map the discrete search space into a latent space, where searching can find results faster.

[0018] The decision maker uses a Bayesian optimization algorithm to perform a multi-objective search in the search space to obtain the optimal DVFS configuration.

[0019] The runtime monitor collects power consumption and latency data during DNN model execution and outputs this information to the CPM engine's modeler. In actual implementation, the system's power monitor is used to collect power consumption, and the Python time library is used to measure execution latency.

[0020] The frequency allocator obtains the DVFS configuration from the CPM engine's decision maker and sets the operating frequency of each layer in real time. The GPU frequency is set and modified through the Devfreq device frequency management framework under the Linux system.

[0021] The load controller controls the execution process of the DNN model by obtaining corresponding information from the operation monitor and the frequency distributor.

[0022] like Figure 2 As shown, this embodiment involves the terminal power consumption management method based on DVFS automatic cross-layer of the above-mentioned system. When the DNN model is executed layer by layer on the AIoT system, the CPM controller monitors the execution status of each layer and collects the execution delay and energy consumption overhead of the layer and outputs it to the CPM engine. The CPM engine calculates the frequency required for the execution of the next layer of the DNN network based on the received execution delay and energy consumption overhead of each layer and feeds back to the CPM controller. The CPM controller sets the frequency of the GPU through Devfreq according to the obtained frequency and the AIoT system continues to execute the next layer of the DNN model.

[0023] This example was validated by embedding this technology on a real edge computing platform and deploying multiple DNN model applications. The Nvidia Jeston Xavier NX was used as the edge computing platform, whose GPU has 15 selectable frequency settings. The system's power monitor was used to collect power consumption, the Python time library was used to measure execution latency, and the Devfreq device frequency management framework in Linux was used to set the GPU frequency. Five different types of DNN models were used as test applications, including AlexNet, GoogleNet, VGGNet, Yolo, and ResNet. AlexNet is a classic image classification model with 21 layers; GoogleNet, which uses the Inception module as a component, has 26 layers. VGGNet, available in two versions, VGG16 and VGG19, uses the 39-layer VGG16; Yolo for object detection has 73 layers; and ResNet uses a residual architecture; the ResNet50 used here has 155 layers. All models were implemented in PyTorch.

[0024] After specific experimental verification, the present invention achieves a significant improvement in energy consumption and execution time, as shown in Table 1. Figure 3 and Figure 4 Figure 2 shows a comparison of DNN computation and DVFS overhead in terms of energy consumption and latency for different power management schemes on Nvidia Jeston Xavier NX edge devices.

[0025] Table 1

[0026]

[0027] The eight technical solutions in Table 1 are respectively represented as follows: (a) Profile-L selects the frequency with the minimum delay for each layer; (b) Profile-E selects the frequency with the minimum energy for each layer; (c) NeuOS uses the LAG algorithm to control the frequency of each layer to meet the latency requirements; (d) Bayesian is a search method that uses the Bayesian optimization algorithm for spatial search; (e) NSGAII is a multi-objective optimization search algorithm based on the genetic algorithm; (f) Sampling is a search method based on random sampling; (g) TPE is a black box optimization search algorithm based on a tree structure; (h) CPM is the automatic power consumption management technology based on DVFS in this embodiment.

[0028] Compared with NeuOS, the best among the other seven solutions, the present invention not only reduces the inference delay by 14.66% to 45.76%, but also reduces energy consumption by 5.71% to 31.58%. At the same time, although Profile-L and Profile-E obtain the lowest computational cost, they cause higher DVFS overhead due to frequent DVFS switching. The other four automatic search methods (Bayesian, NSGAII, Sampling, TPE) also failed to find a good frequency configuration due to the large search space. Although NeuOS has a lower DVFS overhead, the computational cost is relatively large. In summary, the present invention not only obtains a lower computational cost, but also does not cause a higher DVFS overhead, achieving a balance between overhead and computation, thereby obtaining the optimal result.

[0029] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principles and purpose of the present invention. The scope of protection of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. All implementation schemes within its scope shall be subject to the constraints of the present invention.

Claims

1. A DVFS-based automatic cross-layer power consumption management system, characterized in that: include: The CPM engine and CPM controller, where the CPM engine searches for the optimal DVFS configuration for the currently running DNN model, and the CPM controller controls the real-time running process of the DNN model in the AIoT system; The CPM engine includes: a modeler, a merger, an optimizer, and a decision maker, wherein: the modeler models based on the DVFS overhead and the latency and energy consumption of each layer during the DNN model runtime; the merger merges the networks of the DNN model to form network blocks; the optimizer projects the discrete contraction space into the latent space; and the decision maker uses a Bayesian optimization algorithm to search for the optimal DVFS configuration; The CPM controller includes: an operation monitor, a frequency distributor, and a load controller. The operation monitor collects power consumption and latency data during operation and outputs the information to the CPM engine. The frequency distributor obtains the DVFS configuration from the CPM engine and sets the operating frequency of each layer in real time. The load controller controls the execution of the DNN model. The modeler models the energy consumption and latency of the DNN model during runtime as follows: Where: T represents the delay of the entire model, E represents the energy consumption of the entire model; n represents the number of layers of the DNN model, l i represents the i-th layer; represents the frequency of the i-th layer setting; and Respectively represent the i-th layer Delay and energy consumption at frequency; t dvfs and e dvfs Respectively represent the delay and energy consumption of adjusting DVFS; c i Indicates whether the frequency change is sent from layer i-1 to layer i, that is, if and Equal, c i is equal to 0, otherwise c i Equal to 1, the goal of the modeler is to set the corresponding frequency for any i-th layer Make the final T and E as small as possible.

2. The DVFS automatic cross-layer power consumption management system according to claim 1 is characterized in that: The merging means: taking the delay overhead of DVFS switching as the benchmark, merging the neural network layers that are smaller than this benchmark to form a network block, specifically: D = {l1, l2, ..., l n }={b1,b2,…,b m }, each network block Where: D represents the DNN network model, {l1,l2,…,l n } represents the network before merging, with a total of n layers, {b1,b2,…,b m } represents the merged network, which has m network blocks in total; t lj represents the delay of layer j; each network block b i It includes at least one network layer. When it is a single network layer, the delay of this layer is greater than the delay of the DVFS switching frequency; when it is multiple consecutive network layers, the delay of each network layer is less than the delay of the DVFS switching frequency.

3. The DVFS automatic cross-layer power consumption management system according to claim 1 is characterized in that: The optimizer projects the discrete space into a latent space to further optimize the search space, and uses a variational autoencoder / decoder network to map the discrete search space into the latent space, on which the search is performed to find the result faster.

4. The DVFS automatic cross-layer power consumption management system according to claim 1 is characterized in that: The decision maker uses a Bayesian optimization algorithm to perform a multi-objective search in the search space to obtain the optimal DVFS configuration.

5. The DVFS automatic cross-layer power consumption management system according to claim 1 is characterized in that: The operation monitor uses the system's power monitor to collect power consumption, uses the Python time library to measure execution delay, and then outputs the collected results to the modeler of the CPM engine.

6. The DVFS automatic cross-layer power consumption management system according to claim 1 is characterized in that: The frequency allocator obtains the DVFS configuration from the decision maker of the CPM engine and sets the operating frequency of each layer in real time. The frequency of the GPU is set and modified through the Devfreq device frequency management framework under the Linux system.

7. The DVFS automatic cross-layer power consumption management system according to claim 1 is characterized in that: The load controller controls the execution process of the DNN model by obtaining corresponding information from the operation monitor and the frequency distributor.

8. A method for automatic cross-layer terminal power consumption management based on DVFS based on the system according to any one of claims 1 to 7, characterized in that: When the DNN model is executed layer by layer on the AIoT system, the CPM controller monitors the execution status of each layer and collects the execution delay and energy consumption overhead data required for each layer of the network to be executed on the current AIoT system and outputs it to the CPM engine. The CPM engine calculates the frequency required for the execution of the next layer of the DNN network based on the execution delay and energy consumption overhead data and feeds back to the CPM controller. The CPM controller sets the GPU frequency through Devfreq based on the obtained frequency and the AIoT system continues to execute the next layer of the DNN model.

Images