Adaptive operator parallel segmentation method for heterogeneous embedded chip of intelligent Internet of Things

Abstract

The invention discloses a self-adaptive operator parallel segmentation method for an intelligent Internet of Things heterogeneous embedded chip, and the method comprises the steps: firstly carrying out the energy consumption prediction based on an energy consumption prediction model of a random forest regression algorithm; according to various hardware environments and task features, an energy consumption data set is constructed, feature selection is performed, and real-time monitoring and accurate energy consumption prediction of equipment states and task requirements are realized. Secondly, an operator adaptive division method based on a dynamic programming algorithm is used for efficient parallel reasoning of the deep neural network on a multi-heterogeneous processor; on the basis of energy consumption prediction, the dependency relationship between operators, the cross-processor data communication overhead and the dynamic resource condition of the processor are comprehensively considered, and it is ensured that the system achieves the optimal energy efficiency while meeting the performance requirement. Through the method, the energy efficiency and the response speed of deep neural network reasoning on the mobile equipment are remarkably improved, and the problem that a static division method is difficult to adapt to a dynamic environment in the prior art is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet of Things, and in particular relates to an adaptive operator parallel partitioning method for heterogeneous embedded chips of intelligent Internet of Things. Background Art

[0002] With the development of science and technology and the rapid development of social economy, automation and intelligent technology have become an inevitable trend in the development of modern society. Smart IoT devices, especially mobile devices such as smartphones, have been widely popularized around the world and gradually penetrated into all aspects of daily life. While providing rich functions and applications, these devices also put forward higher requirements on their internal computing resources and energy efficiency management.

[0003] The deployment and application of deep neural networks (DNNs) on mobile devices has become a hot topic of research. DNNs have demonstrated powerful capabilities in many fields such as image recognition, speech recognition, natural language processing, and augmented reality. However, due to the high computational complexity of DNNs and the large amount of computing resources required, this poses a huge challenge to the energy efficiency management of battery-powered mobile devices.

[0004] At present, when implementing energy-efficient parallel DNN inference, the existing static partitioning schemes are difficult to adapt to the changing load and processor conditions of mobile devices, resulting in a decrease in performance and energy efficiency. In order to overcome the above challenges, existing research has proposed some solutions. For example, energy efficiency can be improved to a certain extent through methods such as model compression, dynamic voltage frequency scaling (DVFS), and memory access optimization. However, these methods often only focus on a single processor or static environment, and cannot fully cope with the complex interactions between heterogeneous processors and the dynamically changing hardware environment. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the present invention provides an adaptive operator parallel partitioning method for heterogeneous embedded chips of intelligent Internet of Things. First, an energy consumption prediction model based on the random forest regression algorithm is used for energy consumption prediction in a dynamic environment of mobile devices; for a variety of hardware environments and task characteristics, an energy consumption data set is constructed and feature selection is performed. Through offline modeling and online adjustment, real-time monitoring of device status and task requirements and accurate prediction of energy consumption are achieved. Secondly, an operator adaptive partitioning method based on a dynamic programming algorithm is used for efficient parallel reasoning of deep neural networks on multiple heterogeneous processors. On the basis of energy consumption prediction, the dependency between operators, the data communication overhead across processors, and the dynamic resource conditions of the processors are comprehensively considered, and the operator partitioning strategy is optimized in real time to ensure that the system achieves optimal energy efficiency while meeting performance requirements. Through the above methods, the present invention significantly improves the energy efficiency and response speed of deep neural network reasoning on mobile devices, and solves the problem that static partitioning methods in the prior art are difficult to adapt to dynamic environments.

[0006] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0007] Step 1: Construction of dynamic environment energy consumption dataset of mobile devices;

[0008] Collect energy consumption data in various hardware environments for different mobile devices and heterogeneous processors;

[0009] Record the collected data according to the timestamp to ensure the continuity and integrity of the data;

[0010] Preprocess the collected data;

[0011] Normalize the data;

[0012] Finally, the processed data is stored in the specified format to construct the energy consumption dataset of mobile devices in a dynamic environment;

[0013] Step 2: Design of feature selection method for dynamic environment energy consumption prediction of mobile devices;

[0014] Use random forest regression algorithm for feature selection;

[0015] Assume that the energy consumption data set is D = {(X i ,y i )}, where X i is the feature vector, y i The goal of feature selection is to find a set of features F that minimizes the error of the energy consumption prediction model:

[0016]

[0017] in, The energy consumption value predicted by the model;

[0018] Step 3: Energy consumption prediction model training in dynamic environment;

[0019] The random forest regression model is used to predict the energy consumption of the dynamic environment of the mobile device. The energy consumption prediction error is optimized during the training process. The prediction error calculation formula is as follows:

[0020]

[0021] Among them, |D test | is the test data set, y true,i is the actual energy consumption value, y pred,i The energy consumption value predicted by the model;

[0022] By continuously adjusting the model parameters and minimizing the above errors, the optimal energy consumption prediction model is finally obtained;

[0023] Step 4: Operator adaptive partitioning based on dynamic programming;

[0024] Introduce energy consumption feedback to optimize operator allocation by minimizing total energy consumption;

[0025] Considering the dependencies between operators and the dynamic resource conditions of the processor, the dynamic planning table is updated to calculate the execution time and energy consumption of each operator on different processors;

[0026] Use the minimum energy consumption value in the dynamic programming table to determine the optimal operator partitioning strategy and continuously adjust it to adapt to the dynamic environment;

[0027] DP[i][j]=min{DP[i-1][k]+E ij (t)}

[0028] Among them, DP[i][j] represents the minimum energy consumption of allocating the first i operators to the first j processors, E ij (t) represents the operator v i On the processor P j Energy consumption;

[0029] Step 5: Operator scheduling based on adaptive adjustment algorithm;

[0030] Adaptive adjustment algorithm is used to adjust the operator partitioning strategy according to the current resource status and historical status to achieve energy efficiency optimization;

[0031] ΔR=∥R(t)-R k ∥

[0032] Among them, R(t) is the current resource state, R k Historical resource status, ΔR is the deviation between the current status and the historical status;

[0033] By calculating the deviation between the current state and the historical state, determine whether the operator partitioning strategy needs to be adjusted; when the deviation is less than the preset threshold, keep the current operator partitioning strategy unchanged, otherwise adjust the operator partitioning strategy according to the current resource state.

[0034] Preferably, the energy consumption data set includes a feature vector and a corresponding energy consumption value.

[0035] Preferably, the feature vector includes the usage status and task characteristics of the processor.

[0036] Preferably, the usage status of the processor includes the usage rate and frequency of the CPU, GPU, and DSP.

[0037] Preferably, the task characteristics include operator type and operator size.

[0038] The beneficial effects of the present invention are as follows:

[0039] (1) The present invention proposes an adaptive operator parallel partitioning method for heterogeneous embedded chips in intelligent Internet of Things. By combining the random forest regression algorithm and the dynamic programming algorithm, it can accurately predict energy consumption in a dynamic environment and adaptively adjust the operator partitioning strategy, thereby significantly improving the energy efficiency of deep neural network reasoning.

[0040] (2) The present invention adopts an operator scheduling method based on dynamic programming and adaptive adjustment to achieve efficient parallel operation of operators on multiple heterogeneous processors, optimize the performance of deep neural network reasoning, and reduce execution time and energy consumption.

[0041] (3) The present invention monitors the device status and task requirements in real time, utilizes energy consumption feedback and state deviation calculation, and adaptively adjusts the operator partitioning strategy, so as to dynamically adapt to the changing hardware environment and maintain the efficient operation of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is an overall framework diagram of the adaptive operator parallel partitioning method for heterogeneous embedded chips in the intelligent Internet of Things in an example of the present invention.

[0043] Figure 2 This is a diagram of the energy consumption prediction algorithm based on random forest regression of the present invention.

[0044] Figure 3 This is the dynamic programming optimization algorithm diagram of the present invention.

[0045] Figure 4 This is a diagram of the adaptive adjustment algorithm of the present invention. DETAILED DESCRIPTION

[0046] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0047] The present invention provides an adaptive operator parallel partitioning method for heterogeneous embedded chips in intelligent IoT to realize energy-efficient deep neural network inference system. In the real usage scenario of mobile devices, an intelligent scheduling model based on random forest regression and dynamic programming algorithm is proposed.

[0048] like Figures 1 to 4 The technical solution adopted by the present invention to solve the technical problem includes the following steps:

[0049] Step 1: Construction of dynamic environment energy consumption dataset of mobile devices;

[0050] For different mobile devices and heterogeneous processors, energy consumption data in various hardware environments are collected to build an energy consumption data set for mobile devices in a dynamic environment. The collected data includes but is not limited to the processor usage status (such as the usage rate and frequency of the CPU, GPU, and DSP), task characteristics (such as operator type and operator size), etc.

[0051] Step 2: Design of feature selection method for dynamic environment energy consumption prediction of mobile devices;

[0052] The random forest regression algorithm is used for feature selection. Assume that the energy consumption data set is D = {(X i ,y i )}, where X i is the feature vector, y i is the corresponding energy consumption value. The goal of feature selection is to find a set of features F that minimizes the error of the energy consumption prediction model:

[0053]

[0054] in, The energy consumption value predicted by the model;

[0055] Step 3: Energy consumption prediction model training in dynamic environment:

[0056] The random forest regression model is used to predict the energy consumption of mobile devices in dynamic environments. The energy consumption prediction error is mainly optimized during the training process. The prediction error calculation formula is as follows:

[0057]

[0058] Among them, |D test | is the test data set, y true,i is the actual energy consumption value, y pred,i is the energy consumption value predicted by the model. By continuously adjusting the model parameters and minimizing the above error, we can finally get the optimal energy consumption prediction model.

[0059] Step 4: Operator adaptive partitioning based on dynamic programming;

[0060] Introduce energy consumption feedback to optimize operator allocation by minimizing total energy consumption;

[0061] Considering the dependencies between operators and the dynamic resource conditions of the processor, the dynamic planning table is updated to calculate the execution time and energy consumption of each operator on different processors;

[0062] Use the minimum energy consumption value in the dynamic programming table to determine the optimal operator partitioning strategy and continuously adjust it to adapt to the dynamic environment;

[0063] DP[i][j]=min{DP[i-1][k]+Eij (t)}

[0064] Where DP[i][j] represents the minimum energy consumption of allocating the first i operators to the first j processors. ij (t) represents the operator v i On the processor P j energy consumption.

[0065] Step 5: Operator scheduling based on adaptive adjustment algorithm;

[0066] Adaptive adjustment algorithm is used to adjust the operator partitioning strategy according to the current resource status and historical status to achieve energy efficiency optimization;

[0067] ΔR=∥R(t)-R k ∥

[0068] Among them, R(t) is the current resource state, R k Historical resource status, ΔR is the deviation between the current status and the historical status.

[0069] Specifically, by calculating the deviation between the current state and the historical state, it is determined whether the operator partitioning strategy needs to be adjusted. When the deviation is less than the preset threshold, the current operator partitioning strategy is kept unchanged, otherwise the operator partitioning strategy is adjusted according to the current resource state.

[0070] Example:

[0071] The present invention proposes an adaptive operator parallel partitioning method for heterogeneous embedded chips of intelligent Internet of Things, aiming to improve the reasoning energy efficiency and performance of deep neural networks (DNNs) on mobile devices. Through energy consumption prediction and adaptive operator partitioning in a dynamic environment, efficient parallel reasoning of DNNs on multiple heterogeneous processors is achieved. In the implementation process, an energy consumption prediction model is first constructed based on a random forest regression algorithm for energy consumption prediction in a dynamic environment of mobile devices. By collecting energy consumption data under a variety of hardware environments, an energy consumption data set is constructed, and feature selection and model training are performed. Then, based on the dynamic programming algorithm, the operators of DNN are adaptively divided and scheduled, and the dependency between operators, the data communication overhead across processors, and the dynamic resource conditions of the processors are comprehensively considered to optimize the operator partitioning strategy in real time.

[0072] For model data set construction, we collect energy consumption data in various hardware environments for different mobile devices and heterogeneous processors, and build energy consumption data sets in dynamic environments of mobile devices. The collected data includes but is not limited to the usage status of the processor (such as the usage rate and frequency of the CPU, GPU, and DSP), task characteristics (such as operator type and operator size), etc.

[0073] For model feature selection and training, the random forest regression algorithm is used for feature selection to extract key features related to energy consumption prediction to ensure the efficiency of model input features. Combining offline modeling and online adjustment, the random forest regression model is used to monitor the equipment status and task requirements in real time and predict energy consumption to ensure the accuracy and real-time performance of energy consumption prediction.

[0074] For operator adaptive partitioning and scheduling, the dynamic programming algorithm is used to adaptively partition and schedule DNN operators, comprehensively consider the dependencies between operators, data communication overhead across processors, and dynamic resource conditions of processors, and optimize the operator partitioning strategy in real time. The adaptive adjustment algorithm is used to adjust the operator partitioning strategy according to the current resource status and historical status to achieve energy efficiency optimization.

[0075] Through the above method, the present invention improves the energy efficiency and response speed of deep neural network reasoning on mobile devices.

[0076] To achieve the above tasks, the present invention adopts the following technical solutions:

[0077] An adaptive operator parallel partitioning method for heterogeneous embedded chips of intelligent Internet of Things includes the following steps:

[0078] Step 1: Construction of dynamic environment energy consumption dataset for mobile devices: Collect the usage status and energy consumption data of the processor under different hardware environments and task loads. Task loads can include various types of deep neural network inference tasks, such as image recognition, speech recognition, etc. The collected data are recorded according to timestamps to ensure the continuity and integrity of the data. The recorded data should include the usage status of the processor, task characteristics, and corresponding energy consumption values. Preprocess the collected data, including operations such as noise removal and filling missing values, to ensure the quality of the data. Normalize the data to make it suitable for subsequent feature selection and model training. Finally, store the processed data in a specific format to construct an energy consumption dataset. The dataset should include feature vectors and corresponding energy consumption values. The feature vector includes the usage status of the processor (such as the usage rate and frequency of the CPU, GPU, and DSP), task characteristics (such as operator type and operator size), etc., as well as the corresponding energy consumption value.

[0079] Step 2: Design of feature selection method for dynamic environment energy consumption prediction of mobile devices;

[0080] The random forest regression algorithm is used for feature selection. Assume that the energy consumption data set is D = {(X i ,y i )}, where X i is the feature vector, y i is the corresponding energy consumption value. The goal of feature selection is to find a set of features F that minimizes the error of the energy consumption prediction model:

[0081]

[0082] in, The energy consumption value predicted by the model;

[0083] In the feature selection process, the data needs to be preprocessed first, including data cleaning and normalization, to eliminate noise and outliers. Then, the importance of each feature is evaluated using the random forest regression algorithm. Random forest constructs multiple decision trees and evaluates the splitting effect of each feature in each tree to finally obtain the importance score of the feature. Specifically, the feature importance is determined by calculating the average reduced error of each feature at the splitting point in all trees. According to the feature importance score, the features that have the greatest impact on energy consumption prediction are screened out. A threshold is set to select all features with an importance score greater than the threshold. The selected features include processor usage, frequency, temperature, task division ratio, number of channels, number of cores, etc. These features can fully reflect the status and task characteristics of the processor, thereby improving the accuracy and robustness of the energy consumption prediction model.

[0084] Step 3: Energy consumption prediction model training in dynamic environment:

[0085] The random forest regression model is used to predict the energy consumption of mobile devices in a dynamic environment. The energy consumption prediction error is mainly optimized during the training process. First, a training data set and a test data set that have undergone feature selection are prepared. The training data set is used to train the random forest regression model, while the test data set is used to evaluate the prediction performance of the model. During the training process, the random forest regression model is used to fit the training data to construct an energy consumption prediction model. The goal of model training is to optimize the model parameters by minimizing the energy consumption prediction error.

[0086] The calculation formula of prediction error is as follows: The calculation formula of prediction error is as follows:

[0087]

[0088] Among them, |D test | is the test data set, y true,i is the actual energy consumption value, y pred,i is the energy consumption value predicted by the model. By continuously adjusting the model parameters and minimizing the above error, we can finally get the optimal energy consumption prediction model.

[0089] First, initialize the model parameters and select the parameters of the random forest regression model, such as the number of decision trees and the maximum depth, to ensure that the model has sufficient complexity to capture the patterns in the data. Then, use the training data set to train the random forest regression model. The model constructs multiple decision trees and evaluates the splitting effect of each feature in each tree to finally obtain a prediction of energy consumption. Next, use the test data set to evaluate the prediction performance of the model, calculate the prediction error of the model on the test data set, and adjust the model parameters based on the error results. Adjust the model parameters based on the size of the prediction error. If the error is large, reselect the model parameters and repeat the training and evaluation process until the error is minimized.

[0090] Step 4: Operator adaptive partitioning based on dynamic programming;

[0091] Introduce energy consumption feedback to optimize operator allocation by minimizing total energy consumption;

[0092] Considering the dependencies between operators and the dynamic resource conditions of the processor, the dynamic planning table is updated to calculate the execution time and energy consumption of each operator on different processors;

[0093] Use the minimum energy consumption value in the dynamic programming table to determine the optimal operator partitioning strategy and continuously adjust it to adapt to the dynamic environment;

[0094] DP[i][j]=min{DP[i-1][k]+E ij (t)}

[0095] Where DP[i][j] represents the minimum energy consumption of allocating the first i operators to the first j processors. ij (t) represents the operator v i On the processor P j energy consumption.

[0096] During the calculation process, the allocation strategy is first initialized, and the execution time and energy consumption of each operator on the processor are calculated. Then, the optimal allocation strategy for each operator is updated using the dynamic programming table, and the optimal operator partitioning scheme is gradually selected to ensure that the system achieves the best energy efficiency while meeting performance requirements. Specifically, all operators and processors are first traversed, the execution time and energy consumption of the operator on each processor are calculated, and the results are stored in the dynamic programming table. Then, the optimal operator partitioning strategy is found by gradually updating the values ​​in the table. In each step, the allocation scheme with the lowest energy consumption is selected to ensure the optimization of system energy efficiency.

[0097] Step 5: Operator scheduling based on adaptive adjustment algorithm;

[0098] Adaptive adjustment algorithm is used to adjust the operator partitioning strategy according to the current resource status and historical status to achieve energy efficiency optimization;

[0099] Specifically, by calculating the deviation between the current state and the historical state, it is determined whether the operator partitioning strategy needs to be adjusted. When the deviation is less than the preset threshold, the current operator partitioning strategy is kept unchanged; otherwise, the operator partitioning strategy is adjusted according to the current resource state;

[0100] ΔR=∥R(t)-R k ∥

[0101] Among them, R(t) is the current resource state, R k Historical resource status, ΔR is the deviation between the current status and the historical status.

[0102] The workflow of operator scheduling based on the adaptive adjustment algorithm begins with the monitoring of resource status. The current resource status includes dynamic parameters such as battery power, processor load, and processor frequency. The adaptive adjustment algorithm needs to use these real-time data to evaluate the applicability of the current operator partitioning strategy. Before making adaptive adjustments, the module compares the current resource status with the previously saved execution plans and corresponding states. These historical execution plans and states contain the operator partitioning strategies and their performance used in the past under different resource states. By calculating the deviation between the current resource state and the most similar historical state, the system can determine the degree of similarity between the current state and the historical state. The calculation of the deviation is based on the difference between the current resource state and the historical state. If the deviation is small, that is, less than the preset threshold, it means that the current state is similar to the historical state, and the current operator partitioning strategy is still applicable, so the current strategy remains unchanged. This method can avoid frequent policy adjustments and reduce system overhead. When the deviation is large, it means that the current resource state has changed significantly, and the existing operator partitioning strategy may no longer be applicable. In this case, the adaptive adjustment module needs to dynamically adjust the operator partitioning strategy according to the current resource state. By recalculating the execution time and energy consumption of each operator on each processor and using the adjustment function to update the operator partition ratio, it is ensured that the new strategy can be adapted to the current resource status.

Claims

1. An adaptive operator parallel partitioning method for heterogeneous embedded chips of intelligent Internet of Things, characterized in that: The steps include: Step 1: Construction of dynamic environment energy consumption dataset of mobile devices; Collect energy consumption data in various hardware environments for different mobile devices and heterogeneous processors; Record the collected data according to the timestamp to ensure the continuity and integrity of the data; Preprocess the collected data; Normalize the data; Finally, the processed data is stored in the specified format to construct the energy consumption dataset of mobile devices in a dynamic environment; Step 2: Design of feature selection method for dynamic environment energy consumption prediction of mobile devices; Use random forest regression algorithm for feature selection; Assume that the energy consumption data set is D = {(X i ,y i )}, where X i is the feature vector, y i The goal of feature selection is to find a set of features F that minimizes the error of the energy consumption prediction model: in, The energy consumption value predicted by the model; Step 3: Energy consumption prediction model training in dynamic environment; The random forest regression model is used to predict the energy consumption of the dynamic environment of the mobile device. The energy consumption prediction error is optimized during the training process. The prediction error calculation formula is as follows: Among them, |D test | is the test data set, y true,i is the actual energy consumption value, y pred,i The energy consumption value predicted by the model; By continuously adjusting the model parameters and minimizing the above errors, the optimal energy consumption prediction model is finally obtained; Step 4: Operator adaptive partitioning based on dynamic programming; Introduce energy consumption feedback to optimize operator allocation by minimizing total energy consumption; Considering the dependencies between operators and the dynamic resource conditions of the processor, the dynamic planning table is updated to calculate the execution time and energy consumption of each operator on different processors; Use the minimum energy consumption value in the dynamic programming table to determine the optimal operator partitioning strategy and continuously adjust it to adapt to the dynamic environment; DP[i][j]=min{DP[i-1][k]+E ij (t)} Among them, DP[i][j] represents the minimum energy consumption of allocating the first i operators to the first j processors, E ij (t) represents the operator v i On the processor P j Energy consumption; Step 5: Operator scheduling based on adaptive adjustment algorithm; Adaptive adjustment algorithm is used to adjust the operator partitioning strategy according to the current resource status and historical status to achieve energy efficiency optimization; ΔR=∥R(t)-R k ∥ Among them, R(t) is the current resource state, R k Historical resource status, ΔR is the deviation between the current status and the historical status; By calculating the deviation between the current state and the historical state, determine whether the operator partitioning strategy needs to be adjusted; when the deviation is less than the preset threshold, keep the current operator partitioning strategy unchanged, otherwise adjust the operator partitioning strategy according to the current resource state.

2. According to claim 1, the method for adaptive operator parallel partitioning of heterogeneous embedded chips for intelligent Internet of Things is characterized in that: The energy consumption data set includes a feature vector and a corresponding energy consumption value.

3. The method for adaptive operator parallel partitioning of heterogeneous embedded chips for intelligent Internet of Things according to claim 1 is characterized in that: The feature vector includes the usage status and task characteristics of the processor.

4. The method for adaptive operator parallel partitioning of heterogeneous embedded chips for intelligent Internet of Things according to claim 1 is characterized in that: The usage status of the processor includes the usage rate and frequency of the CPU, GPU, and DSP.

5. The method for adaptive operator parallel partitioning of heterogeneous embedded chips for intelligent Internet of Things according to claim 1, characterized in that: The task characteristics include operator type and operator size.

Images