A heterogeneous computing power platform load prediction method

Abstract

The present application relates to the field of electric digital data processing, and in particular to a heterogeneous computing platform load prediction method, which comprises collecting load prediction data to train a prediction model, obtaining a trained prediction model, implementing a curriculum enhancement training strategy, inserting a hidden state memory bank in the process of gradient descent method based on momentum term, collecting load prediction data of the heterogeneous computing platform in the current time window, inputting the load prediction data into the trained prediction model, and obtaining the prediction result of future load. The existing heterogeneous computing platform load prediction method has the problems of low prediction accuracy, poor adaptability and poor stability, but the heterogeneous computing platform load prediction method has high prediction accuracy, good adaptability and good stability.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of digital data processing, in particular to a heterogeneous computing platform load prediction method. BACKGROUND

[0002] With the development of digital economy and the promotion of Internet of Things, the amount of data has increased explosively, and the demand for computing power has also increased accordingly. In order to meet the application requirements in different scenarios and maximize the computing efficiency, people have developed heterogeneous computing power. Heterogeneous computing power refers to the processing system of multiple computing powers such as CPU, GPU, FPGA and ASIC cooperating with each other.

[0003] Heterogeneous computing power realizes efficient cooperation and scheduling based on a heterogeneous computing platform. The resource occupation state and working intensity of various computing units in the heterogeneous computing platform when processing tasks are called "heterogeneous computing platform load". The heterogeneous computing platform load can reflect the computing task pressure, resource utilization efficiency and running state of the heterogeneous computing platform at a certain moment or time period, and is the core index for measuring the performance of the heterogeneous computing platform and optimizing resource scheduling.

[0004] In order to adapt to the dynamic changes of computing task demand and efficiently utilize limited computing units, the heterogeneous computing platform usually dynamically adjusts the load of each computing unit according to the demand of different computing tasks. Dynamic load adjustment usually needs to predict the load first, because the change of computing power demand has the continuity of time dimension, that is, the future load is related to the historical / current load, and resource adjustment needs time, so only according to a certain rule to predict the future load first, can avoid the lag of adjustment, so that the adjustment action is completed before the change of computing power demand.

[0005] The existing heterogeneous computing platform load prediction method mainly has the following problems:

[0006] (1) In the task of predicting the load of the heterogeneous computing platform, the load prediction data mainly includes task attribute data, resource usage state data and environment index data in the historical time window, so the load prediction data usually has the characteristics of multi-modal, non-uniform sampling and scale heterogeneity, which is a kind of multi-source heterogeneous data. At present, the load prediction data is mainly processed by simple splicing after independent feature coding, ignoring the interaction information between modalities, resulting in the loss of interaction information between modalities, the inability to capture cross-domain correlation, and the inability to capture the nonlinear relationship between complex data, so that the training effect of the final prediction model is poor, affecting the prediction accuracy of the prediction model;

[0007] (2) The load fluctuation of the heterogeneous computing platform is dynamically affected by task scheduling, resource contention, environmental interference and other factors. The conventional recurrent neural network usually adopts a static weight update mechanism, which cannot adjust the focus of the prediction model according to real-time features, lacks dynamic selection and adjustment of features at different time steps, and cannot adaptively focus on key feature dimensions, resulting in poor adaptability under different load modes;

[0008] (3) Load prediction needs to capture both short-term fluctuations caused by task scheduling and long-term trends caused by environmental temperature rise. Short-term fluctuations include minute-level fluctuations, and long-term trends include hour-level fluctuations. The modeling capability of traditional LSTM units for long and short-term dependencies is limited to a single time scale when dealing with complex load changes, making it difficult to model both short-term fluctuations and long-term trends, resulting in a prediction model that cannot adapt to rapidly changing loads;

[0009] (4) Temperature and other environmental indicators indirectly affect the supply of computing power through hardware frequency reduction. Traditional methods treat environmental indicators as ordinary features, ignoring the physical constraint effect of environmental changes on resource usage state data, resulting in low load prediction accuracy in extreme environments;

[0010] (5) Different task types on the heterogeneous computing platform have significantly different sensitivities to resource indicators, such as compute-intensive and IO-intensive. However, existing loss functions treat all samples equally, lack sensitivity adjustment for different task types, and result in poor overall load prediction accuracy of the prediction model;

[0011] (6) Load data has significant temporal imbalance, such as a surge in task volume before holidays. However, existing training methods often fail to effectively handle the temporal imbalance of load data, resulting in poor prediction performance of the prediction model when facing sudden loads;

[0012] (7) Conventional random sampling can lead to underfitting of rare patterns by the prediction model. During online training, newly arrived load data may cause catastrophic forgetting of the prediction model, which cannot effectively maintain the memory of historical load patterns, resulting in loss of memory of historical data by the prediction model and affecting the stability of prediction performance;

[0013] Therefore, the existing load prediction method for the heterogeneous computing platform has the problems of low prediction accuracy, poor adaptability and poor stability. SUMMARY

[0014] The technical problem to be solved by the present application is to overcome the above-mentioned shortcomings of the prior art and provide a load prediction method for a heterogeneous computing platform with high prediction accuracy, good adaptability and good stability.

[0015] To solve the above technical problems, the application provides a heterogeneous computing platform load prediction method, which comprises the following steps:

[0016] S1. Collecting a plurality of load prediction data to form a training data set;

[0017] Each piece of load prediction data comprises task attribute data, resource usage state data and environmental indicator data within a time window;

[0018] S2. Constructing a prediction model based on the principle of an improved recurrent neural network, training the prediction model, and obtaining a trained prediction model;

[0019] The improved recurrent neural network refers to the combination of a multi-modal tensor fusion layer, a dynamic feature selection gating mechanism, a double-channel memory separation unit and an environmental coupling residual module on the basis of a traditional LSTM;

[0020] S3. Implementing a curriculum enhancement training strategy to enable the prediction model to learn from a stable load mode;

[0021] S4. Inserting a hidden state memory bank in the process of gradient descent method based on momentum term;

[0022] S5. Collecting the load prediction data of the heterogeneous computing platform in the current time window, inputting the load prediction data into the trained prediction model, and obtaining the prediction result of the future load.

[0023] As a further improvement of the application, the task attribute data comprises task type, task priority and task resource demand.

[0024] As a further improvement of the application, the resource usage state data comprises GPU utilization, memory occupancy and CPU load.

[0025] As a further improvement of the application, the environmental indicator data comprises temperature, power consumption and humidity.

[0026] As a further improvement of the application, the training of the prediction model in S2 comprises:

[0027] S201. Tensor fusion is performed on each piece of load prediction data in the training data set by using a tensor fusion layer;

[0028] S202. Combining feature-level attention weights on the basis of the LSTM gating structure to dynamically adjust the contribution degree of input features;

[0029] S203. Decomposing the memory cell into a fast channel and a slow channel to model the sudden task load and gradual environmental change through the double channels;

[0030] S204. Explicitly modeling the relationship of "temperature-power-computing power" by using an environment-coupled residual term;

[0031] S205. Calculating a task-aware hierarchical loss function;

[0032] S206. Parameter updating of the recurrent neural network based on the gradient descent method of the momentum term;

[0033] S207. Repeating iterations S201-S206 until the preset stopping iteration condition is met.

[0034] Preferably, in S202, the dynamic adjustment of the contribution degree of the input features based on the LSTM gating structure combined with the feature-level attention weight refers to splicing the fusion feature vector at the current moment and the hidden state at the previous moment, generating a gating vector through a fully connected layer and an activation function to dynamically adjust the fusion ratio of the current features and the historical features.

[0035] Preferably, in S205, the calculation formula of the task-aware hierarchical loss function L is:

[0036] ,

[0037] In the formula, N is the total number of samples; is a positive integer; is the weight coefficient of the task type; is the type of the i-th task; is the L2 norm; is the real resource usage state data value of the i-th sample; is the predicted resource usage state data value of the i-th sample; is the environment loss weight; is the L1 norm; is the real environment index data value of the i-th sample; is the predicted environment index data value of the i-th sample. Preferably, the calculation formula of is: ,

[0038] Preferably, the calculation formula of is:

[0039] ,

[0040] In the formula, is an adjustment factor; is an exponential function with a natural constant as the base; indicates the task priority.

[0041] ​​​​​Preferably, the parameter update of the recurrent neural network in S206 based on the gradient descent method of the momentum term comprises dynamically adjusting the momentum coefficient and the learning rate for the gradient of different module parameters, dividing independent parameter groups for the tensor fusion layer, the gating mechanism, the multi-scale memory separation recurrent unit and the residual prediction module, and respectively calculating the momentum buffer and the adaptive learning rate.

[0042] As a further improvement of the application: the prediction result of the future load in S5 is the load prediction value of each computing unit at the next moment.

[0043] The beneficial effects of the application are as follows: the heterogeneous computing power platform load prediction method provided by the application has high prediction accuracy, good adaptability and good stability.

[0044] Firstly, the method adopts a tensor fusion layer to realize cross-modal dynamic correlation modeling (i.e., dynamic correlation modeling of cross-modal data) through feature projection and tensor product operation, and to enhance the nonlinear coupling relationship between task attribute data, resource usage state data and environmental index data by explicitly constructing a cross-modal interaction tensor, thereby improving the utilization efficiency of multi-source heterogeneous data.

[0045] Secondly, the method dynamically adjusts the fusion ratio (contribution degree of input features) of current features and historical features by combining a feature-level attention mechanism with the LSTM gating structure, so that the prediction model can adapt to task load fluctuations and environmental changes, thereby improving the adaptability and flexibility of feature selection.

[0046] Thirdly, the method decomposes the memory cell into a fast channel and a slow channel, models the sudden task load and the gradual environmental change through the two channels respectively, and respectively models the sudden task load and the gradual environmental change, thereby breaking through the limitation of single time scale modeling, solving the problem of insufficient modeling of mixed frequency features by single scale recurrent units, and effectively improving the modeling capability for complex load patterns.

[0047] Fourthly, the method explicitly models the coupling relationship between temperature, power consumption and computing power, improves the accuracy of load prediction through the environmental coupling residual term, and effectively reduces the prediction error in extreme environments such as high temperature.

[0048] Fifthly, the method adopts a hierarchical weighted loss strategy, weights and optimizes the loss according to the task priority, prioritizes the load prediction accuracy of high-priority tasks, enhances the fault tolerance of the prediction model to environmental noise, and improves the overall load prediction accuracy.

[0049] Sixthly, the course enhancement strategy is adopted to make the prediction model start learning from the smooth load pattern and gradually adapt to sudden fluctuations, thereby improving the generalization ability of the prediction model to high fluctuation loads.

[0050] Seventh, the method inserts a hidden state memory bank in the gradient descent method based on the momentum term, avoids catastrophic forgetting, and improves the adaptability and stability of the prediction model to historical data. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 is a schematic diagram of the principles of the present application;

[0052] Figure 2 is a 72-hour load prediction and error analysis diagram;

[0053] Figure 3 is a full-period prediction effect comparison diagram;

[0054] Figure 4 is an analysis diagram of the performance gain of the environment coupling residual module in a high temperature scenario. DETAILED DESCRIPTION

[0055] The specific embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0056] Definitions of related terms involved in the present application:

[0057] (1) Computing power: the ability of a device, cluster, or heterogeneous computing platform to process data;

[0058] (2) Heterogeneous computing platform: a system that integrates multiple computing units with different architectures, functions, and characteristics, and works together to meet diverse computing needs. Computing units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit), NPU (Neural Network Processor), DPU (Data Processing Unit), etc.

[0059] (3) LSTM: an abbreviation of Long Short-Term Memory, i.e., Long Short-Term Memory Network;

[0060] As shown in Figure 1 , the present application provides a heterogeneous computing platform load prediction method, which comprises:

[0061] S1. Collecting and preprocessing a plurality of load prediction data to obtain preprocessed load prediction data, wherein the plurality of preprocessed load prediction data constitutes a training data set;

[0062] Each load prediction data comprises task attribute data, resource usage state data and environmental index data within a time window;

[0063] The task attribute data is acquired from a task scheduling system record, and includes task type, task priority, and task resource requirement, and the task type and the task priority are discrete classification variables;

[0064] The resource usage state data is collected in real time from a hardware monitoring device, and includes GPU utilization, memory occupancy, and CPU load, and the resource usage state data is continuous time series data;

[0065] The environmental index data is collected from an environmental monitoring system monitored by a sensor, and includes temperature, power consumption, and humidity low-frequency sensor data;

[0066] The plurality of pieces of load prediction data are transmitted to a central data storage of the heterogeneous computing platform through an industrial bus or a wireless network for subsequent prediction model training.

[0067] The preprocessing includes standardization, denoising, and missing value filling, the standardization refers to a normalization method to keep the data feature values in the same dimension, the denoising refers to a wavelet transform denoising method to eliminate abnormal values, and the missing value filling refers to a mean filling method to process data missing due to network interruption or equipment failure; the preprocessing can ensure the integrity and accuracy of the load prediction data.

[0068] S2. Constructing a prediction model based on the principle of the improved recurrent neural network, training the prediction model, and obtaining a trained prediction model;

[0069] The improved recurrent neural network refers to a combination of a multi-modal tensor fusion layer, a dynamic feature selection gating mechanism, a double-channel memory separation unit, and an environment coupling residual module based on a traditional LSTM.

[0070] The combination of the multi-modal tensor fusion layer refers to cross-modal interaction of task, resource, and environmental features through the tensor fusion layer after the input layer receives multi-source heterogeneous data; the combination of the dynamic feature selection gating mechanism refers to adaptive weighted fusion of current and historical features through the dynamic gating mechanism; the combination of the double-channel memory separation unit refers to modeling short-term load fluctuations and long-term environmental trends through fast and slow double channels of the memory unit; and the combination of the environment coupling residual module refers to explicitly superimposing the physical constraint effect of environmental factors on resource usage state data in the residual module, and finally generating a load prediction result in the output layer.

[0071] The training of the prediction model in S2 includes:

[0072] S201. Tensor fusion of each piece of load prediction data in the training data set is performed by using a tensor fusion layer, and multi-source heterogeneous data tensor fusion processing is performed;

[0073] Tensor fusion layer is adopted to realize cross-modal dynamic correlation modeling through feature projection and tensor product operation, and cross-modal interaction tensor is explicitly constructed; that is, task attribute data, resource usage state data, and environment index data are respectively mapped to the same dimension through linear projection, the projected results are subjected to tensor product operation to generate a multi-dimensional interaction tensor, and then the interaction tensor is flattened into a one-dimensional vector and stacked with a bias term;

[0074] the cross-modal feature vector at time t after fusion contains dynamic interaction information of tasks, resources, and environments, the dimension is calculated from the unified dimension after projection, the calculation formula is:

[0075] ,

[0076] in the formula, is a projection matrix of the task attribute vector, which maps heterogeneous features to a unified dimension, and the dimension can be ; is a task attribute vector at time t, which contains discrete features such as task type and resource requirement at time t; represents tensor product operation, and generates a three-order tensor, which is converted into a dimensional vector through a Flatten operation, the Flatten operation is an operation of converting multi-dimensional input data (such as two-dimensional or three-dimensional tensor) into a one-dimensional array (vector); is a projection matrix of the resource usage state data vector, which maps heterogeneous features to a unified dimension, and the dimension can be ; is a resource usage state data vector at time t, which contains time series observation values of GPU / CPU utilization and memory occupation; is a projection matrix of the environment index data vector, which maps heterogeneous features to a unified dimension, and the dimension can be ; is an environment index data vector at time t, which contains low-frequency sensor data such as temperature and power consumption; is a fusion bias term, which is used to adjust the reference value of the fusion result;

[0077] and are trainable parameters, which are automatically learned and updated through gradient descent method based on momentum term in the prediction model training process.

[0078] S202. On the basis of LSTM gating structure, a time series dynamic feature selection gating mechanism is constructed by combining feature level attention weight and dynamically adjusting the contribution degree of input features;

[0079] The fusion feature vector at the current time is spliced with the hidden state at the previous time, a gating vector is generated through a full connection layer and an activation function, and the fusion ratio of the current feature and the historical feature is dynamically adjusted;

[0080] The feature selection gating vector at time t The calculation formula is:

[0081] ,

[0082] In the formula, is a Sigmoid activation function; is a gating weight matrix, indicating the importance of each feature in the selection process, and is a trainable parameter, which is automatically learned and updated through the gradient descent method based on the momentum term in the prediction model training process; is the cross-modal feature vector at time t after fusion; is the hidden state at time t, representing the hidden state at the previous time, and storing the compressed representation of historical time series information; is a bias parameter of the gating mechanism, and is a trainable parameter, which is automatically learned and updated through the gradient descent method based on the momentum term in the prediction model training process;

[0083] The current feature and the historical feature are dynamically fused through the gating vector, the weight of the current feature is enhanced when the resource utilization rate suddenly changes, the historical trend is preserved when the environmental index data changes slowly, for example, the weight of the current feature is enhanced when the GPU usage rate suddenly rises; the historical trend is preserved when the GPU usage rate is stable.

[0084] The cross-modal feature vector at time t after fusion The calculation formula is:

[0085] ,

[0086] In the formula, represents Hadamard product; is the cross-modal feature vector at time t after fusion .

[0087] The features at the current time and the historical time are combined through the gating mechanism;

[0088] S203. The memory cells are decomposed into fast channels and slow channels, and the sudden task load and gradual environmental change are modeled through the double channels respectively, and a multi-scale memory separation recurrent unit is constructed;

[0089] The short-term fluctuation and long-term trend are modeled by independent fast channel and slow channel respectively, the fast channel forgetting gate controls the short-term memory update of burst task load, the slow channel forgetting gate controls the long-term memory update of environmental change, and finally the dual-channel memory is fused;

[0090] The fast channel forgetting gate at time t which determines the retention proportion of short-term memory The calculation formula is:

[0091] ,

[0092] In the formula, is the weight matrix of the fast channel, which is a trainable parameter and is automatically learned and updated through the gradient descent method based on the momentum term in the prediction model training process; is the bias term of the fast channel, which is a trainable parameter and is automatically learned and updated through the gradient descent method based on the momentum term in the prediction model training process;

[0093] The slow channel forgetting gate at time t which determines the retention proportion of long-term memory The calculation formula is:

[0094] ,

[0095] In the formula, is the weight matrix of the slow channel, which is a trainable parameter and is automatically learned and updated through the gradient descent method based on the momentum term in the prediction model training process; is the bias term of the slow channel, which is a trainable parameter and is automatically learned and updated through the gradient descent method based on the momentum term in the prediction model training process;

[0096] The memory cell of the fast channel updated at time t The calculation formula is:

[0097] ,

[0098] In the formula, is the memory cell of the fast channel at time t-1, which stores short-term memory information; is the fast channel input gate at time t, which controls the proportion of current time information entering the memory cell of the fast channel; is the fast channel candidate memory cell at time t, which represents the result of the input information at the current time after nonlinear transformation;

[0099] ,

[0100] In the formula, is the memory cell of the slow channel updated at time t, which stores long-term memory information; is the memory cell of the slow channel at time t-1; is the input gate of the slow channel at time t, controlling the proportion of current time information entering the slow channel memory cell; is the candidate memory cell of the slow channel at time t, representing the result of the nonlinear transformation of the input information at the current time;

[0101] The final memory cell is obtained by weighted fusion of the memory cells of the fast channel and the slow channel, which realizes the comprehensive modeling of short-term fluctuations and long-term trends, and is used to update the hidden state The final memory cell after fusion The calculation formula is:

[0102] ,

[0103] In the formula, is an adaptive fusion coefficient that dynamically balances the contributions of the double channels;

[0104] The calculation formula of the adaptive fusion coefficient is:

[0105] ,

[0106] In the formula, is the weight vector of the fusion coefficient, which is a trainable parameter and is automatically learned and updated through the gradient descent method based on the momentum term in the prediction model training process; is the transpose of the weight vector of the fusion coefficient; represents the spliced input of fast channel and slow channel memory; is the bias term of the fusion coefficient, which is a trainable parameter and is automatically learned and updated through the gradient descent method based on the momentum term in the prediction model training process;

[0107] The modeling advantages of the multi-scale memory unit for complex load patterns are analyzed through double-axis time series curve analysis. The primary coordinate axis shows the dynamic following characteristics of the real load curve and the prediction result, and the secondary coordinate axis synchronously displays the absolute error change. As shown in Figure 2 , the prediction curve of the traditional long short-term memory network presents phase lag at the minute-level fluctuation, and obvious amplitude deviation at the hour-level trend turning point, resulting in a periodic peak in the error curve. The prediction curve of the technology can not only closely follow the short-term fluctuation details, but also accurately fit the long-term trend baseline. Its error curve always maintains low and stable fluctuation, especially in the stage of load severe shock, corresponding to the high-frequency segment of the curve, indicating that the technology has stronger robustness.

[0108] As shown in Figure 3As shown, by focusing on the ability of multi-scale memory units to separate and model complex load patterns, and comparing standard long short-term memory networks with temporal convolution methods, the full-time prediction curves show that this technology has the highest fit to slowly changing environmental trend lines, and can quickly respond to the steep rising edge of sudden task loads. In contrast, traditional methods show significant lag at trend inflection points. A magnified view further reveals that during periods of drastic load fluctuations, the phase difference between the predicted curve and the actual value of this technology almost disappears, and the pulse peak capture is more accurate. This is due to the synergistic effect of the fast and slow channels. The fast channel captures instantaneous changes in a timely manner by updating memory cells at high frequency, while the slow channel maintains smooth tracking of the temperature rise trend. The dynamic fusion coefficient automatically adjusts the weights according to the memory state of the two channels, enabling the prediction model to have both short-term sensitivity and long-term stability, breaking through the limitations of single-time-scale modeling.

[0109] S204. Using environmentally coupled residual terms, the relationship between "temperature-power consumption-computing power" is explicitly modeled to construct an environmentally coupled residual prediction module;

[0110] By interpolating environmental indicator data and resource usage status data to calculate residual terms, which are then superimposed onto the basic prediction results, the physical constraint effect of the environment on resources is explicitly modeled.

[0111] Represents the prediction residuals caused by environmental factors The calculation formula is:

[0112] ,

[0113] In the formula, The first coupling weight matrix represents the direct impact weight of environmental indicator data on resource usage status data. It is a trainable parameter that is automatically learned and updated through gradient descent based on momentum terms during the training process of the prediction model. This is a vector of environmental indicator data, such as numerical values ​​for temperature and power consumption. It is the hyperbolic tangent function; The second coupling weight matrix represents the adjustment weight of resource usage status data on the prediction residual. It is a trainable parameter that is automatically learned and updated through gradient descent based on momentum term during the prediction model training process. Let be the resource usage status data vector at time t, through The mapping is obtained, and the mapping method is represented as follows;

[0114] For example, when the temperature exceeds the threshold... Automatically increase GPU load prediction values ​​to reflect thermal throttling effects;

[0115] Resource usage status data vector at time t The calculation formula is:

[0116] ,

[0117] wherein, is a mapping weight vector, is a trainable parameter, which is automatically learned and updated by the momentum-based gradient descent method during the training process of the prediction model; and is a mapping bias term, is a trainable parameter, which is automatically learned and updated by the momentum-based gradient descent method during the training process of the prediction model;

[0118] the final prediction value after superimposing the residual error The calculation formula is:

[0119] ,

[0120] wherein, is an output weight matrix, used to map the hidden state to the prediction value, is a trainable parameter, which is automatically learned and updated by the momentum-based gradient descent method during the training process of the prediction model; is the hidden state at time t, representing the comprehensive time sequence information at the current time.

[0121] The modeling effect of the analysis environment coupling residual error module on the physical constraint is analyzed, and the prediction performance difference with and without the module is compared, as shown in Figure 4 As the temperature rises to the hardware frequency reduction critical point, the growth rate of the technical error is significantly smaller than that of the benchmark method, proving that the residual term successfully captures the “temperature-power-computing power” chain effect, indicating that the residual module explicitly defines the interaction calculation between the environment and the resources, embeds the physical priori knowledge into the neural network, so that the prediction model automatically compensates for the load shift caused by heat dissipation and frequency reduction in the high temperature scene, not only improves the prediction accuracy, but also enhances the generalization ability of the prediction model to extreme working conditions, avoiding the failure risk of the black box prediction model in the data scarce area.

[0122] S205. Calculate a task-aware hierarchical loss function, and hierarchically weight the loss;

[0123] By setting different weights according to the priority of the task, the prediction accuracy of high-priority tasks is given priority, and the accuracy of the overall load prediction model is improved, which is represented as:

[0124] The calculation formula of the task-aware hierarchical loss function L is:

[0125] ,

[0126] wherein, N is the total number of samples; i is a positive integer; is the weight coefficient of the task type ; and is the type of the i-th task, and the type of the task includes computing-intensive and IO-intensive. It is the L2 norm. The characterization is used for regression calculations of resource usage status data; This represents the actual resource usage status data value of the i-th sample, which includes GPU utilization, etc. Let be the predicted resource usage status data value for the i-th sample; The environmental loss weight can be set to 0.4; It is an L1 norm. Characterization is used for robust estimation of environmental indicator data; Let be the real environmental indicator data value of the i-th sample, which includes temperature, etc. Let be the predicted environmental indicator data value for the i-th sample;

[0127] The calculation formula is:

[0128] ,

[0129] In the formula, This is an adjustment factor and can be set to 0.2; It is an exponential function with the natural constant as its base; Indicates task priority. ,when "Time" indicates the highest priority, such as a real-time task; when "Time" indicates the lowest priority, such as batch processing tasks;

[0130] S206. Parameter update of recurrent neural networks based on gradient descent with momentum term;

[0131] The momentum coefficient and learning rate are dynamically adjusted according to the gradient of different module parameters. Independent parameter groups are divided for the tensor fusion layer, gating mechanism, multi-scale memory separation loop unit and residual prediction module. Momentum buffer and adaptive learning rate are calculated separately to solve the problem of unbalanced convergence speed of heterogeneous modules.

[0132] The trainable parameters are divided into 4 groups:

[0133] These are the parameters for the tensor fusion layer;

[0134] For gating mechanism parameters;

[0135] For memory unit parameters;

[0136] These are the parameters for the residual module.

[0137] The gradient magnitudes of different module parameters differ significantly, and independent momentum can prevent fast-converging modules from interfering with slow-converging modules.

[0138] For each set of parameters, the momentum term and adaptive learning rate are maintained independently, and the update process is as follows:

[0139] The momentum term in the e-th iteration The calculation formula is:

[0140] ,

[0141] In the formula, For momentum; This represents the momentum decay rate, such as setting it to 0.9; This is the momentum term for the (e-1)th iteration; Let be the gradient of the task-aware hierarchical loss function with respect to the momentum term in the e-th iteration;

[0142] The learning rate in the e-th iteration The calculation formula is:

[0143] ,

[0144] In the formula, The base learning rate can be set to 0.01; The gradient second moment estimate is obtained using the Adam algorithm; To prevent the denominator from being 0, it should be a small constant, and can be set to 0.0001. This is the task sensitivity factor, which can be set to 0.1; Task type for the current batch of samples The average weight coefficients are used to give batches with a high proportion of high-priority tasks a larger learning rate, thus accelerating the convergence of key task parameters.

[0145] Based on the momentum term and learning rate, the parameters of the recurrent neural network are updated; the parameters of the recurrent neural network in the e-th iteration are... In Can refer to or as well as These four sets of independent parameters are all updated independently using this method;

[0146] Parameters of the recurrent neural network in the e-th iteration The calculation formula is:

[0147] ,

[0148] In the formula, For the first Parameters of the recurrent neural network in the next iteration; Let be the parameters of the recurrent neural network in the e-th iteration; This is the weight decay coefficient, which can be set to 0.01;

[0149] S207. Repeat S201~S206 until the preset stopping iteration condition is met, indicating that the prediction model training is complete. The preset stopping iteration condition is reaching the preset maximum number of iterations, which is set to 1000 times.

[0150] S3. Implement a course-enhanced training strategy to enable the prediction model to learn from a stable load pattern and gradually adapt to sudden fluctuations, thereby improving its generalization ability to extreme loads.

[0151] By dividing the training set chronologically and adjusting the sample order according to the difficulty score, it can be represented as:

[0152] Phase 1: Divide the original training set D into segments according to time sequence. ; This is the first time segment; This is the k-th time segment; This is the Kth time segment;

[0153] Phase 2: Calculate the difficulty score for each segment;

[0154] Difficulty score of the k-th time segment The calculation formula is:

[0155] ,

[0156] In the formula, This represents the true value of the i-th sample. The average load value of the training set is calculated as follows: ;

[0157] Phase 3: By Generate courses by ascending order ; The first segment after sorting by difficulty represents the simplest training segment after sorting by difficulty in ascending order, such as a segment with a stable load. This refers to the j-th segment after sorting by difficulty. This is the Kth segment after sorting by difficulty;

[0158] Phase 4: Each round of training starts from the current training subset. The data is sampled in the middle and high-difficulty samples are gradually added. In the e-th round of training, batch data is uniformly sampled from the first e segments after sorting, and high-volatility load samples are gradually combined.

[0159] S4. Inserting the hidden state memory bank in the process of the momentum-based gradient descent method, deploying the dynamic hidden state playback mechanism to improve the stability and adaptability of the prediction model;

[0160] By combining the hidden state memory bank during online training, the prediction model can avoid catastrophic forgetting when processing new data and maintain the memory of historical load patterns.

[0161] The calculation method of the memory bank M is:

[0162] ,

[0163] In the formula, is the stored mth hidden state; is the stored mth memory cell;

[0164] Randomly select samples from M during each training round, calculate the additional loss, and force the prediction model to retain the memory of historical load patterns when updating parameters, avoiding the degradation of prediction performance caused by new task scheduling strategies;

[0165] The additional loss is calculated as follows:

[0166] ,

[0167] In the formula, is the number of samples randomly sampled from the memory bank M, , denotes the rounding operation;

[0168] By weighting the sum of and L, L is a task-aware hierarchical loss function that drives the prediction model parameter update and retains historical pattern memory.

[0169] S5. Collect the load prediction data of the heterogeneous computing platform in the current time window, input the load prediction data into the trained prediction model, and obtain the prediction result of the future load. The prediction result of the future load is the load prediction value of each computing unit at the next time.

[0170] During collection, automatically adjust the sampling frequency according to the actual running situation of the heterogeneous computing platform. For example, when the load of each computing unit of the heterogeneous computing platform is high, increase the load prediction data collection frequency to ensure that the details of the load prediction data can be captured. Or collect environmental index data at a lower sampling frequency to reflect long-term changes in environmental conditions.

[0171] In prediction, the prediction model uses the historical collected load prediction data as input and combines the memory unit of recurrent neural network to capture time dependence. Taking the GPU load prediction of a certain heterogeneous computing platform as an example, the input includes the task type of the heterogeneous computing platform, the current GPU utilization, temperature sensor data, etc. The next time GPU load prediction value is calculated through the prediction model. By embedding the prediction model into the real-time monitoring system of the heterogeneous computing platform, dynamic prediction and scheduling optimization of the load can be realized. During a high load period, the system can adjust the task scheduling or resource allocation in advance according to the prediction result, thereby improving the resource utilization of the heterogeneous computing platform and reducing the risk of load overload.

Claims

1. A method for load prediction of a heterogeneous computing platform, characterized in that, include: S1. Collect multiple load prediction data to form a training dataset; Each load forecast data includes task attribute data, resource usage status data, and environmental indicator data within a time window; S2. Construct a prediction model based on the principle of an improved recurrent neural network, train the prediction model, and obtain the trained prediction model; Improved recurrent neural networks refer to the combination of multimodal tensor fusion layers, dynamic feature selection gating mechanisms, dual-channel memory separation units, and environment-coupled residual modules on the basis of traditional LSTM. The training prediction model in S2 includes: S201. A tensor fusion layer is used to perform tensor fusion processing on task attribute data, resource usage status data, and environmental indicator data to obtain the cross-modal feature vector at time t after fusion. ; S202. Construct a temporal dynamic feature selection gating mechanism, including: gating the cross-modal feature vectors at time t after fusion. The hidden state at time t-1 The concatenation process involves generating a gating vector through a fully connected layer and activation function. This gating vector is then used to fuse the cross-modal feature vectors at time t. The cross-modal feature vector at time t-1 after fusion The feature vectors are fused together to obtain the fused feature vectors. ; S203. Decompose memory cells into fast channels and slow channels to construct a dual-channel memory separation unit, including: a fast channel forgetting gate to control short-term memory updates for sudden task loads, and a slow channel forgetting gate to control long-term memory updates for environmental changes; The fast-track forgetting gate at time t determines the retention rate of short-term memory. The calculation formula is: ， In the formula, is the weight matrix for the fast channel, which are trainable parameters that are automatically learned and updated through gradient descent based on momentum terms during the training process of the prediction model. The bias term for the fast channel is a trainable parameter that is automatically learned and updated through gradient descent based on the momentum term during the training process of the prediction model. The slow-channel forgetting gate at time t determines the proportion of long-term memory retained. The calculation formula is: ， In the formula, The weight matrix for the slow channel is a trainable parameter that is automatically learned and updated through gradient descent based on the momentum term during the training process of the prediction model. The bias term for the slow channel is a trainable parameter that is automatically learned and updated through gradient descent based on the momentum term during the training process of the prediction model. Forget Gate via the fast passage at time t To calculate the fast-channel update of memory cells at time t Forget gate through the slow channel at time t To calculate the slow channel update of memory cells at time t ; Final weighted fusion and The final memory cells obtained after fusion ; S204. Using environment-coupled residual terms, construct an environment-coupled residual prediction module, including: Represents the prediction residuals caused by environmental factors The calculation formula is: ， In the formula, The first coupling weight matrix represents the direct impact weight of environmental indicator data on resource usage status data. It is a trainable parameter that is automatically learned and updated through gradient descent based on momentum terms during the training process of the prediction model. This is a vector of environmental indicator data, including temperature and power consumption; It is the hyperbolic tangent function; The second coupling weight matrix represents the adjustment weight of resource usage status data on the prediction residual. It is a trainable parameter that is automatically learned and updated through gradient descent based on momentum term during the prediction model training process. Let be the resource usage status data vector at time t, through The mapping yields the resource usage status data vector at time t. The calculation formula is: In the formula, The mapping weight vector is a trainable parameter that is automatically learned and updated through gradient descent based on the momentum term during the training process of the prediction model. The mapping bias term is a trainable parameter that is automatically learned and updated through gradient descent based on the momentum term during the training process of the prediction model. Final predicted value after superimposing residuals The calculation formula is: In the formula, The output weight matrix is ​​used to represent the hidden states. The parameters mapped to the predicted values ​​are trainable parameters that are automatically learned and updated through gradient descent based on the momentum term during the training process of the prediction model. The hidden state at time t represents the comprehensive temporal information at the current time. S205. Calculate the task-aware hierarchical loss function L; S3. Implement a course-enhanced training strategy so that the prediction model learns from a stable load pattern in each training round and gradually incorporates samples with high volatility loads. S4. By incorporating a latent state memory database during online training, the predictive model can avoid catastrophic forgetting when processing new data, maintaining a memory of historical load patterns, including: The calculation method for memory bank M is as follows: , In the formula, This is the m-th hidden state stored. This refers to the m-th memory cell that is stored. During each training round, samples are randomly selected from M, and additional loss is calculated, forcing the prediction model to retain the memory of historical load patterns when updating parameters; Additional losses The calculation formula is: , In the formula, The number of samples randomly sampled from memory M. , Indicates the rounding operation; By The weighted summation with L drives the update of the prediction model parameters while retaining historical pattern memory; S5. Collect load prediction data of heterogeneous computing power platforms within the current time window, input the load prediction data into the trained prediction model, and obtain the prediction results of future load.

2. The method for load prediction of a heterogeneous computing platform according to claim 1, characterized in that, The task attribute data includes task type, task priority, and task resource requirements.

3. The method for load prediction of a heterogeneous computing platform according to claim 1, characterized in that, The resource usage status data includes GPU utilization, memory usage, and CPU load.

4. The method for load prediction of a heterogeneous computing platform according to claim 1, characterized in that, The environmental metrics data include temperature, power consumption, and humidity.

5. The method for load prediction of a heterogeneous computing platform according to claim 1, characterized in that, The training prediction model in S2 also includes: S206. Parameter update of an improved recurrent neural network based on gradient descent with momentum term; S207. Repeat S201~S206 until the preset stopping iteration condition is met.

6. The method for load prediction of a heterogeneous computing platform according to claim 1, characterized in that, The calculation formula for the task-aware hierarchical loss function L in S205 is as follows: , In the formula, N is the total number of samples; It is a positive integer; For task type Weighting coefficients; For the first The type of task; It is an L2 norm; For the first The actual resource usage status data values ​​of each sample; For the first Predicted resource usage status data values ​​for each sample; Assign environmental loss weights; It is an L1 norm; For the first The actual environmental indicator data values ​​of each sample; For the first Predicted environmental indicator data values ​​for each sample.

7. The method for load prediction of a heterogeneous computing platform according to claim 6, characterized in that, The The calculation formula is: ， In the formula, As a regulating factor; It is an exponential function with the natural constant as its base; Indicates task priority.

8. The method for load prediction of a heterogeneous computing platform according to claim 5, characterized in that, The parameter update of the recurrent neural network improved by gradient descent based on momentum term in S206 includes: dividing the multimodal tensor fusion layer, dynamic feature selection gating mechanism, dual-channel memory separation unit and environment coupling residual module into independent parameter groups, and calculating momentum term and adaptive learning rate respectively.

9. The method for load prediction of a heterogeneous computing platform according to claim 1, characterized in that, The predicted result of the future load in S5 is the predicted load value of the prediction model at the next moment.

Images