A time series QoS prediction method based on attention mechanism and multi-task learning

Abstract

The application provides a time series QoS prediction method based on an attention mechanism and multi-task learning, and relates to the technical field of service recommendation and deep learning. The method comprises collecting a QoS dataset, setting a time window length, and constructing an equal-length sequence input time series QoS prediction model based on the QoS dataset to obtain a predicted value of a QoS attribute. The time series QoS prediction model comprises a multi-source feature representation layer, a time series perception layer, an attention specialization layer and a multi-task prediction layer. The model distinguishes the key degree of different time points in the time window by means of the attention mechanism; for any QoS attribute, an independent perception module is set in the multi-task prediction layer to synchronously predict the values of multiple QoS attributes, thereby realizing multi-task prediction. The method can quickly integrate various available effective information into input features, and can realize synchronous prediction of multiple QoS attributes; compared with other prediction methods, the prediction efficiency of the method is significantly improved, and the method is more practical, and the time series QoS prediction model using the method is more robust.

Description

Technical Field

[0001] This invention relates to the fields of service recommendation and deep learning technology, and in particular to a temporal QoS prediction method based on attention mechanism and multi-task learning. Background Technology

[0002] With the rapid development of cloud computing, numerous cloud service providers (such as Google and Amazon) and application developers (such as Salesforce and MobileMe) continue to release various services on the cloud to meet user needs, resulting in a continuous increase in the number of cloud services. The sheer volume of cloud services can already satisfy users' functional requirements, and users' demands for services have upgraded to meeting non-functional needs, primarily focusing on ensuring end-to-end Quality of Service (QoS).

[0003] QoS can be described as a set of non-functional attributes of a service (such as response time and throughput). These attributes directly reflect the degree to which the service meets the user's non-functional needs and affect the overall quality of service delivery. The QoS value obtained by a user when calling cloud services is dynamic. The load of cloud servers fluctuates at different times, users tend to call services during certain periods, and network conditions change frequently due to various hardware and software factors. All of these factors lead to different service qualities obtained by the same user when calling the same service at different times, thus affecting the user experience. How to recommend high-quality services to users in a dynamic and complex cloud environment, and how to select the best alternative service for users when a certain service fails, are key to improving user experience and helping service providers gain a market advantage. Therefore, achieving accurate time-series QoS prediction is an important method to ensure dynamic service quality.

[0004] Temporal QoS prediction forecasts the QoS value for the next moment based on a user's historical experience with cloud services, typically using collaborative filtering and deep learning-based methods. The basic principle of collaborative filtering is to first find a set of users / services with similar experiences to the target user / service, then aggregate the features of these users / services to predict the target QoS value. This is mainly divided into memory-based and model-based methods, and many researchers combine the two or incorporate contextual information for prediction. Deep learning-based methods generally utilize the features of users, services, and historical QoS records, combined with recurrent neural networks to perceive temporal information for prediction. Because deep learning methods can stack deep neural network layers to automatically learn and extract effective features while ignoring irrelevant features, their prediction accuracy is usually higher than that of collaborative filtering-based methods. However, due to the massive number of services in the cloud environment, a user typically only calls a few or even just a few cloud services, and a service is usually only called by a few or even just a few users, generating only a few user-service call QoS records. This results in highly sparse historical QoS data, causing existing methods to rapidly degrade in performance. Furthermore, temporal QoS prediction requires consideration of multiple dimensions of attributes and demands a certain level of prediction efficiency. However, existing methods require multiple models to predict multiple attributes, which cannot be predicted simultaneously, resulting in low overall prediction efficiency and poor model robustness. Summary of the Invention

[0005] To address the aforementioned technical problems in the prior art, this invention provides a temporal QoS prediction method based on attention mechanisms and multi-task learning.

[0006] The method includes:

[0007] S1: Collect QoS dataset;

[0008] S2: Set the length of the time window to w, construct a sequence of length w based on the QoS dataset and input it into the time series QoS prediction model to obtain the predicted value of the QoS attribute;

[0009] The construction process of the time-series QoS prediction model includes:

[0010] A. Construct a multi-source feature representation layer;

[0011] The input at time k is represented as a quadruple consisting of user ID, service ID, time ID, and QoS value, and formalized as (u i ,s j ,t k ,q i,j,k ); where i is the sequence number of the user ID and j is the sequence number of the service ID;

[0012] One-hot encoding is performed on the user ID and service ID, and then the embedding computation is used to map them into d-dimensional hidden vectors respectively.

[0013] The Time2Vec algorithm is used to represent time ID as a d-dimensional vector.

[0014] Calculate the input feature e at time k i,j,k ,

[0015]

[0016] In the formula, [;] represents the vector concatenation operation;

[0017] B. Construct a time-aware layer;

[0018] For a time window that is a time series, the sequence [e] is formed by the input features from time k to time k+w-1. i,j,k ,e i,j,k+1 ,…,e i,j,k+w-1 Input to the time-aware layer, there is

[0019] f k =sigmoid(W f ·[h k-1 ;e i,j,k ]+b f );

[0020] i k =sigmoid(W i ·[h k-1 ;e i,j,k ]+b i );

[0021] o k =sigmoid(W o ·[h k-1 ;e i,j,k ]+b o );

[0022] In the formula, f k i k o k These are the calculation results for the forget gate, input gate, and output gate at time k, respectively; W f W i W o These are the weight matrices for the forget gate, input gate, and output gate, respectively. f b i b o These are the biases for the forget gate, input gate, and output gate, respectively; h k-1 The hidden state output at the previous time step, h0 = 0;

[0023] Cell state C at time k k Aggregate the current state and the previous state.

[0024]

[0025]

[0026] In the formula, Candidate cell state; W c b c These are the weight matrix and bias values ​​for the cell states, respectively;

[0027] Calculate the hidden state h at time k k ,

[0028] h k =o k *tanh(C k ), h k ∈R d×1 ;

[0029] In the formula, R d×1 It is a d-dimensional vector;

[0030] C. Construct an attention-specialized layer;

[0031] Predicted reception time t k+w of and The query vector Q is obtained after a linear transformation.

[0032]

[0033] In the formula, W Q Let R be the weight matrix. d×3d Represents a vector of dimension d×3d;

[0034] For the hidden state vector h l Perform two linear transformations to obtain the key vector G. l Sum vector V l ,

[0035] G l =W G ·h l ;

[0036] V l =W V ·h l ;

[0037] In the formula, W G W V Let W be the weight matrix. G ∈Rd×3d W V ∈R d×3d ;

[0038] Where k≤l≤k+w-1;

[0039] The similarity between the current time step and each historical time step is calculated by scaling the dot product attention.

[0040]

[0041] In the formula, α l For query vector Q relative to key vector G l Similarity; For the key vector G l The transpose of ; d is the hidden state vector h l The dimension;

[0042] Calculate the attention-specific representation a,

[0043]

[0044] D. Construct a multi-task prediction layer;

[0045] Configure a perception module for the xth QoS attribute to perform QoS prediction.

[0046]

[0047]

[0048] …

[0049]

[0050] In the formula, p is the number of perception modules in the prediction layer; and These are the weight matrix and bias of the p-th layer, respectively; Let x be the predicted value for the x-th QoS attribute.

[0051] Preferably, collecting the QoS dataset includes:

[0052] Select several users and several services, and record the QoS value for each call;

[0053] Delete empty and out-of-preset abnormal data, and save the QoS dataset.

[0054] The preferred settings are w=32, d=128;

[0055] The LSTM in the temporal awareness layer has two layers. The neuron structure of the first layer is 386×128, and the neuron structure of the second layer is 128×128.

[0056] The dimension of representation 'a' in the attention specialization layer is 128.

[0057] The number of fully connected layers in a single perception module is set to 2, the neuron structure of the first layer is 128×128, and the neuron structure of the second layer is 128×1.

[0058] Furthermore, the training method for the time-series QoS prediction model includes: predicting the QoS value corresponding to the training data, calculating the loss value of the predicted value relative to the observed value through the MAE function, calculating the gradient of the parameters of the time-series QoS prediction model through the backpropagation algorithm, and implementing the gradient descent algorithm through the AdamW optimizer to optimize the parameters.

[0059] Repeat the above operation until the loss value reaches a stable state or the maximum number of iterations is reached.

[0060] Preferably, the learning rate, the number of training samples per session, and the maximum number of iterations are set to 0.001, 128, and 80, respectively.

[0061] The technical solution provided by this invention can quickly incorporate various available effective information into the input features, and a perception module can be added to the prediction layer to achieve synchronous prediction of multiple QoS attributes. Compared with other time-series QoS prediction methods, this method significantly improves prediction efficiency, is more practical, and the time-series QoS prediction model using this method is more robust. Attached Figure Description

[0062] Figure 1 This is a hierarchical diagram of the time-series QoS prediction model in the service recommendation system of this invention.

[0063] Figure 2 This is a schematic diagram of the temporal QoS prediction framework based on attention mechanism and multi-task learning in this invention.

[0064] Figure 3 This is a schematic diagram of the feature representation calculation process of input information in this invention.

[0065] Figure 4 This is a schematic diagram of the LSTM unit structure of the timing-aware layer in one embodiment of the present invention.

[0066] Figure 5 This is a schematic diagram of the attention specialization layer structure in one embodiment of the present invention.

[0067] Figure 6 This is a schematic diagram of the parameter analysis of dimension d in this invention.

[0068] Figure 7 This is a schematic diagram illustrating the parameter analysis of the sliding window w in this invention.

[0069] Figure 8(a) is a graph showing the average observed value and average predicted value of throughput in one embodiment of the present invention.

[0070] Figure 8(b) is a graph showing the average observed value and average predicted value of the response time in one embodiment of the present invention.

[0071] Figure 9 This diagram illustrates a comparison of the computational efficiency of the present invention with other methods. Detailed Implementation

[0072] To address the technical problem that existing time-series QoS prediction methods require multiple models to predict multiple QoS attributes, resulting in low overall prediction efficiency and poor model robustness, this invention provides a time-series QoS prediction method based on attention mechanisms and multi-task learning.

[0073] like Figure 1 As shown, a real-time service system is operating normally. When a user calls a service, the technical solution provided by this invention predicts the QoS value of the current moment based on the service call records of the previous w moments. The prediction results of this invention can be used to recommend high-quality services to users in real time, and can also provide an overall understanding of the network's QoS status.

[0074] The inventors will now describe the invention in further detail with reference to the accompanying drawings and specific embodiments.

[0075] Example 1

[0076] like Figure 2 As shown, the specific framework of the time-series QoS prediction method provided in this embodiment includes the following four steps.

[0077] Step 1: Collect a QoS dataset. The dataset is collected by selecting several users and services. These users initiate calls to these cloud services over a period of time, and the QoS values ​​(throughput, response time, etc.) for each call are recorded. After collection, erroneous data, including empty data and data outside the predetermined reasonable range, are deleted. The processed QoS dataset is then stored on disk. During model training, the data is loaded into memory all at once to improve training speed.

[0078] Furthermore, this step uses the time-series QoS dataset WS-Dream-dataset2, collected from 142 distributed computers at Planet-Lab, representing real-world network services. This dataset contains QoS values ​​obtained from 142 users calling 4500 services at 64 different times in a real-world scenario, including both throughput and response time QoS attributes. The throughput data ranges from (0 to 1000 kbps), and the response time data ranges from (0 to 20 s). Information about the dataset used in this embodiment is shown in Table 1.

[0079] Table 1. Statistical information of the dataset

[0080]

[0081] Furthermore, to reflect the high sparsity of service calls in the real world, this step randomly deletes QoS records, making the QoS matrix sparse. The matrix density (MD) is set to 10%, 20%, and 30%. Intuitively, when MD = 10%, only 10% of the data is randomly selected as the training set, and the remaining 90% of the data is used as the test set.

[0082] Step 2: Construct the QoS prediction model. This invention comprises four modules: a multi-source feature representation layer, a temporal awareness layer, an attention specialization layer, and a multi-task prediction layer. The structure of each module is as follows:

[0083] 1) First, set the time window size to w, and construct feature inputs by combining user, service, time series and multiple QoS attributes in the form of a sequence according to the window size setting;

[0084] 2) Multi-source feature representation layer: This module represents the features of the input, and its representation process is as follows: Figure 3 As shown. The input at a given time is represented as a four-tuple (User ID, Service ID, Time ID, QoS value), formalized as (u i ,s j ,t k ,q i,j,k User IDs and service IDs are converted to one-hot encodings and mapped to a hidden vector of length d using embedding computation. Each moment is represented as a d-dimensional vector using the Time2Vec algorithm. Finally, the input at a given time is represented as:

[0085]

[0086] Where [;] represents vector concatenation, "representation" refers to feature representation, i is the sequence number of the user ID, and j is the sequence number of the service ID.

[0087] The time-aware layer uses stacked LSTM networks to extract temporal features from the representation sequence. The structure of an LSTM unit is as follows: Figure 4 As shown. The w time steps before the time step to be predicted are selected as the perceptual window. Features from different time steps within the LSTM perceptual window are used, and the hidden state at the current time step is calculated using the output of the previous time step and the input of the current time step. This ensures that the hidden state at subsequent time steps includes features from all previous time steps, ultimately capturing the changing trend of the entire sequence. In this embodiment, a continuous window of length w is modeled as a time series, i.e.:

[0088] The sequence formed by the input features from time k to time k+w-1 [e i,j,k ,e i,j,k+1 ,…,e i,j,k+w-1 [This serves as the input to the time-aware layer.]

[0089] f k =sigmoid(W f ·[h k-1 ;e i,j,k ]+b f );

[0090] i k =sigmoid(W i ·[h k-1 ;e i,j,k ]+b i );

[0091] o k =sigmoid(W o ·[h k-1 ;e i,j,k ]+b o );

[0092] Among them, W f W i W o These are the weight matrices for the forget gate, input gate, and output gate, respectively. f b i b o The biases for these three are: sigmoid is the logistic activation function. h k-1 This represents the hidden state output by the previous loop unit. For the input h0 of the first loop unit, it is initialized to 0 since there is no preceding computation result. k i k o k The results are for the forget gate, input gate, and output gate, respectively.

[0093] Subsequent use of cell state C at time point kBy aggregating this information, and using the outputs of the forget gate and the input gate, a weighted sum is calculated between the current state and the previous state. The process is as follows:

[0094]

[0095]

[0096] Among them, W c b c Here, represents the weight matrix and bias value for the cell state, respectively, and tanh is the hyperbolic tangent activation function. The asterisk (*) indicates the state of a candidate cell, and the asterisk (*) represents the element-wise multiplication of the vectors.

[0097] Finally, the hidden state h of the current loop unit can be obtained. k as follows:

[0098] h k =o k *tanh(C k h k ∈R d×1 ;

[0099] In the formula, R d×1 It is a d-dimensional vector;

[0100] In the final time-aware sequence, the next hidden state h contains information about all the previous hidden states;

[0101] h = [h] k ,h k+1 ,…,h k+w-1 ].

[0102] 4) Attention specialization layer, such as Figure 5 As shown, this module utilizes an attention mechanism to critically distinguish features at different times within the sequence. First, the time t to be predicted is... k+w The user, service, and time representations are concatenated and then linearly transformed to form the query vector Q.

[0103]

[0104] In the formula, W Q Let R be the weight matrix. d×3d Represents a vector of dimension d×3d;

[0105] Then, the hidden state vector h l Perform two distinct linear transformations (k≤l≤k+w-1) to obtain the key vector K. l Sum vector V l Specifically, the QoS attribute value is set to (user, service) as the key.

[0106] G l =W G ·h l ;

[0107] V l =W V ·h l ;

[0108] In the formula, W G W V Let W be the weight matrix. G ∈R d×3d W V ∈R d×3d ;

[0109] This invention utilizes scaled dot product attention to calculate the similarity between the current time-instance features and the sequence features of each historical time-instance, as shown in the following formula:

[0110]

[0111] α l Indicates Q relative to K l The similarity is then calculated. Finally, the representations at each time step are weighted and summed to obtain the attention-specific representation 'a'.

[0112]

[0113] 5) Multi-task prediction layer: Each QoS attribute to be predicted has an independent perception module, but they all share the specialized representation 'a' output by the attention specialization layer. Taking the xth QoS attribute as an example, the calculation process of the perception module is as follows:

[0114]

[0115]

[0116] …

[0117]

[0118] Where p represents the number of perception modules in the prediction layer. and Let represent the weight matrix and bias of the p-th layer, respectively, and ReLU is the linear rectified activation function. This represents the predicted value of the x-th attribute. Different QoS attributes use different weight matrices and biases, and all perception modules share the output 'a' of the same attention specialization layer as input.

[0119] Furthermore, this step initializes the model parameters and fixes the hyperparameters at the start of training, with the following specific settings:

[0120] 1) Time series window size w = 32;

[0121] 2) The embedding dimension d = 128 of the multi-source representation layer, and there are two QoS attributes (throughput and response time);

[0122] 3) The LSTM layer of the temporal awareness layer has 2 layers. The neuron structure of the first layer is 386×128, and the neuron structure of the second layer is 128×128.

[0123] 4) The dimension of representation 'a' in the attention specialization layer is 128.

[0124] 5) The multi-task prediction layer has 2 perception modules, and each perception module has 2 fully connected layers. The neuron structure of the first layer is 128×128, and the neuron structure of the second layer is 128×1.

[0125] Furthermore, to obtain the optimal settings of d and w described above, multiple sets of experiments were designed based on controlling a single variable. The experimental results are as follows: Figure 6 , Figure 7 As shown. Figure 6 As shown, the prediction performance of this invention varies with d, where d∈{16,32,64,128,256} in the experiment. Keeping MD constant, as d increases from 16 to 128, both MAE (Mean Absolute Error) and RMSE (Root Mean Squared Error) gradually decrease, indicating improved prediction performance. However, when d further increases to 256, the improvement is not significant, and some indicators even decline. This is because when there are too many parameters and insufficient training samples, the redundancy of latent factors inevitably leads to overfitting, and excessively large dimensions also reduce the model's computational speed. Therefore, to balance prediction accuracy and computational efficiency, d=128 is chosen. Figure 7 As shown, this invention demonstrates the variation of MAE and RMSE with w when performing time-series throughput prediction and response time prediction, where w ∈ {4, 8, 16, 32, 48, 63}. Keeping MD constant, as w increases from 4 to 32, both MAE and RMSE gradually decrease, resulting in better prediction performance. However, as w gradually increases to 63, MAE and RMSE begin to increase, leading to decreased model performance. When the window size is 63, there are insufficient data samples for training and prediction, resulting in highly unstable results. Therefore, w = 32 is chosen.

[0126] Step 3: Input a batch of training data into the QoS prediction model. The model predicts the corresponding QoS value based on the input, calculates the loss value between the predicted value and the observed value using the MAE function, and optimizes the model parameters using the backpropagation algorithm combined with an optimizer. This step is repeated until the loss value reaches a stationary state or the maximum number of iterations is reached. Specifically, during training, the Mean Absolute Error (MAE) is used as the objective function to calculate the loss between the QoS observation value and the predicted value. After the errors of multiple QoS attributes are superimposed, the model parameters are optimized through the backpropagation algorithm combined with the optimizer to gradually reduce the prediction error. When the model converges, the time-series QoS prediction model with the best prediction results under the current data is obtained.

[0127] The relevant hyperparameters for model training are set as follows:

[0128] 1) AdamW was used as the optimizer during training;

[0129] 2) Learning rate α = 0.001;

[0130] 3) Training batch size, i.e., the number of training samples input by nitrogen blowing is set to 128;

[0131] 4) The maximum number of iterations is set to 80.

[0132] Step 4: After training, evaluate and compare the model using a test set to verify the effectiveness of the invention. Model evaluation and comparison includes two parts: defining evaluation metrics and comparing model results.

[0133] 1) Defining Evaluation Metrics: Time-series QoS prediction tasks focus on the closeness between predicted and true values. Therefore, this invention uses the Mean Absolute Error (MAE) and Root Mean Square Error (RMSE), commonly used in time-series QoS prediction, as evaluation metrics. MAE represents the average absolute deviation between all predicted and true values, avoiding the cancellation of prediction errors from different samples and accurately reflecting the magnitude of the actual prediction error. RMSE represents the standard deviation of the prediction error, reflecting its dispersion. The smaller the MAE and RMSE, the closer the predicted value is to the true value, and the higher the prediction accuracy.

[0134] The definitions of MAE and RMSE are as follows:

[0135]

[0136]

[0137] Where s represents the s-th test sample, and S represents the total number of samples in the test set. and These are the actual value and the predicted value of the sample, respectively.

[0138] 2) Comparison of model results: After model construction in step 2 and model training in step 3, this step made model predictions for time series throughput and time series response time. The final evaluation index values ​​are shown in Table 2.

[0139] Table 2. Prediction results of time-series throughput and response time

[0140]

[0141] As shown in Table 2, compared with various models, this invention demonstrates superior performance in both time-series throughput prediction and time-series response time prediction tasks. In the time-series throughput prediction task, the MAE index of this invention is improved by 44.62%, 35.75%, and 32.22% under different matrix densities, respectively; the average RMSE is improved by 33.93%. In the response time prediction task, the MAE is improved by 22.97%, 19.91%, and 18.25%, respectively; the average RMSE is improved by 11.72%. This is because this invention uses heterogeneous representation processing of users, services, time series, and various QoS attributes in the cloud environment, enabling more accurate extraction of different types of features. Other studies, however, uniformly treat different types of inputs, resulting in models lacking feature discrimination capabilities. More importantly, the multi-task learning approach of this invention not only increases the number of training samples but also allows the prediction layer to capture the different impacts of different QoS attributes on the prediction target, preventing model overfitting and improving generalization ability and computational efficiency.

[0142] Furthermore, as shown in Figures 8(a) and 8(b), the two tables represent the average observed and average predicted values ​​of throughput and response time for over 400,000 test data points at 32 time points, respectively. The trends of the average observed and average predicted value curves in the figures are largely consistent, indicating that the present invention can capture the temporal variation trend of QoS. The average error in throughput is within 0.8 kbps, and the average error in response time is within 0.13 s, also demonstrating that the present invention can effectively improve the accuracy of temporal QoS prediction.

[0143] Furthermore, the computational efficiency of this invention compared to other types of models is as follows: Figure 9As shown, the vertical axis represents the time required for each method to predict a service call at any given time. When predicting only two QoS attributes, the efficiency of this invention is higher than that of DeepTSQP and STCA (both deep learning-based methods), but lower than that of WSPred and AMF (both matrix factorization-based methods). This is because the multi-source feature representation layer of this invention introduces more information, and the LSTM and attention mechanisms also result in a larger number of model parameters compared to other methods. However, these structures make this invention the most accurate in prediction and more competitive in real-world scenarios requiring the prediction of multiple QoS attributes, as multi-task learning ensures that the prediction time of this invention does not increase significantly even when predicting more QoS attributes, while the time cost of other models increases exponentially.

[0144] As can be seen, the technical solution provided by this invention can quickly incorporate various available effective information into the input features, and a perception module can be added to the prediction layer to achieve synchronous prediction of multiple QoS attributes. Compared with other time-series QoS prediction methods, this method significantly improves prediction efficiency, is more practical, and the time-series QoS prediction model using this method is more robust.

[0145] Furthermore, based on the aforementioned beneficial effects, each preferred scheme also achieves the following beneficial effects: storing datasets with deleted abnormal data helps improve the training speed of the model, enabling it to be more efficiently applied to actual prediction work; appropriate parameter settings for the time-series QoS prediction model can balance prediction accuracy and computational efficiency, while ensuring the stability of prediction results; training the time-series QoS prediction model under appropriate parameter settings can improve the model's prediction accuracy and the stability of prediction results.

[0146] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A temporal QoS prediction method based on attention mechanism and multi-task learning, characterized in that, include: S1: Collect QoS dataset; S2: Set the length of the time window to w, construct a sequence of length w based on the QoS dataset and input it into the time series QoS prediction model to obtain the predicted value of the QoS attribute; The construction process of the time-series QoS prediction model includes: A. Construct a multi-source feature representation layer; The input at time k is represented as a quadruple consisting of user ID, service ID, time ID, and QoS value, and formalized as (u i s j , t k q i，j，k ); where i is the sequence number of the user ID and j is the sequence number of the service ID; One-hot encoding is performed on the user ID and service ID, and then the embedding computation is used to map them into d-dimensional hidden vectors respectively. The Time2Vec algorithm is used to represent time ID as a d-dimensional vector. Calculate the input feature e at time k i，j，k , In the formula, [;] represents vector concatenation operation; B. Construct a time-aware layer; For a time window that is a time series, the sequence [e] is formed by the input features from time k to time k+w-1. i，j，k e i，j，k+1 , ..., e i，j，k+w-1 Input to the time-aware layer, there is f k =sigmoid(W f ·[h k-1 ;e i，j，k ]+b f ); i k =sigmoid(W i ·[h k-1 ;e i，j，k ]+b i ); o k =sigmoid(W o ·[h k-1 ;e i，j，k ]+b o ); In the formula, f k i k o k These are the calculation results for the forget gate, input gate, and output gate at time k, respectively; W f W i W o These are the weight matrices for the forget gate, input gate, and output gate, respectively. f b i b o These are the biases for the forget gate, input gate, and output gate, respectively; h k-1 The hidden state output at the previous time step, h0 = 0; Cell state C at time k k Aggregate the current state and the previous state. In the formula, Candidate cell state; W c b c These are the weight matrix and bias values ​​for the cell states, respectively; Calculate the hidden state h at time k k , h k = no k *fish(C) k ),h k ∈R d×1 ; In the formula, R d×1 It is a d-dimensional vector; C. Construct an attention-specialized layer; Predicted reception time t k+w of and The query vector Q is obtained after a linear transformation. In the formula, W Q Let R be the weight matrix. d×3d Represents a vector of dimension d×3d; For the hidden state vector h l Perform two linear transformations to obtain the key vector G. l Sum vector V l , G l =W G ·h l ; V l =WV·h l ; In the formula, W G WV is the weight matrix, W G ∈R d×3d W V ∈R d×3d ; Where k≤l≤k+w-1; The similarity between the current time step and each historical time step is calculated by scaling the dot product attention. In the formula, α l For query vector Q relative to key vector G l Similarity; For the key vector G l The transpose of ; d is the hidden state vector h l The dimension; Calculate the attention-specific representation a, D. Construct a multi-task prediction layer; Configure a perception module for the xth QoS attribute to perform QoS prediction. … In the formula, p is the number of perception modules in the prediction layer; and These are the weight matrix and bias of the p-th layer, respectively; Let x be the predicted value for the x-th QoS attribute.

2. The temporal QoS prediction method based on attention mechanism and multi-task learning as described in claim 1, characterized in that, The collection of QoS datasets includes: Select several users and several services, and record the QoS value for each call; Delete empty and out-of-preset abnormal data, and save the QoS dataset.

3. The temporal QoS prediction method based on attention mechanism and multi-task learning as described in claim 1, characterized in that, Set w=32, d=128; The LSTM in the temporal awareness layer has two layers. The neuron structure of the first layer is 386×128, and the neuron structure of the second layer is 128×128. The dimension of representation 'a' in the attention specialization layer is 128. The number of fully connected layers in a single perception module is set to 2, the neuron structure of the first layer is 128×128, and the neuron structure of the second layer is 128×1.

4. The temporal QoS prediction method based on attention mechanism and multi-task learning as described in claim 1, characterized in that, The training method for the time-series QoS prediction model includes: Predict the QoS value corresponding to the training data, calculate the loss value of the predicted value relative to the observed value through the MAE function, calculate the gradient of the parameters of the time series QoS prediction model through the backpropagation algorithm, and implement the gradient descent algorithm through the AdamW optimizer to optimize the parameters. Repeat the above operation until the loss value reaches a stable state or the maximum number of iterations is reached.

5. The temporal QoS prediction method based on attention mechanism and multi-task learning as described in claim 4, characterized in that, Also includes: The learning rate, number of training samples per session, and maximum number of iterations were set to 0.001, 128, and 80, respectively.

Images