QoS prediction method based on multi-scale feature extraction and multi-task contrast training

Abstract

The application discloses a QoS prediction method based on multi-scale feature extraction and multi-task contrast training, comprising the following steps: S1, obtaining three-dimensional QoS data of a user, and performing dimension reduction on the three-dimensional QoS data to obtain a one-dimensional QoS sequence, wherein the one-dimensional QoS sequence comprises a to-be-predicted value and a historical time sequence; S2, eliminating noise data existing in the historical time sequence through Kalman filtering to obtain a smoothed QoS sequence; S3, for the smoothed QoS sequence, using a QoS distribution-based enhancement method to obtain a QoS enhancement sequence on each time slice; S4, inputting the smoothed QoS sequence and the QoS enhancement sequence on each time slice to a multi-scale feature encoder for coding to extract sequence time features and enhancement sequence time features respectively; and S5, predicting a dynamic service QoS value through multi-task contrast training. The method combines contrast learning and QoS prediction, extracts and continuously optimizes time features of a user from known QoS data, and predicts unknown QoS values.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of network services, and particularly relates to a QoS prediction method based on multi-scale feature extraction and multi-task comparison training. BACKGROUND

[0002] In recent years, with the continuous progress of information and network technologies, Web services are increasingly rich, so that more and more people interact with information through Web services. When a user calls a service, the user first determines the function category of the service according to the needs. Therefore, how to accurately provide high-quality services for the user among a large number of services with different function categories needs to be considered. Since the physical environment such as bandwidth is very limited, the same bandwidth cannot be allocated to all services. In order to solve the problem of the physical environment, the bandwidth of the current service needs to be determined according to the function characteristics of the service, that is, the Quality of Service (QoS) mechanism can be introduced, and the key indicators mainly include availability, throughput, delay, delay variation (including jitter and drift) and loss, to ensure the transmission quality of the service. When the user determines the function category, the user needs to select the most suitable service among a large number of services with similar functions, and this process is realized by predicting the QoS value of the user at this time. However, the current QoS value of the user is largely dependent on the Web service calling context, and the quality of the same Web service may be relatively different for different users. Therefore, it is very important to obtain the personalized QoS value of different users. In addition, due to the continuous change of the overall characteristics of the network caused by the network nodes and links, the quality of the same Web service at different time periods may still change, so the time factor is also very important for the QoS value. Therefore, it becomes a challenging task to obtain a real and effective QoS value related to the user, the service and the time.

[0003] Early QoS prediction methods can mostly only predict QoS values related to users and services, and most methods are achieved through collaborative filtering methods. In recent years, QoS prediction methods related to users, services and time have been widely studied, which can be defined as dynamic service QoS prediction methods. The methods are mainly divided into factor analysis-based methods, time series-based methods and hybrid methods combining the above two methods. Factor analysis-based methods mainly include matrix decomposition, tensor decomposition and the like. However, due to the existence of a large number of missing values in QoS data, the overall QoS value is relatively sparse, so that the information available to the factor decomposition method is incomplete, and the user and time hidden vectors obtained have a certain deviation. Time series-based methods are mainly divided into statistical time series such as ARIMA model and neural network-based time series such as RNN recursive network. There are still some problems: on the one hand, due to the influence of user, service and time factors on QoS value, QoS data is often complex and nonlinear, making it difficult for statistical models to fully capture the internal relationship of the data. On the other hand, due to the unstable network environment, QoS values fluctuate, and there may be a large number of outliers, making deep learning models vulnerable to outliers. Hybrid methods mainly combine matrix decomposition with other methods. Since hybrid methods need to combine the above two methods, they are also affected to a certain extent. These problems cause a certain deviation in the final QoS prediction. SUMMARY

[0004] The purpose of the present application is to propose a QoS prediction method based on multi-scale feature extraction and multi-task contrast training to extract and continuously optimize the time features of users from known QoS data to predict unknown QoS values.

[0005] To solve the above technical problems, the technical scheme of the present application is as follows:

[0006] A QoS prediction method based on multi-scale feature extraction and multi-task contrast training, comprising the following steps:

[0007] S1, obtaining three-dimensional QoS data of a user, and reducing the three-dimensional QoS data to obtain a one-dimensional QoS sequence, the one-dimensional QoS sequence including a to-be-predicted value |seq u and a historical time sequence

[0008] S2, eliminating noise data existing in the historical time sequence through Kalman filtering to obtain a smoothed QoS sequence Seq u ;

[0009] S3, for the smoothed QoS sequence Seq u, using the enhanced method based on QoS distribution, obtaining the QoS enhancement sequence on each time slice

[0010] S4, input the smoothed QoS sequence Seq u and the QoS enhancement sequence on each time slice Up to the scale feature encoder for encoding, to extract sequence time features respectively and the enhancement sequence time feature Z i，t , Z j，t , facilitate the loss of sequence comparison optimization task in multi-task;

[0011] S5, predict the dynamic service QoS value by multi-task contrast training.

[0012] As a preferred, the dimension reduction method in step S1: for different users, extract the QoS values corresponding to the user calling on all services, and form the QoS values into time sequences in turn according to the time slice sequence, divide the time sequence into two parts, the first part is the predicted QoS value calling on the last time slice, and the second part is the historical QoS time sequence on the remaining time slices.

[0013] As a preferred, in step S3, the data enhancement method of QoS time distribution is:

[0014] S3-1, using the overlap cropping method, first randomly select a continuous subsequence, then randomly select QoS values outside the subsequence based on the subsequence, to form two new sequences;

[0015] S3-2, input the two new sequences into the embedding layer to obtain sequence embedding;

[0016] S3-3, use feature Dropout method for sequence embedding, that is, discard part of the neurons in the embedding. Through the overlap cropping and feature Dropout method, the enhanced QoS sequence is obtained. The sequence still retains the time characteristics of the smoothed QoS sequence.

[0017] As a preferred, the multi-scale feature encoder includes WaveNet layer and BiLSTM layer.

[0018] As preferred, in the step S4, the extraction method of the enhanced sequence time feature is: in the WaveNet layer, the QoS enhanced sequence is firstly subjected to a causal convolution layer to obtain a preliminary representation, the preliminary representation is subjected to an expanded convolution and a gated linear unit, and then is subjected to a residual connection with itself, and finally is subjected to an activation function to obtain a final representation, that is, a short-term time feature of the QoS sequence; the preliminary representation is input into a BiLSTM layer to extract a long-term time feature of the QoS sequence, and finally an enhanced sequence time feature Z is output. i，t j，t , which is used for subsequent calculation of the loss of the sequence comparison optimization task in the multi-task.

[0019] As preferred, the extraction method of the sequence time feature is the same as the extraction method of the enhanced sequence time feature, the smoothed QoS sequence is input, and a sequence time feature Z is output. which is used for subsequent calculation of the loss of the sequence prediction task in the multi-task.

[0020] As preferred, the method of the step S5 is:

[0021] S5-1, calculating the loss L of the sequence comparison optimization task in the multi-task cl ;

[0022] S5-2, calculating the loss of the sequence prediction task in the multi-task, inputting the sequence time feature Z and the to-be-predicted value, and using a Smooth L1 to calculate the loss L generated in the sequence prediction. main ;

[0023] S5-3, finally calculating a total loss L total , which is used for joint training of the encoder, the total loss is a weighted sum of the sequence prediction loss and the sequence comparison loss, through the continuous training of the loss, the time feature is continuously optimized, and finally a relatively accurate QoS time feature is obtained, so that the dynamic service QoS value is predicted.

[0024] As preferred, the specific method of the step S5-1 is: firstly, constructing positive and negative samples, in the user dimension, for two QoS enhanced sequences generated for the same user sequence, regarding them as positive samples; for the QoS enhanced sequences generated for other user sequences, regarding them as negative samples; in the time dimension, for the QoS enhanced sequences on the same time slice of the same user, regarding them as positive samples; for the QoS enhanced sequences on different time slices of the same user, regarding them as negative samples; secondly, inputting the enhanced sequence time feature Z respectively calculating the loss in the sequence comparison optimization process in the user and time dimensions; finally, using a MaxPool technology in the time dimension, continuously iterating the number of user time slices, and using the above loss calculation once in each iteration.

[0025] ​In the technical solution, the contrast learning is combined with the QoS prediction. The contrast learning is to compare a sample with a positive sample similar to the sample and a negative sample different from the sample, so that the representation of the samples with similar semantics is closer in the representation space and the representations of the samples with different semantics are farther apart, so as to better extract the model representation. Previous work is mostly used in the field of computer vision and natural language processing, and focuses on feature extraction of the overall sample. However, due to the special time distribution of the time series, the time series field focuses more on the feature representation at a specific time point.

[0026] The application has the following characteristics and beneficial effects:

[0027] By using the above technical solution, the time characteristics of the user are extracted from different time scales by the method of the application, and the characteristics are continuously optimized by the training of the encoder through the multi-task contrast prediction loss, so as to predict the dynamic service QoS value. Specifically, first, the Kalman filter is used to smooth the QoS sequence to eliminate the influence of noise factors. For each smoothed QoS sequence, a QoS enhancement sequence is constructed by using a QoS time distribution-based enhancement method, which is used for the calculation of the sequence contrast optimization task of one of the subsequent multi-task. Secondly, a multi-scale feature encoder is used to preliminarily extract the long-term and short-term time characteristics of the user. Finally, a multi-task contrast training method, i.e., a joint training method of traditional sequence prediction and sequence contrast optimization, is used to train the multi-scale encoder to continuously optimize the time characteristics, so as to accurately infer the unknown QoS value. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only constitute some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0029] Figure 1 A flowchart of a QoS prediction method based on multi-scale feature extraction and multi-task contrast prediction training in an embodiment of the application.

[0030] Figure 2 A schematic diagram of a multi-scale feature extraction method in an embodiment of the application.

[0031] Figure 3 A schematic diagram of a multi-task contrast training method in an embodiment of the application. DETAILED DESCRIPTION

[0032] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other in the case of no conflict.

[0033] In the description of the present application, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element indicated must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second" and the like are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" and the like can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0034] In the description of the present application, it should be noted that, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood through specific circumstances.

[0035] The present application provides a QoS prediction method based on multi-scale feature extraction and multi-task contrast training, as shown in Figure 1 The three-dimensional user, service, and time QoS matrix data are processed by dimension reduction to obtain one-dimensional user QoS data sequence, and Kalman filtering is used to obtain the smoothed user QoS sequence. The smoothed QoS sequence is processed by an enhancement method based on QoS time distribution to obtain a QoS enhanced sequence. The smoothed QoS sequence and the QoS enhanced sequence are respectively input into a multi-scale feature encoder to extract time features. A multi-task contrast loss method, i.e. sequence prediction task and sequence contrast optimization task, is used to jointly train the encoder, and finally the optimized time features are obtained to accurately predict the QoS value.

[0036] In this embodiment, the multi-scale feature extraction method is as shown in Figure 2

[0037] ​Since both the smoothed QoS sequence and the QoS-enhanced sequence input to this layer require the same operation, for ease of explanation, they are collectively referred to as Sequence Seq. Multi-scale feature extraction methods are divided into short-term feature extraction and long-term feature extraction, implemented using WaveNet and BiLSTM networks respectively, as detailed below:

[0038] For the sequence Seq, it is first input into the causal convolutional Conv module in WaveNet, which is a Conv1d convolutional module. The initial representation of the sequence, Seq, is obtained after passing through the causal convolutional module. causal Next, after passing through the dilated convolution module, a new representation Seq is obtained. dilated The calculation process is as follows:

[0039]

[0040] Where padding represents the number of padding elements, d represents the dilation factor, k represents the kernel size, and s represents the stride.

[0041] Seq dilated The outputs of the two activation functions are multiplied together. This result is then staggered with the original input Seq to obtain the output Seq of this layer. l1 The above process is a single-layer operation; the final output Seq is obtained by stacking multiple layers. l .

[0042] Finally, Seq l After final output processing, such as activation function operations, the final output Seq of WaveNet is obtained. wavenet This output represents the local features of the sequence after local feature extraction.

[0043] For this local feature, the input is further processed by a bidirectional LSTM module, or BiLSTM module. The computation process of this module is as follows:

[0044]

[0045]

[0046]

[0047] Among them, w n With v n Let b be the weight matrix at time n. n Let be the bias matrix at time n. and These represent the forward and backward outputs of the BiLSTM at time n, respectively. The final output seq of the BiLSTM is then calculated. bilstm This output represents the long-term features of the sequence after long-term feature extraction.

[0048] In summary, by using short-term and long-term feature extraction methods on the sequence, the final features obtained are the multi-scale features of the sequence.

[0049] The multi-task comparison training method in this embodiment, such as Figure 3 As shown.

[0050] The multi-task approach is divided into sequence prediction and sequence comparison optimization tasks, and the training method is joint training of both tasks.

[0051] The sequence contrast optimization task focuses on further extracting multi-scale temporal features of users. First, positive and negative samples are constructed. In the user dimension, two augmented sequences generated from the same user sequence are considered positive samples; augmented sequences generated from other user sequences are considered negative samples. In the time dimension, augmented sequences from the same user at the same time slice are considered positive samples; augmented sequences from the same user at different time slices are considered negative samples. Next, the loss during the sequence contrast optimization process is calculated using the following formulas in the user and time dimensions respectively:

[0052]

[0053]

[0054] The loss for a single sequence comparison optimization is the sum of the two.

[0055] Finally, the MaxPool technique is used on the time dimension to iterate over the number of user time slices. The loss calculation described above is performed for each iteration until the number of user time slices is 1. The loss at this point is the final sequence contrast optimization loss L. cl .

[0056] Sequence prediction tasks focus on improving the final prediction accuracy. This method uses SmoothL1 loss as the loss L for sequence prediction. main .

[0057] Finally, the loss of the sequence comparison task and the total loss of sequence prediction L are optimized using sequence comparison. total To jointly train a multi-scale feature encoder.

[0058] L total =L main +λL cl

[0059] By jointly training the encoder using the total loss of multiple tasks, the encoder can be further optimized at multiple scales, continuously optimizing temporal features to obtain accurate user representations, thereby predicting dynamic service QoS values.

[0060] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments, including components, without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A QoS prediction method based on multi-scale feature extraction and multi-task contrast training, characterized in that, Comprising the following steps: S1, obtaining three-dimensional QoS data of the user, reducing the three-dimensional QoS data to obtain one-dimensional QoS sequence, the one-dimensional QoS sequence comprising predicted values and historical time series; S2, eliminating noise data existing in the historical time series by Kalman filtering to obtain a smoothed QoS sequence; S3, for the smoothed QoS sequence, using a QoS distribution-based enhancement method to obtain a QoS enhancement sequence at each time slice; In step S3, the data enhancement method of QoS time distribution is: S3-1, using an overlap cropping method, first randomly selecting a continuous subsequence, then randomly selecting QoS values outside the subsequence based on the subsequence to form two new sequences; S3-2, inputting the two new sequences into the embedding layer to obtain sequence embedding; S3-3, using a feature mask method for sequence embedding, that is, randomly masking a part of the embedding values and replacing them with a special item mask to obtain a QoS enhancement sequence; S4, inputting the smoothed QoS sequence and the QoS enhancement sequence at each time slice into a multi-scale feature encoder for encoding to extract sequence time features and enhancement sequence time features respectively; The multi-scale feature encoder comprises a WaveNet layer and a BiLSTM layer; The enhancement sequence time feature extraction method is: in the WaveNet layer, the QoS enhancement sequence first passes through a causal convolution layer to obtain a preliminary representation, which is connected with itself through residual connection after passing through an inflation convolution and a gated linear unit, and finally obtains a final representation through an activation function, which is the short-term time feature of the QoS sequence; the preliminary representation is input into the BiLSTM layer to extract the long-term time feature of the QoS sequence, which is the final output of the enhancement sequence time feature; S5, according to the sequence time features and the enhancement sequence time features, performing multi-task comparison training to obtain a predicted dynamic service QoS value.

2. The QoS prediction method based on multi-scale feature extraction and multi-task contrast training according to claim 1, characterized in that, In step S1, the dimension reduction method: for different users, extract the QoS values called by the corresponding user on all services, and form time series in turn according to the time slice sequence, divide the time series into two parts, the first part is the predicted QoS value called on the last time slice, and the second part is the historical QoS time series on the remaining time slices. 3.The QoS prediction method based on multi-scale feature extraction and multi-task contrast training according to claim 1, characterized in that, The sequence time feature extraction method and the enhancement sequence time feature extraction method are the same, both input the smoothed QoS sequence and output the sequence time feature.

4. The QoS prediction method based on multi-scale feature extraction and multi-task contrast training according to claim 3, characterized in that, The method of step S5 is: S5-1, calculating the loss of the sequence comparison optimization task in the multi-task according to the enhancement sequence time feature; S5-2, calculating the loss of the sequence prediction task in the multi-task, inputting the sequence time feature and the predicted value, and calculating the loss generated in the sequence prediction; S5-3, finally calculating the total loss to jointly train the encoder, the total loss is the weighted sum of the sequence prediction loss and the sequence comparison loss, through the continuous training of the loss, the time feature is continuously optimized, and finally the more accurate QoS time feature is obtained, thereby predicting the dynamic service QoS value.

5. The QoS prediction method based on multi-scale feature extraction and multi-task contrastive training according to claim 4, characterized in that, The specific method of the step S5-1 is: firstly, constructing positive and negative samples, in the user dimension, for the same user sequence, the two QoS enhancement sequences generated are regarded as positive samples; for the QoS enhancement sequences generated by other user sequences, they are regarded as negative samples; in the time dimension, for the same user, the QoS enhancement sequences in the same time slice are regarded as positive samples; for the QoS enhancement sequences in different time slices of the same user, they are regarded as negative samples; secondly, inputting the enhancement sequence time characteristics, calculating the loss in the sequence comparison optimization process in the user and time dimensions respectively; finally, using the MaxPool technology in the time dimension, constantly iterating the number of user time slices, and using the above loss calculation once in each iteration.

6. The QoS prediction method based on multi-scale feature extraction and multi-task contrast training according to claim 5, characterized in that, In the step S5-1, the loss calculation formula of the sequence comparison optimization task is as follows: wherein Z i，t , Z j，t are enhancement sequence time features.

Images