A service system reliability prediction method based on weighted cross dimension reduction

Abstract

The application provides a service system reliability prediction method based on weighted cross dimension reduction, comprising: obtaining service system characteristic data for training, and calculating system reliability value; obtaining basic characteristics and randomly shuffling data in the basic characteristics as noise characteristics, and establishing a characteristic cross window; performing dimension reduction on characteristics in the characteristic cross window based on characteristic screening; judging whether the dimension reduction amplitude meets the requirement according to the ratio of the dimension reduction characteristics to the total characteristic data; performing weighted sampling on the dimension reduction characteristics by using a gradient-based one-sided sampling algorithm, and obtaining residual characteristics of the dimension reduction characteristics; and training an XGBoost model by using the dimension reduction characteristics and the residual characteristics. The application performs weighted cross dimension reduction processing on characteristic data, improves the calculation method of the reliability value, solves the problem of overfitting of the learner caused by too many characteristics of the large service system, and fully utilizes the training advantage of high-dimensional data to improve the prediction accuracy.

Description

Technical Field

[0001] This invention relates to the field of information technology, and more particularly to a method for predicting the reliability of service systems based on weighted cross-dimensionality reduction. Background Technology

[0002] Service systems integrate web services to meet user needs. They are characterized by cross-platform compatibility, loose coupling, and scalability. However, their reliability can fluctuate due to external factors such as location and communication links, as well as internal factors within the service itself. Service system reliability prediction allows for advance understanding of reliability trends, enabling replacement and other measures to be taken before service system failures occur, reducing unreliability and improving the stability of service system operation.

[0003] Currently, most methods for predicting the reliability of service systems are based on statistical or deep learning approaches.

[0004] Statistical reliability prediction methods mainly include CART decision tree algorithm (DT-CART), Hidden Markov Model (HMM), and Collaborative Filtering (CF).

[0005] The DT-CART algorithm is a decision analysis method that evaluates project risk and determines its feasibility by constructing a decision tree based on the Gini coefficient, given the known probabilities of various scenarios. It is a graphical method that intuitively applies probability analysis. However, this algorithm struggles to predict continuous numerical values ​​and requires significant preprocessing for time-series data.

[0006] Hidden Markov Models (HMMs) describe a Markov process with hidden, unknown parameters, determining these hidden parameters from observable parameters. HMMs can further learn patterns in time series; however, they are memoryless and dependent on previous states, thus failing to effectively utilize contextual information, limiting their predictive capabilities.

[0007] Collaborative filtering generates predictions by building a project similarity model and analyzing historical experience data. While simple and effective, this method cannot predict situations beyond historical experience, and defining reliability solely based on the success rate of service calls leads to the "gradient loss" problem.

[0008] Reliability prediction methods based on deep learning mainly include Topology-Aware Neural (TAN), Convolutional Neural Networks (CNN), and Long Short-Term Memory (LSTM).

[0009] The TAN model uses explicit path modeling layers and implicit cross-connection layers to capture path features and endpoint cross-connection features, respectively. It can learn the impact of network topology on prediction results, but it cannot handle complex dynamic topology structures. Therefore, it is only suitable for business scenarios where the network structure is not complex and the response time requirement is not high.

[0010] CNNs can use convolutional and pooling layers to extract features to alleviate the training pressure brought by high-dimensional data, but the training time of the model is long and it is difficult to meet the actual business needs.

[0011] LSTM models can utilize the memory function of neurons to model time series data, but they are difficult to handle high-dimensional data. Furthermore, due to inaccurate calculation of reliability values, the error increases further when predicting high-dimensional data.

[0012] In summary, existing methods for predicting the reliability of service systems still have the following problems:

[0013] The calculation of reliability values ​​is inaccurate. Most recent studies define reliability values ​​solely based on the success rate of service calls. However, metrics such as response time and number of verifications also affect the reliability of a service system. If only a single factor is considered when calculating reliability values, the results will not accurately reflect changes in the actual reliability status of the system.

[0014] Learners are prone to overfitting, resulting in low prediction accuracy and low time efficiency. Most reliability prediction methods cannot properly handle the high-dimensional data collected by the service system, leading to overfitting problems that affect both the accuracy of prediction results and excessively long training and prediction times.

[0015] Improper handling of missing values. Feature data often contains a large number of missing values, and existing linear or nonlinear interpolation methods are not suitable for handling cases where missing values ​​are unevenly distributed. Incorrect imputation of missing values ​​will lead to the training of an incorrect model. Therefore, the handling of missing values ​​directly affects the accuracy of the prediction results. Summary of the Invention

[0016] This invention provides a service system reliability prediction method based on weighted cross-dimensionality reduction. This invention performs weighted cross-dimensionality reduction on feature data, improving the method for calculating reliability values, solving the learner overfitting problem caused by the excessive number of features in large service systems, and fully utilizing the training advantages of high-dimensional data to improve prediction accuracy.

[0017] The technical means employed in this invention are as follows:

[0018] A service system reliability prediction method based on weighted cross-dimensionality reduction includes the following steps:

[0019] S1. Obtain feature data of the training service system, the feature data including at least response time, success rate, and number of calls; calculate the system reliability value based on the response time, success rate, and number of calls;

[0020] S2. Obtain basic features and randomly shuffle the data within the basic features to use as noise features; in the first iteration, use all feature data as basic features, otherwise use the dimensionality reduction features output in the previous iteration as basic features; merge the basic features and noise features horizontally to establish a feature cross window;

[0021] S3. Based on feature filtering, reduce the dimensionality of the features within the feature cross window and output the dimensionality-reduced features;

[0022] S4. Determine whether the dimensionality reduction margin meets the requirements based on the ratio of the dimensionality reduction feature to all feature data. If the requirements are met, the dimensionality reduction ends, the dimensionality reduction feature is output, and S5 is executed. Otherwise, return to S2.

[0023] S5. The dimensionality reduction features are weighted and sampled using a gradient-based one-sided sampling algorithm to obtain the residual features of the dimensionality reduction features.

[0024] S6. Use the dimensionality reduction features and the residual features as feature training data, and input the system reliability value as target training data into the XGBoost model for training. Use the trained XGBoost model to predict the reliability of the service system.

[0025] Furthermore, a system reliability value is calculated based on the response time, success rate, and number of calls, including calculating the system reliability value according to the following formula:

[0026]

[0027] Where rt is the response time, sr is the success rate, cnt is the reliability value, α is a constant used to adjust the contribution of the response time index to the reliability value calculation, and β is the success rate threshold.

[0028] Furthermore, the system reliability value is calculated based on the response time, success rate, and number of calls. The calculation also includes standardizing the calculated system reliability value and mapping it to the range of 0-1.

[0029] Furthermore, based on feature filtering, the features within the feature cross-window are subjected to dimensionality reduction, including:

[0030] Calculate the importance of each feature in the dataset;

[0031] Features are selected based on their importance, and dimensionality reduction is performed to output the dimensionality-reduced features.

[0032] Furthermore, the importance of each feature in the dataset is calculated, including calculating the importance of each feature according to the following formula:

[0033]

[0034] Where i is the feature index, σ is the feature importance calculation function of XGBoost, S is the feature cross window, ψ is the system reliability value, and m is the number of iterations.

[0035] Furthermore, features are selected for dimensionality reduction based on their importance, including selecting features O with importance greater than the mean importance value as the dimensionality reduction result according to the following formula:

[0036]

[0037] Where n is the capacity of S.

[0038] Furthermore, the dimensionality improvement and reduction features are weighted and sampled using a gradient-based one-sided sampling algorithm, including:

[0039] The weights of the residual features are calculated based on the total number of features and the number of dimensionality-reduced features;

[0040] Residual features are sampled from the dimensionality-reduced features according to this weight.

[0041] Compared with the prior art, the present invention has the following advantages:

[0042] This invention proposes a reliability prediction method based on weighted cross-dimensionality reduction. During data processing, constraints are imposed to filter out feature data that negatively impacts reliability prediction. Furthermore, this invention improves the reliability value calculation formula, analyzing the reliability status of the service system from multiple perspectives, including success rate, response time, and number of verifications, providing proactive guidance for the intelligent operation and maintenance of the service system. Experimental results on a real service system demonstrate that, compared to previous methods, this invention fully leverages the training advantages of high-dimensional data, achieving high prediction accuracy while improving training and prediction efficiency. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart of the service system reliability prediction method based on weighted cross-dimensionality reduction in the embodiment.

[0045] Figure 2 This is a schematic diagram of the structure of the feature cross window in the embodiment.

[0046] Figure 3 This represents the actual reliability status of the service system in the example.

[0047] Figure 4 The example compares the service system reliability prediction results with and without weighted cross-dimensionality reduction. (a) shows the XGBoost model with weighted cross-dimensionality reduction, and (b) shows the XGBoost model without weighted cross-dimensionality reduction.

[0048] Figure 5 The following table compares the prediction results of the method of this invention with those of other reliability prediction methods: (a) the method of this invention, (b) the adaBoost algorithm, (c) the catBoost algorithm, (d) the lightGBM algorithm, (e) the CSM-RF model, (f) the Feedback model, (g) the LSTM model, and (h) the BayesNet model. Detailed Implementation

[0049] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0050] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0051] like Figure 1 As shown, this invention provides a service system reliability prediction method based on weighted cross-dimensionality reduction, specifically including the following steps:

[0052] Step 1: Calculate the reliability value.

[0053] The reliability value is calculated using formula (1) with response time rt, success rate sr, and number of calls cnt, where α and β are constants. α is used to adjust the contribution of the response time metric to the reliability value calculation, ranging from 0 to 1, with a default value of 0.1. The smaller α is, the greater the influence of the response time metric on the reliability value calculation. β is the threshold for the success rate, conventionally set to 0.5. When sr < β, ψ < 0, indicating that the service system reliability is low at this time.

[0054]

[0055] Since the reliability values ​​calculated by formula (1) can be positive or negative, in order to more clearly describe the reliability level of the system, the reliability values ​​need to be mapped to the range of 0-1. Therefore, it is necessary to use formula (2) for data standardization. After data standardization, the reliability indicators are of the same order of magnitude, which is suitable for comprehensive evaluation.

[0056]

[0057] Step 2: Create a feature cross window.

[0058] In the first iteration of the algorithm, all features of the dataset are set as base features, the base features are copied as noise features, and the data order within the noise features is randomly shuffled. Finally, the base features and noise features are horizontally merged to establish a feature cross-window (see...). Figure 2 The role of the noise feature is to create random error during iteration, preventing the output from being the same in each iteration. Since each iteration further simplifies the basic features, the size of the cross-window decreases after each iteration.

[0059] If the algorithm is not in its first iteration, copy the results of the previous iteration as the basic feature, and perform the remaining operations as in the previous iteration.

[0060] Step 3: Dimensionality reduction of features within the cross-window based on feature selection

[0061] First, the importance I of each feature in the dataset is calculated using the output of steps 1 and 2 according to formula (3). To reduce error, the average of multiple calculation results is taken. Since random noise has been created in step 2, the calculation results are different each time and no invalid calculation will be generated.

[0062]

[0063] In the above formula, i is the feature index, σ is the feature importance calculation function of XGBoost, S is the feature cross window created in step 2, ψ is the reliability value calculated in step 1, and m is the number of iterations.

[0064] According to formula (4), features O with importance greater than the mean of I are selected as the dimensionality reduction result. Since half of the features in S are noisy features, and these features are artificially created and do not exist in the actual dataset, the output should not include this part.

[0065]

[0066] Where n is the capacity of S.

[0067] Step 4: Determine if the dimensionality reduction is complete.

[0068] Determine whether the ratio of the dimensionality reduction feature output in step 3 to the total features is lower than the threshold, which indicates whether the dimensionality reduction magnitude meets the requirements. If the condition is met, the dimensionality reduction ends, the dimensionality reduction feature is output, and the process proceeds to step 5. If the condition is not met, the dimensionality reduction process continues, and the process returns to step 2.

[0069] Step 5: Weighted Sampling

[0070] To maintain the distribution of samples and improve the diversity of the learner, this invention uses a gradient-based one-side sampling (GOSS) algorithm for weighted sampling. First, based on the total number of features n... a The number of dimensionality-reduced features n output from step 3 b Calculate the weight ω of the residual features (see Formula 5), ​​and then sample the residual features from the remaining features after dimensionality reduction according to the weight.

[0071]

[0072] Residual features help to further learn the residual details of the model. As shown in Equation 5, the more dimensionality reduction features there are, the more residual features there will be. This is because after training a large model with enough features, a relatively large number of residual features are also needed to characterize the details of the model. In other words, the larger the model, the more details need to be considered.

[0073] Step 6: Reliability prediction of the service system.

[0074] The dimensionality reduction features obtained in step 4 and the residual features obtained in step 5 are used as feature data, and the reliability value calculated in step 1 is used as target data and input into the XGBoost model for training and prediction. Since the XGBoost model supports input data with missing values ​​by default, there is no need to consider the handling of missing values.

[0075] The following specific application examples will further illustrate the solution and effects of the present invention.

[0076] All experiments in this invention were conducted on an Intel(R) Xeon(R) Platinum 8260L CPU@2.30GHz 40G server. To verify the model's effectiveness, the dataset provided by the 2020 AIOps competition "Microservice Application System Fault Detection and Root Cause Localization" was used as the experimental dataset. This dataset collected real data from a large application system over a period of half a month, with more than 2,000 feature metrics collected from components such as the operating system, containers, databases, and middleware. The data contained some missing values, and data was collected at a default interval of one minute. To avoid the influence of random errors on the experiments, all experimental data below are the average of the results obtained from five consecutive experiments.

[0077] (1) Evaluation indicators

[0078] This invention utilizes widely used metrics, namely accuracy, mean squared error loss (MSE), and coefficient of determination (R2-score), to evaluate the effectiveness of the method of this invention and various control methods in predicting the reliability of service systems.

[0079] Accuracy is the ratio of the number of samples correctly predicted by the model to the total number of samples. The calculation method is shown in formula (6), where TP is the number of samples correctly predicted by the model, and FP is the number of samples incorrectly predicted by the model. Higher accuracy indicates better model prediction performance.

[0080]

[0081] MSE is the mean of the squared differences between the actual and predicted values. The calculation method is given in formula (7), where N is the experimental prediction sample size, and P... iR represents the actual reliability value in the sample. i This represents the reliability value predicted by the experimental method. The smaller the mean squared error loss, the better the model's prediction performance.

[0082]

[0083] R²-score reflects the proportion of the total variance of the dependent variable that can be explained by the independent variables through the regression relationship. The calculation method is shown in formula (8). Where N is the experimental prediction sample size, P... i R is the reliability value predicted by the experimental method. i This represents the actual reliability value in the sample. This represents the average of the actual reliability values. The closer the R² score is to 1, the better the independent variables explain the dependent variable in the regression analysis, and the better the model learning effect.

[0084]

[0085] (2) Experimental Results and Comparative Analysis

[0086] 1) Comparison of service system reliability prediction methods using and not using weighted cross-dimensionality reduction

[0087] To further illustrate the effectiveness of the weighted cross-dimensionality reduction process in service system reliability prediction, this section compares the service system reliability prediction results using weighted cross-dimensionality reduction with those not using weighted cross-dimensionality reduction (using all features). Both groups are tested on large-scale datasets using the XGBoost model. The actual service system reliability state is as follows: Figure 3 As shown in the figure, the service system reliability prediction results of the comparative experiment are as follows: Figure 4 As shown in (a) and (b), these are cases where the scatter plots in the figure are incorrectly predicted.

[0088] Table 1 compares the service system reliability prediction methods using and not using weighted cross-dimensionality reduction.

[0089]

[0090] Depend on Figure 4As shown in (a) and (b), in the experiment of predicting service system reliability using the same model on the same dataset, the XGBoost model with weighted cross-dimensionality reduction predicted 19 errors, while the XGBoost model without weighted cross-dimensionality reduction predicted 21 errors. This indicates that under the same conditions, the reliability prediction method based on weighted cross-dimensionality reduction predicts values ​​closer to the actual values. This is because not all features collected by the service system affect its reliability. As some features that do not affect the reliability of the service system are included in the training, the training loss of the reliability prediction model gradually accumulates, and the prediction accuracy decreases accordingly. The service system reliability prediction method based on weighted cross-dimensionality reduction effectively improves the accuracy of the prediction results by pre-filtering the feature data.

[0091] As shown in Table 1, with advantages in accuracy, MSE, and R2-score, the XGBoost model using weighted cross-dimensionality reduction consumes only 1 / 3 of the time of the XGBoost model without weighted cross-dimensionality reduction. This indicates that the time efficiency of the service system reliability prediction model is significantly improved after weighted cross-dimensionality reduction.

[0092] The above analysis demonstrates that weighted cross-dimensionality reduction helps achieve a "win-win" situation in terms of both accuracy and time efficiency during service system reliability prediction.

[0093] 2) Comparison of the method of this invention with other service system reliability prediction methods

[0094] To verify the performance of the method of this invention, reliability prediction experiments were conducted under the same conditions with other latest and classic service system reliability research results, including the adaBoost algorithm, catBoost algorithm, lightGBM algorithm, CSM-RF model, Feedback model, LSTM model, and BayesNet model, and various indicators were recorded. Among them, adaBoost, catBoost, and lightGBM are three classic machine learning methods based on the gradient boosting idea; CSM-RF is an ensemble learning method based on the bootstrap aggregating (bagging) idea; and the Feedback model is an improved collaborative filtering algorithm. LSTM is a recurrent neural network, and BayesNet is a probabilistic graphical model based on Bayesian theory. The comparison of prediction results is as follows: Figure 5 As shown in (a), (b), (c), (d), (e), (f), (g), and (h), the scatter points in the figure represent cases of prediction errors.

[0095] Depend on Figure 5 It can be seen that the number of prediction errors in the method of this invention is 19, while that of the adaBoost algorithm is 39, the catBoost algorithm is 50, the lightGBM algorithm is 25, the CSM-RF model is 303, the Feedback model is 321, the LSTM model is 287, and the BayesNet model is 336. This shows that the number of prediction errors in the Boosting-based machine learning method ( Figure 5 (a), (b), (c), (d)) generally have fewer prediction errors than statistical methods. Figure 5 (e), (f), (h)) and deep learning methods Figure 5 (g)). And weak learners ( Figure 5 (f) and (g) are difficult to capture the special patterns of high-dimensional features, which manifest as regularity in the prediction results on the image. The method of this invention has the fewest prediction errors, indicating that the prediction results of this invention are closer to the true values ​​than other reliable prediction methods.

[0096] As shown in Table 2, the method of this invention has a lower time consumption than most reliability prediction methods while maintaining advantages in accuracy, MSE, and R2-score. While the BayesNet model, AdaBoost algorithm, and CSM-RF model have shorter training times (0.70725s, 8.41611s, and 17.5930s, respectively), their low time consumption comes at the cost of lower accuracy (0.35009, 0.92456, and 0.41392). In summary, the method of this invention achieves the best balance between prediction accuracy and time efficiency, making it more suitable for practical applications.

[0097] The above analysis demonstrates that the method of this invention achieves both high accuracy and high time efficiency in predicting the reliability of service systems. This is primarily because the feature data contains special indicators strongly correlated with the reliability of service systems. These indicators are relatively stable under normal conditions but fluctuate significantly during anomalies, exhibiting a low false alarm rate. Furthermore, these indicators demonstrate a tree-like nonlinear relationship. The underlying XGBoost tree structure of this invention effectively adapts to this nonlinear relationship between features, and weighted cross-dimensionality reduction filters out feature data that might amplify model errors. Therefore, this invention has a significant advantage over other methods in predicting the reliability of service systems.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A service system reliability prediction method based on weighted cross-dimensionality reduction, characterized in that, Includes the following steps: S1. Obtain feature data of the training service system, the feature data including at least response time, success rate, and number of calls; calculate the system reliability value based on the response time, success rate, and number of calls; S2. Obtain basic features and randomly shuffle the data within the basic features to use as noise features; in the first iteration, use all feature data as basic features, otherwise use the dimensionality reduction features output in the previous iteration as basic features; merge the basic features and noise features horizontally to establish a feature cross window; S3. Based on feature filtering, reduce the dimensionality of the features within the feature cross window and output the dimensionality-reduced features; Calculate the importance of each feature in the dataset, and then perform dimensionality reduction by selecting features based on their importance, including selecting features with importance greater than the mean importance value using the following formula. As a result of dimensionality reduction: ； in, for The capacity; I i The importance of each feature; S4. Determine whether the dimensionality reduction margin meets the requirements based on the ratio of the dimensionality reduction feature to all feature data. If the requirements are met, the dimensionality reduction ends, the dimensionality reduction feature is output, and S5 is executed. Otherwise, return to S2. S5. The dimensionality reduction features are weighted and sampled using a gradient-based one-sided sampling algorithm to obtain the residual features of the dimensionality reduction features; S6. Use the dimensionality reduction features and the residual features as feature training data, and input the system reliability value as target training data into the XGBoost model for training. Use the trained XGBoost model to predict the reliability of the service system.

2. The service system reliability prediction method based on weighted cross-dimensionality reduction according to claim 1, characterized in that, The system reliability value is calculated based on the response time, success rate, and number of calls, including the calculation of the system reliability value according to the following formula: in, For response time, For the success rate, For the number of calls, This is a constant used to adjust the contribution of the response time metric to the reliability value calculation. It is the threshold for success rate.

3. The service system reliability prediction method based on weighted cross-dimensionality reduction according to claim 2, characterized in that, The system reliability value is calculated based on the response time, success rate, and number of calls. The system reliability value is also standardized and mapped to the range of 0-1.

4. The service system reliability prediction method based on weighted cross-dimensionality reduction according to claim 1, characterized in that, Dimensionality reduction of features within the feature cross-window based on feature filtering includes: Calculate the importance of each feature in the dataset; Features are selected based on their importance, and dimensionality reduction is performed to output the dimensionality-reduced features.

5. The service system reliability prediction method based on weighted cross-dimensionality reduction according to claim 4, characterized in that, Calculate the importance of each feature in the dataset, including calculating the importance of each feature according to the following formula: Among them, China For feature index, This is the feature importance calculation function for XGBoost. For feature cross window, This represents the system reliability value. This represents the number of iterations.

6. The service system reliability prediction method based on weighted cross-dimensionality reduction according to claim 1, characterized in that, The dimensionality-reduced features are weighted by a gradient-based one-sided sampling algorithm, including: The weights of the residual features are calculated based on the total number of features and the number of dimensionality-reduced features; Residual features are sampled from the dimensionality-reduced features according to this weight.

Images