Two-stage algorithm selection and hyper-parameter joint optimization method

Abstract

The invention discloses a two-stage algorithm selection and hyper-parameter joint optimization method, which comprises the following steps of: in the first stage, processing a training set and a test set through row sampling operation and column dimension reduction operation to form a reduced data set; randomly sampling a certain number of configurations in the hyper-parameter space of each candidate algorithm, evaluating the performance of each candidate algorithm by using the reduced data set, and extracting an optimal performance score; in the second stage, a previous algorithm is screened according to the optimal performance score to form a candidate set, and a pruned hyper-parameter search space is formed so as to reduce the calculation complexity of processor hyper-parameter search; and performing hyper-parameter optimization on the pruned hyper-parameter search space by using the original data set, and outputting an optimal algorithm adaptive to the target technical task and hyper-parameter configuration thereof. Algorithm screening and hyper-parameter tuning adaptive to a specific scene are realized through a two-stage optimization strategy, and meanwhile, the method is suitable for a traditional table type dichotomy task and aims at improving the deployment efficiency and performance of a machine learning model.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of automatic machine learning (AutoML) and microservice operation and maintenance, and specifically relates to an algorithm selection and hyperparameter joint optimization technology in a microservice performance fault management scenario, which can be extended and applied to the fields of data processing and machine learning model optimization of traditional table type binary classification tasks. BACKGROUND

[0002] With the wide application of microservice architecture, the academic community has carried out systematic and in-depth research on microservice systems, forming many research results including performance estimation methods, root cause positioning methods and fault classification methods, which provide certain support for the stable operation of microservices.

[0003] However, the dynamic nature and complexity of microservice systems determine that no algorithm and its hyperparameter combination can be suitable for all application scenarios (according to the no free lunch theorem). The hyperparameters of a specific algorithm are often only excellent on a specific dataset or task, and when migrated to a new microservice scenario or binary classification task, the algorithm often needs to be reselected and the hyperparameters optimized, otherwise it is difficult to achieve ideal results.

[0004] In practical applications, there are two major challenges in quickly finding a performance-optimal algorithm and its hyperparameters that adapt to a specific scenario. On the one hand, the actual dataset is large, and the evaluation of a single algorithm (especially deep learning algorithms) is time-consuming, resulting in low efficiency of algorithm selection and hyperparameter joint optimization. On the other hand, there are a large number of candidate algorithms and hyperparameters for the same problem, making the hyperparameter search space extremely large, further increasing the optimization difficulty.

[0005] To address the above challenges, existing technologies have proposed a variety of solutions, but there are also some deficiencies. For example, model-free methods (such as grid search, random search, and manual tuning) are less efficient in high-dimensional space. Bayesian optimization methods (such as SMAC and TPE) can balance exploration and utilization, but still have efficiency problems in high-dimensional space. Multi-fidelity optimization methods (such as continuous halving, Hyperband, and BOHB) improve efficiency through different fidelity evaluations, but do not completely solve the search space redundancy problem. Meta-heuristic methods (such as genetic algorithms and particle swarm optimization) are suitable for complex optimization problems, but their performance is limited when the hyperparameter space is large. Search space pruning methods reduce the search space through meta-learning, historical experimental results, or large language model prior knowledge, but the search efficiency and accuracy still need to be improved when facing large-scale hyperparameter search spaces.

[0006] Therefore, in order to address the problems of low efficiency and insufficient accuracy of existing methods in algorithm selection and hyperparameter joint optimization, there is an urgent need for a joint optimization technique that can efficiently compress the hyperparameter search space, shorten the algorithm evaluation time, and be compatible with microservice operation and maintenance as well as traditional binary classification tasks, so as to meet the needs of rapid deployment of high-performance machine learning models in practical applications. Summary of the Invention

[0007] The main objective of this invention is to provide a two-stage algorithm selection and hyperparameter joint optimization method for tasks such as response time prediction before microservice performance failures, root cause localization after failures, and failure classification. Through a two-stage optimization strategy, the method achieves algorithm selection and hyperparameter tuning adapted to specific scenarios. It is also applicable to traditional tabular binary classification tasks in fields such as medical diagnosis, financial risk control, and network security, aiming to improve the deployment efficiency and performance of machine learning models.

[0008] Based on the first main aspect of this invention, a two-stage algorithm selection and hyperparameter joint optimization method is provided for microservice performance failure response time prediction, root cause localization, or fault classification tasks, as well as algorithm selection and hyperparameter joint optimization for traditional tabular binary classification tasks. This method is executed by the processor of a computer system, and the original dataset is input into the computer system. Original dataset Including training set and test set Candidate Algorithm Set Hyperparameter search space for each algorithm and configuration parameters;

[0009] This method includes one or a combination of the following two stages:

[0010] Phase 1: Row sampling operation Dimensionality reduction operation Processing the training set and test set To form a reduced dataset A certain number of configurations are randomly sampled from the hyperparameter space of each candidate algorithm. The performance of each candidate algorithm is evaluated using a reduced dataset, and the optimal performance score is extracted. ;

[0011] Phase Two: Based on the best performance score Before screening The algorithm constitutes a candidate set. This forms the hyperparameter search space after pruning. To reduce the computational complexity of processor hyperparameter search; in the pruned hyperparameter search space The hyperparameter optimization is performed on the original dataset, and the optimal algorithm suitable for the target technical task is output and the hyperparameter configuration thereof .

[0012] As a further preferred solution, in the foregoing method, the configuration parameters at least include one or a combination of the following parameters: row sampling method , sampling ratio , column dimension reduction method , feature retention ratio , hyperparameter sampling number , and number of retained candidate algorithms .

[0013] As a further preferred solution, in the foregoing method, the random sampling of a certain number of configurations in the hyperparameter space of each candidate algorithm includes:

[0014] randomly sampling configurations in the hyperparameter configuration space of each candidate algorithm :

[0015]

[0016] wherein, denotes the th algorithm and its hyperparameter search space .

[0017] As a further preferred solution, in the foregoing method, the performance evaluation of each candidate algorithm using the reduced dataset includes:

[0018] using the reduced dataset to evaluate the sampled candidate configurations to obtain a performance score set of each algorithm on the reduced dataset, and using the set as an approximate performance of the algorithm on the target dataset :

[0019]

[0020] extracting the optimal performance score of each algorithm to depict the maximum potential of each algorithm:

[0021]

[0022] wherein, the optimal performance score serves as a performance benchmark for subsequent algorithm selection, denotes the number of candidate algorithms.

[0023] As a further preferred solution, in the aforementioned method, in the second stage, based on the optimal performance score of the evaluation result of the first stage , the top performing algorithms are selected to form a candidate set:

[0024]

[0025] wherein, ;

[0026] and the pruned search space is defined as:

[0027]

[0028] The hyperparameter optimization is performed on the pruned search space using the original dataset, and the optimization objective is defined as:

[0029]

[0030] wherein, (P represents the optimal algorithm and its hyperparameter configuration found within the pruned search space.

[0031] As a further preferred solution, in the aforementioned method, the row sampling method is selected from Latin hypercube sampling, random sampling or sampling;

[0032] The sampling ratio is 0.1-0.8, and the data sample size after row sampling is not less than 300 and not more than 3000;

[0033] The row sampling operation maintains the distribution characteristics of the original technical data, and adapts the single data processing throughput of the computer processor, wherein the sampling ratio of random sampling on 28 classification datasets is set to 0.3 by default, and the precision Pearson correlation coefficient between the reduced dataset and the original dataset under this sampling ratio reaches 0.82.

[0034] As a further preferred solution, in the aforementioned method, the column dimension reduction method is selected from , , mutual information estimation, principal component analysis or random forest, and the feature retention ratio is 0.1-0.8, which reduces the dimension by retaining the key features of the technical data to reduce the computer memory occupation and data transmission volume.

[0035] As a further preferred solution, in the aforementioned method, the hyperparameter pruning is performed by selecting the candidate algorithms with better performance in the first stage evaluation, and constructing a reduced configuration space accordingly, so as to efficiently compress the hyperparameter search space.

[0036] The execution process inputs into the computer system: the original configuration space , the optimal performance score of the candidate algorithms in the previous stage , and the number of optimal candidate algorithms to be retained .

[0037] When the computer system performs the hyperparameter pruning, the scores of the candidate algorithms are first paired with the performance scores obtained in the previous stage, and then sorted in descending order of performance, so as to screen out the candidate algorithms with excellent performance; subsequently, the top candidate algorithms are selected to form a pruned candidate algorithm list, and the computer system outputs a reduced configuration space , so that the subsequent hyperparameter optimization stage focuses on the candidate algorithms that have been verified on the reduced dataset.

[0038] Based on the second main aspect of the present application, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;

[0039] The memory stores a computer program, which, when executed by the processor, causes the processor to perform the aforementioned two-stage algorithm selection and hyperparameter joint optimization method.

[0040] Based on the third main aspect of the present application, a computer readable storage medium is provided, which stores a computer program that, when executed, implements the aforementioned two-stage algorithm selection and hyperparameter joint optimization method.

[0041] Compared with the prior art, the present application has the advantages of balancing optimization efficiency and saving of computing resources, and greatly reducing the time cost of algorithm selection and hyperparameter joint optimization. By constructing a reduced dataset, the training time of the algorithm on the proxy dataset is only 1.6% of that on the original dataset, effectively solving the problem of long evaluation time of algorithms on large-scale data. At the same time, the hyperparameter pruning step significantly compresses the search space by retaining the top k high-quality algorithms, thereby reducing the computational complexity of the subsequent optimization and adapting to the computing power of computer hardware.

[0042] The application ensures the stability and reliability of model performance while guaranteeing optimization efficiency. Experimental verification shows that the precision Pearson correlation coefficient of the reduced data set and the original data set reaches 0.82, and the algorithm screening based on the reduced data set can accurately capture the performance potential of high-quality algorithms. On 2 microservice data sets and 28 traditional binary classification data sets, the ROC AUC and accuracy of the application are better than the baseline methods such as Auto-CASH and Bayesian optimization in most scenarios, and the accuracy on 25 data sets is optimal, which fully proves that the two-stage strategy does not sacrifice accuracy, but improves optimization pertinence by eliminating invalid algorithms.

[0043] In addition, the application has strong universality and practicality, and can be adapted to specific tasks such as microservice fault prediction and root cause positioning, as well as traditional binary classification tasks in the medical and financial fields without relying on historical task information or prior knowledge. It supports flexible combination of various row sampling, column dimension reduction methods and hyperparameter optimization algorithms, can be dynamically adjusted and configured according to different data set sizes and task types, and the operation process does not require too much human intervention, which can assist operation personnel or data processing personnel to quickly deploy high-performance models, thereby reducing the actual application threshold. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only some embodiments of the application, and for those skilled in the art, according to these drawings to obtain other drawings without creative labor still belongs to the scope of the application.

[0045] Figure 1 An execution flowchart of a two-stage algorithm selection and hyperparameter joint optimization method in an embodiment of the application is shown.

[0046] Figure 2 A system architecture diagram in an embodiment of the application is shown. DETAILED DESCRIPTION

[0047] The preferred embodiments of the application will be described in detail below in order to more clearly understand the purpose, characteristics and advantages of the application. It should be understood that the following embodiments are not a limitation on the scope of the application, but only to illustrate the essence of the technical scheme of the application.

[0048] In the following description, for purposes of explanation, specific details are set forth in order to provide a thorough understanding of various disclosed embodiments. It will be appreciated, however, that embodiments can be practiced in other ways without one or more of the specific details set forth in this description. In other instances, well-known enclosures, structures, techniques, and applications are not re-described in detail in order to avoid unnecessarily obscuring the description.

[0049] Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0050] In one possible embodiment, the present application proposes a two-stage algorithm selection and hyperparameter joint optimization method (which can be referred to as TS-CASH). The first stage performs algorithm performance evaluation on the reduced dataset, uses the reduced dataset to quickly evaluate the precision distribution of the candidate algorithm on the target dataset, and then selects the candidate algorithm with excellent performance; the second stage is hyperparameter pruning and hyperparameter optimization. In view of the significant linear relationship between the algorithm accuracy on the reduced dataset and the complete dataset, the candidate algorithm with poor performance is directly removed, and in order to avoid mistakenly deleting the candidate algorithm with better performance, only the top

[0051] As shown in Figure 1 , the overall concept of the two-stage algorithm of the present application is as follows:

[0052] The first stage: the training set and are processed through row sampling operation and column dimension reduction operation to form a reduced dataset; a certain number of configurations are randomly sampled in the hyperparameter space of each candidate algorithm, the performance of each candidate algorithm is evaluated using the reduced dataset, and the optimal performance score is extracted;

[0053] The second stage: according to the optimal performance score, the top certain number of algorithms are selected to form a candidate set, and a pruned hyperparameter search space is formed to reduce the computational complexity of processor hyperparameter search; the original dataset is used to optimize the hyperparameters in the pruned hyperparameter search space, and the optimal algorithm and its hyperparameter configuration suitable for the target technical task are output.

[0054] ​The method of the present embodiment refers to the existing CASH method, which is a core technology in the field of automatic machine learning (AutoML) and aims to automatically complete the selection of machine learning algorithms and the tuning of hyperparameters using a unified framework. The core goal of this technology is to automatically identify the most suitable algorithm from a predefined set of candidate algorithms on a given dataset and simultaneously determine its optimal hyperparameter configuration to minimize the loss function, reduce human intervention, and improve the efficiency of the machine learning process.

[0055] The CASH problem can be defined as follows: given a dataset , it is first divided into a training set and a test set . The optimization goal of CASH is to find the optimal algorithm and its corresponding optimal hyperparameter configuration on the training set so that the expected loss on the test set reaches a minimum value:

[0056]

[0057] where denotes the set of candidate algorithms, denotes the hyperparameter search space of algorithm , denotes the algorithm with hyperparameter configuration , denotes the loss value of algorithm on the test set when trained on the training set , and the hyperparameter configuration of the algorithm is . is the optimal solution found under resource constraints, which may be a local optimal solution or there may be multiple solutions. The solution process of this optimization problem needs to search in the discrete space of algorithm selection and the continuous or mixed space of hyperparameter optimization, and the joint search of multiple algorithms makes the search space of hyperparameters too large, which makes the CASH problem a challenging combinatorial optimization problem.

[0058] The improved method in this embodiment employs a two-stage optimization strategy to address the CASH problem. The first stage involves a rapid evaluation of the reduced dataset, comprehensively examining the performance of all candidate algorithms and selecting those that perform best in the given task scenario. The second stage focuses on these selected algorithms and performs hyperparameter optimization on the original dataset to obtain the optimal algorithm and parameter combination. This two-stage process not only improves the efficiency of algorithm selection and parameter tuning but also balances performance and computational resource consumption, ensuring the method has strong generalization ability.

[0059] based on Figure 2 The system framework shown below will be used to explain in detail the specific implementation process of the two stages mentioned above in one possible implementation method.

[0060] Phase 1: Algorithm performance evaluation based on reduced dataset.

[0061] To efficiently evaluate the performance of a large number of candidate algorithms, this embodiment first constructs a computationally efficient dataset proxy (reduced dataset) to lower the cost of algorithm evaluation. The original dataset ( The dataset was transformed into a reduced dataset using row sampling and column dimensionality reduction techniques.

[0062]

[0063] in, Indicates row sampling operation. Using sampling methods (e.g., random sampling or stratified sampling) by sampling ratio Select training samples. The goal is to minimize the number of samples while preserving the distribution characteristics of the dataset. For example, when For random sampling and When, it indicates that a random sampling method is used in the dataset. 10% of the samples were randomly selected. This indicates a column dimensionality reduction operation. Feature processing method (e.g., principal component analysis or feature selection based on random forest importance), and retain the proportion of resulting features. The goal is to minimize the number of features while preserving the dataset's characteristic information. Through row sampling and column dimensionality reduction, the dataset size is significantly reduced, resulting in a shrunken dataset, which in turn lowers the cost of pruning.

[0064] To evaluate each candidate algorithm Performance on reduced datasets requires consideration of its hyperparameter configuration space. Random sampling One configuration:

[0065]

[0066] where, denotes the i-th algorithm in the candidate algorithm space and its hyperparameter search space . Then, the sampled candidate configurations are evaluated on a reduced dataset to obtain a set of performance scores of each algorithm on this reduced dataset, which is used as an approximation of the performance of the algorithm on the target dataset :

[0067]

[0068] To characterize the maximum potential of each algorithm, the optimal performance score of each algorithm is extracted:

[0069]

[0070] where the optimal performance score serves as the performance benchmark for the subsequent algorithm selection, denotes the number of candidate algorithms.

[0071] Stage 2: Hyperparameter space pruning and optimization.

[0072] Based on the optimal performance scores from the evaluation results of the first stage, the top algorithms with the highest performance are selected to form the candidate set:

[0073]

[0074] where, . The pruned search space is defined as:

[0075]

[0076] Finally, hyperparameter optimization is performed on the pruned search space using the original dataset, and the optimization objective of the present embodiment is defined as:

[0077]

[0078] where, denotes the optimal algorithm and its hyperparameter configuration found within the pruned search space. Through this two-stage strategy, the method of the present embodiment can improve the accuracy of optimization while reducing the hyperparameter search space.

[0079] In the above scheme, in the first stage, the present application constructs a reduced proxy dataset by row sampling and column dimensionality reduction techniques , and evaluates the potential performance of each candidate algorithm on this dataset . This enables efficient exploration of candidate algorithm performance on the target dataset even in resource-constrained (e.g. insufficient computing power) situations. The optimal performance score is a vector containing the optimal performance of all candidate algorithms on the reduced dataset. In the subsequent stage, is used as the basis for hyperparameter pruning.

[0080] In the second stage, the present application performs hyperparameter pruning based on the performance of the candidate algorithms obtained in the first stage, i.e. the optimal performance score , to eliminate poorly performing candidate algorithms, and then performs hyperparameter optimization only in the reduced space containing the most promising candidate algorithms and their corresponding hyperparameters. To avoid pruning potentially excellent candidate algorithms, only the top candidate algorithms are retained to continue participating in subsequent optimization.

[0081] In constructing the algorithm of the present application, the idea of the first stage is to use a reduced dataset to prune the hyperparameter search space, improving pruning efficiency. The second stage performs hyperparameter optimization only in the most promising candidate algorithms, improving optimization efficiency.

[0082] In this embodiment, hyperparameter pruning removes poorly performing candidate algorithms, thereby reducing the search space. Hyperparameter optimization is only performed in the pruned hyperparameter space. Because the search space after pruning includes candidate algorithms, the candidate algorithms themselves are treated as a classification hyperparameter during hyperparameter optimization, allowing hyperparameter optimization to be performed on multiple candidate algorithms.

[0083] The following gives a specific example of a two-stage hyperparameter optimization algorithm executed by the processor of a computer system.

[0084] Input: original dataset , hyperparameter search space , row sampling algorithm and sampling ratio , feature processing method and ratio , number of algorithm searches , number of candidate algorithms

[0085] Output: optimal algorithm and hyperparameter configuration

[0086] 1: / / Stage 1: algorithm performance evaluation

[0087] 2:

[0088] 3:

[0089] 4: / / Phase 2: Hyperparameter pruning and optimization

[0090] 5:

[0091] 6: return( )

[0092] In the following possible embodiment, the algorithm performance evaluation process based on the reduced dataset is described.

[0093] The goal of this embodiment is to quickly identify candidate algorithms that perform well on the target dataset using lower cost, providing a basis for subsequent hyperparameter pruning. The specific steps include dataset reduction and candidate algorithm evaluation.

[0094] Data reduction reduces the computational overhead during training by compressing the original dataset, mainly including sample reduction and feature reduction. Among them, sample reduction reduces the number of samples by means of sampling technology, so that the reduced dataset can still reflect the distribution characteristics of the original data; the purpose of feature reduction is to reduce the dimension (dimension is the number of features) of the feature, and common feature dimension reduction methods include principal component analysis, autoencoder, etc. For example, the importance score of each feature can be calculated using the random forest algorithm, and the top features with the highest score are retained. The basis of this reduction strategy is that there is a significant linear relationship between the accuracy of the candidate algorithm on the reduced small-scale dataset and the accuracy on the full-scale dataset.

[0095] The goal of candidate algorithm evaluation is to evaluate each candidate algorithm multiple times on the reduced dataset through random search, so as to characterize the accuracy performance of the candidate algorithm on the target dataset and provide a basis for the hyperparameter pruning in the second phase. The algorithm first traverses each candidate algorithm in the configuration space, and randomly samples in the hyperparameter space of the candidate algorithm. For each algorithm in the candidate space, sample configurations; then, evaluate the loss of each configuration on the reduced dataset, and select the best-performing candidate algorithm according to the loss. The loss refers to an index that can quantify the performance of the model, such as precision or F1 score, etc. Through this strategy, the invention can estimate the performance upper bound of each candidate algorithm with relatively low computational overhead, providing a reliable performance ranking basis for the subsequent hyperparameter pruning step, so as to realize an efficient hyperparameter search space pruning strategy.

[0096] To realize the above process, one possible algorithm implementation is as follows.

[0097] Input: hyperparameter search space , reduced dataset , number of algorithm search times .

[0098] Output: optimal performance score of each candidate algorithm

[0099] 1:

[0100] 2: for m = 1 to do

[0101] 3:

[0102] 4:

[0103] 5: for

[0104] 6:

[0105] 7:

[0106] 8: end for

[0107] 9:

[0108] 10: end for

[0109] 11: return

[0110] where is the empty set symbol, i.e., a set containing no elements.

[0111] On the above basis, the following is one possible implementation to further illustrate hyperparameter pruning and optimization.

[0112] This embodiment is based on the performance of the candidate algorithm, i.e., the optimal performance score , first prunes the hyperparameter search space , and then optimizes the hyperparameters on the reduced . By eliminating poor-performing candidate algorithms in advance, the number of hyperparameters to be optimized is significantly reduced. Specifically, this phase includes two aspects of hyperparameter pruning and hyperparameter optimization.

[0113] The hyperparameter pruning is a key step in the method of the present application. It selects the candidate algorithms with better performance in the first stage of evaluation, and constructs a reduced configuration space accordingly, so as to efficiently compress the hyperparameter search space. The inputs of the algorithm include the original configuration space , the optimal performance score of the candidate algorithm in the previous stage , and the number of optimal candidate algorithms to be reserved . The algorithm is executed by the processor of a computer system. First, the scores of the candidate algorithms are paired with the performance scores obtained in the previous stage, and then sorted in descending order of performance to select the candidate algorithms with excellent performance. Subsequently, the top candidate algorithms are selected to form a pruned candidate algorithm list, so as to exclude the candidate algorithms with poor performance from the subsequent hyperparameter optimization process. The top candidate algorithms are a compromise strategy that maximizes the discovery of high-quality solutions while ensuring search efficiency. Finally, the algorithm outputs a reduced configuration space , so that the subsequent hyperparameter optimization stage focuses on candidate algorithms that have shown better performance on the reduced dataset. This pruning strategy significantly reduces the computational overhead of the subsequent hyperparameter optimization while still retaining the ability to discover high-precision candidate algorithms.

[0114] The goal of hyperparameter optimization is to determine the optimal model and its corresponding hyperparameters in the pruned space . The present application uses the standard Bayesian optimization (BO) method, uses Gaussian process (GP) regression as a proxy model, and uses upper confidence bound (UCB) as the acquisition function. The reason for choosing Bayesian optimization is that its performance is more outstanding when the hyperparameter space is small, and is the pruned hyperparameter search space, and the size of the space is just small. The implementation process of this embodiment can choose a variety of different algorithms. When optimizing, the present application regards the algorithm itself as a classification hyperparameter, so multiple algorithms and their hyperparameters can be optimized simultaneously. In addition, other hyperparameter optimization algorithms can also be used on , such as evolutionary algorithms.

[0115] Overall, the present application has the following advantages. First, the calculation efficiency of the pruning process is high. By performing initial evaluation on the reduced dataset, the calculation time is significantly reduced. Second, by evaluating the candidate algorithms on the reduced dataset and pruning, the search space of the hyperparameters can be greatly reduced. Finally, the present application has strong versatility because the method does not rely on historical task information and is suitable for tasks without prior knowledge.

[0116] The following provides an embodiment of a possible hyperparameter pruning algorithm.

[0117] Input: hyperparameter search space The best performance score of the candidate algorithm on the reduced dataset Number of candidate algorithms

[0118] Output: Pruned hyperparameter search space

[0119] 1:

[0120] 2: for m=1 to do

[0121] 3:

[0122] 4: end for

[0123] 5:

[0124] 6:

[0125] 7: for i=1 to do

[0126] 8:

[0127] 9:

[0128] 10: end for

[0129] 11:

[0130] 12: return

[0131] The time complexity of the present invention is analyzed in one of the following possible implementations.

[0132] The time complexity of this invention is primarily addressed in the first stage, which evaluates all candidate algorithms on a reduced dataset. Assume there are M candidate algorithms, and the hyperparameter space of each candidate algorithm is randomly sampled. If there are 1 configuration, then the complexity of the first stage is 1. ,in This represents the complexity of a single model training and evaluation run on a reduced dataset. Since the reduced dataset is much smaller than the original dataset, It is significantly less complex than training on the full dataset.

[0133] The experimental results on 28 classification datasets show that the training time of the algorithm on the reduced dataset (9±5 minutes) is only 1.6% of the training time on the original dataset (554±693 minutes). The second stage only includes hyperparameter pruning, which is a process of removing poorly performing candidate algorithms in the hyperparameter space. The computational cost of this process is much smaller than the training process, so it can be ignored.

[0134] In the following possible embodiments, the performance of the TS-CASH method of the present application is evaluated.

[0135] First, the precision performance of the present application and the baseline method is evaluated, then the significance of the performance improvement is verified by statistical test, and finally the time overhead introduced by the present application is analyzed. In this embodiment, the present application and four benchmark methods are compared and analyzed in terms of ROC AUC and Accuracy. The specific parameter settings are as follows: is set to 2, indicating that only 2 algorithms are retained after pruning. The row sampling method is set to random sampling, and the row sampling ratio is set to The feature dimension reduction method is set to random forest RF, and the feature retention ratio is set to 0.3. The resource constraint for each method is set to 100 iterations. For traditional binary classification datasets, the accuracy value is the average of five cross-validation. For the two microservice datasets, this embodiment refers to existing work and reports the single evaluation result on the test set.

[0136] In order to verify the effectiveness of TS-CASH in the microservice scenario, this embodiment evaluates TS-CASH on two real microservice datasets D1 and D2. Table 1 shows the comparison results of TS-CASH and baseline methods in terms of precision, recall and F1 score. The precision, recall and F1 score are the values obtained after optimization with these three indicators as optimization objectives.

[0137] The experimental results show that TS-CASH performs well on both microservice datasets. On dataset D1, TS-CASH achieves a precision of 0.9577, a recall of 0.9524, and an F1 score of 0.9526, indicating that TS-CASH outperforms other baseline methods. Among them, Bayesian optimization (BO) is the second best method, with an F1 score of 0.9522, lower than TS-CASH. Random search (RS) performs relatively poorly, with an F1 score of only 0.6855. AutoMHS-GPT performs poorly on this dataset, with an F1 score of only 0.3485, which may be due to the lack of adaptability of its algorithm recommendation strategy in the microservice scenario.

[0138] Table 1 Precision of TS-CASH on 2 microservice datasets

[0139]

[0140] On dataset D2, TS-CASH also maintains good performance with precision 0.7775, recall 0.7593, and F1 score 0.7375. Although the overall performance is slightly lower than that on D1, it is still better than other baseline methods. The F1 score of Bayesian optimization on this dataset is 0.7202, that of random search is 0.6871, and that of AutoMHS-GPT is only 0.1532.

[0141] The results of Auto-CASH are empty because this method relies on metadata to select appropriate algorithms. This embodiment only conducts experiments on two microservice datasets, and these two data do not provide enough metadata, resulting in Auto-CASH being unable to complete the task of algorithm selection, thus not producing effective results on the two datasets.

[0142] The experimental results based on microservice datasets verify the effectiveness of TS-CASH. Compared with general binary classification datasets, microservice datasets have stronger time series and more complex feature distribution, which puts higher requirements on algorithm selection and hyperparameter optimization. TS-CASH can still maintain excellent performance in such a complex environment through its dataset agent mechanism and efficient algorithm pruning strategy.

[0143] In the following possible embodiments, the precision of the TS-CASH method of the present application on 28 classification datasets is described.

[0144] The experimental results show that the ROC AUC of the present application on 21 of the 28 datasets is better than that of the baseline method. For accuracy, the present application is better than the baseline method on 25 of the 28 datasets. This demonstrates that the present application can find the optimal algorithm in most cases. Taking ROC AUC as an example, Auto-CASH only performs better on 2 datasets.

[0145] By analyzing the algorithm and hyperparameter selection results of Auto-CASH, the results show that Auto-CASH tends to select MLP and RF algorithms. In all experiments, the selection ratio of MLP and RF is 60% and 25% respectively. In addition, the results show that Auto-CASH usually only optimizes 1-3 key hyperparameters of the algorithm. The performance of Auto-CASH depends on the characteristics of the target data. If the meta-features of the target data cannot infer the optimal algorithm and hyperparameters, the performance of Auto-CASH will be unsatisfactory.

[0146] For AutoMHS-GPT, it tends to recommend Random Forest and ExtraTrees algorithms on all datasets, with a proportion of 96% and 4%, which is the main reason for the poor performance of AutoMHS-GPT. Random search and Bayesian optimization are only superior to the present invention in a few experiments, mainly because the hyperparameter space is large.

[0147] From the above analysis, it can be seen that the present invention does not obtain the optimal result on all datasets, but this is also expected because no single algorithm can perform well in all cases.

[0148] The present embodiment statistically analyzes the accuracy of the present invention and Auto-CASH on 28 standard classification datasets. The results show that the average difference between the two is 0.021, and the 95% confidence interval is ; Since the interval contains 0, it means that there is no significant improvement in overall performance. However, in terms of ROC-AUC, the present invention outperforms Auto-CASH and other baseline methods in 21 out of 28 experiments; in terms of Accuracy, the present invention exceeds existing baseline algorithms in 25 experiments, highlighting the robustness and relative advantage of the present invention.

[0149] The following embodiment explains the time overhead introduced by the present invention.

[0150] The increased time overhead of the present invention mainly includes row sampling, column dimension reduction and model pre-evaluation. Since the calculation time on small-scale datasets is extremely short, it is difficult to reflect the difference, so this embodiment focuses on the running time of medium-scale and large-scale datasets. The unit of running time is minutes.

[0151] Table 2 Running time of TS-CASH

[0152]

[0153] This embodiment compares the present application with BO, as it can also reflect the time overhead introduced by pruning. The experimental results are shown in Table 2. On large datasets, the total training time of the present application is 231.49 minutes, of which the pruning process takes 14.64 minutes, an increase of about 6.75% compared with the baseline BO (216.85 minutes); on medium-sized datasets, the total training time of the present application is 223.46 minutes, and the pruning takes an additional 8.53 minutes, an increase of about 3.97% compared with BO (214.93 minutes). Overall, the time overhead of the present application compared with BO increases between 4% and 7%, and as the size of the dataset further expands, the ratio remains below 7%. This is because the maximum sample size is limited in the sampling and dimensionality reduction stages, effectively suppressing the growth of time overhead.

[0154] The following embodiment discusses the impact of the data reduction method on the present application (RQ2).

[0155] This embodiment mainly discusses the impact of the row sampling method and sampling rate, the column dimensionality reduction method and feature retention rate on the accuracy of the present application.

[0156] Impact of row sampling method and sampling rate on the present application: This embodiment analyzes the impact of the data sampling algorithm and sampling rate in the first stage on the accuracy of the present application. The reason for introducing the sampling algorithm is to reduce the training dataset at the row level through the row sampling method, thereby reducing the time required for pruning. The amount of data is reflected by the sampling rate. This embodiment selects three sampling algorithms: Random, LHS and Sobol. The sampling rate of each algorithm is set to { , where corresponds to the case without sampling, as the baseline for comparison. is set to 2, indicating that the algorithm retains the two best-performing algorithms during pruning. To avoid the impact of column dimensionality reduction, dimensionality reduction is disabled in the experiment.

[0157] The experimental results show that the sampling method and sampling rate affect the performance of the present application. As the sampling rate increases from 0.1 to 0.5, the ROC AUC of the present application gradually improves when using RS and Sobol, but no such relationship is found when using LHS. The main reason may be that the strategy of LHS is to achieve uniform coverage in the parameter space, but at a low sampling rate, it may not be able to fully represent the diversity of the data. In contrast, RS and Sobol are global exploration methods that can maintain a certain representativeness even at a lower sampling rate. In terms of sampling methods, Sobol performs best.

[0158] Impact of column dimension reduction method and feature retention rate on the accuracy of the present invention: This embodiment analyzes the impact of the column dimension reduction method and the feature retention rate on the accuracy of the present invention. The reason for introducing the dimension reduction method is to reduce the training dataset from the feature level, thereby shortening the time required for pruning. The feature retention rate reflects the degree of retained features. This embodiment considers five dimension reduction algorithms, ParamRepulsor, DensMAP, MIC, PCA, and RF, and sets the feature retention rate for each algorithm to { , where indicates that all features are retained. The parameter is set to 2, indicating that the algorithm retains the top two algorithms in performance during the pruning process. This embodiment disables row sampling to avoid its impact.

[0159] The experimental results show that the traditional PCA method is superior to other algorithms. Regarding the feature retention rate of PCA and RF, the experimental results show that retaining only 30% of the features can achieve an accuracy very close to that using the original dataset.

[0160] The following embodiment illustrates the impact of k on the accuracy of the present invention (RQ3).

[0161] In hyperparameter pruning, the top algorithms determine the size of the hyperparameter space. As increases, the hyperparameter search space also expands. For example, indicates that during the hyperparameter pruning process, the experiment only retains the algorithm with the highest accuracy and its hyperparameters; whereas means that the experiment retains all algorithms and their hyperparameters. To explore the impact of on the accuracy of the present invention, this embodiment designs an experiment to observe the relationship between and the accuracy of the present invention. The number of iterations of the present invention is set to 4,000 times, and the improvement in accuracy performance shows a diminishing marginal utility trend.

[0162] The results show that as increases, the accuracy of the present invention first increases and then decreases. When changes from 1 to 2, both the accuracy and the ROC AUC improve in performance. This indicates that selecting only 1 algorithm cannot identify the best-performing algorithm in the real situation. For accuracy, performance starts to decline when , which means that selecting 2-4 algorithms out of the 14 available algorithms is sufficient. For ROC AUC, performance starts to decline when , which means that selecting 2 algorithms out of the 14 available algorithms for ROC AUC is sufficient. In other words, as The increase in increases the likelihood of selecting important hyperparameters. However, when At that time, whether in terms of accuracy or ROC AUC, the expansion of the hyperparameter space leads to an increase in the number of hyperparameters that need to be optimized, resulting in increased resource consumption and a gradual decrease in accuracy. Using all hyperparameters ( When the accuracy is even lower, the results show that it is sufficient to retain only 2-4 algorithms and their hyperparameters out of the 14 algorithms, without optimizing all 14 algorithms. Considering the trade-off between algorithm accuracy and training time, selecting two algorithms as candidate solutions achieves the highest accuracy.

[0163] The following example illustrates the effect of k on the number of hyperparameters (RQ4).

[0164] Experiments show that the number of hyperparameters increases with... The increase shows a linear growth trend. This positive correlation indicates that... The choice of values ​​directly affects the complexity of the algorithm's configuration space. Therefore, smaller values... A smaller hyperparameter space makes the optimization process more efficient and easier to converge, while a larger one... This results in a larger hyperparameter space, increasing the complexity and computational cost of the search.

[0165] Specifically, when using When the hyperparameter count is 2, the average number of hyperparameters is 20, which provides a good foundation for rapid prototype verification and preliminary experiments. With... As the value gradually increases, the number of hyperparameters exhibits a stable growth pattern. When =14, there are a total of 84 hyperparameters, compared to With a configuration of 2, the hyperparameter space increases by approximately 4.2 times. This significant increase implies a larger search space and higher computational complexity.

[0166] This embodiment mainly focuses on The configuration is set to 2. This choice is not only based on computational efficiency considerations, but more importantly, it achieves effective compression of the hyperparameter space while ensuring algorithm performance. By adopting... =2, compared to With a complete configuration space of 14, the hyperparameter space is reduced by 76%, and this significant dimensionality reduction greatly improves the efficiency of hyperparameter optimization.

[0167] The following example illustrates the impact of resource constraints on the performance of the present invention (RQ5).

[0168] The resource constraint (RC) referred to in this embodiment refers to the number of iterative optimizations performed by the algorithm under a given search space and dataset. The more iterations, the more resources available to find the optimal solution; however, a larger RC also means additional computational resources are required. To explore the impact of different RCs on the accuracy of the present application and BO, this section conducts experiments on a small dataset group and sets 10 different values. The values are:

[0169]

[0170] These values correspond to different resource constraint scenarios. At the same time, this embodiment also gives the accuracy performance of BO under the same conditions as a baseline for comparison.

[0171] The results show that under different resource constraints and accuracy indicators, the accuracy of the present application is always better than that of BO. This shows that the present application can improve accuracy under various resource constraints. In addition, in a search space containing 84 hyperparameters, random search requires more than 4500 iterations to achieve the best global accuracy, because when the algorithm accuracy tends to be stable from 4500 to 5000.

[0172] In the above embodiments, the technical terms, technical principles or technical means related to the technical solutions of the present application are not described in detail in the above content, which are all known to those skilled in the art. Technical or conventional means.

[0173] The above shows and describes the basic principles and main features of the present application and the advantages of the present application. Those skilled in the art should understand that the present application is not limited by the above embodiments, and the above embodiments and descriptions in the specification are only to illustrate the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A two-stage algorithm selection and hyperparameter joint optimization method for algorithm selection and hyperparameter joint optimization of microservice performance fault response time prediction, root cause localization or fault classification tasks, and traditional table type binary classification tasks, characterized in that, The method is performed by a processor of a computer system and inputs an original dataset into the computer system The original dataset includes a training set and a test set , a candidate algorithm set , a hyperparameter search space corresponding to each algorithm , and configuration parameters The method comprises one of the following two stages or a combination thereof: First stage: by row sampling operation and column dimensionality reduction operation Processing training set and test set , forming reduced data set ; randomly sampling a certain number of configurations in the hyperparameter space of each candidate algorithm, evaluating the performance of each candidate algorithm using the reduced data set, and extracting the optimal performance score ; Second stage: according to the optimal performance score Before screening Individual algorithm constitutes a candidate set , form a pruned hyperparameter search space To reduce the computational complexity of processor hyperparameter search; in the pruned hyperparameter search space Hyperparameter optimization is carried out using the original data set, and the optimal algorithm and hyperparameter configuration suitable for the target technical task are output .​ 2. The two-stage algorithm selection and hyperparameter joint optimization method according to claim 1, characterized in that, The configuration parameters at least include one or a combination of the following parameters: a row sampling method , a sampling ratio , a column dimension reduction method , a feature retention ratio , a hyperparameter sampling number , and a candidate algorithm retention number . 3.The two-stage algorithm selection and hyper-parameter joint optimization method of claim 1, wherein, The random sampling of a certain number of configurations in the hyperparameter space of each candidate algorithm comprises: In each candidate algorithm hyperparameter configuration space randomly sample configurations: ； wherein, represents the candidate algorithm space of the algorithm and its hyperparameter search space .

4. The two-stage algorithm selection and hyperparameter joint optimization method according to claim 3, characterized in that, The performance evaluation of each candidate algorithm using the reduced dataset comprises: Using a reduced data set Evaluating sampled candidate configurations , obtaining a set of performance scores for each algorithm on the reduced data set, using the set as an approximation of the algorithm's performance on the target data set ​ ； The optimal performance score of each algorithm is extracted to depict the maximum potential of each algorithm: ； wherein the optimal performance score as a performance benchmark for the selection of subsequent algorithms, denotes the number of candidate algorithms.

5. The two-stage algorithm selection and hyperparameter joint optimization method according to claim 4, characterized in that, In the second phase, the best performance score is based on the evaluation results of the first phase. Choose the highest performance front The algorithm constitutes a candidate set: ； wherein ; And the pruned search space is defined as: ； pruned search space Hyperparameter optimization was performed using the original dataset, with the optimization objective defined as: ； wherein, represents the optimal algorithm found within the pruned search space and its hyperparameter configuration.

6. The two-stage algorithm selection and hyperparameter joint optimization method according to claim 1, characterized in that, The row sampling method selected from Latin hypercube sampling, random sampling, or Sampling; The sampling ratio The value is 0.1-0.8, the data sample quantity after line sampling is not less than 300 and not more than 3000; The row sampling operation By keeping the distribution characteristics of the original technical data, the single data processing throughput of the adaptive computer processor is adapted, wherein the sampling proportion of random sampling on the 28 classification data sets is set to 0.3 by default, and the precision Pearson correlation coefficient between the reduced data set and the original data set reaches 0.82 under the sampling proportion.

7. The two-stage algorithm selection and hyperparameter joint optimization method according to claim 1, characterized in that, The column dimension reduction method selected from , , mutual information estimation, principal component analysis or random forest, feature retention ratio The value is 0.1-0.8, reduce the dimension by retaining the key features of technical data to reduce computer memory occupation and data transmission.

8. The two-stage algorithm selection and hyperparameter joint optimization method according to claim 1, characterized in that, The hyperparameter pruning compresses the hyperparameter search space efficiently by selecting the candidate algorithm with better performance in the first stage evaluation and constructing a simplified configuration space accordingly; The execution process inputs into the computer system: the original configuration space , the optimal performance score of the previous stage candidate algorithm , and the number of optimal candidate algorithms desired to be retained ; When the computer system performs hyperparameter pruning, the scores of the candidate algorithms are first paired with the performance scores obtained in the previous stage, and then sorted in descending order of performance to screen out candidate algorithms with superior performance; then, the top candidate algorithms are selected to form a pruned candidate algorithm list, and the computer system outputs the reduced configuration space so that the subsequent hyperparameter optimization stage focuses on candidate algorithms that have been verified on the reduced dataset.

9. An electronic device comprising: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory stores a computer program, and when the program is executed by the processor, the processor executes the two-stage algorithm selection and hyperparameter joint optimization method of any one of claims 1-8.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed to implement the two-stage algorithm selection and hyperparameter joint optimization method of any one of claims 1-8.

Images