Hyper-parameter space analysis method and device, electronic equipment and storage medium

Through the combination of deep learning model and CMA-ES algorithm, the problem of insufficient interpretability and optimization efficiency of hyperparameter space analysis in the existing technology is solved, and efficient analysis of hyperparameter space and evaluation of feature importance are realized, which significantly improves the The effect of model tuning.

CN119089175AInactive Publication Date: 2024-12-06NANKAI UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411273484.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The prior art has shortcomings in the assessment of interpretability, optimization efficiency and feature importance of hyperparameter data, and it is impossible to effectively find the boundary information of the hyperparameter space, affecting model tuning.

Method used

Deep learning model is used for spatial modeling, combined with Sharpley value calculation and covariance matrix adaptive evolution CMA-ES algorithm, to find optimization to obtain the boundary eigenvalues ​​of hyperparameter space, and realize efficient analysis of hyperparameter space.

Benefits of technology

It provides a more comprehensive, transparent and efficient hyperparameter spatial analysis solution, which improves the interpretability and optimization efficiency of data, and can more accurately evaluate the importance of features.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119089175A_ABST
    Figure CN119089175A_ABST
Patent Text Reader

Abstract

The invention provides a hyper-parameter space analysis method and device, electronic equipment and a storage medium. The method comprises the steps that hyper-parameter data is obtained, the hyper-parameter data has multiple dimensions, and each dimension represents one hyper-parameter; performing spatial modeling through a deep learning model by using the hyper-parameter data, so that the deep learning model represents a mapping relationship between a first dimension and a second dimension in a hyper-parameter space corresponding to the hyper-parameter data; calculating a Sharpley value by using a deep learning model, wherein the Sharpley value is used for indicating the influence degree of the second dimension on the first dimension; and using a covariance matrix adaptive evolution CMA-ES algorithm to perform spatial optimization in the hyper-parameter space for the deep learning model to obtain a boundary feature value, wherein the boundary feature value is used for indicating the position of a point which changes fastest in the first dimension in the hyper-parameter space. The embodiment of the invention provides a more comprehensive, transparent and efficient hyper-parameter space analysis scheme.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing, and specifically to a hyperparameter space analysis method, device, electronic device and storage medium. Background Art

[0002] In-depth exploration and research of hyperparameter space is one of the important topics in the field of multidimensional big data analysis, and its goal is to deeply understand the correlation and structural information hidden in the data.

[0003] In the current field of data analysis, various correlation analysis methods, such as the Pearson correlation coefficient and the Spearman rank correlation coefficient, are often used to reveal relationships between data features. In addition, methods such as causal relationship analysis, regression analysis, factor analysis, association rule mining, and time series analysis are also widely used. However, these methods have limitations when dealing with complex data relationships. They are unable to identify the boundaries of the hyperparameter space, which is of great significance in helping researchers tune models and is crucial for hyperparameter space analysis. Summary of the Invention

[0004] In view of this, the embodiments of the present application are committed to providing a hyperparameter space analysis method, device, electronic device and storage medium to achieve accurate and efficient analysis of the hyperparameter space.

[0005] In one aspect of the present application, a hyperparameter space analysis method is provided, comprising:

[0006] Obtaining hyperparameter data, where the hyperparameter data has multiple dimensions, each dimension representing a hyperparameter;

[0007] Performing spatial modeling using a deep learning model using the hyperparameter data, so that the deep learning model represents a mapping relationship between a first dimension and a second dimension in a hyperparameter space corresponding to the hyperparameter data;

[0008] Calculating a Shapley value using the deep learning model, where the Shapley value is used to indicate the degree of influence of the second dimension on the first dimension;

[0009] Using a covariance matrix adaptive evolution CMA-ES algorithm to perform spatial optimization in a hyperparameter space for the deep learning model to obtain a boundary eigenvalue, where the boundary eigenvalue is used to indicate the position of the point with the fastest change in the first dimension in the hyperparameter space;

[0010] The first dimension includes one or more different dimensions in the hyperparameter data, and the second dimension includes all dimensions in the hyperparameter data except the first dimension.

[0011] In some implementations, the method further includes: performing data preprocessing on the hyperparameter data, where the data preprocessing includes one or more of the following: data cleaning and normalization.

[0012] In some embodiments, the method further includes: using the hyperparameter data to train and test the deep learning model so that the deep learning model can represent the mapping relationship between the first dimension and the second dimension in the hyperparameter space corresponding to the hyperparameter data.

[0013] In some embodiments, the use of hyperparameter data to train and test the deep learning model includes: dividing the hyperparameter data into a training data set and a test data set; training the deep learning model with the values ​​of the data on the first dimension of the hyperparameter data in the training data set as the true values ​​of the output features and the values ​​of the data on the second dimension of the hyperparameter data in the training data set as the input features, and the output features of the deep learning model are the predicted values ​​of the data on the first dimension; inputting the values ​​of the data on the second dimension of the hyperparameter data in the test data set into the deep learning model to obtain the predicted values ​​of the data on the first dimension, and evaluating the deep learning model based on the values ​​of the data on the first dimension of the hyperparameter data in the test data set and the predicted values ​​of the data on the first dimension.

[0014] In some implementations, the deep learning model is constructed by one of the following:

[0015] Multilayer Perceptron;

[0016] Convolutional neural networks;

[0017] Recurrent Neural Networks;

[0018] Long Short-Term Memory Networks;

[0019] Attention mechanism model;

[0020] An ensemble learning model that includes machine learning decision trees.

[0021] In some embodiments, calculating the Shapley value using the deep learning model includes: creating an interpreter, which is used to estimate the Shapley value of the deep learning model; using the interpreter to call the explainer.shap_values ​​method to perform Shapley calculation in the hyperparameter space described by the deep learning model to obtain a Shapley value sequence for each dimension in the second dimension, wherein the Shapley value sequence is used to indicate the degree of influence of the numerical value of data in one dimension in the second dimension on the numerical value of data in the first dimension.

[0022] In some implementations, the method further includes: visualizing a Shapley value sequence of each dimension in the second dimension.

[0023] In one aspect of the present application, a hyperparameter space analysis device is provided, comprising:

[0024] An acquisition unit, configured to acquire hyperparameter data, wherein the hyperparameter data has multiple dimensions, each dimension representing a hyperparameter;

[0025] A modeling unit, configured to perform spatial modeling using a deep learning model using the hyperparameter data, so that the deep learning model represents a mapping relationship between a first dimension and a second dimension in a hyperparameter space corresponding to the hyperparameter data;

[0026] a Shapley calculation unit, configured to calculate a Shapley value using the deep learning model and the hyperparameter data, wherein the Shapley value is used to indicate the degree of influence of the second dimension on the first dimension;

[0027] an eigenvalue determination unit, configured to perform spatial optimization in a hyperparameter space for the deep learning model using a covariance matrix adaptive evolution (CMA-ES) algorithm to obtain a boundary eigenvalue, wherein the boundary eigenvalue is used to indicate a position of a point with the fastest change in a first dimension in the hyperparameter space;

[0028] The first dimension includes one or more different dimensions in the hyperparameter data, and the second dimension includes all dimensions in the hyperparameter data except the first dimension.

[0029] In one aspect of the present application, an electronic device is provided, comprising: a processor and a memory;

[0030] The memory is connected to the processor, and is used to store computer programs;

[0031] The processor is configured to implement the above method by running the computer program stored in the memory.

[0032] In one aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above method is implemented.

[0033] According to the embodiments of the present application, by combining deep learning models, CMA-ES algorithms and SHAP technology, the shortcomings of existing technologies in the interpretability of hyperparameter data, optimization efficiency and feature importance evaluation are systematically solved, providing a more comprehensive, transparent and efficient hyperparameter space analysis solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0035] Figure 1 Schematic diagram of the process of the hyperparameter space analysis method provided in the embodiment of the present application;

[0036] Figure 2 This is an exemplary implementation flowchart of using hyperparameter data to perform spatial modeling through a deep learning model according to an embodiment of the present application;

[0037] Figure 3 Schematic diagram of the structure of the hyperparameter space analysis device provided in an embodiment of the present application;

[0038] Figure 4 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0040] Explanation of terms:

[0041] Hyperparameter space: Hyperparameters, also known as hyperparameter space, are parameters that control the learning process in machine learning. Unlike model parameters, which are automatically learned from data during the learning process (e.g., the weights of a neural network), hyperparameters are typically set before learning begins and remain constant throughout the learning process. The hyperparameter space refers to the set of all possible hyperparameter values. In the hyperparameter space, each dimension represents a hyperparameter, and different dimensions have varying degrees of correlation, with each point representing a specific set of hyperparameter settings. For example, if there are two hyperparameters: learning rate and batch size, then the hyperparameter space is a two-dimensional space, with one axis representing the learning rate and the other representing the batch size. The size and complexity of the hyperparameter space depends on the number of hyperparameters and the range of their possible values. A high-dimensional space or hyperparameters with a large number of possible values ​​makes searching for the optimal hyperparameter configuration more difficult, a phenomenon known as the "curse of dimensionality." Therefore, choosing an appropriate hyperparameter space is crucial for model tuning.

[0042] Hyperparameter space boundaries: These are the boundaries of the hyperparameter space. These boundaries refer to the complete range of possible values ​​for a hyperparameter and have a significant impact on the optimization and selection of machine learning models. In the disclosed embodiments, the hyperparameter space boundary refers to the sampling interval where the other dimensions in the hyperparameter space have the greatest impact on a specific dimension.

[0043] Shapley Additive Explanations (SHAP): A widely used explanation technique. SHAP quantifies and explains the contribution of features to model outputs based on Shapley values, providing a consistent and fair explanation framework that allows the impact of each feature on model outputs to be understood and explained.

[0044] Exemplary Methods

[0045] Figure 1 FIG2 shows a flow chart of the hyperparameter space analysis method provided by the embodiment of the present disclosure. Figure 1 , the method may include the following steps:

[0046] Step 101: Obtain hyperparameter data. The hyperparameter data has multiple dimensions, and each dimension represents a hyperparameter.

[0047] Step 102: Using the hyperparameter data to perform spatial modeling through a deep learning model, so that the deep learning model represents the mapping relationship between the first dimension and the second dimension in the hyperparameter space corresponding to the hyperparameter data;

[0048] The first dimension includes one or more different dimensions in the hyperparameter data, and the second dimension includes all dimensions in the hyperparameter data except the first dimension.

[0049] Step 103: Calculate the Shapley value using the deep learning model and hyperparameter data. The Shapley value is used to indicate the degree of influence of the second dimension on the first dimension.

[0050] Step 104: Use the covariance matrix adaptive evolution CMA-ES algorithm to perform spatial optimization in the hyperparameter space for the deep learning model to obtain boundary eigenvalues. The boundary eigenvalues ​​are used to indicate the position of the point with the fastest change in the first dimension in the hyperparameter space.

[0051] In the disclosed embodiment, the hyperparameter data is a data set of hyperparameters of a model, and the model can be any type of machine learning model. In some embodiments, if there are N kinds of hyperparameters of a model, and N is an integer greater than 1, the hyperparameter data of the model is N-dimensional data, and the hyperparameter space corresponding to the hyperparameter data is N-dimensional space. If the model has thousands or more hyperparameters, its hyperparameter data is high-dimensional data of thousands or ten thousand dimensions, and the corresponding hyperparameter space is a high-dimensional space such as thousands of dimensions and ten thousand dimensions. Exemplarily, the hyperparameter data can be represented as multidimensional feature data (X1, X2, ..., Xn), where n represents the total number of dimensions of the hyperparameter data, and n is an integer greater than 1. Xi (i = 1, 2, ..., n) represents the data on dimension i, that is, the possible values ​​of hyperparameter i.

[0052] In step 101, the hyper-parameter data can be input by a staff member or come from other electronic devices. The embodiment of the present disclosure does not limit the specific method of obtaining the hyper-parameter data.

[0053] In some implementations, before step 102, the process may also include preprocessing the hyperparameter data. This preprocessing may include one or more of the following: data cleaning and normalization. Data preprocessing can effectively improve the quality of the hyperparameter data, reduce the risk of errors, and provide a reliable foundation for subsequent processing.

[0054] In some examples, data cleaning may include checking, correcting, and screening the first hyperparameter data to ensure the accuracy and completeness of the first hyperparameter data.

[0055] In some examples, the normalization process may include unifying the units and scales of data of different dimensions in the first hyperparameter data. By normalizing the first hyperparameter data, dimensional differences between different features can be eliminated.

[0056] In other implementations, data preprocessing may also include: inconsistency processing, missing value processing, etc. The embodiments of the present disclosure do not limit the specific process and implementation of data preprocessing.

[0057] In step 102, using hyperparameter data to perform spatial modeling using a deep learning model involves steps such as parameter initialization, hyperparameter optimization, training, and testing of the deep learning model. Parameter initialization and hyperparameter optimization are optional. That is, step 102 may include training and testing the deep learning model using the hyperparameter data so that the deep learning model can represent the mapping relationship between the first dimension and the second dimension in the hyperparameter space corresponding to the hyperparameter data.

[0058] Figure 2 FIG1 shows an exemplary implementation flow chart of using hyperparameter data to perform spatial modeling through a deep learning model in step 102. Figure 2 In some implementations, step 102 may include:

[0059] Step 201: divide the hyperparameter data into a training data set and a test data set;

[0060] Specifically, the hyperparameter data can be randomly divided into a training dataset and a test dataset according to a pre-set ratio. That is, the training dataset contains some hyperparameter data, and the test dataset contains another portion of hyperparameter data. Proper division of the training and test datasets can prevent overfitting of deep learning models.

[0061] Step 202: Using the values ​​of the data on the first dimension of the hyperparameter data in the training dataset as the true values ​​of the output features and the values ​​of the data on the second dimension of the hyperparameter data in the training dataset as the input features to train a deep learning model, the output features of the deep learning model being the predicted values ​​of the data on the first dimension;

[0062] Specifically, grid search and other methods can be used to determine the hyperparameter values ​​of the deep learning model, and then the hyperparameters are introduced into the deep learning model. Then, the deep learning model is trained with the values ​​of the data on the first dimension of the hyperparameter data in the training dataset as the true values ​​of the output features and the values ​​of the data on the second dimension of the hyperparameter data in the training dataset as the input features, until the loss value reaches the requirement and converges.

[0063] Step 203: Input the values ​​of the data on the second dimension of the hyperparameter data in the test data set into the deep learning model to obtain the predicted values ​​of the data on the first dimension, and evaluate the deep learning model based on the values ​​of the data on the first dimension of the hyperparameter data in the test data set and the predicted values ​​of the data on the first dimension.

[0064] Specifically, after training is completed, the trained deep learning model can be evaluated on a test data set to verify the performance of the deep learning model on data it has not seen. Specifically, the trained deep learning model is used to predict the test data set, and the loss value between the prediction result (i.e., the predicted value of the data on the first dimension) and the true label (i.e., the value of the data on the first dimension in the test data set) is calculated. If the loss value is less than the current best loss value, the parameters of the deep learning model are updated until the convergence condition is met. In some embodiments, the loss function used to calculate the loss value can be, but is not limited to, the mean square error.

[0065] Step 204: Save the trained deep learning model.

[0066] Specifically, saving the deep learning model means saving all parameters of the deep learning model, including model parameters and hyperparameters of the deep learning model.

[0067] In some implementations, after training is completed, the parameters of the deep learning model can be saved as a file of a specified type and the file can be stored in a specified path for use in subsequent processing.

[0068] In specific applications, the deep learning model in step 102 can be constructed using one of the following: a multilayer perceptron (MLP), a convolutional neural network, a recurrent neural network, a long short-term memory network, an attention mechanism model, an ensemble learning model of a machine learning decision tree such as eXtreme Gradient Boosting (XGBoost), etc. For example, the deep learning model in step 102 can be an MLP model constructed using an MLP.

[0069] Taking the deep learning model as an MLP model as an example, the specific implementation process of step 102 may include: randomly dividing the preprocessed hyperparameter data into a training data set and a test data set according to an appropriate proportion, selecting a multilayer perceptron (MLP) as a modeling tool and using a deep learning framework to build an MLP model, using the training data set to train the MLP model, and testing and evaluating the trained MLP model using the test data set, thereby completing the modeling of the hyperparameter space.

[0070] The training process of the MLP model may include: first, initializing the MLP model, optimizing the hyperparameters of the MLP model to determine the hyperparameters of the MLP model, using the hyperparameters of the MLP model to train and test the MLP model, and when the MLP model meets the requirements, completing the training and saving the parameters of the MLP model.

[0071] In step 102, spatial modeling is performed through a deep learning model to obtain a continuous hyperparameter space, which contains not only the original hyperparameter data, but also the hyperparameter data estimated by the deep learning model. Thus, the discretized hyperparameter data is converted into continuously distributed hyperparameter data through spatial modeling, that is, the coarse-grained hyperparameter data is converted into fine-grained hyperparameters, so that subsequent steps 103 and 104 can perform related processing based on the continuously distributed hyperparameter data, thereby obtaining Shapley values ​​and boundary eigenvalues ​​with finer granularity and higher accuracy.

[0072] In some implementations, step 103 is implemented by SHAP. Specifically, step 103 may include the following steps a1 to a2:

[0073] Step a1: Create an interpreter, which is used to estimate the Shapley value of the deep learning model;

[0074] SHAP supports many types of interpreters, such as deep, gradient, kernel, linear, tree, and sampling. You can choose the appropriate interpreter for your specific application. In some examples, step a1 can use the Kernel Explainer as the interpreter for the deep learning model.

[0075] Specifically, a file containing deep learning model parameters is loaded to generate a function expression for the deep learning model. Based on this function expression, a KernelExplainer interpreter is created. When creating the interpreter, two parameters are passed in: the function expression for the deep learning model and its input data. The model's prediction function is used to make predictions based on the input data, which is typically preprocessed and standardized input feature data.

[0076] Specifically, you can construct a single-sample explanation object and a full-data explanation object. By creating these two explanation objects, you can conveniently store the explanation information of the model prediction for subsequent analysis and visualization.

[0077] In step a2, the explainer is called to perform a Shapley calculation in the hyperparameter space described by the deep learning model using the explainer.shap_values ​​method to obtain a Shapley value sequence for each dimension in the second dimension. The Shapley value sequence is used to indicate the degree of influence of the numerical value of the data in one dimension in the second dimension on the numerical value of the data in the first dimension.

[0078] Specifically, using the created KernelExplainer interpreter, call the explainer.shap_values ​​method to calculate the Shapley value. This method requires input data and calculates the Shapley value for all input data. After the calculation is complete, a sequence of Shapley values ​​for each dimension in the second dimension is obtained. The Shapley value sequence indicates the degree to which the value of a data point in the second dimension affects the value of the data point in the first dimension. These Shapley values ​​can help understand the formation process of model prediction results and the contribution of different features to the model prediction.

[0079] In some implementations, after step 103, the process may further include: visualizing the Shapley value sequence of each dimension in the second dimension, thereby allowing the user to intuitively view the degree of influence of other dimensions on a specified dimension.

[0080] For example, visualizing the Shapley value sequence of each dimension in the second dimension may include: generating a histogram and an explanation report indicating the Shapley value sequence of each dimension in the second dimension, and displaying the histogram and the explanation report so that the user can intuitively and clearly view the SHAP calculation results.

[0081] In the embodiment of the present disclosure, the Shapley value obtained in step 103 not only provides an importance ranking of the values ​​of the data on the second dimension in the hyperparameter data, but also dynamically reveals the changes in the importance of the numerical values ​​of the data on the first dimension in the hyperparameter data in different scenarios, thereby providing a more in-depth and refined hyperparameter space analysis. This dynamic evaluation method can truly reflect the complex interactions between data of different dimensions in the hyperparameter space (i.e., different hyperparameter values), thereby improving the comprehensiveness and accuracy of the hyperparameter space importance analysis.

[0082] In some implementations, step 104 may include the following steps b1 to b2:

[0083] Step b1, initializing the parameters of the CMA-ES algorithm according to the mapping relationship indicated by the deep learning model;

[0084] The parameters of the CMA-ES algorithm include: the number of parent individuals, the number of search space dimensions, the upper and lower bounds of the search space, the standard deviation of the population, and adaptive parameters. The search space belongs to the hyperparameter space, and the adaptive parameters are used to control the update of the covariance matrix, the update of the standard deviation, and the damping coefficient.

[0085] Specifically, step b1 may include the following:

[0086] 1) According to the incoming population size, calculate the number of parent individuals mu selected, which is usually half of the population size.

[0087] 2) Initialize the search space dimension n_dim.

[0088] 3) Initialize the upper and lower bounds lb and ub of the search space.

[0089] 4) Initialize the population standard deviation sigma.

[0090] 5) Initialize the initial mean xmean of the population. If not specified, it is randomly sampled from the upper and lower bounds of the search space.

[0091] 6) Initialize and select the weight vector weights of the parent individual, which is calculated using the weight formula specific to the CMA-ES algorithm.

[0092] 7) The weighted average of the fitness of individuals in the population, mueff, is calculated.

[0093] 8) Initialize the adaptive parameters cc, cs, c1, cmu and damps. These parameters control the update of the covariance matrix, the update of the standard deviation and the damping coefficient, respectively, and play an important role in the convergence and stability of the algorithm.

[0094] Step b2: Perform the following processing until the step size reaches the standard deviation of the population to determine the boundary eigenvalue:

[0095] First, calculate the transformation matrix of the population: Calculate the transformation matrix of the population. The transformation matrix is ​​the product of matrix B and matrix D. Matrix B is the eigenvector matrix of the covariance matrix, and matrix D is the eigenvalue diagonal matrix.

[0096] Secondly, generate samples of lambda_pop individuals through a loop, use the sample of each individual to obtain a random vector by sampling from the standard normal distribution, and use the covariance matrix and the random vector to calculate the true sample vector and its fitness value. The sample vector is within the boundary of the search space;

[0097] Third, sort the samples according to their fitness values, select the mu individuals with the highest fitness as the parent individuals, and update the adaptive parameters in the CMA-ES algorithm, such as the adaptive learning rates cc, cs, c1, cmu, and damps, as well as the eigenvectors and eigenvalues ​​of the covariance matrix.

[0098] Fourth, calculate the new mean vector xmean and the new transformation matrix based on the parent individuals, and calculate the new step length sigma based on the cumulative sum of the evolutionary path;

[0099] Finally, the covariance matrix is ​​decomposed into eigenvalues ​​to update the transformation matrix and adjust the step size sigma. If a flat area is found in the fitness function, the step size sigma is adjusted to try to jump out of the flat area.

[0100] Assuming that the hyperparameter data is represented as multidimensional feature data (X1, X2, ..., Xn), the boundary feature value can be expressed as [x1, x2, ..., xn]. Where Xi represents the feature of a certain dimension, and xi represents the position of the point with the fastest change in a certain dimension i.

[0101] Compared with traditional evolutionary strategy algorithms, the CMA-ES algorithm has stronger adaptability and global search capabilities, and is particularly suitable for optimization problems that are high-dimensional, non-convex, and have complex structures. CMA-ES is a gradient-free optimization and does not use gradient information. It performs well on complex optimization problems such as non-separable, ill-conditioned, orrugged / multi-modal. In the embodiment of the present disclosure, step 104 combines the CMA-ES algorithm to find the point in the output space of the deep learning model that causes the fastest change in the output of the deep learning model, and can more accurately find the value of the data in the second dimension that has the greatest impact on the value of the data in the first dimension. Since the CMA-ES algorithm can efficiently explore high-dimensional space and concentrate on searching the areas that are most sensitive to the output of the deep learning model, it significantly improves the efficiency and accuracy of the spatial optimization process in the hyperparameter space, while also reducing the waste of computing resources. It is particularly suitable for spatial optimization of hyperparameter data with a high number of dimensions.

[0102] In the embodiment of the present disclosure, the hyperparameter data is first used to perform spatial modeling through a deep learning model, and the feature contribution of the model is analyzed through the SHAP technology to better explain the mapping relationship between other dimensions (i.e., the second dimension) and the specified dimension (i.e., the first dimension) in the hyperparameter data, and the CMA-ES algorithm is used to perform spatial optimization of the hyperparameter space for the trained deep learning model to find the spatial optimal point, which is the point where the value of the first dimension in the hyperparameter space changes the fastest. It can be seen that the embodiment of the present disclosure systematically solves the shortcomings of the existing technology in the interpretability, optimization efficiency and feature importance evaluation of hyperparameter data by combining deep learning models, CMA-ES algorithms and SHAP technology, and provides a more comprehensive, transparent and efficient hyperparameter space analysis solution.

[0103] Exemplary devices

[0104] Figure 3 FIG1 shows a schematic diagram of the structure of a hyperparameter data analysis device provided by an embodiment of the present application. Figure 3 , the hyper-parameter data analysis device 300 may include:

[0105] An acquisition unit 301 is used to acquire hyperparameter data, where the hyperparameter data has multiple dimensions, each dimension representing a hyperparameter;

[0106] A modeling unit 302 is configured to perform spatial modeling using a deep learning model using the hyperparameter data, so that the deep learning model represents a mapping relationship between a first dimension and a second dimension in a hyperparameter space corresponding to the hyperparameter data;

[0107] A Shapley calculation unit 303 is used to calculate a Shapley value using the deep learning model and hyperparameter data. The Shapley value is used to indicate the degree of influence of the second dimension on the first dimension.

[0108] An eigenvalue determination unit 304 is configured to use a covariance matrix adaptive evolution (CMA-ES) algorithm to perform spatial optimization in a hyperparameter space for a deep learning model to obtain a boundary eigenvalue, where the boundary eigenvalue indicates the position of the point with the fastest change in the first dimension in the hyperparameter space.

[0109] The first dimension includes one or more different dimensions in the hyperparameter data, and the second dimension includes all dimensions in the hyperparameter data except the first dimension.

[0110] In some implementations, the hyper-parameter data analysis device 300 may further include: a preprocessing unit 305 for performing data preprocessing on the hyper-parameter data, where the data preprocessing includes one or more of the following: data cleaning and normalization.

[0111] In some embodiments, the modeling unit 302 can be specifically used to: divide the hyperparameter data into a training data set and a test data set; use the values ​​of the data on the first dimension of the hyperparameter data in the training data set as the true values ​​of the output features and the values ​​of the data on the second dimension of the hyperparameter data in the training data set as the input features to train the deep learning model, and the output features of the deep learning model are the predicted values ​​of the data on the first dimension; input the values ​​of the data on the second dimension of the hyperparameter data in the test data set into the deep learning model to obtain the predicted values ​​of the data on the first dimension, and evaluate the deep learning model based on the values ​​of the data on the first dimension of the hyperparameter data in the test data set and the predicted values ​​of the data on the first dimension.

[0112] In some embodiments, the Shapley calculation unit 303 can be specifically used to: create an interpreter, which is used to estimate the Shapley value of the deep learning model; use the interpreter to call the explainer.shap_values ​​method to perform Shapley calculation in the hyperparameter space described by the deep learning model to obtain a Shapley value sequence for each dimension in the second dimension, and the Shapley value sequence is used to indicate the degree of influence of the numerical value of data in one dimension in the second dimension on the numerical value of data in the first dimension.

[0113] In some implementations, the hyperparameter data analysis device 300 may further include: a visualization unit 306, configured to visualize the Shapley value sequence of each dimension in the second dimension.

[0114] Other technical details of the hyper-parameter data analysis device 300 in the embodiment of the present application can be found in the method section above and will not be repeated here.

[0115] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0116] electronic devices

[0117] Figure 4 Schematic diagram of the structure of the electronic device provided by the embodiment of the present application is shown. Figure 4 As shown, the electronic device may include: one or more processors 401, and a memory 402 for storing one or more programs, which are executed by the one or more processors 401 to implement the method flow shown in the above embodiments of the present disclosure and / or program units corresponding to each unit in the device.

[0118] The various components are interconnected using different buses and can be mounted on a common motherboard or in other ways as needed. The processor 401 can process instructions executed within the electronic device, including instructions stored in or on the memory to display graphical information of the user interface on an external input / output device (such as a display device coupled to the interface). In other embodiments, if desired, multiple processors and / or multiple buses can be used with multiple memories and multiple memories.

[0119] The processor 401 may include one or more single-core processors or multi-core processors. The processor 401 may include any combination of general-purpose processors or dedicated processors (such as image processors, application processors, baseband processors, etc.).

[0120] The memory 402 is a computer-readable storage medium provided by the present disclosure, which can be used to store non-transient software programs, non-transient computer executable programs and units, such as the embodiment of the present disclosure. Figure 1The processor 401 executes the non-transient software programs, instructions and units stored in the memory 402, thereby executing the above method embodiments. Figure 1 The methods shown correspond to programs, instructions, and units.

[0121] The electronic device may further include: an input device 403 and an output device 404. The processor 401, the memory 402, the input device 403 and the output device 404 may be connected via a bus or other means. Figure 4 The bus connection is taken as an example.

[0122] The input device 403 can receive input digital or character information and generate signal input related to user settings and function control of the hyperparameter space analysis device, such as input devices such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, and a joystick. The output device 404 may include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0123] The programs (also referred to as software, software applications, or code) described above include machine instructions for a programmable processor, and these computer programs may be implemented using an object-oriented programming language, assembly, or machine language.

[0124] As time goes by and technology develops, the meaning of medium becomes more and more extensive, and the propagation path of computer programs is no longer limited to tangible media, but can also be downloaded directly from the Internet, etc. Any combination of one or more computer-readable storage media can be used. Computer-readable storage media can adopt but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or components, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium that contains or stores a program that can be used by or in combination with an instruction execution system, device or device.

[0125] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, the program including instructions, which, when executed by one or more processors of a computing device, execute the steps of any one of the method embodiments described above.

[0126] An embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the steps of any one of the methods described in the aforementioned method embodiments.

[0127] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A hyperparameter space analysis method, comprising: Obtaining hyperparameter data, where the hyperparameter data has multiple dimensions, each dimension representing a hyperparameter; Using the hyperparameter data to perform spatial modeling through a deep learning model, so that the deep learning model represents a mapping relationship between a first dimension and a second dimension in a hyperparameter space corresponding to the hyperparameter data; Calculating a Shapley value using the deep learning model, where the Shapley value is used to indicate the degree of influence of the second dimension on the first dimension; Using a covariance matrix adaptive evolution CMA-ES algorithm to perform spatial optimization in a hyperparameter space for the deep learning model to obtain a boundary eigenvalue, wherein the boundary eigenvalue is used to indicate the position of the point with the fastest change in the first dimension in the hyperparameter space; The first dimension includes one or more different dimensions in the hyperparameter data, and the second dimension includes all dimensions in the hyperparameter data except the first dimension.

2. The method according to claim 1, characterized in that The method further comprises: The hyperparameter data is preprocessed, and the data preprocessing includes one or more of the following: data cleaning and normalization.

3. The method according to claim 1, characterized in that The method also includes: using the hyperparameter data to train and test the deep learning model so that the deep learning model can characterize the mapping relationship between the first dimension and the second dimension in the hyperparameter space corresponding to the hyperparameter data.

4. The method according to claim 3, characterized in that The use of hyperparameter data to train and test the deep learning model includes: Dividing the hyperparameter data into a training data set and a test data set; The deep learning model is trained by taking the values ​​of the data on the first dimension of the hyperparameter data in the training data set as the true values ​​of the output features and the values ​​of the data on the second dimension of the hyperparameter data in the training data set as the input features, and the output features of the deep learning model are the predicted values ​​of the data on the first dimension; The values ​​of the data on the second dimension of the hyperparameter data in the test data set are input into the deep learning model to obtain the predicted values ​​of the data on the first dimension, and the deep learning model is evaluated based on the values ​​of the data on the first dimension of the hyperparameter data in the test data set and the predicted values ​​of the data on the first dimension.

5. The method according to claim 1, characterized in that The deep learning model is constructed by one of the following: Multilayer Perceptron; Convolutional Neural Networks; Recurrent Neural Networks; Long short-term memory networks; Attention mechanism model; An ensemble learning model that includes machine learning decision trees.

6. The method according to claim 1, characterized in that The calculating of the Shapley value by using the deep learning model includes: Creating an interpreter for estimating the Shapley value of the deep learning model; The explainer is used to call the explainer.shap_values ​​method to perform Shapley calculation in the hyperparameter space described by the deep learning model to obtain a Shapley value sequence for each dimension in the second dimension, wherein the Shapley value sequence is used to indicate the degree of influence of the numerical value of data in one dimension in the second dimension on the numerical value of data in the first dimension.

7. The method according to claim 5, characterized in that Also includes: Visualize the sequence of Shapley values ​​for each of the second dimensions.

8. A hyperparameter spatial analysis device, characterized in that: include: An acquisition unit, used to acquire hyperparameter data, wherein the hyperparameter data has multiple dimensions, each dimension representing a hyperparameter; A modeling unit, configured to perform spatial modeling using a deep learning model using the hyperparameter data, so that the deep learning model represents a mapping relationship between a first dimension and a second dimension in a hyperparameter space corresponding to the hyperparameter data; A Shapley calculation unit, configured to calculate a Shapley value using the deep learning model and the hyperparameter data, wherein the Shapley value is used to indicate the degree of influence of the second dimension on the first dimension; An eigenvalue determination unit, used to use a covariance matrix adaptive evolution CMA-ES algorithm to perform spatial optimization in a hyperparameter space for the deep learning model to obtain a boundary eigenvalue, wherein the boundary eigenvalue is used to indicate the position of the point with the fastest change in the first dimension in the hyperparameter space; The first dimension includes one or more different dimensions in the hyperparameter data, and the second dimension includes all dimensions in the hyperparameter data except the first dimension.

9. An electronic device, characterized in that: include: Processor and memory; Wherein, the memory is connected to the processor, and the memory is used to store a computer program; The processor is configured to implement the method according to any one of claims 1 to 7 by running the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Citation Information

Cited By

  • Hyper-parameter space analysis method and device, electronic equipment and storage medium

    CN119988930A