Obtaining explanations of a bayesian optimization process

Abstract

Embodiments of the present disclosure relate to a computer-readable data carrier, an apparatus, a computer program, and a method for obtaining explanations of a Bayesian optimization process. The method comprises sampling the search space of the Bayesian optimization process to obtain sampled points and obtaining an empirical approximation of a posterior distribution of a surrogate model of the Bayesian optimization process based on the sampled points. Further, the method comprises obtaining rule-based explanations of the Bayesian optimization process from the posterior distributions.

Description

[0001] OBTAINING EXPLANATIONS OF A BAYESIAN OPTIMIZATION PROCESS

[0002] Description

[0003] Computer-readable data carrier, apparatus, computer program, and method for obtaining explanations of a Bayesian optimization process

[0004] Embodiments of the present disclosure relate to a computer-readable data carrier, an apparatus, a computer program, and a method for obtaining explanations of a Bayesian optimization process.

[0005] Bayesian optimization processes are applied in various fields of technology and science. In particular, it is applied for hyper parameter optimization (HPO), e.g., for training neural networks and / or tuning cyber physical systems. In doing so, e.g., various parameters may be adjusted based on a result of the Bayesian optimization process as well as on an experience of a user.

[0006] In automotive applications, in this way, pinch protection solutions of automatic car windows or trunk doors may be adjusted.

[0007] However, it is not understandable to humans how or why a Bayesian optimization process arrived at a certain optimization result. So, Bayesian optimization process may lack explainability to its users (e.g., as for neural networks). Therefore, is difficult or even impossible to evaluate the optimization results. This is particularly problematic in validations or certifications of cyber physical systems.

[0008] Hence, there may be a demand for a concept for obtaining explanations of a Bayesian optimization process.

[0009] Embodiments of the present disclosure are based on the finding that rules indicating an optimization behavior may be obtained from posterior distributions of a surrogate model of a Bayesian optimization process.

[0010] Embodiments provide a method for obtaining explanations of a Bayesian optimization process. The method comprises sampling the search space of the Bayesian optimization process to obtain sampled points and obtaining an empirical approximation of a posterior distribution of a surrogate model of the Bayesian optimization process based on the sampled points. Further, the method comprises obtaining rule-based explanations of the Bayesian optimization process from the posterior distributions.

[0011] This way, the Bayesian optimization process may become understandable to a user from the rule-based explanations. This, e.g., helps to user to validate and / or evaluate results of the Bayesian optimization process. The proposed approach, e.g., may be applied for a parameterization of cyber physical systems and / or training neural network. In practice, the proposed approach, e.g., may be applied for setting and / or adjusting hyper parameters.

[0012] In some embodiments, obtaining the rule-based explanations comprises applying a hierarchical clustering mechanism for finding clusters in the posterior distributions and obtaining the rule-based explanations from the clusters. This facilitates, e.g., to find appropriate clusters for obtaining the rule-based explanations. In practice, the clustering, e.g., allows to obtain a dendrogram for the clusters.

[0013] In practice, the clusters, e.g., can be pruned accordingly. So, in some embodiments, the method comprises pruning the obtained clusters using a pruning criterion. In this way, e.g., reasonable explanations can be extracted from the pruned clusters.

[0014] In some embodiments the method further comprises obtaining a coverage, support, confidence, and relevance of the explanations. Further, it comprises obtaining an interestingness of the explanations based on the coverage, support, confidence, and relevance and selecting one or more explanations based on their interestingness. In this way, the most relevant explanations indicating rules determining the Bayesian optimization process may be extracted.

[0015] Optionally, the method further comprises providing the explanations for adapting the Bayesian optimization process. In doing so, the explanations, e.g., are displayed to user. This may help the user to validate the results of the Bayesian optimization process, i.e., to see if the results address or meet certain requirements, e.g., if the results address / meet predefined safety requirements of cyber physical system.

[0016] Optionally, the adapted Bayesian optimization process is applied for training a machine-learning based model.

[0017] In practice, the trained machine-learning based model may be applied for controlling a cyber physical system.

[0018] Further embodiments of the present disclosure provide a computer program comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out the proposed method.

[0019] Other embodiments provide an apparatus comprising one or more interfaces for communication and a data processing circuit configured to execute the proposed method.

[0020] Further embodiments provide a computer-readable data carrier having stored thereon the proposed computer program.

[0021] Further, embodiments are now described with reference to the attached drawings. It should be noted that the embodiments illustrated by the referenced drawings show merely optional embodiments as an example and that the scope of the present disclosure is by no means limited to the embodiments presented:

[0022] Brief description of the drawings

[0023] Fig. 1 shows a flow chart schematically illustrating an embodiment of a method for obtaining explanations of a Bayesian optimization process;

[0024] Fig. 2a - 2e illustrates a use case of the proposed approach; and Fig. 3 shows a block diagram schematically illustrating an embodiment of an apparatus according to the present disclosure.

[0025] Black-box optimization plays a pivotal role in optimizing complex parameter spaces with unknown relations to the optimization target. One commonly encountered example for this problem type is hyperparameter optimization, which is substantial for the performance of nearly all machine learning models.

[0026] However, the inner workings of Bayesian optimization, such as the surrogate model, acquisition function, and sequential decision-making process, can be opaque and challenging to interpret. This lack of interpretability poses limitations in understanding the decision-making process and the factors influencing the optimization results. This means, a user is not able to validate results or learn from them for future tasks.

[0027] The present disclosure provides an approach to obtain human-understandable explanations of the inner workings of a Bayesian optimization process.

[0028] In a nutshell, the presented method in this disclosure provides explanations for Bayesian optimization processes.

[0029] In the present disclosure, the proposed approach is explained in more detail in connection with a use case for hyper parameter tuning. The presented method can be understood as a post hoc explainability method of a Bayesian optimization process (also referred to as “Bayesian optimization model”). In particular, the proposed approach may be applied to Bayesian optimization processes with a Gaussian process surrogate backbone.

[0030] In practice, the proposed approach may be applied for the optimization of complex models, so-called “black boxes”. The proposed approach, therefore, may be understood as a concept of explainable black box optimization (XBO). In context of the present disclosure, explainable black box optimization refers to the development of optimization processes that can provide human-understandable explanations for their results. The goal of XBO is to create optimization systems that are transparent and interpretable, so that their decision-making processes can be understood and verified by humans. This is important for several reasons, including increasing trust in the results, reducing the risk of local minima, and enabling better regulatory oversight. XBO can also help researchers and developers better understand and improve the inner workings of black-box optimization systems.

[0031] The presented method is a rule-based post hoc explainability system designed for black-box Bayesian optimization task. After the optimization run is complete, the disclosed method accesses the Gaussian process surrogate model of the Bayesian optimization framework and explains this model to the end user as definitive global rules or local rules, as per need.

[0032] An exemplary embodiment of the proposed method is now explained in more detail below with reference to Fig. 1 schematically illustrating the proposed method and Fig. 2a - 2e illustrating an exemplary use case.

[0033] As can be seen from Fig. 1 , the Bayesian optimization process may executed as per a standard procedure. In the present example, the Bayesian optimization process is applied for determining an extreme value of function 211 (see Fig. 2, diagram 210).

[0034] The Bayesian optimization process may involve generating of initial data (see step 101 ), evaluating of the initial data on a simulator (see step 102), and creating a surrogate model (see step 103). Consequently, acquisition function of the Bayesian optimization process is optimized (see step 104) in further data / points are queried (see Fig. 2b, data points 212) based on the optimized acquisition function (see step 105 and function 213) iteratively until a predefined stopping criterion is reached (see 106), e.g., until a predefined accuracy is reached. Resulting surrogate model is frozen for a following explainability process (see step 107). In the present example, the Bayesian optimization, e.g., comprises a Gaussian process (GP) surrogate model. The results of this optimization may be collected once the optimization process is complete (see step 108). These results contain the evaluated points and a score. These evaluated points are approximately located near the optimal results.

[0035] Then, an explanation data set including the sampled points is obtained. To this end, e.g., a uniform sampling is applied inthe search space (see step 109).

[0036] This explanation dataset (i.e. the sampled points) is then used to query the posterior distribution of the learned GP surrogate model to produce an (empirical) approximation of the full distribution, now including the predicted output values. This is done until a predefined number of samples of the posterior distribution is reached (see step 111 and posterior distributions 214 in Fig. 2c). In the present example, e.g., 500 samples are generated. The samples of the posterior distribution are referred to herein as “distribution data set”.

[0037] Consequently, rule-based explanations are generated (see step 112).

[0038] For this, the bounded optimization space is clustered based on the posterior distribution and using hierarchical clustering based on a pair wise distance metric e.g. Euclidean distance. In doing so, clusters 216 at generated based on the posterior distributions 214 (see Fig. 2e). The clusters 216 may be configured such that a cluster includes at least a predefined amount of posterior distributions between two or more data points 212.

[0039] After that rules are created for these clusters using a suitable pruning criterion, in the present example, e.g., a distance-based cutoff threshold. In other embodiments, the pruning criterion may be based on the variance, the entropy, the impurity, and / or the like. The pruning criterion controls the granularity of rules covering the optimization space. Lower distance would indicate highly specific rules, while higher distances would indicate highly general rules. One would like to balance this threshold depending on the kind of rules necessary. For the ease of selecting this cutoff, a dendogram based graph may be used (see dendrogram 215 in Fig. 2d). In the present example, the threshold is approximately 2 to obtain a reasonable granularity. For selecting the threshold, the dendrogram 215, e.g., is presented to user and the user may select the threshold based on experience and / or preferences for a particular use case.

[0040] Then for each of these rules, samples of the posterior satisfying them are selected for a relevance metric computation which is the upper bound of the log likelihood of the samples as computed from the GP surrogate model. This implies a consideration of the aforementioned implicit meaning of the output.

[0041] Then one or more metrics (see step 113) supported by the data to gauge how well the clusters fit the data are applied to sort / filter the rules. The metrics may be based on parameters such as coverage, support, and confidence.

[0042] Coverage is the area of the posterior distribution as covered by the antecedent part of the rule. Support is the area of the posterior distribution as covered by both the antecedent and consequent part of the rule. And confidence is the ratio of support by coverage. These area based metrics are computed with empirical cumulative probability distribution computation, i.e. how many samples cover each rule by total number of samples.

[0043] Finally, a weighted sum of relevance, support, coverage, and confidence is computed to create the rule induction metric called interestingness.

[0044] Then rules are sorted by this and displayed as a global explanation (see step 114). To display local explanation, one just needs to first provide the sample for which an explanation is demanded, then a search needs to take place over the global rules for finding the rule which is satisfied by the queried sample and display that rule as explanation. Here, one needs to note that the optimization results are only true within a bounded and evaluated space. So, if one provides samples which are out of the evaluated space, the explainability algorithm will detect that and display ‘no rules found, due to sample out of bounds’. Which is one key difference from other rule based explainability algorithms traditionally used for regression based XAI.

[0045] In the present disclosure, the rules may be as follows:

[0046] Rule 0: if -1 .6363636363636362 <= x < -1 .1515151515151514 then consequent [0.12543198936310448, 0.5636343812260803] | interestingness: 0.3055036188742452, Confidence: 0.9999750006249843, Support: 0.04, Coverage: 0.04

[0047] Rule 1 : if -1 .0303030303030303 <= x < -0.5454545454545454 then consequent [0.42271111945645473, 0.8246553252578734] | interestingness: 0.3696005669425466, Confidence: 0.9999750006249843, Support: 0.04, Coverage: 0.04

[0048] Rule 2: if -1 .878787878787879 <= x < -1 .7575757575757576 then consequent [0.1284637621204146, 0.3198622261980227] | interestingness: 0.26040669908014485, Confidence: 0.9999000099990002, Support: 0.01 , Coverage: 0.01

[0049] Rule 3: if -2.0 <= x < -2.0 then consequent [0.19976402670045354, 0.2037015963729745] | interestingness: 0.04552165020540445, Confidence: 0.0, Support: 0.0, Coverage: 0.0

[0050] Rule 4: if 8.666666666666668 <= x < 10.0 then consequent [0.15483598075232685, 0.5353577645084758] | interestingness: 0.3213653392135032, Confidence: 0.9999909091735529, Support: 0.11 , Coverage: 0.11

[0051] Rule 5: if 9.151515151515152 <= x < 9.636363636363637 then consequent [0.09785683783122504, 0.48533927738567556] | interestingness: 0.29194995431803383, Confidence: 0.9999750006249843, Support: 0.04, Coverage: 0.04 Rule 6: if 8.545454545454545 <= x < 9.757575757575758 then consequent

[0052] [0.11615606280531035, 0.609195235856448] | interestingness:

[0053] 0.332427557727707, Confidence: 0.9999900000999989, Support: 0.1 , Coverage: 0.1

[0054] Rule 7: if 7.575757575757576 <= x < 8.424242424242424 then consequent

[0055] [0.4871272427974106, 0.852667559443693] | interestingness:

[0056] 0.38768450382659214, Confidence: 0.999985714489793, Support: 0.07,

[0057] Coverage: 0.07

[0058] Rule 8: if 1 .5151515151515151 <= x < 2.4848484848484853 then consequent

[0059] [1.2037149428842666, 1.4022472561725452] | interestingness:

[0060] 0.7236619813276911 , Confidence: 0.999987500156248, Support: 0.08, Coverage: 0.08

[0061] Rule 9: if 0.3030303030303032 <= x < 2.8484848484848486 then consequent

[0062] [0.9422912301049835, 1.1626786872605048] | interestingness:

[0063] 0.46915187572535116, Confidence: 0.5714258503530936, Support: 0.12, Coverage: 0.21

[0064] Rule 10: if 3.212121212121212 <= x < 4.666666666666667 then consequent

[0065] [0.6167223148966493, 0.9397461657171042] | interestingness:

[0066] 0.43439674323989713, Confidence: 0.9999916667361105, Support: 0.12, Coverage: 0.12

[0067] Rule 11 : if 5.151515151515151 <= x < 6.484848484848484 then consequent

[0068] [0.925994790370685, 1.0736522623778915] | interestingness:

[0069] 0.4923250676966593, Confidence: 0.9999909091735529, Support: 0.11 , Coverage: 0.11

[0070] Rule 12: if 2.9696969696969697 <= x < 4.181818181818182 then consequent [0.748586470129623, 0.8987935243656516] | interestingness: 0.28079540483456605, Confidence: 0.3999960000399996, Support: 0.04,

[0071] Coverage: 0.1

[0072] Rule 13: if 4.787878787878788 <= x < 7.818181818181818 then consequent

[0073] [0.7081362839988092, 1.0420305742772198] | interestingness:

[0074] 0.5184332337748623, Confidence: 0.9999960000160001 , Support: 0.25, Coverage: 0.25

[0075] Rule 14: if 6.8484848484848495 <= x < 7.212121212121213 then consequent

[0076] [0.7222059842760059, 1.0868539118684286] | interestingness:

[0077] 0.47525849837431877, Confidence: 0.9999666677777407, Support: 0.03, Coverage: 0.03

[0078] Rule 15: if 6.6060606060606055 <= x < 7.454545454545455 then consequent

[0079] [0.7183049460559934, 1.0821653045848727] | interestingness:

[0080] 0.4847771522628339, Confidence: 0.999985714489793, Support: 0.07, Coverage: 0.07

[0081] Rule 16: if -0.4242424242424243 <= x < -0.06060606060606055 then consequent [0.6861163011871731 , 1.041723786391222] | interestingness:

[0082] 0.45227804791608, Confidence: 0.9999666677777407, Support: 0.03, Coverage: 0.03

[0083] The disclosed method has multiple advantages:

[0084] • Providing explanations of Bayesian optimization

[0085] • Increasing oversight of this algorithm for regulators

[0086] • Increasing the trust over this algorithm for various stakeholders who need to use the results provided by this algorithm

[0087] • Relevance based on GP likelihood to identify interesting areas of the optimization space

[0088] • Weighted interestingness metric for rule induction to identify most interesting rules based on the explored space • Identifying out of bounds points and excluding an explanation for them, the concept of ‘I don’t know’ since the points are not evaluated

[0089] The proposed approach can be used in various applications in various fields of technology that profit from such explanations to be produced (and displayed to the user or regulators), based on black box optimization. Besides the mentioned Hyperparameter Optimization, the proposed approach may be also used for parameter optimization of cyber physical systems, like pinch protections, sensor fusion algorithms or safety measurements triggering algorithms.

[0090] In doing so, e.g., parameters of a pinch protection system for an automatic window, automatic sunroof, or automatic trunk lid of a vehicle may be adjusted more accurately. This way, the cyber physical system may act as desired. For example, this way the parameters of a pinch protection system for an automatic trunk lid may be adapted more accurately such that it behaves properly under various circumstances (e.g., wind or snow on the trunk lid).

[0091] Even if the proposed approach is described herein only in connection with a few possible implementations, the skilled person having profit of the present disclosure will appreciate that the proposed approach may be applied in various other applications. As well, the skilled person will appreciate that equivalents of the explained features may be used (e.g., other clustering algorithms, other surrogate models, other Bayesian optimization processes, other rule extraction metrics / algorithms).

[0092] What has been described herein in connection with a method, may be analogously implemented in a computer implemented method, a computer program, and / or an apparatus. An exemplary apparatus is described in more detail below with reference to Fig. 3.

[0093] Fig. 3 shows a block diagram schematically illustrating an embodiment of such an apparatus 300. The apparatus comprises one or more interfaces 310 for communication and a data processing circuit 320 configured to execute the proposed method.

[0094] In embodiments, the one or more interfaces 310 may comprise wired and / or wireless interfaces for transmitting and / or receiving communication signals in connection with the execution of the proposed concept. In practice, the interfaces, e.g., comprise pins, wires, antennas, and / or the like. As well, the interfaces may comprise means for (analog and / or digital) signal or data processing in connection with the communication, e.g., filters, samples, analog-to-digital converters, signal acquisition and / or reconstruction means as well as signal amplifiers, compressors and / or any encryption / decryption means.

[0095] The data processing circuit 320 may correspond to or comprise any type of programable hardware. So, examples of the data processing circuit 320, e.g., comprise a memory, microcontroller, field programable gate arrays, one or more central, and / or graphical processing units. To execute the proposed method, the data processing circuit 320 may be configured to access or retrieve an appropriate computer program for the execution of the proposed method from a memory of the data processing circuit 320 or a separate memory which is communicatively coupled to the data processing circuit 320.

[0096] In the foregoing description, it can be seen that various features are grouped together in examples for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed examples require more features than are expressly recited in each claim. Rather, as the following claims reflect, subject matter may lie in less than all features of a single disclosed example. Thus, the following claims are hereby incorporated into the description, where each claim may stand on its own as a separate example. While each claim may stand on its own as a separate example, it is to be noted that, although a dependent claim may refer in the claims to a specific combination with one or more other claims, other examples may also include a combination of the dependent claim with the subject matter of each other dependent claim or a combination of each feature with other dependent or independent claims. Such combinations are proposed herein unless it is stated that a specific combination is not intended. Furthermore, it is intended to include also features of a claim to any other independent claim even if this claim is not directly made dependent to the independent claim.

[0097] Although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that a variety of alternate and / or equivalent implementations may be substituted for the specific embodiments shown and described without departing from the scope of the present embodiments. This application is intended to cover any adaptations or variations of the specific embodiments discussed herein. Therefore, it is intended that the embodiments be limited only by the claims and the equivalents thereof.

Claims

Patent claims1 . A method for obtaining explanations of a Bayesian optimization process, the method comprising: sampling (109) the search space of the Bayesian optimization process to obtain sampled points; obtaining (111 ) an empirical approximation of a posterior distribution of a surrogate model of the Bayesian optimization process based on the sampled points; and obtaining (112) rule-based explanations of the Bayesian optimization process from the posterior distributions.

2. The method of claim 1 , wherein obtaining the rule-based explanations comprises applying a hierarchical clustering mechanism for finding clusters in the posterior distributions and obtaining the rule-based explanations from the clusters.

3. The method of claim 2, wherein the method comprises pruning the obtained clusters using a pruning criterion.

4. The method of any one of the preceding claims, wherein the method further comprises: obtaining a coverage, support, confidence, and relevance of the explanations; obtaining an interestingness of the explanations based on the coverage, support, confidence, and relevance; and selecting one or more explanations based on their interestingness.

5. The method of any one of the preceding claims, wherein the method further comprises providing the explanations for adapting the Bayesian optimization process.

6. The method of claim 5, wherein the method further comprises applying the adapted Bayesian optimization process for training a machine-learning based model.

7. The method of claim 6, wherein the method further comprises applying the trained machine-learning based model for controlling a cyber physical system.

8. A computer program comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out the method of any one of the claims 1 to 7.

9. An apparatus (300) comprising: one or more interfaces (310) for communication; and a data processing circuit (320) configured to execute the method of any one of the claims 1 to 7.

10. A computer-readable data carrier having stored thereon the computer program of claim 8.

Images