Computer-readable data carrier, device, computer program and method for obtaining explanations of a Bayesian optimization process

Abstract

Embodiments of the present disclosure relate to a computer-readable data carrier, a device, a computer program, and a method for obtaining explanations of a Bayesian optimization process. The method comprises taking samples in the search space of the Bayesian optimization process to obtain tried points and obtaining an empirical approximation of an a posteriori distribution of a surrogate model of the Bayesian optimization process based on the tried points. Furthermore, the method comprises obtaining rule-based explanations of the Bayesian optimization process from the a posteriori distributions.

Description

[0001] Embodiments of the present disclosure relate to a computer-readable medium, an apparatus, a computer program, and a method for obtaining explanations of a Bayesian optimization process.

[0002] Bayesian optimization processes are applied in various engineering and scientific fields. In particular, they are used for hyperparameter optimization (HPO), e.g., for training neural networks and / or tuning cyber-physical systems. For example, various parameters can be adjusted based on the results of the Bayesian optimization process as well as a user's experience.

[0003] In automotive applications, anti-pinch solutions for automatic car windows or tailgates / trunk lids can be adjusted in this way.

[0004] However, it is not understandable to humans how or why a Bayesian optimization process arrived at a particular optimization result. For example, a Bayesian optimization process may lack explainability for its users (e.g., with neural networks). Therefore, it is difficult or even impossible to evaluate the optimization results. This is particularly problematic in the validation or certification of cyber-physical systems.

[0005] Consequently, there may be a need for a concept for obtaining explanations of a Bayesian optimization process.

[0006] Embodiments of the present disclosure are based on the recognition that rules specifying optimization behavior can be obtained from posterior distributions of a surrogate model of a Bayesian optimization process.

[0007] Embodiments provide a method for obtaining explanations of a Bayesian optimization process. The method includes sampling the search space of the Bayesian optimization process to obtain sampled points and obtaining an empirical approximation of a posterior distribution of a surrogate model of the Bayesian optimization process based on the sampled points. The method further includes obtaining rule-based explanations of the Bayesian optimization process from the posterior distributions.

[0008] In this way, the Bayesian optimization process can be understood by the user from the rule-based explanations. This helps the user, for example, to validate and / or evaluate the results of the Bayesian optimization process. The proposed approach can be applied, for example, to the parameterization of cyber-physical systems and / or the training of a neural network. In practice, the proposed approach can be applied, for example, to the setting and / or tuning of hyperparameters.

[0009] In some embodiments, obtaining the rule-based explanations comprises applying a hierarchical cluster analysis mechanism to find clusters in the posterior distributions and obtaining the rule-based explanations from the clusters. This facilitates, for example, finding suitable clusters for obtaining the rule-based explanations. In practice, the clustering method allows, for example, obtaining a dendrogram for the clusters.

[0010] In practice, the clusters can be pruned accordingly, for example. In some embodiments, the method includes pruning the obtained clusters using a pruning criterion. In this way, for example, comprehensible explanations can be extracted from the pruned clusters.

[0011] In some embodiments, the method further comprises obtaining coverage, support, confidence, and relevance of the explanations. It further comprises obtaining an interestingness of the explanations based on the coverage, support, confidence, and relevance, and selecting one or more explanations based on their interestingness. In this way, the most relevant explanations, which specify rules governing the Bayesian optimization process, can be extracted.

[0012] Optionally, the method further comprises providing explanations for adapting the Bayesian optimization process. The explanations are displayed, for example, to the user. This can help the user validate the results of the Bayesian optimization process, i.e., see whether the results address or satisfy certain requirements, e.g., whether the results address / satisfy predefined security requirements of the cyber-physical system.

[0013] Optionally, the customized Bayesian optimization process is applied to train a machine learning-based model.

[0014] In practice, the trained machine learning-based model can be applied to control a cyber-physical system.

[0015] Further embodiments of the present disclosure provide a computer program comprising instructions that, when the computer program is executed by a computer, cause the computer to perform the proposed method.

[0016] Other embodiments provide a device comprising one or more interfaces for communication and a data processing circuit configured to carry out the proposed method.

[0017] Further embodiments provide a computer-readable data carrier on which the proposed computer program is stored.

[0018] Further embodiments will now be described with reference to the accompanying drawings. It should be noted that the embodiments illustrated by the drawings to which reference is made merely show optional embodiments by way of example, and that the scope of the present disclosure is by no means limited to the embodiments set forth. Short description of the drawings Fig. 1 shows a flow diagram schematically illustrating one embodiment of a method for obtaining explanations of a Bayesian optimization process; Fig. 2a - 2e illustrate a use case of the proposed approach; and Fig. 3 shows a block diagram schematically illustrating an embodiment of an apparatus according to the present disclosure.

[0019] Black-box optimization plays a key role in optimizing complex parameter spaces with unknown relationships to the optimization objective. A common example of this problem type is hyperparameter optimization, which is essential to the performance of almost all machine learning models.

[0020] However, the inner workings of Bayesian optimization, such as the surrogate model, the detection function, and the sequential decision-making process, can be opaque and difficult to interpret. This lack of interpretability limits the understanding of the decision-making process and the factors that influence optimization results. This means that a user is unable to validate results or learn from them for future tasks.

[0021] The present disclosure provides an approach for obtaining human-understandable explanations of the inner workings of a Bayesian optimization process.

[0022] In short, the method presented in this disclosure provides explanations for Bayesian optimization processes.

[0023] In the present disclosure, the proposed approach is further explained in the context of a hyperparameter tuning use case. The presented method can be understood as a post-hoc explainability procedure of a Bayesian optimization process (also referred to as a "Bayesian optimization model"). In particular, the proposed approach can be applied to Bayesian optimization processes with a Gaussian process surrogate backbone.

[0024] In practice, the proposed approach can be applied to the optimization of complex models, so-called "black boxes." The proposed approach can therefore be understood as a concept of explainable black box optimization (XBO).

[0025] In the context of the present disclosure, explainable black-box optimization concerns the development of optimization processes that can provide human-understandable explanations for their results. The goal of XBO is to create optimization systems that are transparent and interpretable, so that their decision-making processes can be understood and verified by humans. This is important for several reasons, including increasing confidence in the results, reducing the risk of local minima, and enabling better regulatory oversight. XBO can also help researchers and developers better understand and improve the inner workings of black-box optimization systems.

[0026] The presented method is a rule-based post-hoc explainability system designed for the Bayesian optimization task using a black box. After the optimization process is complete, the disclosed method accesses the Gaussian process surrogate model of the Bayesian optimization framework and explains this model to the end user as definitive global rules or local rules, as required.

[0027] An embodiment of the proposed method will now be explained in more detail below, with reference to Fig. 1, which schematically illustrates the proposed method, and Fig. 2a - 2e, which illustrate an example use case.

[0028] As from Fig. 1, the Bayesian optimization process can be carried out according to a standard procedure. In this example, the Bayesian optimization process is used to determine an extreme value of function 211 (see Fig. 2, Diagram 210).

[0029] The Bayesian optimization process may include generating initial data (see step 101), evaluating the initial data on a simulator (see step 102), and creating a surrogate model (see step 103). The acquisition function of the Bayesian optimization process is then optimized on additional data (see step 104) / points are queried (see Fig. 2b, data points 212), based on the optimized detection function (see step 105 and function 213) in an iterative manner until a predefined stopping criterion is reached (see 106), e.g., until a predefined accuracy is achieved. The resulting surrogate model is frozen for a subsequent explainability process (see step 107).

[0030] In the present example, the Bayesian optimization includes a Gaussian process (GP) surrogate model. The results of this optimization can be collected once the optimization process is complete (see step 108). These results include the evaluated points and a score. These evaluated points are approximately close to the optimal results.

[0031] Then, an explanatory dataset containing the sampled points is obtained. For this purpose, uniform sampling in the search space is used (see step 109).

[0032] This explanatory dataset (i.e., the sampled points) is then used to query the posterior distribution of the learned GP surrogate model to generate an (empirical) approximation of the full distribution, which now has the predicted output values. This is done until a predefined number of samples of the posterior distribution is reached (see step 111 and posterior distributions 214 in Fig. 2c). In this example, 500 samples are generated. The samples of the posterior distribution are referred to herein as the "distribution data set."

[0033] Rule-based explanations are then generated (see step 112).

[0034] For this purpose, the restricted optimization space is clustered based on the posterior distribution and using hierarchical cluster analysis based on a pairwise distance metric, e.g., Euclidean distance. 214 clusters 216 are generated based on the posterior distributions (see Fig. 2e). The clusters 216 may be configured such that a cluster has at least a predefined set of posterior distributions between two or more data points 212.

[0035] Rules are then created for these clusters using a suitable pruning criterion, in the present example, for example, a distance-based threshold. In other embodiments, the pruning criterion may be based on variance, entropy, impurity, and / or the like. The pruning criterion controls the level of detail of rules covering the optimization space. A lower distance would indicate highly specific rules, while higher distances would indicate very general rules. One would like to balance this threshold depending on the type of rules required. To easily select this threshold, a dendrogram-based graph can be used (see dendrogram 215 in Fig. 2d). In the present example, the threshold is approximately 2 to achieve an appropriate level of detail. To select the threshold, the dendrogram 215 is presented to the user, for example, and the user can select the threshold based on experience and / or preferences for a particular use case.

[0036] Then, for each of these rules, samples of the posterior distribution that satisfy them are selected for a relevance metric calculation, which is the upper bound of the log-likelihood of the samples, as calculated from the GP surrogate model. This implies consideration of the aforementioned implicit importance of the output.

[0037] Then, one or more metrics (see step 113), supported by the data to measure how well the clusters fit the data, are applied to sort / filter the rules. The metrics can be based on parameters such as coverage, support, and confidence.

[0038] Coverage is the range of the posterior distribution covered by the preceding part of the rule. Support is the range of the posterior distribution covered by both the preceding and subsequent parts of the rule. Confidence is the ratio of support to coverage. These range-based metrics are calculated using an empirical cumulative probability distribution calculation, i.e., how many samples of the total number of samples cover each rule.

[0039] Finally, a weighted sum of relevance, support, coverage, and confidence is calculated to create the rule induction metric called interestingness.

[0040] Rules are then sorted and displayed as a global explanation (see step 114). To display a local explanation, one simply needs to first provide the sample for which an explanation is requested, then search the global rules to find the rule satisfied by the requested sample, and display this rule as an explanation. Note that the optimization results are only true within a bounded and evaluated space. Therefore, if samples are provided that lie outside the evaluated space, the explainability algorithm will detect this and display the following message: "No rules found because the sample is outside the bounds." This is a key difference from other rule-based explainability algorithms traditionally used for regression-based XAI.

[0041] In the present disclosure, the rules may be as follows: Rule 0: if -1.6363636363636362 <= x < -1.1515151515151514, then [0.12543198936310448, 0.5636343812260803] | Interestingness: 0.3055036188742452, Confidence: 0.9999750006249843, Support: 0.04, Coverage: 0.04 Rule 1: if -1.0303030303030303 <= x < -0.5454545454545454, then [0.42271111945645473, 0.8246553252578734] | Interestingness: 0.3696005669425466, Confidence: 0.9999750006249843, Support: 0.04, Coverage: 0.04 Rule 2: if -1.878787878787879 <= x < -1.7575757575757576, then [0.1284637621204146, 0.3198622261980227] | Interestingness: 0.26040669908014485, Confidence: 0.9999000099990002, Support: 0.01, Coverage: 0.01 Rule 3: if -2.0 <= x < -2.0, then [0.19976402670045354, 0.2037015963729745] | Interestingness: 0.04552165020540445, Confidence: 0.0, Support: 0.0, Coverage: 0.0 Rule 4: if 8.666666666666668 <= x < 10.0, then [0.15483598075232685, 0.5353577645084758] | Interestingness: 0.3213653392135032, Confidence: 0.9999909091735529, Support: 0.11, Coverage: 0.11 Rule 5: if 9.151515151515152 <= x < 9.636363636363637, then [0.09785683783122504, 0.48533927738567556] | Interestingness: 0.29194995431803383, Confidence: 0.9999750006249843, Support: 0.04, Coverage: 0.04 Rule 6: if 8.545454545454545 <= x < 9.757575757575758, then [0.11615606280531035, 0.609195235856448] | Interestingness: 0.332427557727707, Confidence: 0.9999900000999989, Support: 0.1, Coverage: 0.1 Rule 7: if 7.575757575757576 <= x < 8.424242424242424, then [0.4871272427974106, 0.852667559443693] | Interestingness: 0.38768450382659214, Confidence: 0.999985714489793, Support: 0.07, Coverage: 0.07 Rule 8: if 1.5151515151515151 <= x < 2.4848484848484853, then [1.2037149428842666, 1.4022472561725452] follows | Interestingness: 0.7236619813276911, Confidence: 0.999987500156248, Support: 0.08, Coverage: 0.08 Rule 9: if 0.3030303030303032 <= x < 2.8484848484848486, then [0.9422912301049835, 1.1626786872605048] | Interestingness: 0.46915187572535116, Confidence: 0.5714258503530936, Support: 0.12, Coverage: 0.21 Rule 10: if 3.212121212121212 <= x < 4.666666666666667, then [0.6167223148966493, 0.9397461657171042] | Interestingness: 0.43439674323989713, Confidence: 0.9999916667361105, Support: 0.12, Coverage: 0.12 Rule 11: if 5.15151515151515151 <= x < 6.484848484848484, then [0.925994790370685, 1.0736522623778915] | Interestingness: 0.4923250676966593, Confidence: 0.9999909091735529, Support: 0.11, Coverage: 0.11 Rule 12: if 2.9696969696969697 <= x < 4.181818181818182, then [0.748586470129623, 0.8987935243656516] | Interestingness: 0.28079540483456605, Confidence: 0.3999960000399996, Support: 0.04, Coverage: 0.1 Rule 13: if 4.787878787878788 <= x < 7.818181818181818, then [0.7081362839988092, 1.0420305742772198] | Interestingness: 0.5184332337748623, Confidence: 0.9999960000160001, Support: 0.25, Coverage: 0.25 Rule 14: if 6.8484848484848495 <= x < 7.212121212121213, then [0.7222059842760059, 1.0868539118684286] | Interestingness: 0.47525849837431877, Confidence: 0.9999666677777407, Support: 0.03, Coverage: 0.03 Rule 15: if 6.6060606060606055 <= x < 7.454545454545455, then [0.7183049460559934, 1.0821653045848727] | Interestingness: 0.4847771522628339, Confidence: 0.999985714489793, Support: 0.07, Coverage: 0.07 Rule 16: if -0.4242424242424243 <= x < -0.06060606060606055, then [0.6861163011871731, 1.041723786391222] | Interestingness: 0.45227804791608, Confidence: 0.9999666677777407, Support: 0.03, Coverage: 0.03

[0042] The disclosed method has several advantages: • Providing explanations of Bayesian optimization • Expand oversight of this algorithm for regulators • Increasing trust in this algorithm for various relevant actors who need to use the results provided by this algorithm • Relevance based on GP likelihood to identify interesting areas of the optimization space • Weighted interestingness metric for rule induction to identify the most interesting rules based on the explored space • Identifying points outside the bounds and excluding an explanation for them, the concept of “I don’t know” because the points are not evaluated

[0043] The proposed approach can be used in various applications in various engineering fields that benefit from generating such explanations (and displaying them to users or regulators) based on black-box optimization. In addition to the aforementioned hyperparameter optimization, the proposed approach can also be used for parameter optimization of cyber-physical systems, such as anti-entrapment systems, sensor fusion algorithms, or algorithms for triggering safety measurements.

[0044] For example, the parameters of an anti-pinch system for an automatic window, an automatic sunroof, or an automatic trunk lid of a vehicle can be adjusted more precisely. This allows the cyber-physical system to function as desired. For example, the parameters of an anti-pinch system for an automatic trunk lid can be adjusted more precisely so that it behaves correctly under various conditions (e.g., wind or snow on the trunk lid).

[0045] Although the proposed approach is described herein only in the context of some possible implementations, those skilled in the art who benefit from the present disclosure will recognize that the proposed approach can be applied in various other applications. Furthermore, those skilled in the art will recognize that equivalents of the discussed features (e.g., other clustering algorithms, other surrogate models, other Bayesian optimization processes, other rule extraction metrics / algorithms) may be used.

[0046] What has been described herein in connection with a method can be implemented analogously in a computer-implemented method, a computer program and / or a device. An exemplary device is described below with reference to Fig. 3 described in more detail.

[0047] Fig.Figure 3 shows a block diagram schematically illustrating one embodiment of such a device 300. The device comprises one or more interfaces 310 for communication and a data processing circuit 320 configured to execute the proposed method.

[0048] In embodiments, the one or more interfaces 310 may comprise wired and / or wireless interfaces for transmitting and / or receiving communication signals related to the implementation of the proposed concept. In practice, the interfaces comprise, for example, pins, wires, antennas, and / or the like. Furthermore, the interfaces may comprise means for (analog and / or digital) signal or data processing related to communication, such as filters, samples, analog-to-digital converters, signal acquisition and / or reconstruction means, as well as signal amplifiers, compressors, and / or any encryption / decryption means.

[0049] Data processing circuitry 320 may correspond to or include any programmable hardware type. Examples of data processing circuitry 320 include, for example, a memory, a microcontroller, field-programmable gate arrays, one or more main and / or graphics processors. To carry out the proposed method, data processing circuitry 320 may be configured to access or retrieve a suitable computer program for executing the proposed method from a memory of data processing circuitry 320 or a separate memory communicatively coupled to data processing circuitry 320.

[0050] In the foregoing description, it will be appreciated that, for the purpose of streamlining the disclosure, various features are grouped into examples. This method of disclosure should not be interpreted as reflecting an intent that the claimed examples require more features than those expressly recited in each claim. Rather, as the following claims reflect, the subject matter may consist in fewer than all of the features of a single disclosed example. Thus, the following claims are hereby incorporated into the description, where each claim may stand on its own as a separate example.While each claim may stand on its own as a separate example, it should be noted that although a dependent claim in the claims may refer to a specific combination with one or more other claim(s), other examples may also include a combination of the dependent claim with the subject matter of any other dependent claim, or a combination of any feature with other dependent or independent claims. Such combinations are suggested herein unless it is stated that a specific combination is not intended. Furthermore, it is intended to include features of a claim relating to any other independent claim, even if that claim is not directly made a dependent of the independent claim.

[0051] Although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that the specific embodiments shown and described may be replaced with a variety of alternative and / or equivalent implementations without departing from the scope of the present embodiments. This application is intended to cover any adaptations or modifications of the specific embodiments discussed herein. Therefore, it is intended that the embodiments be limited only by the claims and their equivalents.

Claims

[1] A method for obtaining explanations of a Bayesian optimization process, the method comprising: Taking samples (109) in the search space of the Bayesian optimization process to obtain sampled points; Obtaining (111) an empirical approximation of a posterior distribution of a surrogate model of the Bayesian optimization process based on the sampled points; and Obtaining (112) rule-based explanations of the Bayesian optimization process from the posterior distributions. [2] The method of claim 1, wherein obtaining the rule-based explanations comprises applying a hierarchical cluster analysis mechanism for finding clusters in the posterior distributions and obtaining the rule-based explanations from the clusters. [3] The method of claim 2, wherein the method comprises pruning the obtained clusters using a pruning criterion. [4] A method according to any one of the preceding claims, the method further comprising: Obtaining coverage, support, confidence and relevance of the explanations; Obtaining an interestingness of the explanations based on coverage, support, confidence, and relevance; and selecting one or more explanations based on their interestingness. [5] A method according to any one of the preceding claims, wherein the method further comprises providing the explanations for adapting the Bayesian optimization process. [6] The method of claim 5, further comprising applying the adapted Bayesian optimization process to train a machine learning-based model. [7] The method of claim 6, wherein the method further comprises applying the trained machine learning-based model to control a cyber-physical system. [8] A computer program comprising instructions which, when the computer program is executed by a computer, cause the computer to perform the method according to any one of claims 1 to 7. [9] Device (300) comprising: one or more interfaces (310) for communication; and a data processing circuit (320) configured to carry out the method according to one of claims 1 to 7. [10] A computer-readable data carrier on which the computer program according to claim 8 is stored.

Images