Machine learning model error detection

Abstract

A system includes a memory having instructions therein; and at least one processor in communication with the memory, wherein the at least one processor is configured to execute the instructions to determine a global-level importance magnitude value for a global-level importance of an explainable feature of a machine learning base model based on a first prediction of the machine learning base model. The at least one processor is further configured to execute the instructions to determine a global-level importance direction label for the global-level importance of the explainable feature based on the first prediction. The at least one processor is further configured to execute the instructions to generate a communication for presentation to a user based on a second prediction of the machine learning base model, based on the global-level importance magnitude value, and based on the global-level importance direction label.

Description

Background Technology

[0001] This invention relates generally to the field of artificial intelligence, and more specifically to improving the accuracy of machine learning models.

[0002] The various abilities of machines to acquire and apply knowledge and skills have been categorized as artificial intelligence (“AI”). Machine learning is considered a form of AI. Machine learning has employed algorithms and statistical models that enable computer systems to perform tasks primarily based on data patterns and associated reasoning rather than explicit instructions. Some machine learning models have performed classification and / or other predictive tasks. Providing actionable insights into when and how such models are compromised is challenging. Summary of the Invention

[0003] A method for correcting erroneous predictions of a machine learning base model for a user is disclosed. The method includes: determining a global importance magnitude value for the global importance of interpretable features of the machine learning base model to the machine learning base model based on a first prediction of the machine learning base model. The method further includes: determining a global importance direction label for the global importance of interpretable features of the machine learning base model to the machine learning base model based on the first prediction of the machine learning base model. The method also includes: generating a delivery for presentation to the user based on a second prediction of the machine learning base model, the global importance magnitude value, and the global importance direction label.

[0004] A method for alerting a user to erroneous predictions of a machine learning base model is also disclosed. The method includes: running the machine learning base model on a first input dataset to generate baseline prediction pairs, and determining the local importance of a first interpretable feature of the machine learning base model to the prediction class of the machine learning base model. The method further includes: determining the global importance of the first interpretable feature of the machine learning base model based on the local importance of the first interpretable feature. The method also includes: running the machine learning base model on a second input dataset to generate new predictions. The method further includes: determining an error specification for the new prediction based on both the local and global importance of the first interpretable feature of the machine learning base model. Finally, the method includes: transmitting the new prediction and an indication of the error specification for the new prediction to the user.

[0005] A system for correcting erroneous predictions of a machine learning base model for a user is also disclosed. The system includes: a memory having instructions; and at least one processor communicating with the memory, wherein the at least one processor is configured to execute instructions to: determine a global importance magnitude value of the global importance of interpretable features of the machine learning base model to the machine learning base model based on a first prediction of the machine learning base model. The at least one processor is also configured to execute instructions to determine a global importance direction label of the global importance of interpretable features of the machine learning base model to the machine learning base model based on the first prediction of the machine learning base model. The at least one processor is further configured to execute instructions to generate a delivery for presentation to the user based on a second prediction of the machine learning base model, based on the global importance magnitude value, and based on the global importance direction label.

[0006] A system for alerting a user to erroneous predictions of a machine learning base model is also disclosed. The system includes: a memory having instructions; and at least one processor communicating with the memory, wherein the at least one processor is configured to execute instructions to: run the machine learning base model on a first input dataset to generate baseline prediction pairs from the machine learning base model, and determine the local importance of a first interpretable feature of the machine learning base model to the prediction class of the machine learning base model. The at least one processor is also configured to execute instructions to determine the global importance of the first interpretable feature of the machine learning base model based on the local importance of the first interpretable feature. The at least one processor is further configured to execute instructions to run the machine learning base model on a second input dataset to generate new predictions from the machine learning base model. The at least one processor is further configured to execute instructions to determine an error specification for the new prediction based on both the local importance and the global importance of the first interpretable feature of the machine learning base model. The at least one processor is also configured to execute instructions to transmit the new prediction and an indication of the error specification for the new prediction for presentation to the user.

[0007] A computer program product for alerting a user to erroneous predictions of a machine learning base model is also disclosed. The computer program product includes a computer-readable storage medium having program instructions contained therein, executable by at least one processor to cause the at least one processor to: run the machine learning base model on a first input dataset to generate baseline prediction pairs from the machine learning base model, and determine the local importance of a first interpretable feature of the machine learning base model to the prediction class of the machine learning base model. The program instructions are also executable by at least one processor to cause the at least one processor to: determine the global importance of the first interpretable feature of the machine learning base model based on the local importance of the first interpretable feature of the machine learning base model. The program instructions are also executable by at least one processor to cause the at least one processor to: run the machine learning base model on a second input dataset to generate new predictions from the machine learning base model. The program instructions are also executable by at least one processor to cause the at least one processor to: determine an error specification for the new prediction based on both the local importance and the global importance of the first interpretable feature of the machine learning base model. The program instructions are also executable by at least one processor to cause the at least one processor to: transmit the new prediction and an indication of the error specification for the new prediction for presentation to a user. Attached Figure Description

[0008] To gain a more complete understanding of this disclosure, reference is now made to the following brief description in conjunction with the accompanying drawings and detailed description, wherein the same reference numerals denote the same parts.

[0009] Figure 1 This is a tabular illustration of the contribution or importance of example local features generated by LIME analysis of an example sentiment classification model (not shown) based on various aspects of this disclosure.

[0010] Figure 2 This is a block diagram illustrating a machine learning prediction system based on various aspects of this disclosure.

[0011] Figure 3 This is a data flow diagram illustrating how, in some cases, global importance magnitude and direction are calculated for uninterpretable features such as, for example, “insufficiency”, according to various aspects of this disclosure.

[0012] Figure 4 It is a flowchart illustrating the machine learning prediction process according to various aspects of this disclosure.

[0013] Figure 5 This is a block diagram illustrating the hardware architecture of a data processing system based on various aspects of this disclosure.

[0014] The diagrams shown are merely illustrative and are not intended to assert or imply any limitation regarding the environment, architecture, design, or process in which different embodiments may be implemented. Detailed Implementation

[0015] First, it should be understood that although illustrative implementations of one or more embodiments are provided below, the disclosed systems, computer program products, and / or methods can be implemented using any number of currently known or existing techniques. This disclosure should in no way be limited to the illustrative implementations, drawings, and techniques shown below, including the exemplary designs and implementations shown and described herein, but modifications can be made within the full scope of the appended claims and their equivalents.

[0016] As used in the written disclosure and claims, the terms “comprising” and “including” (and variations thereof) are used in an open-ended manner and should therefore be construed as meaning “including, but not limited to”. Unless otherwise specified, “or” as used throughout this document does not require mutual exclusivity, and the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise.

[0017] The term "module" or "unit" (and variations thereof) as used herein includes one or more hardware or electronic components, such as circuits, processors, and memory, which may be specifically configured to perform a particular function. Memory may include volatile or non-volatile memory storing data, such as, but not limited to, computer-executable instructions, machine code, and various other forms of data. A module or unit may be configured to use data to execute one or more instructions to perform one or more tasks. In some cases, a module or unit may also refer to a specific set of functions, software instructions, or circuits configured to perform a particular task. For example, a module or unit may include software components, such as, but not limited to, data access objects, service components, user interface components, application programming interface ("API") components; hardware components, such as circuits, processors, and memory; and / or combinations thereof. As referenced herein, computer-executable instructions may be of any form, including but not limited to machine code, assembly code, and high-level programming code written in any programming language.

[0018] Furthermore, as used herein, the term "transmission" (and variations thereof) means receiving and / or sending data or information via a transmission link. A transmission link may include wired and wireless links, and may include direct links, or may include multiple links via one or more transmission networks or network devices (e.g., but not limited to routers, firewalls, servers, and switches). A transmission network may include any type of wired or wireless network. The network may include private networks and / or public networks such as the Internet. Additionally, in some embodiments, the term "transmission" may also include internal transmission between various components of the system and / or with external input / output devices such as keyboards or display devices.

[0019] The term "interpretable features" (and its variations) used in this paper refers to data representations that are understandable to humans. Intuitive interpretable features for text data are words (or other single characters) or phrases, although the corresponding machine learning models can be built at embedding time. For images, possible interpretable features could be superpixels, although pixel-level features can be used to train the corresponding models.

[0020] As used herein, "local-level interpretability," "local interpretability," "local-level," "local," etc. (and variations thereof) refer to explanations used to justify why a given machine learning model makes a specific prediction for a single instance. In some embodiments, aspects of this disclosure employ the Locally Interpretable Model-Unknown Interpretation ("LIME") technique for determining local-level interpretability. LIME works by perturbing the interpretable features of the given data input to the machine learning model and recording the corresponding changes in the model's predicted probabilities. Based on the perturbed data input and the corresponding predicted output, LIME uses linear regression to assign relative weights to the importance of the interpretable features.

[0021] Figure 1This is a tabular illustration 100 showing the contribution or importance of example local-level features generated by LIME analysis of an example sentiment classification model (not shown) based on various aspects of this disclosure. In the described example, the model predicts "positive" in the sentence "Panera gives me hiccups."2(a). The unary feature "panera" contributes positively to the positive prediction with an amplitude of 0.576, while "hiccup" contributes negatively with an amplitude of 0.159. It should be understood that by observing the contributions of local-level features, it is possible to determine that the model is producing prediction errors by considering the word "panera" as a significant indicator of positive, thus causing the model to assign a positive label to a negative sentence. Various embodiments of this disclosure may employ LIME analysis to determine the local-level importance of interpretable features of a machine learning model. However, it should also be understood that other embodiments may employ any of several other suitable local-level determinism techniques. Non-limiting examples of suitable alternative methods for performing local-level interpretation include Shapley additive interpretation ("SHAP") and black-box interpretation via transparent approximation ("BETA").

[0022] The terms “global interpretability,” “global,” “global,” “global,” etc. (and their variations) used in this paper refer to how interpretable features affect the predictions of the corresponding machine learning model about the entire or whole input dataset, rather than its effect on predictions made from individual data instances from the entire dataset. In some cases, a machine learning model may have been trained on millions of data instances, while humans may only be able to label it with a more limited number of inputs based on local interpretations. It should be understood that global interpretability can allow individuals to gain a more general understanding of a machine learning model without knowing the detailed instance-level predictions, and in this sense, global interpretability can help extract more knowledge for less human intervention.

[0023] This disclosure provides a system for detecting one or more errors in a machine learning model. Given a pre-trained black-box machine learning model (“machine learning base model” or “base model”) and a first input dataset, the system runs the machine learning base model on the first input dataset using a data perturbation process to generate baseline predictions from the machine learning base model and determines the local importance of interpretable features of the machine learning base model to each prediction class of the machine learning base model for each baseline prediction. For a corresponding prediction class of the machine learning base model, each local importance corresponds to the difference between a first prediction probability and a second prediction probability, the first prediction probability being generated from running the machine learning base model on a first input including the corresponding interpretable feature, and the second prediction probability being generated from running the machine learning base model on a second input excluding the corresponding interpretable feature. The system aggregates the local importance of the interpretable features of the machine learning base model to determine the global importance of the interpretable features to the entire machine learning base model. Each global importance of a corresponding interpretable feature includes an amplitude value (“global importance amplitude value”) and a direction label (“global importance direction label”). The global importance magnitude value corresponds to the highest relative importance magnitude of the corresponding interpretable feature in the prediction class of the machine learning base model, and the global importance orientation label corresponds to the prediction class of the machine learning base model associated with that magnitude value. The system ranks the interpretable features according to their corresponding global importance magnitude values, transmits one or more features from the highest-ranked interpretable features, and transmits the corresponding global importance orientation label (for presentation to one or more human evaluators) for each of these highest-ranked interpretable features. Based on the corresponding consistent human evaluation, the system receives the transmission of each feature from the one or more highest-ranked interpretable features, which has an erroneous global importance orientation. The system also receives a second input dataset. The system runs the machine learning base model on the second input dataset to generate one or more new predictions. The system calculates a local error score for each new prediction as a normalized version of the cumulative error introduced into each new prediction by the highest-ranked interpretable feature, which has a corresponding erroneous global importance orientation based on the corresponding consistent human evaluation. The system determines the corresponding error designation for each prediction in the new prediction based on whether the corresponding local error score for the new prediction exceeds a threshold (e.g., "problematic" or "suspected error," rather than "no problem" or "non-suspected error"). In some embodiments, the system transmits each new prediction and an indication of the corresponding error designation for the new prediction (for presentation to one or more system developers or one or more other users).It should be understood that one or more system developers or one or more other users can further analyze any one or more problematic new predictions, and / or analyze any associated data or other characteristics of the machine learning base model, and can modify the machine learning base model and / or take one or more other improvement and / or correction actions. In some embodiments, the system transmits one or more corrected predictions (for presentation to one or more users). Each such corrected prediction is based on a corresponding prediction in one or more new predictions and an error specification for the new prediction. More specifically, when the error specification for the new prediction is "problematic" or "suspected error," the corresponding corrected prediction is the inverse prediction of that new prediction. Conversely, when the error specification for the new prediction is "no problem" or "no suspected error," the corresponding corrected prediction is the same as the new prediction.

[0024] Figure 2 This is a block diagram illustration of a machine learning prediction system 200 according to various aspects of the present disclosure. The machine learning prediction system 200 is configured to implement a machine learning prediction process 400 according to various aspects of the present disclosure (the machine learning prediction process 400 itself is not described in the diagram). Figure 2 It is clearly shown in the document, but see also Figure 4 The machine learning prediction system 200 includes a local importance generation module 212. The local importance generation module 212 is configured to receive a machine learning base model 216 and a first input dataset 220. It should be understood that the machine learning base model 216 may be a pre-trained classification model for tasks such as sentiment analysis, intent prediction, image classification, etc., or may be any other pre-trained machine learning model that can be considered a black box. It should also be understood that the machine learning base model 216 may include logistic regression, support vector machines (“SVM”), random forests, ensemble neural networks (“CNN”), recurrent neural networks (“RNN”), and / or any other one or more types of machine learning and / or deep learning models. The local importance generation module 212 is also configured to generate local importance from each data instance d containing interpretable features j. i Each individual interpretable feature j is masked one at a time in i∈{0,1,...,N} to run perturbation-based LIME analysis at the local level on all data instances of the first input dataset 220, and is configured to compute the absolute change in the predicted probability of the machine learning base model 216 associated with each class label k∈{0,1,...,K} of the machine learning base model 216 as:

[0025] Where P(y=k|d) i ) represents the predicted probability of the machine learning base model 216 with interpretable features j, where Let represent the predicted probability of the machine learning base model 216 that does not have interpretable features j, and where This indicates that interpretable feature j is relevant to data instance d. i The local importance associated with class k.

[0026] The local importance generation module 212 is also configured to obtain The set is transmitted as local importance 224. In some embodiments, the local importance generation module 212 may include one or more corresponding data processing systems in the data processing system, such as data processing system 500 (data processing system 500 itself is not in the local importance generation module 224). Figure 2 It is clearly shown in the document, but see also Figure 5 ).

[0027] The machine learning prediction system 200 also includes a global importance generation module 228. The global importance generation module 228 is transportively coupled to the local importance generation module 212. The global importance generation module 228 is configured to receive the transmission of local importance 224. The global importance generation module 228 is also configured to, for all N data instances d containing each interpretable feature j... i By calculating the following formula, the local importance of interpretable feature j is aggregated into all N data instances d at the global level. i :

[0028]

[0029] Where k* represents the class label with the largest average probability change and represents the direction of the global importance of the interpretable feature j', and where The relationship between interpretable feature j' and each data instance d i The local importance associated with each class k. The global importance generation module 228 is also configured to calculate the association magnitude of the global importance of each interpretable feature j as follows:

[0030]

[0031] Among the related Indicates the magnitude of global importance.

[0032] Figure 3 The data flow diagram 300 illustrates, in some cases, how to calculate the magnitude and direction of global importance for a univariate interpretable feature (e.g., "insufficiency") based on various aspects of this disclosure. It should be understood that global importance measurement can be viewed as an aggregation of local importance, where the basic assumption is that if removing a feature can more or less significantly change the predicted probability, then that feature is more or less important.

[0033] Refer again Figure 2 The global importance generation module 228 is also configured to transmit the result set of k* as the corresponding global importance direction 232, and to associate it with... The result set is transmitted as the corresponding global importance magnitude 236. In some embodiments, the global importance generation module 228 may include one or more corresponding data processing systems in the data processing system, such as data processing system 500 (data processing system 500 itself is not in the data processing system). Figure 2 It is clearly shown in the document, but see also Figure 5 ).

[0034] The machine learning prediction system 200 also includes an interpretable feature ranking module 240. The interpretable feature ranking module 240 is transmissively coupled to the global importance generation module 228. The interpretable feature ranking module 240 is configured to receive transmissions of global importance directions 232 and global importance magnitudes 236. The interpretable feature ranking module 240 is also configured to sort interpretable features j in descending order based on their respective global importance magnitudes 236. The interpretable feature ranking module 240 is further configured to transmit the number T of one or more features in this highest ranking of interpretable features j, along with their respective global importance directions 232, as T highest-ranked interpretable features 244 and T global importance directions 248, respectively, for final presentation to one or more human evaluators 252. In some embodiments, the interpretable feature ranking module 240 may include one or more corresponding data processing systems, such as data processing system 500 (data processing system 500 itself is not included in...). Figure 2 It is clearly shown in the document, but see also Figure 5 ).

[0035] Human evaluator 252 may include experts, online population workers, and / or anyone else capable of identifying those features among the T highest-ranked interpretable features 244, where any one or more of the corresponding directions among the T global importance directions 248 are not what human evaluator 252 believes they should be. For example, such an evaluation task may include human evaluator 252 making a yes-or-no judgment (e.g., affirmative or negative in the case of a sentiment analysis task) for each feature in the T highest-ranked interpretable features 244 and each corresponding direction among the T global importance directions 248, where human evaluator 252 is asked whether each corresponding direction among the T global importance directions 248 appears correct. An example task question might be "Is the word 'panera' positive sentiment polarization?", where the evaluator expects to choose "no" as the answer. Among the T highest-ranked interpretable features 244, the consensus of the human evaluator 252 can determine the number E of one or more of them to have the corresponding orientation among the T global importance orientations 248 that are problematic or incorrect (i.e., not what the consensus of the human evaluator 252 believes they should be). As further described below, the machine learning prediction system 200 can use the corresponding identifiers of the E problematic interpretable features 260.

[0036] The machine learning prediction system 200 also includes a new prediction generation module 264. The new prediction generation module 264 is configured to receive a machine learning base model 216 and a second input dataset 268. The new prediction generation module 264 is also configured to run the machine learning base model 216 on the second input dataset 268 to generate a new prediction 272 based on the second input dataset 268. The new prediction generation module 264 is also configured to transmit the new prediction 272. In some embodiments, the new prediction generation module 264 may include one or more corresponding data processing systems, such as data processing system 500 (data processing system 500 itself is not in...). Figure 2 It is clearly shown in the document, but see also Figure 5 ).

[0037] The machine learning prediction system 200 also includes a local error score generation module 276. The local error score generation module 276 is transmissively coupled to a local importance generation module 212, a global importance generation module 228, and a new prediction generation module 264. The local error score generation module 276 is configured to receive transmissions of local importance 224, transmissions of global importance direction 232, transmissions of new predictions 272, and transmissions of identifiers of E problematic interpretable features 260. It should be understood that even if the T highest-ranked interpretable features 244 can help identify problematic predictions on unlabeled instances, in some instances, labeling errors at the instance level solely based on the problematic interpretable features identified by the identifiers of the E problematic interpretable features 260 can sometimes be unreliable. For example, as... Figure 1 As shown, noting that the incorrect learning of "panera" as "positive" can help accurately identify the incorrect prediction in sentence 2(a); however, its incorrect impact on sentence 2(b) is masked by the presence of another positive feature, "good," which can be correctly learned at the global level. See again Figure 2 To more accurately identify problematic predictions, the local error score generation module 276 is also configured to calculate a metric called the local error score e to address the relative impact of global error features on local levels. More specifically, the local error score generation module 276 is configured to calculate the local error score e for each new prediction in the new prediction 272, as a normalized version of the cumulative error contribution of the problematic interpretable features identified by the identifier of the problematic interpretable feature 260 in the new prediction 272, as follows:

[0038]

[0039] in Let represent the local contribution of the erroneous explainable feature j on a specific instance, where m represents the total number of erroneous features identified from a global perspective. This represents the local contribution of explainable feature i, where the global importance direction of i is the same as the corresponding prediction of the new prediction 272, and where n represents the total number of explainable features with positive contributions. For example, refer to... Figure 1 As can be seen, sentence 2(a) yields a much higher local error score (0.926) than sentence 2(b) (0.502). It should be understood that each local error score e will have a value between -∞ and 1.

[0040] See again Figure 2The local error score generation module 276 is further configured to associate each local error score in the local error score e with a corresponding new prediction in the new prediction 272, and transmit the resulting set as a prediction 280 for the score e. In some embodiments, the local error score generation module 276 may include one or more data processing systems corresponding to a data processing system, such as data processing system 500 (data processing system 500 itself is not in...). Figure 2 It is clearly stated in the document, but see also Figure 5 ).

[0041] The machine learning prediction system 200 also includes an output logic module 284. The output logic module 284 is transmissively coupled to the local error score generation module 276. The output logic module 284 is configured to receive the transmission of e-score predictions 280. The output logic module 284 is also configured to determine a corresponding error designation for each prediction in the e-score predictions 280 (and therefore also for each prediction in the associated new predictions 272) based on whether the corresponding local error score e for the prediction in the e-score predictions 280 exceeds a predefined threshold τ. For example, in some embodiments, the output logic module 284 is configured to designate each prediction in the e-score predictions 280 (and therefore each of the associated new predictions 272) that has a local error score e exceeding the predefined threshold τ as "problematic" or "problematic new prediction". The output logic module 284 is also configured to automatically generate a corresponding prediction in the new predictions 288 for correction of each prediction in the corresponding e-score predictions 280 based on whether the prediction in the corresponding e-score predictions 280 exceeds a predetermined threshold τ. For example, when a specific prediction in e-score prediction 280 is "no" or "negative" and the local error score e for the prediction in e-score prediction 280 exceeds a predefined threshold τ, output logic module 284 can automatically generate a "yes" or "positive" prediction as the corresponding prediction in the corrected new prediction 288. Conversely, when a specific prediction in e-score prediction 280 is "yes" or "positive" and the local error score e for the prediction in e-score prediction 280 exceeds a predefined threshold τ, output logic module 284 can automatically generate a "no" or "negative" prediction as the corresponding prediction in the corrected new prediction 288. Output logic module 284 is configured such that for each prediction in the corrected new prediction 288, the prediction in the corrected new prediction 288 is the same as the prediction in the new prediction 272, wherein for the corresponding prediction in the corrected new prediction 288, the corresponding prediction in e-score prediction 280 does not exceed a predetermined threshold τ.

[0042] Output logic module 284 is also configured to transmit corrected new predictions 288. Output logic module 284 is also configured to transmit any e-score prediction 280 (and therefore any associated new prediction 272) with a corresponding local error score e exceeding a predefined threshold T as a corresponding set of one or more problematic new predictions 292. In some embodiments, output logic module 284 may include one or more corresponding data processing systems in a data processing system, such as data processing system 500 (data processing system 500 itself is not in...). Figure 2 It is clearly shown in the document, but see also Figure 5 ).

[0043] Figure 4 This is a flowchart illustration of a machine learning prediction process 400 according to various aspects of this disclosure. In some cases, one or more steps of the machine learning prediction process 400 may be performed by one or more components of the machine learning prediction system 200 and / or one or more other systems, components, methods, and / or processes described herein. For clarity, the following description of the machine learning prediction process 400 may refer to one or more such systems, components, methods, and / or processes. However, it should be understood that the machine learning prediction process 400 and / or any one or more particular steps thereof may be performed by any suitable system(s), component(s), method(s), and / or process(s). It should also be understood that the machine learning prediction process 400 may be performed simultaneously or substantially simultaneously with any other method(s) and / or process(s) described herein.

[0044] At step 412, the machine learning prediction process 400 receives the machine learning base model. Therefore, in some cases, the local importance generation module 212 and / or the new prediction generation module 264 may receive the transmission of the machine learning base model 216. From step 412, the machine learning prediction process 400 proceeds to (and continues to) step 418.

[0045] At step 418, the machine learning prediction process 400 receives the first input dataset. Therefore, in some cases, the local importance generation module 212 may receive the transmission of the first input dataset 220. From step 418, the machine learning prediction process 400 proceeds to (and continues to) step 424.

[0046] At step 424, the machine learning prediction process 400 uses a data perturbation process to run a machine learning base model on the first input dataset to generate baseline predictions through the machine learning base model, and determines the local importance of the interpretable features of the machine learning base model for each prediction class of the machine learning base model for each baseline prediction. Therefore, in some cases, the local importance generation module 212 may use the LIME data perturbation technique to run a machine learning base model 216 on the first input dataset 220 to determine local importance 224. From step 424, the machine learning prediction process 400 proceeds to (and continues to) step 430.

[0047] At step 430, the machine learning prediction process 400 aggregates the local importance of interpretable features of the machine learning base model to determine the global importance of the interpretable features to the entire machine learning base model. Therefore, in some cases, the global importance generation module 228 can use the local importance 224 to determine the global importance direction 232 and the global importance magnitude 236. From step 430, the machine learning prediction process 400 proceeds to (and continues to) step 436.

[0048] At step 436, the machine learning prediction process 400 sorts the interpretable features according to their corresponding global importance magnitude values. Therefore, in some cases, the interpretable feature sorting module 240 can sort the interpretable features in descending order based on their respective global importance magnitudes 236. From step 436, the machine learning prediction process 400 proceeds to (and continues to) step 442.

[0049] At step 442, the machine learning prediction process 400 transmits one or more features from the highest-ranked interpretable features, and transmits the corresponding global importance direction label for each of these highest-ranked interpretable features. Therefore, in some cases, the interpretable feature ranking module 240 may transmit one or more of the highest-ranked interpretable features j, along with their corresponding global importance directions 232, as T highest-ranked interpretable features 244 and T global importance directions 248, respectively, to be ultimately presented to one or more human evaluators 252. From step 442, the machine learning prediction process 400 proceeds to (and continues to) step 448.

[0050] At step 448, the machine learning prediction process 400 receives a transmission of each of one or more of the highest-ranked interpretable features, which have an incorrect global importance orientation according to one or more consistent human assessments. Thus, in some instances, the local error score generation module 276 may receive a transmission of an identifier of the problematic interpretable feature 260. From step 448, the machine learning prediction process 400 proceeds to (and continues to) step 454.

[0051] In step 454, the machine learning prediction process 400 receives a second input dataset. Therefore, in some cases, the new prediction generation module 264 may receive the transmission of the second input dataset 268. From step 454, the machine learning prediction process 400 proceeds to (and continues to) step 460.

[0052] In step 460, the machine learning prediction process 400 runs a machine learning base model on the second input dataset to generate one or more new predictions. Therefore, in some cases, the new prediction generation module 264 may run the machine learning base model 216 on the second input dataset 268 to generate a new prediction 272. From step 460, the machine learning prediction process 400 proceeds to (and continues to) step 466.

[0053] In step 466, the machine learning prediction process 400 calculates a local error score for each new prediction in the new predictions of the machine learning base model, as a normalized version of the cumulative error introduced into each new prediction by the highest-ranked interpretable feature, which has a corresponding error global importance orientation according to the corresponding consensus human assessment. Thus, in some instances, the local error score generation module 276 can calculate a local error score e for each prediction in the new predictions 272, as a normalized version of the cumulative error contribution in each prediction of the new predictions 272 introduced by the identifier of the problematic interpretable feature 260. From step 466, the machine learning prediction process 400 proceeds to (and continues to) step 472.

[0054] At step 472, the machine learning prediction process 400 determines the corresponding error specification of the prediction in the new prediction based on whether the corresponding local error score of each new prediction in the machine learning base model exceeds a threshold. Therefore, in some instances, the output logic module 284 may determine the corresponding error specification of each prediction in e-score prediction 280 (and thus also for each prediction in the associated new prediction 272) based on whether the corresponding local error score e of the prediction in e-score prediction 280 exceeds a predefined threshold τ. From step 472, the machine learning prediction process 400 proceeds to (and continues to) step 478.

[0055] At step 478, the machine learning prediction process 400 transmits each prediction in the new predictions of the machine learning base model and an indication of the corresponding error specification of that new prediction, and / or transmits each corresponding corrected prediction. Therefore, in some cases, the output logic module 284 may transmit a corrected new prediction 288 and / or transmit one or more corresponding problematic new predictions 292.

[0056] Figure 5 This is a block diagram illustrating the hardware architecture of a data processing system 500 according to various aspects of this disclosure. In some embodiments, one or more systems and / or components described herein (e.g., machine learning prediction system 200 and / or one or more components thereof) may be implemented using a corresponding one or more data processing systems 500. Furthermore, the data processing system 500 may be configured to store and execute one or more instructions for performing one or more steps of the machine learning prediction process 400 and / or for performing one or more steps of any other methods and / or processes described herein.

[0057] The data processing system 500 employs a hub architecture including a northbridge and memory controller hub (“NB / MCH”) 506 and a southbridge and input / output (“I / O”) controller hub (“SB / ICH”) 510. A processor 502, main memory 504, and graphics processor 508 are connected to the NB / MCH 506. The graphics processor 508 can be connected to the NB / MCH 506 via an Accelerated Graphics Port (“AGP”). A computer bus, such as bus 532 or bus 534, can be implemented using any type of transport structure or architecture that provides data transfer between different components or devices attached to that structure or architecture.

[0058] Network adapter 516 is connected to SB / ICH 510. Audio adapter 530, keyboard and mouse adapter 522, modem 524, read-only memory (“ROM”) 526, hard disk drive (“HDD”) 512, optical disc read-only memory (“CD-ROM”) drive 514, universal serial bus (“USB”) port and other transport ports 518, and peripheral component interconnect / peripheral component interconnect fast (“PCI / PCIe”) device 520 are connected to SB / ICH 510 via buses 532 and 534. PCI / PCIe devices may include, for example, Ethernet adapters, interposer cards, and personal computing (“PC”) cards for notebook computers. PCI uses a card bus controller, while PCIe does not. ROM 526 may include, for example, a flash basic input / output system (“BIOS”). Modem 524 or network adapter 516 can be used to send and receive data over a network.

[0059] HDD 512 and CD-ROM drive 514 are connected to SB / ICH 510 via bus 534. HDD 512 and CD-ROM drive 514 may use, for example, an integrated drive electronics (“IDE”) or Serial Advanced Technology Attachment (“SATA”) interface. In some embodiments, HDD 512 may be replaced by other forms of data storage devices, including but not limited to solid-state drives (“SSD”). Super I / O (“SIO”) device 528 may be connected to SB / ICH 510. SIO device 528 may include a chip on the motherboard configured to help perform less demanding controller functions for SB / ICH 510, such as controlling a printer port, controlling a fan, and / or controlling small light-emitting diodes (“LEDs”) of the data processing system 500.

[0060] The data processing system 500 may include a single processor 502 or may include multiple processors 502. Additionally, the processor 502 may have multiple cores. In some embodiments, the data processing system 500 may employ a large number of processors 502, including hundreds or thousands of processor cores. In some embodiments, the processors 502 may be configured to perform coordinated computation sets in parallel.

[0061] The operating system is executed on the data processing system 500 using processor 502. The operating system coordinates and provides control over various components within the data processing system 500. Various applications and services can run in conjunction with the operating system. Instructions for the operating system, applications, and other data reside on storage devices such as one or more HDDs, including HDD 512, and can be loaded into main memory 504 for execution by processor 502. In some embodiments, additional instructions or data may be stored on one or more external devices. The processes described herein with respect to illustrative embodiments can be executed by processor(s) 502 using computer-usable program code, which may reside in memory such as main memory 504, ROM 526, or on one or more peripheral devices.

[0062] This invention can be a system, method, and / or computer program product at any possible level of integration technical detail. A computer program product may include a computer-readable storage medium (or medium) having computer-readable program instructions thereon for causing a processor to execute aspects of the invention.

[0063] Computer-readable storage media can be tangible devices capable of retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanically encoded devices (such as punched cards or raised structures in slots on which instructions are recorded), and any suitable combination of the foregoing. Computer-readable storage media as used herein should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0064] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device, or downloaded via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network) to an external computer or external storage device. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.

[0065] Computer-readable program instructions for performing the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or connected to an external computer (e.g., via the Internet provided by an Internet service provider). According to aspects of the invention, electronic circuits including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) can be personalized to execute computer-readable program instructions by utilizing state information from the computer-readable program instructions in order to perform aspects of the invention.

[0066] This document describes aspects of the invention with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0067] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more boxes of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, and / or other apparatus to function in a particular manner, such that the computer-readable storage medium in which the instructions are stored includes an article of writing comprising instructions for implementing aspects of the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0068] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device, thereby producing a computer-implemented method, such that the instructions that execute on the computer, other programmable apparatus or other device implement the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0069] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each box in a flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing a specified logical function(s). In some alternative implementations, the functions indicated in the boxes may not occur in the order shown in the figures. For example, two boxes shown consecutively may actually be completed as a single step, executed concurrently or substantially concurrently in a manner that overlaps partially or entirely in time, or these boxes may sometimes be executed in reverse order, depending on the functions involved. It will also be noted that each box in the block diagrams and / or flowcharts, and combinations of boxes in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0070] Various embodiments of the invention have been described for illustrative purposes, but these descriptions are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. Furthermore, the steps of the methods described herein can be performed in any suitable order, or simultaneously where appropriate. The terminology used herein is chosen to best explain the principles of the embodiments, their practical application, or technical improvements to existing technologies in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for correcting erroneous predictions of a machine learning base model for a user, the method comprising: Determine multiple global importance magnitude values ​​representing multiple interpretable features of the machine learning base model to the global importance of the machine learning base model, wherein the machine learning base model includes a pre-trained classification model for sentiment analysis, intent prediction, image classification tasks, or other pre-trained machine learning models that are considered black boxes, and the interpretable features include at least one of words or phrases of text data, superpixel or pixel-level features of images. Determine multiple global importance direction labels representing the global importance of the interpretable features of the machine learning base model to the machine learning base model; The interpretable features are ranked according to the global importance magnitude value associated with them; Generate a delivery for presentation to the user, wherein the delivery includes one or more of the highest-ranked interpretable features from the ranked interpretable features, and the global importance direction label associated with the highest-ranked interpretable feature; as well as Based on the global importance directional labels of one or more errors associated with the one or more highest-ranked interpretable features, a local error score is calculated for one or more new predictions of the machine learning base model.

2. The method according to claim 1, further comprising: Receive error assessments that identify the global importance direction labels of the one or more errors.

3. The method according to claim 2, further comprising: Based on the local error scores, the one or more new predictions are corrected or the underlying machine learning model is modified.

4. The method of claim 1, wherein receiving an error assessment of the global importance direction label comprises: Receive human error assessments of the global importance direction labels.

5. A method for alerting a user to erroneous predictions of a machine learning base model, the method comprising: The machine learning base model is run on a first input dataset to generate baseline prediction pairs from the machine learning base model, and the local importance of a first interpretable feature of the machine learning base model to the predicted class of the machine learning base model is determined, wherein the machine learning base model includes a pre-trained classification model for sentiment analysis, intent prediction, image classification tasks, or other pre-trained machine learning models that are treated as black boxes, and the first interpretable feature includes at least one of words or phrases in text data, superpixel or pixel-level features of images; Based on the local importance of the first interpretable feature of the machine learning base model, the global importance of the first interpretable feature of the machine learning base model to the machine learning base model is determined. Run the machine learning base model on the second input dataset to generate new predictions from the machine learning base model; Based on the local importance of the first interpretable feature of the machine learning base model and the global importance of the first interpretable feature of the machine learning base model, an error specification for the new prediction is determined. as well as The new prediction and an indication of the error specified in the new prediction are transmitted for presentation to the user.

6. The method according to claim 5, further comprising: Based on the global importance magnitude value, the ranking of the first interpretable feature of the machine learning base model relative to the second interpretable feature of the machine learning base model is determined; Based on the ranking of the first interpretable features of the machine learning base model, the first interpretable features of the machine learning base model are transmitted. Transmit global importance direction labels; Receive the transmission of the first interpretable feature of the machine learning base model; as well as Calculate the local error score for the new prediction as a normalized version of the error introduced into the new prediction by the first interpretable feature of the machine learning base model. The global importance of the first interpretable feature of the machine learning base model includes the global importance magnitude value and the global importance direction label. According to human assessment, the global importance direction label is incorrect, and Determining the error designation for the new prediction includes: determining the error designation for each new prediction based on whether the local error score for the new prediction exceeds a threshold.

7. The method of claim 6, wherein running the machine learning base model on the first input dataset to generate the baseline prediction pair from the machine learning base model, and determining the local importance of the first interpretable feature of the machine learning base model to the prediction class of the machine learning base model comprises: The data perturbation process is used to generate the baseline prediction pairs from the machine learning base model, and the local importance of the first interpretable feature of the machine learning base model to the prediction class of the machine learning base model is determined.

8. The method of claim 7, wherein the data perturbation process is used to generate the baseline prediction pair from the machine learning base model, and determining the local importance of the first interpretable feature of the machine learning base model to the prediction class of the machine learning base model comprises: calculate in j This indicates the first interpretable feature. in This represents the first data instance in the first input dataset. in Including the first interpretable feature j , in In addition to the first interpretable feature j All other first data instances , in This represents the prediction class corresponding to the machine learning base model. in This indicates that the machine learning base model is derived from the first data instance. The first prediction probability for the prediction class k corresponding to the machine learning base model is generated by running the machine learning base model on it. in This indicates that the machine learning base model originates from... The second prediction probability for the prediction class k corresponding to the machine learning base model is generated by running the machine learning base model on it, and in This represents the first interpretable feature associated with the corresponding prediction class k. j For the first data instance The aforementioned local importance.

9. The method of claim 8, wherein determining the global importance of the first interpretable feature of the machine learning base model to the machine learning base model based on the local importance of the first interpretable feature of the machine learning base model comprises: calculate Where N represents the cardinality of the data instance set in the first input dataset. Each data instance in the data instance set includes the first interpretable feature, and in The global importance direction label representing the global importance of the first interpretable feature.

10. The method of claim 9, wherein determining the global importance of the first interpretable feature of the machine learning base model to the machine learning base model based on the local importance of the first interpretable feature of the machine learning base model comprises: calculate in The magnitude of the third predicted probability represents the global importance directional label generated by the machine learning base model and associated with the global importance of the first interpretable feature. in The global importance magnitude value representing the global importance of the first interpretable feature.

11. The method of claim 10, wherein calculating the local error score for the new prediction comprises: calculate in Indicates the first corresponding interpretable feature j The contribution of the machine learning base model to the predicted probabilities generated by running the machine learning base model on a second data instance in the second input dataset. According to the aforementioned human assessment, each globally important directional label in the global importance directional label set is incorrect. in m This represents the cardinality of the global importance direction label set. in Indicates the second corresponding interpretable feature i The contribution of the predicted probability generated by the machine learning base model from the second data instance in the second input dataset. Wherein the second corresponding interpretable feature i The direction of contribution is consistent with the new prediction. in n This represents the cardinality of the interpretable feature set, and The contribution direction of each interpretable feature in the interpretable feature set is consistent with the new prediction.

12. A system for correcting erroneous predictions of a user's underlying machine learning model, the system comprising: Memory containing instructions; as well as At least one processor communicating with the memory, wherein the at least one processor is configured to execute the instructions to: Determine multiple global importance magnitude values ​​representing multiple interpretable features of the machine learning base model to the global importance of the machine learning base model, wherein the machine learning base model includes a pre-trained classification model for sentiment analysis, intent prediction, image classification tasks, or other pre-trained machine learning models that are considered black boxes, and the interpretable features include at least one of words or phrases of text data, superpixel or pixel-level features of images. Determine multiple global importance direction labels representing the global importance of the interpretable features of the machine learning base model to the machine learning base model; The interpretable features are ranked according to the global importance magnitude value associated with them; Generate a delivery for presentation to the user, wherein the delivery includes one or more of the highest-ranked interpretable features from the ranked interpretable features, and the global importance direction label associated with the highest-ranked interpretable feature; as well as Based on the global importance directional labels of one or more errors associated with the one or more highest-ranked interpretable features, a local error score is calculated for one or more new predictions of the machine learning base model.

13. The system of claim 12, wherein the at least one processor is further configured to execute the instructions to: Receive error assessments that identify the global importance direction labels of the one or more errors.

14. The system of claim 13, wherein the at least one processor is further configured to execute the instructions to correct the one or more new predictions or modify the machine learning base model based on the local error score.

15. The system of claim 12, wherein the at least one processor is further configured to execute the instructions to receive a human error assessment of the global importance direction label.

16. A system for alerting a user to erroneous predictions of a machine learning underlying model, the system comprising: Memory containing instructions; as well as At least one processor communicating with the memory, wherein the at least one processor is configured to execute the instructions to: The machine learning base model is run on a first input dataset to generate baseline prediction pairs from the machine learning base model, and the local importance of a first interpretable feature of the machine learning base model to the predicted class of the machine learning base model is determined, wherein the machine learning base model includes a pre-trained classification model for sentiment analysis, intent prediction, image classification tasks, or other pre-trained machine learning models that are treated as black boxes, and the first interpretable feature includes at least one of words or phrases in text data, superpixel or pixel-level features of images; Based on the local importance of the first interpretable feature of the machine learning base model, the global importance of the first interpretable feature of the machine learning base model to the machine learning base model is determined. Run the machine learning base model on the second input dataset to generate new predictions from the machine learning base model; Based on the local importance of the first interpretable feature of the machine learning base model and the global importance of the first interpretable feature of the machine learning base model, an error specification for the new prediction is determined. as well as The new prediction and an indication of errors in the new prediction are transmitted for presentation to the user.

17. The system of claim 16, wherein the at least one processor is further configured to execute the instructions to: Based on the global level importance magnitude value, the ranking of the first interpretable feature of the machine learning base model relative to the second interpretable feature of the machine learning base model is determined; Based on the ranking of the first interpretable features of the machine learning base model, the first interpretable features of the machine learning base model are transmitted. Transmit global importance direction labels; Receive the transmission of the first interpretable feature of the machine learning base model; Calculate the local error score for the new prediction as a normalized version of the error introduced into the new prediction by the first interpretable feature of the machine learning base model; and The error assignment for each new prediction is determined based on whether the local error score for the new prediction exceeds a threshold. The global importance of the first interpretable feature of the machine learning base model includes: the global importance magnitude value and the global importance direction label. The global importance direction label mentioned therein has been identified as incorrect.

18. The system of claim 17, wherein the at least one processor is further configured to execute the instructions to generate the baseline prediction pair from the machine learning base model using a data perturbation process, and to determine the local importance of the first interpretable feature of the machine learning base model to the prediction class of the machine learning base model.

19. The system of claim 18, wherein the at least one processor is further configured to execute the instructions to calculate: in j This indicates the first interpretable feature. in This represents the first data instance in the first input dataset. in Including the first interpretable feature j , in In addition to the first interpretable feature j All other first data instances , in This represents the prediction class corresponding to the machine learning base model. in This indicates that the machine learning base model is derived from the first data instance. The first prediction probability for the prediction class k corresponding to the machine learning base model is generated by running the machine learning base model on it. in This indicates that the machine learning base model originates from... The second prediction probability for the prediction class k corresponding to the machine learning base model is generated by running the machine learning base model on it, and in This represents the first interpretable feature associated with the corresponding prediction class k. j For the first data instance The aforementioned local importance.

20. The system of claim 19, wherein the at least one processor is further configured to execute the instructions to calculate: Where N represents the cardinality of the data instance set in the first input dataset. Each data instance in the data instance set includes the first interpretable feature, and in The global importance direction label representing the global importance of the first interpretable feature.

21. The system of claim 20, wherein the at least one processor is further configured to execute the instructions to calculate: in The magnitude of the third predicted probability represents the global importance directional label generated by the machine learning base model and associated with the global importance of the first interpretable feature. in The global importance magnitude value representing the global importance of the first interpretable feature.

22. The system of claim 21, wherein the at least one processor is further configured to execute the instructions to calculate: in Indicates the first corresponding interpretable feature j The contribution of the machine learning base model to the predicted probabilities generated by running the machine learning base model on a second data instance in the second input dataset. According to human assessment, every globally important directional label in the global importance directional label set is incorrect. in m This represents the cardinality of the global importance direction label set. in Indicates the second corresponding interpretable feature i The contribution of the predicted probability generated by the machine learning base model from the second data instance in the second input dataset. Wherein the second corresponding interpretable feature i The direction of contribution is consistent with the new prediction. in n This represents the cardinality of the interpretable feature set, and The contribution direction of each interpretable feature in the interpretable feature set is consistent with the new prediction.

23. A computer program product for alerting a user to erroneous predictions of a machine learning base model, the computer program product comprising a computer-readable storage medium having program instructions contained therein, the program instructions being executable by at least one processor to cause the at least one processor to: The machine learning base model is run on a first input dataset to generate baseline prediction pairs from the machine learning base model, and the local importance of a first interpretable feature of the machine learning base model to the predicted class of the machine learning base model is determined, wherein the machine learning base model includes a pre-trained classification model for sentiment analysis, intent prediction, image classification tasks, or other pre-trained machine learning models that are treated as black boxes, and the first interpretable feature includes at least one of words or phrases in text data, superpixel or pixel-level features of images; Based on the local importance of the first interpretable feature of the machine learning base model, the global importance of the first interpretable feature of the machine learning base model to the machine learning base model is determined. Run the machine learning base model on the second input dataset to generate new predictions from the machine learning base model; The error specification for the new prediction is determined based on the local importance of the first interpretable feature of the machine learning base model and the global importance of the first interpretable feature of the machine learning base model. as well as The new prediction and an indication of the error in the new prediction are transmitted for presentation to the user.

24. The computer program product of claim 23, wherein the program instructions are further executable by the at least one processor to cause the at least one processor to: Based on the global importance magnitude value, the ranking of the first interpretable feature of the machine learning base model relative to the second interpretable feature of the machine learning base model is determined; Based on the ranking of the first interpretable features of the machine learning base model, the first interpretable features of the machine learning base model are transmitted. Transmit global importance direction labels; Receive the transmission of the first interpretable feature of the machine learning base model; Calculate the local error score for the new prediction as a normalized version of the error introduced into the new prediction by the first interpretable feature of the machine learning base model; and The error assignment for each new prediction is determined based on whether the local error score for the new prediction exceeds a threshold, and The global importance of the first interpretable feature of the machine learning base model includes the global importance magnitude value and the global importance direction label, and The global importance direction label mentioned therein has been identified as incorrect.

25. The computer program product of claim 24, wherein the program instructions are further executable by the at least one processor to cause the at least one processor to use a data perturbation process to generate the baseline prediction pair from the machine learning base model, and to determine the local importance of the first interpretable feature of the machine learning base model to the prediction class of the machine learning base model.

Images