Removing unwanted inferences from machine learning models

Abstract

A method and system (100) for removing unwanted inferences from a machine learning model (102) is provided. The system includes a search component (118) configured to receive rejected explanations (110) of a model output (106) provided by the machine learning model (102), identify data samples (120) to reject by selecting training samples from training data (104) used to train the machine learning model (102), the selected training samples being associated with explanations that are similar to the rejected explanation according to a calculated similarity measure, and pass the data samples (120) to a machine learning rejection unit (122).

Description

[Technical field]

[0001] The present invention relates to a system and method for removing unwanted inferences from machine learning models. [Background technology]

[0002] The lack of insight provided regarding the "reasoning" of machine learning (ML) models still gives end users no confidence and limits the acceptance of ML in industry, even when the end users are domain experts. Systems have been proposed to explain and contextualize ML output in technical systems in industrial applications, which promotes the acceptance of ML decision support. However, domain experts may still lose confidence in ML models if either or both of the model output and its explanation are perceived to be inaccurate. ML models are typically trained using historical data, and over time, it is not uncommon that previously inferred rules may no longer be valid. At any given time, the rules learned by an ML model may contradict the domain expert's expertise. Machine learning unlearning (MuL) has been proposed as a way to "unlearn" or remove undesirable inferences, as described, for example, in L. BOURTOULE et al., Machine Unlearning. arXiv:1912.03817, 2020. Summary of the Invention

[0003] There is a need to identify sources of unwanted inferences that should be removed from ML models. This need is met by the subject matter of the independent claims. Optional features are set out in the dependent claims and the following description.

[0004] According to a first aspect, a system for removing undesirable inferences from a machine learning model is provided. The system includes a search component configured to receive rejected explanations of a model output provided by the machine learning model, identify data samples to be rejected by selecting training samples from training data used to train the machine learning model, the selected training samples being related to explanations similar to the rejected explanations (e.g., according to a calculated similarity measure, the calculated similarity measure meets (e.g., exceeds) a predetermined threshold), and pass the data samples to be rejected to a machine learning rejection unit.

[0005] In this way, a machine learning pipeline may be provided that includes not only an explainer path (to explain the model output) but also a feedback path (to correct or improve the model), providing domain experts with the ability to tune the ML model by leveraging the model output explanations. The system may be applied in industrial environments to improve the quality of the ML model output. The system promises high potential in the field of transfer learning, where trained machine learning models are used as a starting point for training new industrial processes, since it offers the possibility to learn and reject predictions and explanations that do not apply to the new process.

[0006] In one implementation, the search component may be configured to identify the data samples to be learned and rejected by obtaining a training explanation associated with each selected candidate training sample from the training data, calculating an explanation similarity measure between the rejected explanation and the training explanation, comparing the calculated explanation similarity measure to an explanation similarity threshold, and selecting the training sample for inclusion in the data samples to be learned and rejected in response to the calculated explanation similarity measure satisfying the explanation similarity threshold. The search component may be further configured to calculate a sample similarity measure between the rejected sample and the training sample, compare the calculated sample similarity measure to the sample similarity threshold, and identify as the data samples to be learned and rejected only those selected training samples whose calculated sample similarity measure also satisfies the sample similarity threshold. The search component may be further configured to repeat the obtaining, calculating, comparing, and selecting for one or more additional training samples in the training data.

[0007] "Similarity measure" means any suitable metric for expressing similarity. In some implementations, the search component may be configured to calculate the explanation similarity measure as either (i) a distance metric between the rejected explanations and the training explanations, the distance metric including one or more of Euclidean distance, city block distance, Mahalanobis distance, Huber distance, or elastic distance measures such as dynamic time warping or Levenshtein distance, or (ii) a Jaccard similarity coefficient. Sample similarity measures described herein may be calculated in the same manner.

[0008] The explanations may be rule-based, in which case the search component may be configured to identify an explanation associated with the training sample as similar to the rejected explanation in response to one or more of: (i) the premise, (ii) the structure, and (iii) the decision value of the rule-based explanation being the same or similar (e.g., within a predetermined similarity threshold).

[0009] If the explanation includes a prototype, the search component may be configured to identify explanations associated with the training sample as similar to the rejected explanation in response to the same prototype being generated.

[0010] The machine learning rejection unit may be configured to remove unwanted inferences from the machine learning model by assigning weights of the training loss function to data samples to be rejected before retraining the machine learning model using an updated set of training data. The weights may be set to 0 to achieve complete rejection of the unwanted data samples, or any weight between 0 and 1 may be used for advantageous partial rejection, if this is desired.

[0011] The weights assigned to data samples for training rejection may be based on a similarity measure calculated for those data samples.

[0012] Weights may be assigned to some, but not all, of the multiple signals in the training data sample, allowing useful signals in the training data sample to be retained for further training of the model.

[0013] The machine learning rejection unit may be configured to incrementally modify the weights based on the implicit function theorem applied to the optimum of the machine model training loss minimization problem to avoid full retraining in favor of incremental training.

[0014] To facilitate complete removal of unwanted inferences, the machine learning rejection unit may be configured to remove unwanted inferences from the machine learning model by retraining the machine learning model using an updated set of training data from which the rejected data samples have been removed.

[0015] Instead of completely removing undesirable samples or reducing their weighting, the machine learning rejection unit may be configured to modify and reject data samples by finding perturbations to the data samples, so that explanations derived from the perturbed data samples match the non-rejected explanations. This offers the expert the possibility to generate artificial training data from existing training data in order to modify the decision boundary of the model in such a way that the obtained explanations better comply with the expert's expertise.

[0016] According to a second aspect, there is provided a method for removing unwanted inferences from a machine learning model, the method comprising: receiving rejected explanations of a model output provided by a machine learning model, identifying data samples to be rejected by selecting training samples from training data used to train the machine learning model, the selected training samples being associated with explanations similar to the rejected explanations according to a calculated similarity measure, and passing the data samples to be rejected to a machine learning rejection unit.

[0017] According to a second aspect of the invention there is provided a computer program product comprising instructions which, when executed by a computer, cause the computer to perform the method of the second aspect.

[0018] According to a fourth aspect, there is provided a non-transitory computer storage medium having stored thereon the computer program product of the third aspect.

[0019] Any optional feature or sub-embodiment of any embodiment may be applied to any other embodiment, as appropriate.

[0020] The above-mentioned aspects and examples will become apparent from and be elucidated with reference to the following detailed description. [Brief description of the drawings]

[0021] Illustrative examples will now be described with reference to the following drawings: [Figure 1] Figure 1 shows a machine learning system that includes not only an explainer pathway but also a feedback pathway. [Diagram 2] FIG. 2 shows the feedback paths in the system of FIG. 1 in more detail. [Diagram 3] FIG. 3 illustrates a method for identifying samples to be rejected. [Figure 4] FIG. 4 illustrates a method for identifying samples to be rejected. [Diagram 5] FIG. 5 illustrates a use case of the system in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] FIG. 1 shows a high-level diagram of a machine learning system 100 that includes an explainer path as well as a feedback path. The system 100 may be applicable independent of the type of ML model whenever an ML model is used in an industrial application. The ML model 102 is trained based on training data 104 to provide an ML model output 106 to an end user 112 when a sample 105 is input to the model 102. The model output 106 may include a recommendation to the end user 112 to take a predefined action. The explainer 108 also receives the model output 106 and provides an explanation 110 of the model output 106 to the end user 112. The path model-explainer-end user constitutes the explainer path. The explanation 110 can be created in any known manner. An example of providing an explanation of an ML model output is described, for example, in the related European Patent Application No. 20 196 232.1. According to the present disclosure, the path end user model constitutes a feedback path, whereby the end user 112 can provide feedback 114 to the ML model 102.

[0023] FIG. 2 illustrates the feedback path in the ML system 100 in more detail. In this example, the end user 112 is a control room operator as the domain expert for which the ML model 102 was created. The operator 112 can use his own expertise to evaluate the explanation 110 provided to him by the explainer 108 and provide feedback 114 as a result. The operator 112 can consider the model output 106 and / or the explanation 110 to be inaccurate and thus use the feedback path to have this undesirable inference removed from the system 100. The operator 112 may, for example, choose whether the MuL unit 122 removes this undesirable inference completely from the system 100 or consider it less important (in the latter case, the affected training data 104 may not be removed but rather assigned a smaller weight in the ML model 102), as will be explained further below.

[0024] An example of an inference that an operator may want to remove may include, for example, an erroneous inference made by the ML model 102 based on training data 104 collected as a result of trainee operator actions. Another example relates to changes in plant equipment or production processes implemented over a period of time such that older rules inferred by the ML model 102 no longer reflect the new reality. Yet another example involves rules inferred by the ML model 102 that are incorrect from an expert point of view, such as rules that were erroneously learned based on training data 104 provided when the plant equipment was in maintenance mode.

[0025] Following the decision to remove the undesirable inference, operator 112 initiates a removal process 116 to remove the undesirable inference and designate explanation 110 as a rejected explanation.

[0026] Because the operator 112 only looks at the model outputs 106 and explanations 110 when requesting unwanted inferences to be removed, a search component 118 is provided to aggregate all raw data points that contributed to the unwanted inference as one or more data samples 120 to be learned and rejected.

[0027] 3 and 4, FIG. 4 illustrates step 302 of FIG. 3 in more detail, where the search component 118 is configured to automatically identify data samples 120 to reject by selecting training samples from the training data 104 that are associated with explanations similar to the rejected explanation.

[0028] For example, the search component 118 may 1. Receive 301 rejected explanations identified by an operator 112 when initiating a removal process 116, the rejected explanations being derived from or related to at least one rejected sample; 2. For example, i. obtaining 401 a training explanation associated with each selected candidate training sample from the training data 104, for example by computing an explanation for the training sample (e.g., by inputting the training sample into the ML model 102 to generate a model output and inputting the model output into the explainer 108) and / or by retrieving a pre-computed explanation associated with the training sample, the pre-computed explanation having been pre-computed for one or more versions of the ML model 102 and stored for later retrieval; ii. calculating an explanation similarity measure between the rejected explanations and the training explanations, and optionally also a sample similarity measure between the rejected samples and the training samples (402); iii. comparing the calculated explanation similarity measure to an explanation similarity threshold and, optionally, comparing the calculated sample similarity measure to a sample similarity threshold (403); iv. selecting (404) training samples for inclusion in the training-rejected data samples 120 in response to the calculated explanation similarity measure satisfying the explanation similarity threshold, the selected training-rejected data samples 120 being optionally further restricted to include only training samples whose calculated sample similarity measures also satisfy the sample similarity threshold; v. Optionally, by repeating 405 steps 401 through 404 for one or more additional training samples, e.g., by traversing all training samples in the training data 104 or by traversing selected representative samples (e.g., cluster centroids derived by grouping all training samples into clusters using an unsupervised clustering algorithm); Identifying (302) data samples 120 to reject by selecting from the training data 105 one or more training samples that are associated with explanations similar to the rejected explanation (and thus contributed to the undesired inference); 3. It is configured to pass (303) the data samples 120 to be rejected for learning to the MuL unit 122.

[0029] If the sample 105 is also used as training data for further development of the ML model 102 , the sample 105 may be incorporated into the learning reject data sample 120 .

[0030] Therefore, as shown in Figures 3 and 4, the process of selecting training samples to be learned and rejected is based on a similarity measure. For both explanations and samples, a respective similarity measure can be used. Generally speaking, if a training sample similar to a rejected sample with a rejected explanation yields an explanation similar to the rejected explanation, it may be a good candidate for learning and rejection. Therefore, two similarity measures can be defined: a sample similarity measure and an explanation similarity measure. In the context of this disclosure, the term "similarity measure" may also be interpreted as including a distance measure as a measure of the difference between two samples or explanations.

[0031] Regarding sample similarity measures, a sample in machine learning can be described as a vector or matrix s with dimension s i (For matrices, x i,j ) often have the same numeric data type

number

number

[0032] With respect to explanation similarity measures, the question of how to measure the similarity of two explanations 110 may depend on the structure of the explanations. Using the same notation, we consider two explanations e 1 and e 2 The similarity (e.g., distance) measure between 1 ,e 2 ) may also be expressed as

[0033] The explanation 110 can include feature explanations. For example, feature explanation methods such as LIME (MTRIBEIRO et al., “Why Should I Trust You?” Explaining the Predictions of Any Classifier) ​​or SHAPE (E. STRUMBELJ et al., Explaining prediction models and individual predictions with feature contributions. Knowledge and Information Systems 41.3, 2014, pages 647-665) can be used to explain each feature x i,j Importance weight w i,jIn this case, feature similarity can be measured in the same way as sample similarity, as described above.

[0034] Another way to provide explanation 100 is by using rules generated by, for example, the ANCHORs method (Anchors: High-precision model-agnostic explanations Thirty-Second AAAI Conference on Artificial Intelligence, 2018) or by a decision tree surrogate model. An example of a rule-based explanation 110 is as follows: Rule 1: Temperature t>100°C and flow rate f<1m 3 / s, the product quality is good.

[0035] Two rule-based explanations may be considered similar if they have the same antecedents and similar decision values ​​within the same structure (considered as features and comparison operators). For example, the following rule may be considered similar to rule 1: Rule 2: Temperature t>105°C and flow rate f<0.8m 3 / s, the product quality is good. The determination of similarity can be expanded by allowing for additional or missing antecedents. For example, the following rules may be considered to be similar to rule 1 and rule 2: Rule 3: If the temperature t>105 degrees, the product quality is good. Methods such as ANCHORs attempt to find rules that apply to a large subset of the training data 104 and therefore generate a small number of rules, and therefore the rule-based process described above is sufficient for most practical cases.

[0036] Another way to provide an explanation 110 is by prototype. In this case, the explainer 108 presents a prototype that represents how machine learning internally represents the sample, and optionally how similar the internal representation of the sample is to the prototype. If the explainer 108 presents only the prototype, then two explanations 110 are similar or identical if the same prototype was created. If the explainer 108 also provides a measure of similarity to the prototype, then this similarity to the prototype can be used. If the similarity or distance to all prototypes is given, a measure like Euclidean distance can be used.

[0037] As described above in connection with Figures 3 and 4, the training explanations for the training samples may be generated on demand or may be generated in advance. t and rejected samples r Similarities between the training and the rejected explanations t and e r In the above-described method of measuring similarity between training samples 120 and 122 (as described above with reference to steps 403 and 404), one way to select training samples for inclusion in the data samples to be learned 120 is by applying two thresholds using the following rules: d(s r ,s t ) <t s And d(e r ,e t ) <t e If so, s t Reject learning, Here, t s is the sample similarity threshold, and t e is the explanation similarity threshold. As explained below, d(s r ,s t ) and d(e r ,e t ) may also be used as weights (or to calculate weights) in the learning rejection process.

[0038] 2, the unlearned combined data samples 120 are passed to a MuL unit 122 that uses the MuL algorithm (as described in Machine Unlearning by BOURTOULE et al.) for unlearning. MuL ​​unit 122 may use any one or more of a number of techniques for unlearning the samples, as described next.

[0039] One easily implementable form of unlearning the unnecessary samples 120 involves the MuL unit 122 retraining the ML model 102 using an updated set of training data 104 in which the unnecessary samples 120 have been removed.

[0040] To increase the flexibility of the forgetting mechanism, as an alternative to the complete removal of the unwanted samples 120, the MuL unit 122 can apply memory factors to the unwanted samples 120, each of which multiplies the term of its corresponding unwanted sample 120 in the training loss function. Choosing a memory factor of 0 is equivalent to removing the unwanted sample 120 from the training data 104, and choosing a memory factor of 1 is equivalent to keeping the sample as before. Further flexibility can be obtained by choosing a memory factor between 0 and 1, for example based on a similarity measure, which means that for example the unwanted samples 120 can also be half-forgotten. For the mapping of the similarity measure to the memory factors, an inverse sigmoid function or a thresholding step function with pre-specified scale and gradient parameters can be used.

[0041] Further flexibility can be obtained by using even finer grained storage coefficients that can vary for different components of the label of the unwanted samples 120. These storage coefficients can be used in part to mask the unwanted samples 120. In the exemplary case of a factory where the temperature sensor location has been modified, instead of completely discarding the unwanted samples 120, the fine grained storage coefficients allow to mask only the now invalid temperature signals and preserve all remaining information of the unwanted samples 120 for training. This can be achieved by setting the storage coefficients corresponding to the temperature signal data to 0 and keeping the storage coefficients of all other signals at 1.

[0042] In addition to the increased flexibility that memory coefficients provide, they can also be exploited by the MuL unit 122 to reduce the computational ML training effort by using a mathematical sensitivity analysis of the optimal ML weights (here, "weights" refers to the degrees of freedom in the ML model 102, not the feature weights of the description 110) with respect to successive memory coefficients, based on the implicit function theorem applied to the optimality of the ML training loss minimization problem. Thus, the MuL unit 122 can modify the optimal weights in response to changes in the memory coefficients to avoid full retraining in favor of incremental training. The application of the implicit function theorem provides that the inverse of the so-called Hessian matrix of the training objective function provides the correct scaling of the increments of the memory coefficients to the optimal ML weight increments. Thus, the approximately optimal ML weight increments can be calculated by iterative linear algebra methods, such as conjugate gradient methods, which require only a Hessian matrix-vector product that modern ML frameworks can efficiently compute for mini-batch approximations of the Hessian matrix.

[0043] Additionally or alternatively, an approach using additive learning and learning rejection may be used by the MuL unit 122, as presented in Towards Making Systems Forget with Machine Unlearning by Y. CAO et al. This approach involves converting the learning algorithm to an additive form. To forget the unnecessary samples 120, the approach only needs to update a small number of additives, which is asymptotically faster than retraining from scratch. This approach is general in that the additive form is from statistical query learning, which can implement many machine learning algorithms.

[0044] FIG. 5 illustrates how an ML system 100 as depicted in FIG. 2 can be applied in an industrial environment. The training data 104 includes two features, temperature and pressure, that determine whether a valve in a monitored system (e.g., factory equipment) should be opened or remain closed (for simplicity, only an excerpt of the training data 104 is shown in FIG. 5). Considering an example where a monitored system provides a temperature of 80,0° C. and a pressure of 3.1 bar as samples 105, the ML model 102 provides a recommendation to open the valve as model output 106. The corresponding inference, presented to a control room operator 112 as explanation 110, is the rule "Temperature >= 80,0 and Pressure > 3.0". Based on the operator's expertise, the operator 112 may consider this explanation 110 to be inaccurate (or no longer valid or applicable) and may therefore wish for this undesired inference to be removed from the system 100 entirely. Therefore, operator 112 initiates pruning process 116, after which search component 118 retrieves all raw data points that contributed to this undesired inference in the manner described above. Figure 5 shows that three data samples 120 were selected for learning rejection from the base training data 104 that met the criteria provided by description 110. These data samples 120 are passed to MuL unit 122 for learning rejection.

[0045] In addition to learning and rejecting the samples 120 that generated rejected explanations, the MuL unit 122 can modify the samples 120 to generate explanations that fit the opinion of the end user 112. To do so, the end user 112 is given the option to modify the explanation 110 or to annotate the relevant samples 105 to capture domain insights. The system 100 then perturbs the data samples so that the output of the explainer 108 matches the modified explanation. Sample perturbation (as described in MTRIBEIRO et al., “Why Should I Trust You?” Explaining the Predictions of Any Classifier”) combined with an optimization or meta-learning process (as described in European Patent Application No. 20 196 232.1) can be used to modify the samples 120 and result in the rejected explanation. If the process is successful in generating a useful new data sample, the original sample 120 can be preserved. The new data sample affects the decision boundary of the ML model 102 so that the explanation 110 provided by the explainer 108 also changes.

[0046] The end user 112 as a domain expert may be replaced by a system (e.g., an expert system) that encapsulates domain knowledge and provides feedback 114 instead of a human domain expert. The expert system may be another ML model. Thus, the decision to completely remove unwanted samples 120 or to de-weight them (as described above) may be made automatically by the expert system based on predefined rules, based on thresholds of the proximity of the explanations to acceptable explanations, or based on additional information about the samples in the affected training data 104. The predefined rules may be based, for example, on the data / actions / knowledge of already modeled human domain experts.

[0047] From the above, it is apparent that one or more computer programs may be provided that include machine-readable instructions that, when executed on one or more computers, cause the one or more computers to perform the methods described and claimed. It is also apparent from the above that a non-transitory computer storage medium and / or download product may store one or more computer programs thereon. One or more computers may then operate with the one or more computer programs. One or more computers may comprise a non-transitory computer storage medium and / or download product.

[0048] While the invention has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description is to be considered exemplary or illustrative and not restrictive. The invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. The following is a summary of the claims as originally filed: [C1] A system (100) for removing unwanted inferences from a machine learning model (102), the system comprising a search component (118), the search component comprising: receiving rejected explanations (110) of the model output (106) provided by the machine learning model (102); identifying data samples (120) to reject by selecting training samples from training data (104) used to train the machine learning model (102), the selected training samples being associated with explanations that are similar to the rejected explanations according to a calculated similarity measure; The system (100) is configured to pass the data samples (120) to be rejected to a machine learning rejection unit (122). [C2] The search component (118) obtaining (401) a training description associated with each candidate training sample selected from the training data (104); calculating (402) an explanation similarity measure between the rejected explanations and the training explanations; comparing (403) the calculated explanation similarity measure to an explanation similarity threshold; and in response to the calculated explanation similarity measure satisfying the explanation similarity threshold, selecting (404) the training sample for inclusion in the learning reject data sample (120). [C3] The system of claim 2, wherein the search component (118) is further configured to calculate a sample similarity measure between the rejected samples and the training samples, compare the calculated sample similarity measure to a sample similarity threshold, and identify as training-rejected data samples (120) only those of the selected training samples whose calculated sample similarity measure also satisfies the sample similarity threshold. [C4] The system of claim 2 or 3, wherein the search component (118) is further configured to repeat (705) the obtaining (401), the calculating (402), the comparing (403), and the selecting (404) for one or more additional training samples in the training data (104). [C5] 5. The system of claim 2, wherein the search component (118) is configured to calculate (402) the explanation similarity measure as either (i) a distance metric between the rejected explanations and the training explanations, the distance metric including one or more of Euclidean distance, city block distance, Mahalanobis distance, Huber distance, or an elastic distance measure such as dynamic time warping or Levenshtein distance, or (ii) a Jaccard similarity coefficient. [C6] The system of any one of claims 1 to 5, wherein the explanations are rule-based, and the search component (118) is configured to identify explanations associated with the training sample as similar to the rejected explanations in response to one or more of the rule-based explanations having the same or similar (i) premise, (ii) structure, and (iii) decision value. [C7] The system of any one of claims 1 to 6, wherein the explanations include prototypes, and the search component (118) is configured to identify explanations associated with the training samples as similar to the rejected explanations in response to the same prototype being generated. [C8] 8. The system of any one of claims 1 to 7, further comprising the machine learning rejection unit (122), configured to remove unwanted inferences from the machine learning model (102) by assigning training loss function weights to the rejected data samples (120) before retraining the machine learning model (102) using an updated set of training data (104). [C9] The system of claim 8, wherein the weights assigned to the rejected data samples (120) are based on the similarity measure calculated for those data samples. [C10] 10. The system of claim 8 or 9, wherein the weights are assigned to some but not all of a plurality of signals in the training discard data samples (120). [C11] 11. The system of any one of C8 to C10, wherein the machine learning rejection unit (122) is configured to incrementally change the weights based on an implicit function theorem applied to an optimality of a machine model training loss minimization problem. [C12] 8. The system of claim 1, further comprising: the machine learning rejection unit (122), configured to remove unwanted inferences from the machine learning model (102) by retraining the machine learning model (102) using an updated set of the training data (104) from which the rejected data samples (120) have been removed. [C13] 8. The system of any one of claims 1 to 7, further comprising the machine learning rejection unit (122), the machine learning rejection unit configured to modify the learning to reject data samples (120) by finding perturbations to the data samples (120), such that explanations derived from the perturbed data samples are consistent with non-rejected explanations. [C14] A method for removing unwanted inferences from a machine learning model (102), comprising: receiving (301) rejected explanations (110) of a model output (106) provided by the machine learning model (102); identifying (302) data samples (120) to reject by selecting training samples from training data (104) used to train the machine learning model (102); the selected training samples are associated with explanations that are similar to the rejected explanations according to a calculated similarity measure; and passing (303) the data samples to be rejected (120) to a machine learning rejection unit (122). [C15] A computer program product comprising instructions which, when executed by a computer, cause the computer to perform the method of C14.

Claims

1. A system (100) for removing unwanted inferences from a machine learning model (102), the system providing model outputs (106) and explanations (110) to an operator (112) by the machine learning model (102), the system comprising a search component (118), the search component comprising: receiving rejected explanations (110) of the model outputs (106) provided by the machine learning model (102), where the rejected explanations (110) correspond to the model outputs (106) that are deemed inaccurate by the operator (112); identifying data samples (120) to reject by selecting training samples from training data (104) used to train the machine learning model (102), the selected training samples being associated with explanations that are similar to the rejected explanations according to a calculated similarity measure; The system (100) is configured to pass the rejected data samples (120) to a machine learning rejection unit (122), where the machine learning rejection unit (122) is configured to remove unwanted inferences from the machine learning model (102) by retraining the machine learning model (102) using an updated set of training data (104) from which the rejected data samples (120) have been removed.

2. The search component (118) obtaining (401) a training description associated with each candidate training sample selected from the training data (104); Computing (402) an explanation similarity measure between the rejected explanations and the training explanations; comparing (403) the calculated explanation similarity measure to an explanation similarity threshold; and selecting (404) the training sample for inclusion in the data sample to be rejected in response to the calculated explanation similarity measure satisfying the explanation similarity threshold.

3. 3. The system of claim 2, wherein the search component (118) is further configured to: calculate a sample similarity measure between the rejected samples associated with the rejected explanations (110) and the training samples; compare the calculated sample similarity measure to a sample similarity threshold; and identify as data samples (120) to be learned and rejected only the selected training samples whose calculated sample similarity measure also satisfies the sample similarity threshold.

4. 4. The system of claim 2 or 3, wherein the searching component (118) is further configured to repeat (705) the obtaining (401), the calculating (402), the comparing (403), and the selecting (404) for one or more additional training samples in the training data (104).

5. 5. The system of claim 2, wherein the search component (118) is configured to calculate (402) the explanation similarity measure as either (i) a distance metric between the rejected explanations and the training explanations, the distance metric including one or more of Euclidean distance, city block distance, Mahalanobis distance, Huber distance, or elastic distance measures such as dynamic time warping or Levenshtein distance, or (ii) a Jaccard similarity coefficient.

6. 6. The system of claim 1, wherein the explanations are rule-based, and the search component (118) is configured to identify explanations associated with the training samples as similar to the rejected explanations in response to one or more of the rule-based explanations having the same or similar (i) premise, (ii) structure, and (iii) decision value.

7. 7. The system of claim 1, wherein the explanations include prototypes, and the search component is configured to identify explanations associated with the training samples as similar to the rejected explanations in response to the same prototype being generated.

8. 8. The system of claim 1, further comprising the machine learning rejection unit (122), configured to remove unwanted inferences from the machine learning model (102) by assigning training loss function weights to the rejected data samples (120) before retraining the machine learning model (102) using an updated set of training data (104).

9. The system of claim 8 , wherein the weights assigned to the rejected data samples (120) are based on the similarity measure calculated for those data samples.

10. The system of claim 8 or 9, wherein the weights are assigned to some but not all of a plurality of signals in the training discard data samples (120).

11. 11. The system of claim 8, wherein the machine learning rejection unit (122) is configured to incrementally modify the weights based on an implicit function theorem applied to an optimum of a machine model training loss minimization problem.

12. A system (100) for removing unwanted inferences from a machine learning model (102), the system providing model outputs (106) and explanations (110) to an operator (112) by the machine learning model (102), the system comprising a search component (118), the search component comprising: receiving rejected explanations (110) of the model outputs (106) provided by the machine learning model (102), where the rejected explanations (110) correspond to the model outputs (106) that are deemed inaccurate by the operator (112); identifying data samples (120) to reject by selecting training samples from training data (104) used to train the machine learning model (102), the selected training samples being associated with explanations that are similar to the rejected explanations according to a calculated similarity measure; The system (100) is configured to pass the learning reject data samples (120) to a machine learning rejection unit (122), where the machine learning rejection unit (122) is configured to modify the learning reject data samples (120) by finding perturbations to the data samples (120) such that explanations derived from the perturbed data samples match explanations that are not rejected.

13. A method for removing unwanted inferences from a machine learning model (102), performed by a system, comprising: providing model outputs (106) and explanations (110) to an operator (112) by the machine learning model (102); receiving (301) rejected explanations (110) of the model outputs (106) provided by the machine learning model (102), wherein the rejected explanations (110) correspond to the model outputs (106) that are deemed inaccurate by the operator (112); identifying (302) training-reject data samples (120) by selecting training samples from training data (104) used to train the machine learning model (102), the selected training samples being associated with explanations similar to the rejected explanations according to a calculated similarity measure; and passing (303) the training-reject data samples (120) to a machine learning rejection unit (122), wherein the machine learning rejection unit (122) removes undesirable inferences from the machine learning model (102) by retraining the machine learning model (102) using an updated set of the training data (104) from which the training-reject data samples (120) have been removed.

14. A method for removing unwanted inferences from a machine learning model (102), performed by a system, comprising: providing model outputs (106) and explanations (110) to an operator (112) by the machine learning model (102); receiving (301) rejected explanations (110) of the model outputs (106) provided by the machine learning model (102), wherein the rejected explanations (110) correspond to the model outputs (106) that are deemed inaccurate by the operator (112); 1. A method for identifying (302) a data sample to reject by selecting training samples from training data (104) used to train the machine learning model (102), the selected training samples being associated with explanations that are similar to the rejected explanations according to a calculated similarity measure, and passing (303) the data sample to reject to a machine learning rejection unit (122), wherein the machine learning rejection unit (122) modifies the data sample to reject by finding a perturbation to the data sample (120), such that an explanation derived from the perturbed data sample matches an explanation that is not rejected.

15. A computer program comprising instructions which, when executed by a computer, cause the computer to carry out the method of claim 13 or 14.

Images