Computer-implemented method, computer system, and apparatus for transforming a black-box model to generate a white-box model based on sets of one or more rules derived from the black-box model, observed data, or synthetic data, and related computer readable medium and low-power chip, and apparatus and computer program product for white-box model induction

Abstract

A model induction method for explainable artificial intelligence (XAI) may be shown and described. A model of a black-box AI may be an input to the model induction method, along with a set of sample input data. A linear or non-linear predictor function may be used to predict the output of the black-box model, producing a set of data points. The data points may be partitioned by a partitioning function, and each partition may represent one or more rules. The data may also be transformed using a number of transformation functions, such as a polynomial expansion. A local model may be fitted to the transformed function or functions. A set of rules may be interpreted from the local models and may form a white-box AI model. Linear or non-linear data may be modeled by the white-box model. Further, the white-box model may be implemented on a low- power device.

Description

[Technical Field]

[0001] A method for extracting a model from an existing machine learning model can be shown and described. [Previous Technology]

[0002] Recent advancements in machine learning have yielded a plethora of new implementations of artificial intelligence, offering numerous benefits and advantages. AI algorithms have outperformed humans in new application areas. Despite this success, AI may not be perfect and may still be prone to errors, just like humans. These errors are often undetectable, unpredictable, and may not have direct intervention solutions. Despite these limitations, AI can still be implemented in mission-critical environments, where errors could have significant consequences.

[0003] Artificial intelligence and machine learning algorithms are often unable to explain their decisions and actions to human users. Therefore, when errors occur, they may be difficult to detect and correct. Thus, a method may be needed in this field to explain the inferences behind an AI's conclusions.

[0004] One proposed solution is a generalized linear rule model. This method uses a linear model that leverages rule-based features for both regression and classification. This method takes a linear combination of decision rules within the framework of a generalized linear model (GLM). The generalized linear model approach can become problematic as the number of models and rules increases and as the linear model underfits a set of rules.

[0005] Another possible solution can be found in attribute-based methods that provide instance-level feature importance proportional to the feature contribution in the model prediction. Systems like LIME and Shapley values ​​detect the behavior of perturbations to an input and then use that data to construct a locally linear model—a simplified proxy for the complete model in the neighborhood of that input. Gradient-based attribute-based methods, like LRP, SmoothGrad, DeepLIFT, and DeConvNet, use at least one forward and / or backward pass through the system to generate feature attributes. The output from the attribute-based method can be a set of features, along with their weights, that explains which features are most important for a particular prediction.

[0006] In addition, several rule extraction methods exist. Some of these methods can operate at the neuron level, rather than the entire network architecture level, and are referred to as decomposition techniques. These methods are limited by their specific model type. Instead, other rule extraction methods can employ teaching techniques, which can be applied to any black-box model, regardless of the underlying architecture. Teaching methods can generate artificial data points to feed into the black box to observe how the rules are applied.

[0007] Regardless of the technology, the models described above may not be able to model nonlinear data without generating fine-grained rules. This could result in a highly complex rule-based model that is potentially slower or less efficient than the original model. Furthermore, modeling nonlinear data using a typical model extraction method may produce inaccurate results or no results at all. Additionally, the generated model may require significant storage and processing power to become effective. Network connectivity may be required for the model to process the data. [Summary of the Invention]

[0008] A model induction method for interpretable artificial intelligence (XAI) can be demonstrated and described. A black-box AI model, along with a sample input data set, such as training data or synthetically generated data, can be used as input to the model induction method. The model induction method can perturb the input data in various ways using a suitable perturbation method, and can combine the perturbated data with the sample input data. Next, a predictor function can be used to predict the output of the black-box model. The predictor function can be linear or non-linear. A set of data points can be created by the predictor function. The data points can be segmented by a segmentation function such that a partition of the data points can represent one or more rules.

[0009] These segmented data points can be filtered. These segmented data points can represent data from a local model. A local model can be defined as a model applied only to one partition, while a global model is a model applied to the entire dataset. A local model can be fitted to each partition. The local model can be linear. Several transformation functions, such as a polynomial expansion, can also be used to transform the data. A local model can be fitted to this or such transformation functions. Finally, the transformed and fitted functions can be interpreted from the model to form rules that can be represented by symbolic logic. A combination of multiple local models can be used in a global model creation program.

[0010] The resulting set of rules can form a white-box AI model. Compared to the original black-box model, the white-box model can be smaller in size and requires less processing power to operate. Therefore, the white-box model can operate as fast as or faster than the black-box model without sacrificing accuracy. Linear or nonlinear data can be modeled by the white-box model. Furthermore, the white-box model can be implemented on a low-power device and may not require internet connectivity.

[0011] In a first embodiment, the present invention provides a computer-implemented method for transforming a black-box model to generate a white-box model based on a plurality of sets of one or more rules derived from the black-box model. The method includes: receiving the black-box model; generating a data point set associated with the black-box model, wherein the black-box model is configured to derive the data point set; segmenting the data point set using a segmentation function to obtain a segmented data point set, wherein the segmented data points are assigned to one or more partitions generated by the segmentation function, wherein each partition of the one or more partitions represents a set of one or more rules; identifying a transformation function of the segmented data point set based on the segmented data point set, wherein the set of one or more rules is interpreted from a local model of the identified transformation function; and providing the plurality of sets of one or more rules associated with the partitions obtained from the segmentation function to generate the white-box model by applying the plurality of sets of one or more rules, wherein the plurality of sets of one or more rules includes the set of one or more rules representing each partition.

[0012] In a second embodiment, the present invention provides a computer system for transforming a black-box model to generate a white-box model based on a plurality of sets of one or more rules derived from the black-box model. The system includes: at least one circuit configured to execute a sequence of actions of a programmable instruction set executed by at least one processor, wherein the programmable instruction set is stored in the form of a computer-readable storage medium, such that the execution of the sequence of actions enables the at least one processor to: receive the black-box model and / or an input set; and generate a set of data points associated with the black-box model based on the input set, wherein the black-box model is configured to... The data point set is derived; the data point set is segmented using a segmentation function to obtain a segmented data point set, wherein the segmented data points are assigned to one or more partitions generated by the segmentation function, wherein each of the one or more partitions represents a set of one or more rules; based on the segmented data point set, a transformation function of the segmented data point set is identified, wherein the set of one or more rules is interpreted from a local model of the identified transformation function; and the sets of one or more rules associated with the partitions obtained from the segmentation function are provided to generate the white-box model, wherein the sets of one or more rules include the sets of one or more rules representing each partition.

[0013] In other cases, it is an apparatus and computer product for inducing white box models, which includes one or more processors configured to implement the methods or systems described herein.

[0014] The methods described herein can be executed by software in a machine-readable form on a tangible storage medium, such as a computer program comprising computer code components adapted to perform all steps of any method described herein when the program is run on a computer and when the computer program can be embodied on a computer-readable medium. Examples of tangible (or non-transitory) storage media include magnetic disks, USB flash drives, memory cards, etc., and do not contain propagating signals. The software may be adapted to execute on a parallel processor or a serial processor, such that the method steps can be executed in any suitable order or simultaneously.

[0015] This application recognizes firmware and software as valuable, separately tradable commodities. It is intended to encompass software that runs on standard hardware to perform desired functions. It is also intended to encompass software that "describes" or defines hardware configurations, such as HDL (Hardware Description Language) software used to design silicon wafers or to configure general-purpose programmable chips to perform desired functions. In this case, a low-power silicon wafer is configured to apply a white-box model based on one or more sets of rules derived from a black-box model. The white-box model is configured to execute a programmable instruction set via one or more processors on the wafer.

[0016] It should be further understood that various options of the present invention can be appropriately combined, as will be obvious to those skilled in the art, and can be combined with any form of the present invention.

Implementation Method

[0026] The following description and related figures, relating to specific embodiments of the invention, disclose the features of the invention. Those skilled in the art will recognize that alternative embodiments can be conceived without departing from the spirit or scope of the invention claims. Furthermore, well-known elements of exemplary embodiments of the invention will not be described in detail or will be omitted to avoid obscuring the relevant details of the invention.

[0027] As used herein, the term "exemplary" means "serving as an example, illustration, or diagram." The embodiments described herein are not limiting but merely exemplary. It should be understood that the described embodiments are not necessarily to be construed as preferred or superior to other embodiments. Furthermore, the terms "embodiment of the invention," "embodiment," or "invention" do not require that all embodiments of the invention encompass the discussed features, advantages, or modes of operation.

[0028] Furthermore, the numerous embodiments described herein are described in terms of sequences of actions executed, for example, by elements of a computing device. Those skilled in the art will recognize that the various sequences of actions described herein can be executed by specific circuitry (e.g., application-specific integrated circuits (ASICs)) and / or by program instructions executed by at least one processor. Additionally, the sequences of actions described herein can be fully embodied in any form of computer-readable storage medium, such that execution of the sequences of actions enables the processor to perform the functions described herein. Therefore, various embodiments of the invention can be embodied in several different forms, all of which are considered within the scope of the claimed subject matter. Furthermore, for each of the embodiments described herein, any corresponding form of such embodiments can be described herein as, for example, a "computer" configured to perform the described actions.

[0029] Identifying and understanding the behavior of a black-box system can be useful in a variety of applications. This behavior can be modeled using a set of rules or a suitable logical equivalent. Information derived from the behavioral model can provide a human user with better insight into how the black-box system arrives at a particular conclusion. Knowledge of these rules can allow a user to identify potential risks and errors associated with machine learning systems. As used herein, a black-box system can mean any machine learning model that has at least one uninterpretable or unexplainable or otherwise inaccessible component.

[0030] A method, system, or computer program for modeling data may be disclosed. In one exemplary embodiment, the system may use one of the rules to combine and model nonlinear data. These rules may be in any form, such as disjunctive normal form (DNF) or conjunctive normal form (CNF). Nonlinear data may be modeled by combining linear methods with nonlinear transformations (such as polynomial transformations). The method may be model-independent, making it applicable to any black-box system, regardless of the underlying model.

[0031] An example program can utilize an existing black-box model as an input predictor for an algorithm. Given some input data, the predictor model can be used to output a prediction. This technique can provide useful insights in several applications. For example, a machine learning model can be analyzed to determine any incidental patterns in the selected training data that the model may have missed, and then such errors can be corrected. Furthermore, the analysis can provide insights into a specific domain of the machine learning model. A machine learning model providing medical diagnosis can be analyzed and interpreted to provide a physician with further insights into which symptoms of a patient lead to a diagnosis. In addition to these examples, the system can also be implemented on any artificial intelligence or machine learning model.

[0032] An instance system for extracting an instance of a model may involve a first step of creating input data. The input data can be any type of data, such as training data and / or synthetic data. Each synthetic data sample can be used to test predictions. The synthetic data can be wide enough to test the entire black-box machine learning algorithm. Each synthetic base data can be a single variable, feature, or value. The system then applies the synthetic data as input to a black-box system to receive a variation of the model's output. In an instance embodiment, the system may additionally apply a perturbation to the synthetic dataset. The output can be recorded, and the algorithm can identify how each input affects the output without otherwise analyzing the internal structure of the predictor model. Thus, even if only a small portion of the training data was initially supplied, the newly generated model can have a complete view of the entire dataset.

[0033] Referring now to exemplary Figure 4, Figure 4 illustrates an implementation scheme of a model-induced method. Consider an example of a black-box model used to detect abnormal patterns of data packets within a telecommunications network and take appropriate action, such as allowing a user to maintain connection, discarding portions of data packets, or modifying the network's routing priorities to implement faster or slower transmission. For all such cases, our white-box model provides an explanation of why this action is necessary, while a black-box model would simply recommend the action without any explanation. Both are useful for telecommunications operators and customers to understand why the model arrives at a conclusion. Using a white-box model, we can understand what conditions and characteristics lead to the result. Both sides have different objectives. On the one hand, telecommunications operators are interested in minimizing security risks and maximizing network utilization, while customers are interested in uptime and reliability. In one case, a customer can be disconnected based on a suspicious current data access pattern, and the customer must close or delete the application that generated such a suspicious data pattern before being allowed to reconnect. This explanation helps the customer understand how to correct their settings to comply with the telecom operator's services and helps the telecom operator avoid losing the customer entirely, while still minimizing risk. The telecom operator can also benefit from this explanation. They can observe that the customer was rejected due to a recurring security vulnerability caused by a specific application, which may indicate a high probability that the customer represents an unacceptable security risk within the current parameters of their applied security policy. The operator can also benefit from observing an accompanying justification that can arise with this explanation, containing a meta-interpretation of how the explanation itself was generated from a white-box model. Furthermore, a third party can also benefit from this explanation: the creator of the telecom security model. The model's creator can observe biases in the model that cause it to over-prioritize the fast reconnection count variable over other, more important variables, and can modify the model to account for these biases.

[0034] Referring again to the exemplary Figure 4, the illustrated system may consider various factors. For example, in the illustrated system, these factors may include the number of connections in the last hour, the bandwidth consumed by both upload and download, connection speed, connection and reconnection counts, access point information, access point statistics, operating system information, device information, location information, the number of parallel applications, application usage information, access patterns in the last day, weekly or monthly, billing information, etc. These factors may be weighted differently according to the telecommunications network model 404. The training and testing data 406 may include various values ​​of these variables to sample a wide range of data. The training and testing data 406 may further include synthetically generated data and may also be perturbed. It may also be considered that the model induction method 408 may only require training data, while the test data may only be used to evaluate the interpretable model or XAI rule 410. The training and testing data, together with the telecommunications network model, can be used as input to the model induction method. The model induction method 408 can use training and testing data 406 to query the telecommunications network model 404 in order to obtain an interpretable model or XAI rule 410.

[0035] In a black-box environment where predictive models typically produce high accuracy, the model simply returns a binary outcome with a probability distribution, such as 70% yes and 30% no. Some solutions using interpretable models such as decision trees can return a stacked trace of the so-called root cause of the outcome; however, such models typically lack the predictive power provided by black-box models. An exemplary embodiment presents comparable predictive capabilities of black-box models while achieving interpretability.

[0036] Another example illustrating some principles used in an exemplary embodiment of the present invention is the analysis of a black-box machine learning algorithm for classifying images. Synthetic data can be applied to the black-box system and can take various input forms. For example, synthetic data can include various images along with modifications to those images containing optional features that can be extracted from them. The data can contain multiple images for each classification category. The resulting XAI model retains the classification capabilities of the black-box model while enabling interpretation capabilities to understand the underlying procedures used to arrive at the classification answer, including but not limited to regions of interest, feature patterns, heatmaps, transformations, etc. The input to the induction method can be in the form of input data points and their respective labels or responses (predictor model outputs), along with the predictor model itself. Labels or responses can be obtained via (a) training data and / or (b) synthetic data; and / or (c) by dynamically querying a combination of sampled, perturbed data points obtained from the predictor model.

[0037] One exemplary embodiment of the modeling system may input data into a black-box system and record the output corresponding to each input. In another embodiment, the modeling system may use a common predictor function that returns labels or responses to all or part of the input dataset (this is equivalent to running an induction method directly on the training data), such as a simple mapping function. In another embodiment, the modeling system may use a suitable automatic predictor building method to create a predictor model directly from the training data. Adding a black-box or predictor model as input to the induction method can help reduce or eliminate errors and / or noise from the original input dataset.

[0038] Data points can initially be obtained from all or part of the training data, or, depending on the case, combined with synthetically generated data. The synthetically generated data can be generated via an external program, such as a simulation system, a world model, etc. The input data can also be an empty set; in this case, the induction method will work using perturbed data obtained from the input predictor model. An optional data quality assurance function can also be applied as a whole to the input data points and / or a subset of the input data before executing the induction method. The induction method constructs a global model by dynamically creating perturbed sample data points (then injecting these perturbed sample data points into the input predictor model, which can be a black box), and then annotates the output and combines it with all other available data points from the training and / or synthetic data. It is possible to utilize the perturbed data independently without using the original training data. An exemplary embodiment of one perturbation method may utilize a combination of the following: random noise addition methods, rotational perturbation, projection perturbation, k-anonymization, generative adversarial networks (GANs), or any other suitable perturbation method. The resulting combined data points may form the input to the segmentation function of the inducing method. Data points from the predictor model may be continuous, discrete, or categorical values. The predictor model outputting a classification result may apply this classification to: (i.) a discretized output via a suitable transformation function assigning appropriate class labels, or (ii.) a probability output that can be used directly or via a suitable transformation function.

[0039] The next exemplary step may identify a hierarchy of "partitions". Each partition may cover a set or cluster of data points that can be used for combined data extraction in an induction method. Each partition may define zero, one, or more rules and may contain sub-partitions with their respective associated rules. Extracted partitions may be merged into overlapping and non-overlapping partitions. In the case of overlapping partitions, a priority function may be used to determine which partition to initiate. In an alternative embodiment, if more than one partition is initiated, an aggregation function may be used to combine the results from multiple initiated partitions. Various methods may be implemented to find rules or clusters of data points about a rule, as long as each partition has enough data to fit a linear model and is not so complex that a linear model may not perform well. Nonlinear transformations, including but not limited to polynomial expansion, rotation, dimensional and dimensionless scaling, state-space and phase-space transformations, integer / real / complex / quaternion / octon transformations, Fourier transform, Walsh function, continuous data bucketing, Haar and non-Haar wavelets, generalized L2 function, fractal-based transformations, Hadamard transform, Type I and Type II fuzzy logic, knowledge graph networks, classification coding, difference analysis of data and conditional features, and normalization / standardization can be applied to individual partitions before linear fitting to enhance model performance.

[0040] A partition can be a cluster that groups data points according to a rule and / or a distance similarity function, as appropriate. Each partition can represent a concept or a unique data category. A partition represented by exactly one rule has a linear model that outputs a predicted or categorical value. Since the model can be modeled linearly, the coefficients of the linear model can be used to score the features by their importance. The underlying features can represent a combination of linear and nonlinear fits, because the inductive method handles both linear and nonlinear data and / or linear and nonlinear predictor models.

[0041] For example, the following is a partition that can be created in the telecommunications network security control application shown in Figure 4: Ÿ IF Upload_Bandwidth > 10000 AND Reconnect_Count <= 3000 THEN Connection_Approval =.... Ÿ IF Upload_Bandwidth > 10000 AND Reconnect_Count > 3000 THEN Connection_Approval =... Ÿ IF Banwidth_In_The_Last_10_Minutes >= 500000 THEN Connection_Approval =... Ÿ IF Device_Status = “Idle” AND Concurrent_Applications < 10 THEN Connection_Approval =... Ÿ etc. The following is an example of a linear model used to predict the probability of connection approval. Each coefficient represents the importance of each feature in determining the final output, where i can represent the feature index. In this example, the sigmoid function is used as an optional initiation function because it is a binary classification case. Another rule can be incorporated into nonlinear transformations, such as polynomial expansions, which can be one of the features in the rule equation.

[0042] Referring to previous examples involving image classification, a hierarchical partitioning may contain rules for identifying specific trends in the outputs of a black-box system corresponding to a specific input. An instance-based modeling system can recognize that whenever a specific shape, such as a paperclip, is given as an input to the black-box system, the black-box system always or almost always returns the same classification as the output, such as office supplies. It should be noted that the system may not recognize or label the paperclip in this way, but may interpret the configuration of gray pixels in that unique shape as one of the inputs that typically leads to the output classification of "office supplies". The system can calculate a percentage of the number of times that the paperclip shape, as one of the inputs, returns office supplies as an output, and generate a weight associated with the paperclip shape. Furthermore, a white-box algorithm can output details of the object that leads to the classification. For example, in Figure 2B, the paperclip is identified and labeled as such because the algorithm detects (i.) that the object contains four straight parallel lines, (ii.) that the object contains three curves, each forming a semicircle, and (iii.) that the objects in (i.) and (ii.) are continuous. This interpretation can be accompanied by a percentage output. This percentage can also be incorporated into the accuracy with which the algorithm can predict the correct identification of the detected object, including an error range or confidence interval representing the degree of certainty or uncertainty. The output weights or percentages can be presented to the user as an interpretation of the conclusion or output classification. Thus, the user can now understand why the black-box system returns an output classification of "office supplies" corresponding to the input image. In an exemplary embodiment, the system can return an image in which the paperclip is highlighted or indicated, along with a percentage or weight given to the highlighted object.

[0043] Furthermore, once the black-box image classifier has been analyzed by the system, a white-box model of the system can be extracted, allowing a user to use the white-box model instead of the original black-box model. The white-box model can perform the same function as the black-box model, while providing an explanation for its returned classification or prediction. This explanation can be presented or transformed into various types of machine- and human-readable explanations, such as text, images, or graphs. This explanation may be accompanied by a reason, which is a meta-explanation demonstrating how the white-box model generated the explanation. Additionally, the white-box model can be an artificial intelligence or machine learning algorithm or system.

[0044] An exemplary embodiment can be illustrated using mathematical terminology. In an exemplary embodiment, X may represent an input dataset, and Predict(x) may represent a predictor model function with n outputs. X may be a matrix with m dimensions. Each dimension (m) may represent an input feature of X, and each output may represent the number of categories (n). The input dataset may be a combination of training data and synthesized data. Additionally, in an exemplary embodiment, Perturbate(x) may be any perturbation function used to generate sample perturbation data given some input training or test data X, where X may be a subset of the input data, a neighborhood of the input data, a suitable embedding of the input / output / explanation data, or even a plain empty set, in which case the perturbation function needs to use some form of random or non-random query predictor model. Xp may represent the combination of the perturbated and original input datasets, and Y may represent the output from the predictor model function, such that, and.

[0045] The hierarchy of partitions can be represented by P through an external function Partition, such that, where k is equal to the number of rules in which each partition is defined by exactly one rule.

[0046] Ri(x) can represent each rule in the i-th partition. Each rule (Ri(x)) can consist of multiple conditions, such that , where q can represent the number of conditions in the rule, and Cj can be the j-th condition relative to the input matrix x of the i-th rule (i.e., the i-th partition). The perturbed input matrix XP can be divided into k matrices, where k equals the number of rules, such that XL represents a locally filtered matrix defined such that the union is true when the partitions do not overlap. The union is true when the partitions overlap. When the partitions overlap, a ranking function needs to be applied to select the most relevant rule or a probability-weighted combination method of some form is used. In an alternative embodiment, if more than one partition is started, an aggregation function is used to combine the results from multiple partitions.

[0047] A third instance step may involve fitting a local model to the partitions defined by Pi. First, XL may represent the input found in both XP and Ri(x). In other words, XL may be the input or feature data applied to the i-th rule. Then, XL may be perturbed, and XLP may represent a combination of the local perturbed datasets of XL and Perburate(XL). Therefore, YL=Predict(XLP). Finally, XT may be computed by finding a transformation of XLP such that XT=Transform(XLP). The transformation function may be a pipeline of transformations, including but not limited to polynomial expansion, rotation, dimensional and dimensionless scaling, Fourier transform, integer / real / complex / quaternion / octon transformations, Walsh function, state-space and phase-space transformations, Haar and non-Haar wavelets, generalized L2 function, fractal-based transformations, Hadamard transform, type 1 and type 2 fuzzy logic, knowledge graph networks, classification coding, data difference analysis, and normalization / standardization. The transformation function pipeline can further include analysis of transformations of data sequences ordered according to the values ​​of one or more variables, including time-ordered data sequences. XT can generate new features such that z represents the total number of features of the transformation function.

[0048] For each j in the range of 1 to n (i.e., the number of outputs), the system can compute a local model in a linear manner. YL is a matrix consisting of n vectors, each outputting a vector, such that YLj represents the j-th output vector. The linear model can be defined by Yij = Fit(XT, YLj), where Fit is a function that fits a linear model, such as linear regression, logistic regression, kernel methods, etc. The linear model defines a set of coefficients such that... In an alternative embodiment, parallel formalization or gradient descent techniques or a suitable equivalent can be used to achieve the fitting of the local model in an atomic fitting function. It should be noted that xi can be a transformed feature, such as a polynomial (e.g., x²), an intersection (e.g., xy), a conditional feature (e.g., x > 10 and y < 2), a Fourier transform, etc. It should be noted that xi can be positive or negative, corresponding to the positive or negative contribution of the relevant transformed feature to the model. Positive or negative contributions are also commonly referred to as excitation and inhibition effects. Compression or regularization techniques can also be applied to simplify the fitting of local models, which in turn simplifies the interpretation.

[0049] Finally, rules can be constructed from various combinations of Yij. If, then if, then if, then if, then if, then

[0050] In addition, extra functions can be applied to the final output of the rules. For example, the softmax function can be used to define rules for a multi-class classification system such that if , then , and

[0051] In an alternative exemplary embodiment, the sigmoid function can be applied to the output for use in a binary classification algorithm, such that: if, then

[0052] Referring now to exemplary Figure 5, Figure 5 depicts the model induction procedure. In an exemplary embodiment, training data X502 with m dimensions can be created. Additionally, a predictor model Predict(x) with n outputs can be initially created, where Predict(x) has n outputs 504. In a next exemplary step, the input training data can be perturbed and various samples can be merged 506. A predictor model, such as Y = Predict(Xp), can predict the sample outputs in another step 508. Furthermore, the samples can be partitioned into k hierarchical partitions 510, such that P = Partition(Xp, Y) = {Pi, ..., Pi, ..., Pk}. Partition(X,Y) can be a function configured to partition the data samples. This partitioning function can consist of a clustering algorithm, such as k-means, Bayesian, connectivity-based, centroid-based, assignment-based, grid-based, density-based, fuzzy logic-based, entropy-based, a mutual information (MI)-based method, or any other logically suitable method. The segmentation function may also include a set of methods that will result in several overlapping or non-overlapping partitions. In the case of overlapping partitions, an aggregation function can be used to combine the results from multiple partitions. Alternatively, the segmentation function may include an association-based algorithm, a causal-based segmentation, or other logically suitable partitioning implementation. Next, the k partitions can be fitted to a local model 511. This can be done by first filtering local samples 512, then locally perturbing the samples, and combining the perturbed samples with the filtered local samples 514. The model can then use a predictor model to locally predict the perturbed data 516. Alternatively, the data can be transformed using one of a variety of transformation formulas 518. A local model can be fitted for each j-th output such that YLj = Fit(XT) 520. As the final step in fitting the local partitions, coefficients 522 can be extracted from the local model YLj. In an alternative embodiment, the fitting 511 can be reconfigured to implement the internal function in parallel or as an atomic operation. For example, the perturbation of each local model can be implemented in a single step, while the transformation and fitting of all local models can be implemented in another single step. In an exemplary implementation, it is conceivable that multiple local models representing individual partitions can be fitted simultaneously using gradient descent techniques. In an alternative embodiment, the fitting of local models can also be used as feedback to improve the segmentation procedure in a global manner, thus achieving a global refinement procedure. The feedback can be implemented using gradient descent methods. In the last exemplary step, a XAI model 524 with k rules can be created using local partitions and extracted coefficients.

[0053] The example mathematics demonstrated in Figure 5 can be further illustrated by the following example. Given m = 2, it can represent the number of dimensions (2 dimensions, i.e., the x-axis and y-axis). Furthermore, in this example, k = 4, which can represent 4 rules R0, R1, R2, and R3. Extracting partitions based on these four rules can reveal that these rules will, for example, the partition assignment function depends on the choice of the partitioning function or several partitioning functions and is a key choice for the interpretability of the induced model. The partition type and characteristics can also be determined by the choice of the partitioning function or several partitioning functions. Suitable partition types include linear, Bayesian, and curvilinear partitions, including continuous and non-continuous segmented partitions, such as Bayesian curve segments, and other suitable partition types that give themselves interpretability and interpretability. In topological terms, model induction methods can use partitioning functions or several partitioning functions to transform the predictor model computational data space from a T0 (Kergoff), T1 (Freche), T2 / T3 / T4 / T5 / T6 (Hausdorf) and T3½ (Tikhonov) space into a T2 / T3 / T4 / T5 / T6 (Hausdorf) and T3½ (Tikhonov) space used by the induced white-box model.

[0054] In this exemplary embodiment, the transformation function can be applied as a polynomial expansion. In addition to the original features x and y, the polynomial expansion can introduce additional features defined by xy, such as x², y² features, and their intersection. After the transformation, a local model can be applied using a linear method. In this exemplary embodiment, a logistic regression can be applied, as would be used in a classification algorithm. Therefore, the following local fit can be produced:

[0055] Finally, the above local fittings can be combined with these local fittings to produce the following rule-based model:

[0056] The interpretation of these equations can be used as a method to generate explanations. Explanations can be in the form of text, images, audiovisual materials, etc. Transformation functions can be used to extract explanations via, for example, coefficients and underlying equations. In the above example, coefficients indicate feature importance. In one example, in the XAI model function defined by fr(5,20), setting x = 5 and y = 20 will trigger the first rule, i.e., when x ≤ 10. Expanding the equations will yield: From this equation, the product of the coefficients and variable combinations can be extracted into a set defined by the equations. For the purpose of feature importance, B0 can be ignored. When R is sorted, the most important coefficient / feature combinations can be determined. This "ranking" of coefficients / features can be used to generate explanations in the form of a textual format, a heatmap as an image, or any other considered form. Other explanation methods that utilize the explanation coefficients in a different way can be used. The underlying equations may affect the order and filtering applied to the resulting explanations. The use of causal models will further influence the generation of explanations, including intervention and counterfactual explanations. The basic explanation can then be further modified based on a user and a world model that combines a target action plan and other relevant information available to the system. Referring now to illustrative Figure 6, Figure 6 illustrates the above rules for hierarchical partitioning.

[0057] The induced XAI model enables several additional AI use cases beyond rule-based models, including bias detection, causal analysis, explanation generation, conversion into an interpretable neural network, interpretable neural network, interpretable transducer and transformer, interpretable spike network, interpretable memory network, interpretable reinforcement learning, deployment on edge hardware, and integration with expert systems for human-assisted collaborative AI. The induced model can be expressed in a general format or can be used directly. It is conceivable that a logically equivalent procedure to this induced method or a portion thereof can be recreated using a suitable machine learning system that has been appropriately trained, for example, via gradient descent techniques (such as neural networks, transducers, transformers, autoencoders, spike networks, memory networks, and / or reinforcement learning systems).

[0058] Summarization techniques can provide a method for simplifying interpretations. In the case of high-order polynomials (2 or higher), this technique can extract simpler features. For example, an equation may have features x, x², y, y², y³, xy with their respective coefficients. The resulting feature importance is an ordered set R of elements. For the purpose of feature importance and the summarization interpretation, the elements can be grouped regardless of the polynomial order. In this instance, the simplified result set is formed. A simplified interpretation may also include a threshold value such that only the first n features are considered, where n is a static number or percentage value. Other summarization techniques can be used on nonlinear equations, including but not limited to polynomial equations, state-space and phase-space transformations, Fourier transforms, etc.

[0059] The interpretation of the XAI model can be used to generate both human-readable and machine-readable interpretations. Human-readable interpretations can be generated in various formats, including natural language text files, images, graphs, audio, speech, video, verbal, 3D data, and other appropriate formats. Machine-readable interpretations can be represented using a general format or any other logically equivalent format. Furthermore, the resulting model can be a white-box AI or machine learning model that accurately captures the original model, which can be a non-linear black-box model, such as a deep learning or ensemble method. Any model or method that can be queried and produces a result (such as a classification, regression, or prediction result) can be a source for generating a corresponding white-box interpretable model. This source can have any underlying structure, as there is no need to analyze the internal structure.

[0060] Furthermore, the extracted model can be global, thus capturing the entire multidimensional space of the original model. The global model may be fully interpretable, even if the original model is not fully interpretable, and may still maintain the performance or accuracy of a high-order model. Interpretation can be completed simultaneously and in parallel with the answer, and there may be no performance loss.

[0061] The extracted white-box model possesses advantages not present in the original model. This extracted model is suitable for applications requiring low latency, such as those used in autonomous vehicles or medical surgical robots, in real-time or near-real-time environments. The system can use a spatially efficient transformation to store the model as compactly as possible using a level of detail that is scaled up or down as required in the underlying model. Therefore, it can be deployed on hardware with low memory and limited processing power. This may be particularly advantageous in various applications. For example, a low-power chip can be implemented in a vehicle. Adding a low-power chip can be significantly cheaper than implementing a black-box model on a higher-power chip. Furthermore, a model requiring internet connectivity can be converted into a white-box model that can be implemented on a low-power chip that does not require internet connectivity. Moreover, the resulting white-box model can be embodied in both software and hardware. Since the resulting white-box model is a complete representation of the original input model, it requires no network or online processing and can operate entirely offline. It is possible to consider using a hardware embodiment of the induction method and / or the resulting white-box model, which can be implemented using a flexible architecture such as an FPGA or a more static architecture such as an ASIC or analog / digital electronic device.

[0062] The resulting XAI (interpretable AI) model can be represented using first-order symbolic logic, making it computationally equivalent to a known programming language and applicable to any computing device. The extracted properties of the model can form the basis for several XAI methods, causal logic extraction methods, human knowledge incorporation, and adjustment / feedback techniques.

[0063] In one exemplary embodiment, the system can automatically adapt to different detail areas.

[0064] Referring now to exemplary Figure 1, an exemplary method for extracting an interpretable white-box model from a black-box machine learning algorithm can be illustrated. In an exemplary first step, synthetic data and / or training data 102 can be created or obtained. A perturbation of the synthetic data and / or training data set can also be created, allowing for a larger dataset without increasing the need for additional synthetic data and / or training data, thus saving resources. It is conceivable to normalize the synthetic data and / or training data using a suitable normalization method (such as minimum-maximum scaling, standard scaling, or other suitable methods) before step 102 or within any of steps 102, 104, 106, 108, 110, 112, and 114. It is further conceivable to apply a reverse normalization method, appropriately reversed, before step 102 or within any of steps 102, 104, 106, 108, 110, 112, and 114. Next, synthetic data and / or training data can be loaded as input into the black-box system 104. The black-box system can be any underlying architecture of a machine learning algorithm. In an exemplary embodiment, the machine learning algorithm can be a deep neural network (DNN). The black-box system may also contain non-linear modeling data. The underlying architecture and structure of the black-box model may not be important, as it may not be directly analyzable. Instead, synthetic data and / or training data can be loaded as input 104, and the output can be recorded as data point predictions or classifications 106. Because a large amount of synthetic data and / or training data is loaded as input, the output data point predictions or classifications can provide a comprehensive view of the black-box algorithm. In an alternative embodiment, the black-box model can be split in a suitable manner such that only a portion of it is induced by the induction method. For example, in the case of a spiral neural network (CNN), only fully connected non-spiral layers are induced, leaving the aforementioned layers (spiral and pooling layers) as transformed inputs to the induction method.

[0065] Still referring to the exemplary Figure 1, the method can continue by aggregating data points into hierarchical partitions 108 based on predictions or classifications. Rule conditions can be obtained from the hierarchical partitions. The procedure used to find partitions or the boundary system of partitions is an external function defined by Partition(X). Partition(X) can be a configured function for segmenting similar data and can be used to create rules. The segmentation function can include a cluster algorithm, such as k-means, Bayesian, connectivity-based, centroid-based, distribution-based, grid-based, density-based, fuzzy logic-based, entropy-based, or a mutual information (MI)-based method. Alternatively, the segmentation function can include correlation-based algorithms, causal segmentation, or other logically suitable segmentation implementations.

[0066] Hierarchical partitioning can organize output data points in various ways. In an exemplary embodiment, data points can be aggregated such that each partition represents a rule or a set of rules. Then, the hierarchical partitions can be modeled using mathematical transformations and linear models. Any transformation function or combination of transformation functions can be used, including but not limited to polynomial expansion, cyclotron filters, fuzzy subordination functions, integer / real / complex / quaternion / octon transformations, Fourier transforms, etc. This transformation function or combination of transformation functions can be applied before step 108 and / or during step 110. An exemplary embodiment may apply a polynomial expansion. Within step 108, the hierarchical partitions may also undergo one or more iterative optimization steps, depending on the circumstances, involving merging and splitting the hierarchical partitions using a suitable aggregation, splitting, or optimization method. A suitable optimization method seeks to find the topological space of all path connections within the predictor's computational data space while providing an optimal specification fixation that minimizes one-to-one ratio of the total number of partitions. Furthermore, a linear fitting model can be applied to partition 110. Depending on the application of the black-box model, such as softmax or a sigmoid function, additional functions and transformations may be applied before linear fitting. The computed linear model obtained from the partition can be used to construct rules or some other logically equivalent representation 112. Rules can be stored in any medium. For example, rules can be stored as mathematical equations or represented using first-order symbolic logic. Such storage of rules allows the extracted model to be applied to any known programming language and to any computing device. Finally, the rules can be applied to the white-box model 114. The white-box model can store the rules of the black-box model, thereby allowing it to mimic the functions of the black-box model while providing interpretations that the black-box model may not have provided. Furthermore, the extracted white-box model can be similar to the original black-box model in performance, efficiency, and accuracy. The extracted white-box model W, as a result of transforming the input predictor model U via the induced method, is an approximate or perfect mutual simulation of U, i.e., W ~ U. The degree of mutual simulation between the induced white-box model and the original predictor model is a parameter that can be set during steps 108, 110, 112, and 114. In terms of model and output interpretability, mutual simulation typically involves the interpretability of the output in the case of the extracted model as the final result of the induction method.

[0067] Referring now to exemplary Figures 2A and 2B, Figure 2A shows the output of a black-box image classification algorithm, and Figure 2B shows the output of a white-box image classification algorithm extracted from the black-box algorithm of Figure 2A. The same image 200 can be loaded as input into both the black-box algorithm of Figure 2A and the white-box algorithm of Figure 2B. As seen in exemplary Figure 2A, the black-box algorithm outputs a label for the image classification; in this case, the label is "office supplies." Additionally, the black-box algorithm may or may not present a percentage probability of accurate prediction. In this embodiment, the black-box model does not provide any other information as output besides classification.

[0068] Figure 2B shows the output of the white-box model extracted from the black-box model. The white-box model returns a classified image along with a classification label 202. The white-box model also classifies the image as "office supplies". In addition, the white-box model returns a number 204 representing the accuracy of the model. The accuracy can be presented as a percentage probability of the labeled classification system being accurate. In this case, the model calculates an 89% probability that the returned classification is correctly labeled as "office supplies". Furthermore, the white-box model returns an explanation 206 for label 202. This explanation can consist of multiple levels of detail, down to the individual object level. The explanation is generated mathematically and then converted into a human-readable format using a separate program, thus allowing for flexibility and generality in the generation of the explanation itself. The explanation can take the form of text, graphics, images, etc. An instance-specific detailed explanation 206 corresponds to label 202.

[0069] In this exemplary embodiment, explanation 206 may consist of a list of variables or factors that lead to the final classification. For example, the algorithm may identify a notebook 210 within the image. Notebook 210 may indicate a 24% probability that the image should be classified as "office supplies." The 24% can be calculated from rules extracted from the black-box model. Additionally, other factors leading to the conclusion that the image should be classified as "office supplies" may also be listed. In this exemplary embodiment, elements may be listed in order of importance or influence on the final decision, allowing a user to quickly identify key factors leading to the conclusion. The algorithm may also identify objects that reduce the probability of a correct classification, such as a baseball 212. In this case, the exemplary white-box model calculates that baseball 212 reduces the probability of a correct classification by up to 7%. This rule and percentage can be obtained from the rules of the white-box model. In this exemplary embodiment, additional features of the input image 200, such as a pen, marker, stapler, or other notebook, may have been identified; however, these may have a minor impact on the output. The output interpretation can be configured to display only the features that have the greatest impact on the final decision, thus avoiding presenting a user with a large amount of irrelevant information and preventing the user from being overwhelmed by it. Consider identifying all objects within the input image 200 and presenting only relevant features to the user. The threshold values ​​for which features are presented can be set by the user or the algorithm. The threshold value can be a specific number of features, such as the top 6 features. Alternatively, the threshold value can be a percentage; for example, the algorithm can present any feature that affects the final result by more than 5%. The level of detail presented to the user may also be influenced by a user model known to the system and can depend on a combination of the user's expertise, familiarity with the model domain, current objective, current stage of work, user and world model, and other relevant information that can be used in the personalization of the interpretation. The interpretation can be hidden or shown, or interactively collapsed and expanded, for the user to maintain the correct level of detail.

[0070] In a further exemplary embodiment, the white-box model may depict an interpretation 206 of the input image, such as by enclosing the identified factors in a box. For example, a baseball 212 may have a box 214 surrounding it along with the calculated percentage of the baseball's contribution to the overall search. In an exemplary case involving images, the underlying visual features of the guiding system marker object may also be presented. The visual representation may take any form. The most important factors that contribute most to the conclusion may be highlighted, while other factors that contribute only slightly to the conclusion may be ignored in order to provide the user with an appropriate (rather than excessive) amount of information to process.

[0071] Figures 2A and 2B provide only one example of how a white-box interpretable artificial intelligence model can provide both textual and visual explanations of rules extracted from a black-box model. The extraction method can be applied to any black-box model that may be far more complex than an image classification algorithm. The output can be in the form of percentages and factors, as shown in Figure 2B, or alternatively in the form of a list of rules, variables, differences, or any other data that can help a user understand the conclusions.

[0072] Referring now to exemplary Figure 3, Figure 3 illustrates the interaction between the white-box model extractor 302 and the black-box model 304. In one exemplary embodiment, the black-box model can be any black-box machine learning algorithm. Furthermore, the model does not necessarily have to be a black-box model. The model can be another white-box model for which the user seeks further analysis. The white-box model extractor 302 can provide one of the inputs to the black-box machine learning algorithm 304. The white-box model can be displayed or at least partially displayed via an interface. Synthetic data can be generated by the model extractor 302 or by an external program and can be combined with the input data. Alternatively, training data can be used as input, or the input can come from another program, such as a simulation system or a reinforcement learning model. The input can be perturbed. The black-box machine learning algorithm 304 can then execute as it is designed and can output any output it is designed to return. The output can be in any form and can be used for data point prediction and classification 308 that the white-box model extractor 302 can process. After processing the data points to define a set of rules corresponding to the rules of the black-box machine learning algorithm 304, the white-box model extractor 302 can apply those rules to the white-box model.

[0073] The procedure can be iterative, and additional input data 306 can then be loaded into the black-box model 304 to generate additional data point predictions and classifications 308. The additional data points can differ from the original data points to cover a new part of the black-box machine learning algorithm 304 and generate or discover a new rule associated with it. Input data 306 can be loaded continuously, and corresponding data point predictions and classifications 308 can be processed continuously until the white-box model is completely similar to the black-box model. The two models can have different underlying structures or architectures, but may still follow the same rules, such that an input will produce the same output from both models. The white-box model extractor can have a module that compares the rules of the two models. This module can take a dataset into account and check to ensure that the outputs of the two models are the same. If the outputs of the two models are different, the white-box model extractor can generate additional synthetic data 306 and restart the procedure to define unknown rules. Additional synthetic data 306 can be guided to generate data point predictions and classifications 308 from the black-box model 304 that are different from the previously obtained data points, so as to define different rules.

[0074] In a further exemplary embodiment, the white-box model can be converted into a general interpretable neural network. The performance and accuracy of the original white-box model can be maintained. The lossless transformation preserves all low-level features of the original model. Furthermore, various performance enhancements can be applied to the resulting interpretable neural network output. This allows for the utilization of all existing artificial neural network libraries, including deep learning and other related methods.

[0075] In another exemplary embodiment, the extracted model can be transformed or integrated into a causal machine learning algorithm or XAI. By integrating a causal logic model, explanations can be supported by causal relationships. By implementing causal relationships, explanations can include logical explanations, such as "what-if" and "what-if-not" analyses, and use causal logic tools such as interventions and counterfactuals. Subsequently, the AI ​​can have the ability to infer causes and effects while still retaining the advantages of the XAI architecture.

[0076] The extracted white-box model itself can be a standalone white-box interpretable AI system. Referring now to Figure 7, Figure 7 depicts the internal structure of an interpretable AI system. An image of a dog can be the input 700 of the AI ​​system. In this exemplary embodiment, the AI ​​can be an image classification algorithm. The system can classify the image by applying a set of rules to the input image and analyzing whether the rules are satisfied. This exemplary embodiment can implement a hierarchical structure for the rules, although a linear or other structure can also be applied. An exemplary embodiment can first analyze a part of the dog, such as a leg 702. R1 can be a rule triggered when an object with four legs is detected. Since R1 is triggered by the input image, the system can be guided to apply rules R2 and R3 to the input image. Rule R2 can be a conditional rule that can be triggered by the appearance of a tail 704. This rule can be parsed, stored, or defined. Rule R2 can be triggered by a condition such as a thin object that also satisfies another, thicker object of the same or similar color (subject) as another object in R1 (the rule for recognizing legs). Once rule R2 is satisfied, the AI ​​can be guided to rules R4 and R5. Rule R4 can recognize the dog's nose and mouth 706. It can be triggered by multiple conditions, such as the detected presence of a round nose, eyes, and mouth similar to those of other dogs. Rule R5 can be an additional rule triggered by the presence of a collar 708. Rule R5 may not be necessary for classification; however, R5 can increase the probability of making an accurate classification.

[0077] Additionally, after triggering R1, the system may have been directed to rule R3. Rule R3 analyzes the dog's ears in the input image. In this instance, rule R3 may be triggered by the presence of cat-like ears 710. The system can detect that the dog in the input image has pointed ears, a characteristic shared by many cats. Therefore, the input image can be further compared with common cat features. Since rule R3 may have been triggered by the pointed ears in the input image, the system may be directed to rules R6 and R7. Rule R6 compares the nose 712 in the input image with the noses of various cats, or the common shapes of cat noses and mouths. The conditions for rule R6 may include a short nose and mouth with whiskers and sharp teeth. This rule may not be triggered by the input image. Furthermore, rule R7 compares the unique features of a cat's eyes with the eyes 714 in the input image. This rule may be triggered by the unique vertical pupils found in cats. Since R6 and R7 may not be triggered by the input image, the system can recognize that the image may not be correctly classified as a cat, even if it may have cat-like ears.

[0078] Still referring to one of the exemplary embodiments of interpretable AI illustrated in Figure 7, the output may be a classification of an input image, such as "dog". The interpretable AI can provide additional information along with the classification. Various rules can be provided or explained to offer insights into the classification. For example, the output may demonstrate that the input image has four legs, thus guiding it to various animals. Furthermore, the output may show the tail of the input image and how it compares to the tails of other various dogs or contains features similar to common dog tails (R2). Next, the output may show a comparison of the muzzle in R5 with common dog muzzles, and the collar analyzed in R6, which may have increased the probability that the input image is a dog. The probability of accuracy can be presented along with the various interpretations of these rules. For example, the output may show an 85% probability that the part of the object analyzed in R2 is a tail. The output may further show the predicted accuracy of the output. For example, the output can show that the tail alone indicates that the object is one of the dogs with a 12% probability, or that the tail, when detected together with the nose, muzzle, and collar, indicates that the input image is one of the dogs with a 78% probability.

[0079] In addition, the output can illustrate why the category is not another object, such as a cat. The output can show that although R3 indicates, for example, due to pointed ears, there is a 9% probability that the object is a cat, R7 did not detect cat eyes and R6 did not detect a cat-like nose and mouth, so the probability that the input image is a cat is reduced to 2%.

[0080] By demonstrating why a particular output is chosen over another, one user of the artificial intelligence system can be better equipped to apply real-world results. Furthermore, the user can analyze the output to analyze the underlying artificial intelligence system. The explanation provides insight into specific biases and occasional errors found in the system, which, once identified, can be corrected by one of the system's users or creators.

[0081] The causal model can further incorporate strength and bias data. Additionally, inferential data from taxonomic and external knowledge sources can be implemented. Inputs can be analyzed, and a causal model can be generated that explains possible circumstances regarding the history of the inputs. For example, if the input is the occurrence of an event, the causal model can explain events that may have contributed to the occurrence of the event. Furthermore, appropriate causal models can be semi-automatically identified, allowing them to rely on both labeled and unlabeled training data. Therefore, causal relationships can be inferred in a supervised or unsupervised manner. This enables the XAI model to analyze causal models based on the characteristics of the underlying input data.

[0082] In a further embodiment, an XAI model may incorporate human input to develop a causal model. An alternative exemplary embodiment of XAI may be implemented in a bias detection system. This system can detect biases and further correct unintentional biases. Due to the white-box nature of the XAI model, an exemplary embodiment may suggest corrections in the model to reduce or eliminate biases. Additionally, intentional biases may be purposefully introduced into the data due to cause and effect.

[0083] In one embodiment, a computer-implemented method for transforming a black-box model to generate a white-box model based on a plurality of sets of one or more rules derived from the black-box model is provided. The method includes: receiving the black-box model; generating a data point set associated with the black-box model, wherein the black-box model is configured to derive the data point set; dividing the data point set using a segmentation function to obtain a segmented data point set, wherein the segmented data points are assigned to one or more partitions generated by the segmentation function, wherein each partition of the one or more partitions represents a set of one or more rules; identifying a transformation function of the segmented data point set based on the segmented data point set, wherein the set of one or more rules is interpreted from a local model of the identified transformation function; and providing the plurality of sets of one or more rules associated with the partitions obtained from the segmentation function to generate the white-box model by applying the plurality of sets of one or more rules, wherein the plurality of sets of one or more rules includes the set of one or more rules representing each partition.

[0084] As an option, the method further includes: receiving an input set, wherein the data point set is generated based on the input set.

[0085] As another option, the input set is perturbed to produce a perturbed data set.

[0086] As another option, the input set includes training data synthesized from an internal or external program.

[0087] As another option, the method further includes: merging the perturbed data set with the input set.

[0088] As another option, the generated data point set can be derived by applying a classification, a regression and / or one or more prediction techniques.

[0089] As another option, the generated data point set includes a single value, multiple values ​​and / or sequence ordered according to one or more variables, wherein the one or more variables contain time-ordered data points.

[0090] As another option, the method further includes: fitting a global model to the generated data point set.

[0091] As another option, the method further includes: wherein the partitioning function includes at least one clustering algorithm.

[0092] As another option, the method includes: configuring the partitions obtained from the partitioning function within a level of the partitions.

[0093] As another option, the method further includes: applying an aggregation function to aggregate the results from the partitions.

[0094] As another option, the method further includes: applying a splitting function to split the results from the partitions.

[0095] As another option, the identified transformation function is a linear or nonlinear function.

[0096] As another option, the method further includes: fitting the local model to the identified transformation function by applying one or more programming techniques or algorithms.

[0097] As another option, the method further includes: identifying a linear or nonlinear start function and / or ranking function for the partitions and / or the segmented data points, wherein the ranking function is configured to provide a ranking of the partitions and / or the segmented data points; wherein the ranking function is a priority function configured to determine which partition should be started.

[0098] As another option, the group of one or more rules is presented in a logical representation format, wherein the logical representation format includes machine- and human-readable formats.

[0099] As another option, the logical representation format is in disjunctive normal form and / or conjunctive normal form.

[0100] As another option, the logical representation format corresponds to linear or nonlinear.

[0101] As another option, it is determined that the set of one or more rules can be interpreted in conjunction with one of the related contexts of the application of the transformation function.

[0102] As another option, one of the outputs of the transformation function provides an interpretation of the data point set derived from the white-box model as an answer, which is associated with the set of one or more rules interpreted from the local model.

[0103] As another option, each interpretation includes a confidence interval of one percentage related to the accuracy of the interpretation; wherein the accuracy of the interpretation is measured as the degree of certainty with respect to one or more rules.

[0104] As another option, each answer and each explanation are associated with an answer context and an explanation context, respectively, so that the answers and explanations can be customized for specific situations.

[0105] As another option, the method further includes: applying a spatially efficient transformation to store the white box model in a compact manner.

[0106] As another option, the space-efficient transformation is configured to reduce the delay of the white-box model.

[0107] As another option, the space efficiency transformation is deployed to implement the white box model in a low memory and low power hardware.

[0108] In another embodiment, there is a computer system for transforming a black-box model to generate a white-box model based on a plurality of rules derived from the black-box model. The system includes: at least one circuit configured to execute a sequence of actions as a programmable instruction set executable by at least one processor, wherein the programmable instruction set is stored in the form of a computer-readable storage medium, such that the execution of the sequence of actions enables the at least one processor to: receive the black-box model and / or an input set; and generate a set of data points associated with the black-box model based on the input set, wherein the black-box model is configured to derive the white-box model. A data point set; the data point set is segmented using a segmentation function to obtain a segmented data point set, wherein the segmented data points are assigned to one or more partitions generated by the segmentation function, wherein each of the one or more partitions represents a set of one or more rules; based on the segmented data point set, a transformation function of the segmented data point set is identified, wherein the set of one or more rules is interpreted from a local model of the identified transformation function; and the sets of one or more rules associated with the partitions obtained from the segmentation function are provided to generate the white-box model, wherein the sets of one or more rules include the sets of one or more rules representing each partition.

[0109] As an option, the system further includes: wherein the white box model is generated by applying the plurality of sets of one or more rules and at least one local model.

[0110] As an option, the at least one processor is further configured to receive multiple factors associated with the black box model.

[0111] As another option, each of the multiple factors is weighted individually.

[0112] As another option, the generated white-box model retains the classification, regression and / or prediction capabilities of the black-box model and incorporates the interpretability for producing one or more types of results.

[0113] As another option, the system is configured to perform the method of any of the aforementioned options.

[0114] As another option, the system or method further includes: a quality assurance function configured to be applied as a whole to a subset of the input data prior to the execution of one of the transformations or inductions.

[0115] As another option, the system or method further includes: a reason, which accompanies the generated white-box model in the form of a meta-interpretation.

[0116] As another option, the generated white-box model is an artificial intelligence or a machine learning algorithm.

[0117] As another option, the system or method further includes: a ranking function configured to select at least one rule or apply a probability and / or aggregation method in response to overlapping of the partitions.

[0118] As another option, the system or method further includes: wherein the transformation function includes one or more transformations that analyze a sequence of data points ordered according to one or more values ​​of one or more variables, including a time-ordered data sequence.

[0119] As another option, these partitions are fitted to one or more local partitions.

[0120] As another option, the fit in one or more local partitions is reconfigured to implement an internal function in parallel or as an atomic operation.

[0121] As another option, the fit to one or more local partitions is configured to be used as feedback to the segmentation procedure performed by the segmentation function.

[0122] As another option, the white-box model includes one or more sets of rules accompanying the coefficients extracted from the local model, wherein the one or more sets of rules include a module that compares at least two of the one or more sets of rules.

[0123] As another option, the white-box model is generated based on the one or more machine learning models, wherein the one or more machine learning models are at least partially configured to apply a logical equivalent procedure to perform the transformation of the black-box model.

[0124] As another option, the black box model is at least partially split to transform it into the white box model.

[0125] As another option, the system or method further includes: a user model associated with a user, the user model being configured to adapt the white-box model according to the user to maintain a level of detail.

[0126] As another option, at least a portion of the white box model is displayed through an interface.

[0127] As another option, the white-box model is configured to generate a causal model based on user input, wherein the causal model provides one inference from taxonomic and external knowledge sources.

[0128] As another option, the white box model is configured to detect deviations, wherein the deviations are corrected based on the white box model.

[0129] As another option, the system or method further includes: applying a transformation function or applying a transformation function to the data point set before the segmentation function segments the data point set.

[0130] As another option, the system or method further includes: a predictor function configured to identify or recognize a relationship between at least one input and at least one output of the black box model to derive the data point set, wherein the predictor function includes at least one linear or nonlinear function.

[0131] As another option, the black box model is adapted to have at least one inaccessible internal component.

[0132] In another form, it is a computer-readable medium that includes instructions that, when implemented in a computing system, cause the system to perform a method or system as described in any of the foregoing options.

[0133] In another aspect, there is an apparatus for transforming a black-box model to generate a white-box model based on a set of one or more rules derived from the black-box model, the apparatus being configured to implement a method or system as described in any of the foregoing options.

[0134] In another aspect, there is an apparatus for inducing white-box models, which includes one or more processors configured to implement any of the aforementioned options.

[0135] In another state, there is a low-power chip configured to apply a white-box model generated based on a set of one or more rules derived from a black-box model, wherein the white-box model is configured to execute a programmable instruction set via one or more processors of the chip.

[0136] In another embodiment, there is a computer program product for inducing white-box models, which is configured to implement methods or systems as described in any of the foregoing options.

[0137] As an option, the product is configured to be implemented on a low-power chip.

[0138] The foregoing description and accompanying drawings illustrate the principles, preferred embodiments, and operating modes of the present invention. However, the present invention should not be construed as limited to the specific embodiments discussed above. Those skilled in the art will understand additional variations of the embodiments discussed above.

[0139] Therefore, the above embodiments should be considered illustrative rather than restrictive. Accordingly, it should be understood that modifications to these embodiments can be made by those skilled in the art without departing from the scope of the invention as defined by the following claims. [Simplified Explanation of the Diagram]

[0017] The advantages of the embodiments of the present invention will become apparent from the following detailed description of exemplary embodiments thereof, which should be considered in conjunction with the accompanying drawings, wherein similar element symbols indicate similar elements, wherein:

[0018] Figure 1 is an instance method for extracting an interpretable white-box model from a black-box machine learning algorithm.

[0019] Figure 2A is an example of one of the outputs of a black box image classification algorithm.

[0020] Figure 2B is an example of one of the outputs of a white box image classification algorithm.

[0021] Figure 3 is an example flowchart depicting one of the interactions between a white box model extractor and a black box model.

[0022] Figure 4 is an example flowchart illustrating one of the implementation schemes of an example model induction method.

[0023] Figure 5 is an example diagram illustrating one of the model induction procedures.

[0024] Figure 6 is an instance diagram illustrating one of the hierarchical partitions of a set of instance rules.

[0025] Figure 7 is an example diagram illustrating the internal structure of an example interpretable AI system.

Claims

1. A computer device comprising: At least one processor; At least one memory of the at least one processor, wherein the at least one memory has a reduced capacity; At least one hardware computer-readable storage medium storing executable instructions, which are executable by at least one processor constrained by at least one memory, and the at least one hardware computer-readable storage medium being configured to cause a computer device to execute: A data point set is generated from a black-box model, the black-box model being configured to derive the data point set; The data point set is partitioned using a partitioning function to obtain a partitioned data point set, wherein the partitioned data points are assigned to one or more partitions generated by the partitioning function, wherein each of the one or more partitions represents a set of one or more rules; Based on the partitioned data point set, a transformation function of the partitioned data point set is identified. The function interprets one or more rules from a local model of the identified transformation function; and provides one or more sets of rules associated with the partitions obtained from the segmentation function to generate a white-box model by applying one or more sets of rules, wherein the one or more sets of rules include one or more sets of rules that represent each partition.

2. The computer device as claimed in claim 1, further comprising: At least one interface hardware is coupled to the computer device to transmit input to and receive output from the computer device.

3. The computer device of claim 2, wherein the at least one interface hardware is configured to receive the black-box model and output the white-box model generated from the black-box model.

4. The computer device of claim 2 or 3, wherein the at least one interface hardware is configured to allow a user to input sample data and provide an output based on sample data associated with the black box model.

5. The computer device of claim 4, wherein the at least one interface hardware is configured to display the white box model as the output.

6. A computer device as described in any of claims 1 to 3, wherein the at least one processor is a flexible architecture, a static architecture, or a processor used in conjunction with an analog or digital electronic device.

7. A computer device as claimed in any of claims 1 to 3, wherein the transformation function provides one or more rules of the set of space-efficient transformation to enable the at least one memory to compactly store the local model for use during the space-efficient transformation.

8. The computer device as requested in item 7, wherein the space-efficient transformation is enabled when the computer device receives reduced power.

9. A computer device as claimed in any of claims 1 to 3, wherein the at least one processor is compatible with a low-power chip having low memory requirements.

10. A system for assisting the operation of a vehicle or a robot in real-time or quasi-real-time, based on the output of a computer device as claimed in any one of claims 1 to 9, the system comprising: An interface module coupled to the computer device or a hardware component associated with the computer device, wherein the interface module is configured to present a user of the system having the output of the computer device; and a control module configured to receive instructions from the user to operate the vehicle or the robot, the control module being guided by the output of the computer device and configured to provide such instructions to the vehicle or the robot in an environment requiring real-time or near-real-time user input.

Images