Explainable Transducers and Transformers

Abstract

An explainable transducer-transformer (XTT) can be a finite-state transducer along with an explainable transformer. Variations of XTT can include an explainable transformer-encoder and an explainable transformer-decoder. An exemplary explainable transducer can be used as a part replacement in a trained explainable neural network (XNN) architecture or a logically equivalent architecture. An explainable transformer can replace the black-box model components of a transformer with their white-box model equivalents in both the encoder and decoder sublayers of the transformer. XTT can utilize an explanation and interpretation generation system (EIGS) that generates explanations and filters such explanations to generate an answer, explanation, and its justification interpretation.

Description

[Technical field]

[0001] Exemplary embodiments relate to the field of machine learning systems. [Background technology]

[0002] A transducer is one of the fundamental models that allows machine learning systems to translate, map, and transform data from an input language to a second output language. A finite state transducer (FST) is a finite state automaton that operates on two or more tapes, and is more general than a plain finite state automata (FSA) that operates on a single tape. FSTs specify relationships between sets of strings, and therefore can be treated as a basic translator or transformer between strings that occur in two distinct input and output sets.

[0003] A finite state transducer (FST) is a 6-tuple, as shown in Figure 1.

number

number

number

[0004] Finite-State Transducers (FSTs) can be a basic component for translating an input language into another language. Transformers (Vaswani et al., 2017) can be a natural evolution of FSTs with attention-based architectures that provide state-of-the-art results in various domains, such as Natural Language Processing (NLP) and image classification. Transformers require significantly less time to train compared to other architectures, such as LSTM and CNN architectures, due to the parallelization of its components, such as simultaneously computing queries, keys, and values.

[0005] The transformer architecture may be divided into an encoder component and a decoder component.

[0006] The input to the encoder component may include the sum of the input embedding and the positional encoding of the input tokens. Positional encoding is required because there is no concept of word order in the Transformer (Vaswani et al., 2017) unlike sequential architectures such as recurrent neural networks where the input tokens are inserted sequentially and therefore preserve the order of the input tokens. The encoder layer architecture may include two sublayers as shown in the Transformer (Vaswani et al., 2017) architecture. The first sublayer may include a multi-head attention component 215 followed by a summation and normalization component 217 as shown in FIG. 2. The second sublayer may include a feedforward neural network component 230 followed by a summation and normalization component 232. The multi-head attention component may include multiple instances of scaled dot product attention (Vaswani et al., 2017), where each instance has its own weight to improve the generalization of the model. Each instance {z0, ..., z n The output matrices of} are concatenated and multiplied by a weighting matrix W0 to yield the output matrix.

[0007] The architecture of the decoder layer can include three sublayers, as shown in the Transformer (Vaswani et al., 2017) architecture. The first sublayer consists of a masked multi-head attention component 250 followed by a summing and normalizing component 252. The second sublayer consists of a multi-head attention (encoder-decoder) component 240 followed by a summing and normalizing component 242. The third sublayer consists of a feedforward network component 270 followed by a summing and normalizing component 272, as shown in Figure 2. The encoder-decoder attention component is similar to the multi-head attention component. However, the query vector Q is from the sublayer immediately preceding the decoder layer, and the key vector K and value vector V are taken from the output of the final encoder layer. The masked multi-head attention component is a multi-head attention component with a modification such that the self-attention layer is only allowed to respond to earlier positions of the input token. The output of the decoder layer may be connected to a linear layer 280 followed by a SoftMax computation to generate output vocabulary probabilities 290 representing the predicted tokens. Inputs to the decoder component may include token embeddings and positional encodings 295 of the output tokens 260.

[0008] The core component of the Transformer (Vaswani et al., 2017) architecture is the attention component. A Transformer can have three types of attention mechanisms: encoder self-attention, decoder self-attention, and encoder-decoder self-attention. The input of the encoder self-attention is the source input token of the Transformer or the output of the previous encoder layer. The encoder self-attention component has no masking and each token has a global dependency with other input tokens. The decoder self-attention component uses the output token of the Transformer or the output of the previous decoder layer as input token. In decoder self-attention, the input token depends on the previous input token. In the encoder-decoder attention component, the query is retrieved from the previous component of the decoder layer, and the keys and values ​​are retrieved from the output of the encoder.

[0009] Input for self-attention inp q , inp w , and inp k is the same in the transformer besides the encoder-decoder attention. The self-attention presented in the transformer can be shown as shown in Equation 1.

[0010]

number

[0011] Here, Q is a matrix of a set of queries, K is a matrix of a set of keys, V is a matrix of a set of values, and d k refers to the key and query dimensions.

[0012] The work in Tsai et al. (2019) presents a new formulation of the attention mechanism in the Transformer from the lens of kernels. (2019) use a filtering function M as a masking mechanism in the kernel formulation. The filtering function M may be required for masking self-attention in the decoder module of the Transformer. (2019) allows for a better understanding of the attention mechanism by introducing customizability and attention components such as kernel feature space, kernel construction, position embedding, value function, and incorporation of filtering functions.

[0013] Bi-directional Encoder Representations from Transformers (BERT) (Devlin et al., 2018) is an architecture based on the encoder component of the Transformer (Vaswani et al., 2017) designed to construct bi-directional contextual representations of input tokens in a self-supervised approach.

[0014] BERT (Devlin et al., 2018) uses masked language modeling to randomly mask random input tokens and set a pre-training objective to predict the original token of a masked input token. BERT typically randomly masks 15% of the input tokens. A randomly masked input token has an 80% chance of being replaced by the [MASK] token, a 10% chance of being replaced by another random token, and a 10% chance of remaining unchanged. BERT (Devlin et al., 2018) can also use next sentence prediction (NSP) as a pre-training objective to predict whether a second input sentence is the next sentence of a first input sentence or not. This is useful for multiple downstream tasks such as Question-Answering prediction. The NSP pre-training objective can have a 50% chance that the second sentence in the input is the actual sentence following the first input sentence, and a 50% chance that the second sentence in the input is a random sentence from the training dataset.

[0015] BERT uses word piece embeddings (Wu et al., 2016) for input tokens. The word piece embeddings of the input tokens are summed with segment and position embeddings to create the BERT input. BERT can be similarly fine-tuned for downstream tasks using approaches such as supervised classification to build a fine-tuned model for the selected task.

[0016] Embeddings can also represent non-textual information, such as Contextualized Object Embeddings (COBE), which are extracted from narrated educational videos that use a visual detector to predict contextualized word embeddings of objects and their associated narration (Bertasius and Torresani, 2020).

[0017] ELMo (Peters et al., 2018) builds word representations that take into account linguistic context as well as syntactic and semantic properties. The generated word representations are drawn from the internal state of a two-way language model.

[0018] Generative pre-trained Transformer (GPT) (Radford et al., 2018) is an architecture based on the decoder component of a variant (Liu et al., 2018) of the Transformer (Vaswani et al., 2017) that is designed to train a model on a large corpus of unlabeled text and uses the model for fine-tuning to adapt it to classification tasks using labeled data. The GPT pre-training objective is to maximize the likelihood, as shown in Equation 2. Σ i logP(u i |u i-k , ..., u i-1 :Θ) (2) where P is the conditional probability modeled using a neural network architecture with respective parameters Θ, and u is the set of U={u1, ..., u n}, and k is the input context window of the GPT architecture.

[0019] The trained GPT architecture is fine-tuned for the supervision task by adding linear layers to predict the classification labels and maximize the objective shown in Equation 3. Σ (x,y) logP(y|x1, ..., x m ) (3) where P is the conditional probability, y is the classification label, {x1, ..., x m} is the input token.

[0020] Hoover et al. (2019) present EXBERT, an interactive tool used to understand the BERT attention mechanism. EXBERT can show potential inductive biases by providing insight into learned contextual representations. s (Here, input s =[token1, token2, ..., token n Attention in []) can be specified as an n x n matrix. EXBERT shows the attention relationship of the input sequence as an n x n matrix at any encoder layer, attention head, or aggregate attention value, along with the ability to search for specific tokens in the corpus using cosine similarity to retrieve the top 50 matches, and a summary display of matched metadata such as POS and NER in the searched corpus.

[0021] The Tree Transformer (Wang et al., 2019) is an encoder-based transformer architecture with constraints on the attention heads to follow a tree-based hierarchical structure. The constraints are implemented using a constituent attention module to learn a tree structure from a corpus. The tree transformer layer consists of a multi-head attention sublayer, a constituent attention module, and a feedforward sublayer. Constituent priors are computed by the constituent attention module. The attention of the multi-head attention is guided by the constituent priors. The constituent attention module calculates a sequence u = {u1, u2, ..., u nWe generate a configuration prior by generating {\displaystyle \mathbb {u}\left(u\left)}\right. The elements in u indicate the probability that each input token (e.g., w1) and its neighboring tokens (e.g., w2) belong to the same configuration block. Breakpoints are indicated by small probability values ​​and trigger the construction of new configuration blocks. The sequence u is constructed using neighborhood attention and hierarchical constraints. Tree transformer methods can be classified as gray-box methods, and thus tracing activation paths is not possible. Our proposed method introduces a white-box component into a transformer-based architecture to generate global and local explanations.

[0022] The EQ Transformer (Mousavi et al., 2020) uses an encoder and three decoders for seismic detection based on a hierarchical attentive model. The input to the encoder includes the seismic signal to generate a contextual representation of the time dependence. The decoder uses this learned information to map the information to a probability vector associated with the presence of the seismic signal, P phase, and S phase for each time point.

[0023] Transformer architectures can be used for image synthesis (Weissenborn et al., 2020), multiple object tracking (Zhang et al., 2020), music generation (Huang et al., 2018), 3D object detection (Yang et al., 2019), point cloud processing (Guo et al., 2020), time series forecasting (Lim et al., 2020), vision language modeling (Li et al., 2019), and end-to-end object detection (Zhu et al., 2020).

[0024] The Transformer architecture is a black-box machine learning model. The black-box feedforward neural network components in Transformer (Vaswani et al., 2017), BERT (Devlin et al., 2018), and GPT (Radford et al., 2018) cannot provide explanations for the generated predictions. The attention learned inside the Transformer during pre-training remains a black-box and cannot capture human intuition of hierarchical structures. EXBERT provides some degree of post-mortem analysis of the learned attention, however, the Transformer-based architecture remains a black-box model. Our proposed method introduces a white-box component within the Transformer-based architecture to generate global and local explanations. Summary of the Invention

[0025] In accordance with at least one exemplary embodiment, a method, system, and apparatus for an accountable transducer and an accountable transformer may be shown and described.

[0026] Transducers can enable machine learning systems to translate, map, and convert data from an input language into a second output language. Transducers are often implemented as Finite State Transducers (FSTs).

[0027] An exemplary Transformer can be an attention-based architecture based on a combination of decoder and encoder components and can provide state-of-the-art results in various machine learning applications. The Transformer can be thought of as a sophisticated evolution of the FST with an attention-based architecture.

[0028] Exemplary embodiments may combine an FST with an explainable transformer to generate an exemplary explainable transducer transformer (XTT). XTT variants such as explainable transformer-encoder and explainable transformer-decoder may be contemplated.

[0029] The exemplary explainable transducers may be used as part replacements in a trained Explainable Neural Network (XNN) architecture or a logically equivalent architecture.

[0030] The exemplary explainable transformer replaces the black-box model components of the transformer with white-box model equivalents in the sub-layers of both the encoder and decoder layers of the transformer.

[0031] XTT can utilize an Explanation and Interpretation Generation System (EIGS) that generates explanations and filters explanations to generate interpretations of answers, explanations, and their justifications.

[0032] In a first aspect, the present disclosure provides an explainable machine learning system, the explainable machine learning system comprising a finite state transducer, FST, configured to enable the machine learning system to translate, map, and transform data from an input language into a second output language, the finite state transducer being adapted to receive data from the input language including one or more input features used for transformation or to transform the one or more input features, the finite state transducer being further configured to provide translated, mapped, and / or transformed data as a second output language based on the one or more input features, and / or comprising an explainable transformer having an attention-based architecture based on a combination of decoder and encoder components formed in association with the data from the input language, the finite state transducer and the explainable transformer being configured to be combined to generate an explainable transducer-transformer, XTT, configured to translate, map, and / or transform the data from the input language.

[0033] In a second aspect, the disclosure provides a system for providing an explainable transducer transformer, the explainable transducer transformer comprising: an input layer configured to receive an input and identify one or more input features; a conditional network, the conditional layer configured to model the input features based on one or more partitions, each of the one or more partitions comprising a rule; an aggregation layer configured to aggregate the one or more rules into one or more aggregate partitions; and a switch output layer configured to selectively pool aggregate partitions from the aggregation layer having one or more partitions from the conditional layer; and a feature generation and prediction network, the feature generation and prediction network comprising one or more transformation neurons configured to apply one or more transformations to the input features. the prediction network comprising: a transformation network; a fit layer configured to combine features transformed by the feature generation and transformation network to identify one or more features and one or more coefficients associated with at least one of the one or more partitions; a value output layer configured to analyze the one or more coefficients and to output values ​​associated with the one or more features or at least one of the one or more partitions; an encoder layer and a decoder layer, at least one of the encoder layer and the decoder layer comprising an explainable architecture formed from inputs; and an output layer configured to generate outputs interpretable and explainable by at least one of a machine program or a human, where one or more execution paths through the partitions are identifiable by an external process.

[0034] In a third aspect, the present disclosure provides a computer-implemented method for deploying the system described in the first and / or second aspects.

[0035] The methods described herein may be implemented by software on a tangible storage medium in machine-readable form, for example in the form of a computer program, which may be embodied on a computer readable medium, comprising computer program code means adapted to perform all the steps of any of the methods described herein when the program is run on a computer. Examples of tangible (or non-transitory) storage media include disks, thumb drives, memory cards, etc., and do not include propagated signals. The software may be suitable for execution on a parallel or serial processor such that the method steps may be performed in any suitable order or simultaneously.

[0036] The preferred features may be combined as appropriate and in any of the aspects of the invention as will be apparent to one skilled in the art.

[0037] Advantages of embodiments of the present invention will become apparent from the following detailed description of illustrative embodiments of the invention, which description should be taken in conjunction with the accompanying drawings, in which like numerals indicate like elements and in which: [Brief description of the drawings]

[0038] [Figure 1] FIG. 1 is a diagram of an exemplary embodiment of a finite state transducer (prior art). [Diagram 2] FIG. 1 is a diagram of an exemplary embodiment of a transformer (prior art) adapted from Vaswani et al. (2017). [Diagram 3] FIG. 1 is a diagram of an exemplary explainable neural network (XNN) architecture. [Figure 4] FIG. 1 is a diagram of an exemplary XNN architecture. [Diagram 5] FIG. 1 is a diagram of an exemplary Interpretable Neural Network (INN) architecture. [Figure 6]FIG. 1 is a diagram of an example embodiment of an accountable transformer architecture that uses the accountable architecture in encoder and decoder components with partition information. [Figure 7] FIG. 1 is a diagram of an example embodiment of an explainable transformer architecture having parallel explainable encoder components and parallel explainable decoder components. [Figure 8] FIG. 1 illustrates (a.) an exemplary embodiment of a Transformer Encoder (prior art) and (b.) an exemplary embodiment of an Explainable Transformer Encoder (XTT-Encoder). [Figure 9] FIG. 1 is a diagram of (a.) an exemplary embodiment of a Transformer Decoder (prior art) and (b.) an exemplary embodiment of an Explainable Transformer Decoder (XTT-Decoder). [Figure 10] FIG. 2 is a diagram of (a.) an example embodiment of an XTT-encoder with a downstream classification task and (b.) an example embodiment of an XTT-decoder with a downstream classification task. [Figure 11] FIG. 1 is a diagram of an exemplary embodiment of a distributed XTT training system. [Figure 12] FIG. 2 is a diagram of an exemplary embodiment of an explainable finite state transducer. [Figure 13] FIG. 1 illustrates an exemplary embodiment of a system for embedding rule-based knowledge within an exemplary XNN. [Figure 14] 1 is an exemplary schematic flow chart illustrating an exemplary explanation and interpretation process. [Figure 15] 1 is an exemplary schematic flow chart illustrating an exemplary recursion method. [Figure 16] FIG. 1 is a diagram of an example embodiment of a CNN-XTT architecture. [Figure 17] FIG. 1 is a diagram of an example embodiment of CNN-XTT implementing an inverse indexing mechanism. [Figure 18] FIG. 1 is a diagram of an example behavioral model for safety and assurance in an ML-based system. [Figure 19] FIG. 1 is a diagram of an exemplary XTT with a causal model. [Figure 20] FIG. 1 is a diagram of an exemplary explainable system having a multitask crossover learning structure. [Figure 21] 1A and 1B are diagrams of an exemplary multi-tasking crossover learning structure having (a.) one-way crossover links and (b.) two-way crossover links. [Figure 22] FIG. 1 is a diagram of an exemplary Explanation Structure Model (ESM) for document processing using XTT within an Identify-Assess-Recommend-Resolve (IAR) framework. [Diagram 23] FIG. 1 illustrates an example automated draft causal model creation from document data using an example XTT system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0039] Aspects of the invention are disclosed in the following description and associated drawings directed to specific embodiments of the invention. Alternate embodiments may be devised without departing from the spirit or scope of the invention. Additionally, well-known elements of exemplary embodiments of the invention will not be described in detail or will be omitted so as not to obscure the relevant details of the invention. Additionally, to facilitate understanding of the description, an explanatory discussion of some terms used herein follows.

[0040] As used herein, the word "exemplary" means "serving as an example, instance or illustration." The embodiments described herein are not limiting, but rather merely exemplary. It is to be understood that the described embodiments are not necessarily to be construed as preferred or advantageous over other embodiments. Furthermore, the term "embodiments of the invention," "embodiments," or "invention" does not require that all embodiments of the invention include the discussed feature, advantage, or mode of operation.

[0041] Furthermore, many of the embodiments described herein are described in terms of sequences of actions performed by, for example, elements of a computing device. It should be recognized by those skilled in the art that various sequences of actions described herein may be performed by specific circuitry (e.g., an application specific integrated circuit (ASIC)) and / or by program instructions executed by at least one processor. Moreover, the sequences of actions described herein may be embodied entirely within any form of computer-readable storage medium, whereby execution of the sequences of actions enables at least one processor to perform the functions described herein. Moreover, the sequences of actions described herein may be embodied in a combination of hardware and software. As such, various aspects of the invention may be embodied in many different forms, all of which are contemplated to be within the scope of the claimed subject matter. Moreover, for each of the embodiments described herein, any such embodiment in a corresponding form may be described herein as, for example, a computer configured to perform the described actions.

[0042] The terms interpretable and explainable can have different meanings. Interpretability can be a property that may need to be specified in terms of an interpreter. An interpreter can be an agent that interprets system outputs or artifacts using a combination of (i.) its own knowledge and beliefs, (ii.) a goal-action plan, (iii.) a context, and (iv.) the world environment. An exemplary interpreter can be a knowledgeable human being.

[0043] An alternative to a knowledgeable human interpreter may be a suitable automated system, e.g., a narrow domain expert system, that may be capable of interpreting the output or artifacts for a limited range of applications. For example, a medical expert system, or some logical equivalent such as an end-to-end machine learning system, may be capable of outputting valid interpretations of medical results in a specific set of medical application domains.

[0044] It may be contemplated that non-human interpreters may be created in the future that can partially or completely replace the role of human interpreters and / or extend interpretation capabilities to a wider range of application domains.

[0045] There can be two distinct types of interpretability: (i.) model interpretability, which measures how interpretable any form of automated or mechanical model is, along with its subcomponents, structure, and behavior, and (ii.) output interpretability, which measures how interpretable the output from any form of automated or mechanical model is.

[0046] Thus, interpretability is not a simple binary characteristic, but may be assessed on a sliding scale ranging from completely interpretable to uninterpretable. Model interpretability may be the interpretability of the underlying embodiment, implementation, and / or process that produces the output, while output interpretability may be the interpretability of the output itself or whatever artifact is being examined.

[0047] A machine learning system or suitable alternative embodiment may include multiple model components. Model components may be model interpretable if their internal behavior and function can be fully understood and correctly predicted by an interpreter for a subset of possible inputs. In one embodiment, the behavior and function of the model components may be implemented and represented in various ways, such as state transition charts, process flow charts or process descriptions, behavioral models, or any other suitable manner. Model components may be output interpretable if their outputs can be understood and correctly predicted by an interpreter for a subset of possible inputs.

[0048] An example machine learning system, or suitable alternative embodiment, may be (i.) globally interpretable if it is fully model-interpretable (i.e., all of its components are model-interpretable), or (ii.) modularly interpretable if it is partially model-interpretable (i.e., only some of its components are model-interpretable). Additionally, a machine learning system, or suitable alternative embodiment, may be locally interpretable if all its outputs are output-interpretable.

[0049] A grey-box, being a hybrid mix of black-box and white-box properties, can have the properties of a white-box in terms of its output, but the properties of a black-box in terms of its internal behaviour or functionality.

[0050] A white-box can be a fully model-interpretable and output-interpretable system that can achieve both local and global explainability. Thus, a fully white-box system can be fully explainable and fully interpretable, both in terms of its internal functioning and its outputs.

[0051] A black box can be output interpretable but not model interpretable, and can achieve limited local explainability, making the black box at least explainable with little or no explainability capability and minimal understanding in terms of its internal functioning. Deep learning neural networks can be output interpretable but model uninterpretable systems.

[0052] A grey box can be a system that is partially model-interpretable and output-interpretable, and can be partially explainable in terms of internal functionality and interpretable in terms of outputs. Thus, an exemplary grey box can be between a white box and a black box on a scale from mostly explainable and interpretable (white box) to slightly explainable and interpretable (black box). Grey box systems can have a level of modular interpretability because some of their components can be model-interpretable.

[0053] Explainable architectures utilized in the Explainable Transformer XTT model include, but are not limited to, eXplainable artificial intelligence (XAI) models, Interpretable Neural Nets (INN), Explainable Neural Nets (XNN), Explainable Spiking Nets (XSN), and Explainable Memory Nets (XMN) models. Further exemplary embodiments can present methods for detecting bias both globally and locally by leveraging the white-box nature of eXplainable Reinforcement Learning (XRL).

[0054] Although some examples may refer specifically to one or more of these (e.g., only XAI or XNN), it may be contemplated that any of the embodiments described herein may be applied interchangeably to XAI, XNN, XTT, XSN, or XMN. Another exemplary embodiment may relate to bias detection in interpretable neural nets (INNs) and related gray-box models, which may be hybrid models between black-box and white-box models. The exemplary embodiment may be fully applicable to the white-box portion of the gray-box model, and may be applied to at least some portion of the black-box portion of the gray-box model. It may be contemplated that any of the embodiments described herein may be applied interchangeably to INNs as well.

[0055] Explainable Neural Nets (XNNs) are a new type of Artificial Neural Networks (ANNs) that are inherently interpretable and explainable. The main concept behind XNNs is that the internal network structure is fully interpretable. The interpretability is built into the architecture itself, but it functions like a standard neural network. This removes the need to apply further techniques or processing to interpret the results of the neural network. XNNs compute both the answer and its explanation in a single feed-forward step, without the need for simulations, iterations, perturbations, etc. XNNs are similarly designed to be easily implementable efficiently in hardware as well as software, resulting in considerable improvements in speed and space.

[0056] XNN can combine multiple local models to become one global model. Local models analyze small areas in the entire search space. Global models roughly understand the model from a holistic perspective. XNN can do both - multiple partitions represent local zones and multiple linear models that explain each partition, and when combined, they constitute a global model. XNN supports both linear and nonlinear data by embedding transformations within the neural network itself, while preserving explainability. Each layer, neuron, and connection in XNN has a precise, well-known, understandable function, unlike standard ANNs, which are black boxes. Thus, XNN is the first fully white-box ANN known to date, and brings a new category of neural network that is understandable and interpretable.

[0057] Referring now to the exemplary FIG. 13, FIG. 13 may be a schematic flow chart illustrating rule-based knowledge or logically equivalent knowledge embedded in the XNN. First, the partition condition 2902 may be selected using a location method that may reference multiple rules and encoded knowledge. The partitions may be non-overlapping or overlapping. For non-overlapping partitions, the XNN may adopt a single path in feed-forward mode. For overlapping partitions, the XNN may adopt multiple paths in feed-forward mode and may calculate a probability or ranking score for each path. In an alternative embodiment, overlapping partitions may also utilize an aggregation function that combines results from activated partitions. The partition condition 2902 may be interpreted as focusing the XNN on a particular area of ​​the model shown. The partition location method may typically be implemented in a form given by a template 2904 in which various features 2906 are iteratively compared to real numbers 2908 using logical equivalents such as conjunctive normal form (CNF) or disjunctive normal form (DNF). In alternative embodiments, other non-Boolean logic systems may be utilized such as type 1 or type 2 fuzzy logic systems, modal logic, quantum logic, probabilistic logic, or other suitable types of logic systems for the expression of logical or similar statements.

[0058] The values, conditions, and basis equations of the localization method may be selected and identified using an external process, such as XAI model induction or a logically equivalent method. In an alternative embodiment, the values, conditions, and basis equations of the localization may be partially or fully inducted using an end-to-end approach using a gradient descent method, such as backpropagation. The selected values ​​are iteratively tested and varied until a certain level of accuracy is obtained by relative testing or when fitting the model to the provided synthetic data, and / or by iteratively questioning an initial black-box predictor model. The XNN may have four exemplary components within the localization or centralization module that may be part of the conditional network: an input layer 2910, a conditional layer 2912, a value layer 2914, and an output layer 2916.

[0059] The input layer 2910 may be constructed to receive various features that need to be processed by the XNN. The input layer 2910 feeds the processed features through a conditional layer 2912 where each activation excites a group of neurons. The conditional layer requires that a condition be met before passing along to the output. Additionally, the input may be further analyzed by a value layer 2914. The value (in the case of computations such as integers or real numbers) or class (such as in classification applications) X of the output X is given by an equation Xe calculated by the value layer 2914. The Xe function result may be used to generate the output 2916. It may be contemplated that the conditional layer and the value layer may occur in any order or simultaneously.

[0060] Referring now to the exemplary FIG. 3, FIG. 3 may show a schematic diagram of an exemplary high-level XNN architecture. The input layer 402 may be input to both the conditional network 410 and the predictive network 420, possibly simultaneously. The conditional network 410 may include a conditional layer 412, an aggregation layer 414, and a switch output layer (outputting conditional values) 416. The predictive network 420 may include a feature generation and transformation 422, a fit layer 424, and a predictive output layer (outputting values) 426. The layers may be analyzed by a selection and ranking layer 428, which may multiply the switch output by the value output to generate a ranked or scored output 430. Explanations and answers may be calculated by the XNN simultaneously by the conditional network and the predictive network. The selection and ranking layer 428 may ensure that the answers and explanations are correctly matched, ranked, and properly scored before being sent to the output 430.

[0061] It is contemplated that the processing of the conditional network 410 and the predictive network 420 may be in any order. Depending on the particular application of the XNN, it may be contemplated that some of the components of the conditional network 410, such as components 412, 414, and 416, may be optional or may be replaced with trivial implementations. Depending on the particular application of the XNN, it may further be contemplated that some of the components of the predictive network 420, such as components 422, 424, and 426, may be optional or may be replaced with trivial implementations.

[0062] It may further be contemplated that in some circumstances, the selection and ranking layer 418 and output 430 may be combined to be one integrated component. For optimization, the XNN may be similarly implemented with the conditional network 410 and the predictive network 420 both merged into one network with all their components. This merged conditional and predictive network may also be merged with the combined selection and ranking layer 428 and output 430. This optimization would still result in a logically equivalent XNN that may be faster for feedforward processing.

[0063] Thus, the XNN may be implemented such that there is an input layer 402 and a combination of a conditional network 410 and a predictive network 420 including a conditional layer 412, an aggregation layer 414, a switch output layer 416, a feature generation and transformation layer 422, a fit layer 424, a predictive layer 426, and a ranking layer 428 leading to an output 430. This combination may be applied to all embodiments and implementations of the XNN, including both software and hardware implementations. The transformation capabilities of the XNN in this regard are unique and unmatched in other neural network implementations, as the white-box nature of the XNN allows for flexibility and extreme merging to be implemented without affecting the logical behavior of the XNN, which may affect various attributes of a practical implementation, such as size / space usage, performance, resource usage, trainability, and overall throughput.

[0064]

[0033] Referring now to Figure 4, Figure 4 can show an example XNN architecture that combines results from a switch output layer with a value output layer. The example shown in Figure 4 is logically equivalent to the following example rule set:

number

[0065] The rule set may be found following the activation function in the output layer 520. The example architecture of FIG. 4 may begin with an input 400. This input 400 may be processed in an appropriate input layer, such as item 402 shown in FIG. 3. The input may then be used as an input to the conditional network 410 and the predictive network 420. As shown in FIG. 4, the predictive network may include a feature generation and transformation layer 422, a fit layer 424, and a value output layer 426. The value output layer 426 may provide an equation that weights different features of the input. Additionally, the input 400 may be used as an input to the conditional network 410, as shown in FIG. 3. Again, the conditional layer 412 and the aggregation layer 414 may generate conjunctive rules or other logical equivalents or partitions, as shown in the switch output layer 416.

[0066] The outputs of the value output layer 426 and the switch output layer 416 may be combined in an appropriate output layer, such as the output layer 430 shown in Figure 3. Once the output layer 430 is formed, a sigmoid or other activation function 520 may be applied to the results 518, depending on the application. It is further contemplated that a ranking layer 516 may similarly be applied to the results before and / or after output combination, depending on the application.

[0067] XNNs can present an intuitive way to build interpretable models while still harnessing the power of ANNs and related methods such as deep learning. Once a model is trained through backpropagation or similar methods, the resulting neural network can be used to aid in predictions, and the internal structure of the XNN can be used to build rules.

[0068] Interpretable Neural Networks (INNs) provide an architecture that can automatically generate explanations using existing deep learning techniques. INNs can take advantage of existing software infrastructure and hardware used for neural networks, and can also remain fully compatible with backpropagation training techniques.

[0069] The architecture can include a feature transformer that transforms the input into several hidden features and multiple relevance estimators that transform the hidden features into feature weights. The feature weights are then combined with the transformed input to extract the attribution of each input and transformed feature. The resulting feature contributions are then aggregated for the result. Feature contributions can be extracted at different levels. In the simplest form, the contributions can be directly linked to the original input. In other cases, such as CNNs, feature contributions can be similarly calculated for high-level features typically found in kernels and filters. Furthermore, INNs can partition the model with various partitions, thus allowing a higher level of flexibility and interpretability by allowing a mix of local or compartmentalized explainability. In some cases, INNs can provide global explainability as well.

[0070] Referring now to FIG. 5, FIG. 5 may show an exemplary architecture for an interpretable neural network. The exemplary embodiment may apply architectural constraints for standard neural networks to model data. The exemplary embodiment may start with some input vectors X 601. The inputs may be connected to a feature generation and transformation network 602 and a k-relevance estimator 604. The transformed features may be abstract or high-level features, which may have been computed using deep neural networks such as CNNs, nonlinear mathematical functions such as polynomial expansions, Fourier transforms, continuous data bucketization, causal operators, or some other form of generative features that may be discrete or continuous. It is further contemplated that the feature and transformation network can be a pipeline of transformations including, but not limited to, polynomial expansions, rotations, dimensional and non-dimensional scaling, Fourier transforms, Walsh functions, state space and phase space transformations, Haar and non-Haar wavelets, generalized L2 functions, fractal-based transformations, Hadamard transforms, Type 1 and Type 2 fuzzy logic, knowledge graph networks, categorical encoding, topological transformations of Kolmogorov / Fréchet / Hausdorff / Tychonoff spaces, dissimilarity analysis, and data normalization / standardization. The relevance estimator 604 can compute, at least in a local manner, coefficients for each transformed feature.

[0071] In mathematical terms, a transformation network can be represented as a function T(X). Similarly, θ j (X) denotes the relevance function of the j-th partition. If X→T(X) returns a vector with z transformed dimension, then X→θ j (X) similarly returns a vector with z coefficients or relevance weights. |T(X)|=|θ j It is assumed that (X)|=z.

[0072] INNs can be flexible enough to allow modeling of complexity through various options and configurations. i (X) can be a deep neural network that allows to model complex abstract features. The network can also be an XNN to have a white-box estimator. T(X) and θ j It may be noted that the combinations of (X) may represent various embodiments of explainable models that may be implemented using the INN architecture.

[0073] The neural network may include a conditional network 606. The conditional network 606 may evaluate rules in the form of IF conditions to activate one or more partitions. The output of Ci(X) may be binary. It may be noted that partitions may be static or dynamic and may be discovered through an external partitioning process, through a connected neural network, or through an internal process. It may also be noted that the INN may work with only one partition, where Ci(X) is always 1 for all values ​​of X. This is equivalent to having zero partitions. In this case, there is no need to apply a partitioning method to find a suitable partition.

[0074] The feature contribution 608 can calculate the contribution of each transformed feature activated by the associated partition. The relevance contribution can multiply the result of the calculated coefficient by the transformed feature. In mathematical terms, the feature contribution 608 can be expressed as θ j (X)T(X). The output of layer 608 can serve as the basis for explanation generation. Values ​​from this layer can be used to generate feature contribution graphs, heat maps, textual explanations, or other forms of explanations.

[0075] In the aggregation layer 610, the neural network aggregates the results for each partition (the predicted results for the activated partitions). In mathematical terms, an exemplary aggregation function is A j (θ j (X)T(X)). In an exemplary embodiment, the aggregation function can be a simple sum of the feature contributions. This is equivalent to a linear function, resulting in R j =θ j (X)1T(X)+…+θ j (X) Z At least it works in a local way so that T(X).

[0076] It may be contemplated that a model may have overlapping partitions. In this case, a ranking function may be applied to resolve the overlapping partitions. Let R define the rank function, and P j If,specifies the output of the jth partition, then,the generalized model can be written as, f(x)=R((P1, P2, ..., P j , ..., P k )) leads to.

[0077] Finally, the switch layer 620 selects the activated partition. If more than one partition is activated, some ranking function 625 may need to be applied. The results are generated through the result layer 630.

[0078] In a practical embodiment, the INN model includes an estimator θ j There may be some optimizations in (x), in the aggregation function A, in the ranking function R, or in some other component. INNs can provide a higher level of flexibility by allowing partitions to be customized depending on the level of simplicity or complexity required.

[0079] Referring now to the exemplary embodiment of FIG. 14, FIG. 14 may present an exemplary explanation generation pipeline. The pipeline may begin with an input query 902 indicating a data sample, scenario, or other question currently being examined, which may then be processed through three components: an explainable or interpretable system / model 904, an explainer 908, and an interpreter 912. Each of the three main components may generate different outputs that may be inputs to the next component in the pipeline. It may be contemplated that the input query 902 is not limited to only a particular data sample or scenario, but may relate to the entire model (global model explanation), or a modular explanation that addresses a particular component of the interpretable model.

[0080] The explainable or interpretable system / model 904 can generate an explainable or interpretable output 906 consisting of an answer 9061 with an optional model explanation 9062 of the answer, and optionally an answer justification 9063 and / or the model explanation. The answer justification 9063 and / or the model explanation is an explanation of the model explanation (i.e., a meta-explanation) and gives further information about the assumptions, process, and decision-making adopted by the explainable or interpretable system / model 904 when outputting the answer and / or the model explanation. The explainer 908 can generate an explainer output 910 consisting of an Explanation Scaffolding 9101 and optional candidate explanations 9102. In an exemplary embodiment, the candidate explanations are generated as a result of the processing of the Interpretation Scaffolding 9101 and may be used in the process of evaluation of the contents of the explanation scaffold in some form of iterative process including unsupervised or supervised learning and optimization. The filter 911 is an optional component that primarily transforms and filters the explanation scaffold 9101 and interpretation scaffold 9111, for example, after the explainer 908 has generated output for the current step, but before the interpreter 912 begins.

[0081] The interpreter 912 can use the interpretation scaffold 9111 to generate an explanation 9141 along with an optional interpretation 9142 of the explanation.

[0082] The end result 914 of the explanation process, including explanations and / or interpretations, may be provided to a human user, another application, or another system component forming part of a larger embodiment or some other automated system.

[0083] 14 may be one exemplary pipeline, and it should be noted that the pipeline may be implemented in a variety of alternative ways that may omit or combine one or more components and / or execute them in different orders and sequences. For example, in an exemplary practical implementation embodiment, it may be contemplated that both the explainer 908 and the interpreter 912 may be omitted, along with their respective outputs 908 and 914.

[0084] In another exemplary embodiment, components 908 and 912 may be combined as a single component that produces a combination of outputs 908 and 914 .

[0085] In another contemplated embodiment, a simplified implementation of interpreter 914 simply takes candidate explanation 9102 and outputs it as explanation 9141 .

[0086] It may further be contemplated that for global (i.e., model-level) description or query-less application embodiments, even the input query 902 may be omitted without loss of logical equivalence to the example pipeline of FIG. 14.

[0087] It may be contemplated that the outputs of certain combinations in FIG. 14 may be combined together or omitted in practical embodiments. For example, it may be envisioned that the justification 9063 may be considered optional in some applications, while the interpretation 9142 may be left for a human interpreter or expert to create, instead of an automated system. An interpretable machine learning system may have its function and internal behavior (model interpretability), the answer (output interpretability) and the explanation fully understood and interpreted by the interpreter. Although the final explanation with the answer may not require further post-processing on their core structure, further processing cycles that add syntactic and semantic information, contextualize and personalize the explanation are accepted as shown in FIG. 14, and the explanation 9141 goes through different stages of transformation and enrichment before reaching the final stage.

[0088] Explainable reinforcement learning (XRL) introduces explainable reinforcement learning systems by introducing the concepts of explanations and explanation models to actions and the environment in which the XRL system is deployed. An action can refer to an input provided to the environment, computed by applying a policy to the current state. It can be discrete or continuous. The set of all possible actions is called the action space.

[0089] RL / XRL agents can handle action selection and ranking, i.e., some action a for a current state s that results in state s'. The reward is denoted by r. RL agents can have a simulation of the environment that is used in the action selection process. Models can have further connection points depending on the structure of the model itself.

[0090] The first XRL modification introduces an explanation x as part of the model / environment model, i.e., the world model can return a partial or complete explanation for the state s' and reward r, which can be expressed as xe The second XRL modification is in the action space and is defined as a, x, which indicate the association description, i.e., the action and the description of the action, respectively. a A policy can refer to a mapping from past experience to actions. A policy П in XRL is defined as x , and P x where, P x ≒s → a, x a P x ≒s, x s → a, x a It is an explainable mapping such that

[0091] In behavioral FSM terms, each (state, action) pair can have certain connection points before taking the action after selecting the state, during the action, and after the action is taken. For RL and XRL, another connection point is before, during, and after the selection of the action under policy π. This is applicable when the action space and / or state space are discrete or continuous. Explanations as part of the XRL learning process can result in better safety and control mechanisms because they can enable a better understanding of the inner workings of the system, which may require tuning, monitoring, and automatic / manual intervention. XRL agents can be modeled similarly using FSTs instead of FSMs.

[0092] Exemplary Explainable Transducers (XFSTs) can partially replace pre-trained XNN architectures. Explainable Transducer-Transformer (XTT) embodiments can replace the black-box model components of Transformers (Vaswani et al., 2017) with white-box components by using an explainable architecture x (where x ∈ {XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture. Thus, XFSTs denote white-box machine learning architectures. Exemplary embodiments can provide an explainable version of the finite-state transducer with an explainable transformer, and can also include two main explainable transformer variants, namely, explainable transformer-encoder and explainable transformer-decoder.

[0093] Exemplary embodiments can provide an explainable transformer-encoder by introducing an explainable model into a sub-layer of the encoder layer of the transformer.

[0094] Exemplary embodiments may also provide an explainable Transformer-Decoder by introducing explainable models into sub-layers of the decoder layer of the Transformer.

[0095] The explainable transformer architecture can utilize an explanation framework to generate explanations and filter such explanations to generate interpretability for an interpreter.

[0096] In an explainable Transformer architecture, the inputs vary according to the components selected for induction of an explainable architecture x, where x∈{XAI, XNN, INN, XRL, XSN, XMN}, or a logically equivalent or similar architecture. In an exemplary embodiment, once an explainable model component is selected by induction, the actual inputs for the Transformer are fixed, and padding may be used for inputs of shorter length if necessary.

[0097] An explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture can be reducted in the explainable transducer transformer (XTT) to the feedforward network component 230 in the encoder layer and the feedforward network component 270 in the decoder layer to create two explainable architectures 700 and 710, as shown in FIG. 6.

[0098] The induced white-box model 700 constructs an explanatory and explainable path trace from the output of the summation and normalization component 217 to the output of the black-box feedforward component 230. The induced white-box model 710 can construct an explanatory and explainable path trace from the output of the summation and normalization component 242 to the output of the black-box feedforward network component 270.

[0099] XTT, which utilizes explainable models supporting wide learning models such as XNN, itself a class of Wide Neural Networks (WNN), can exhibit unique practical properties, especially when combined with the embedding nature of the Transformer architecture, which adds the generalization capabilities of deep learning to the more accurate and universal learning capabilities of wide learning. Our use of hierarchical partitioning techniques or other logically equivalent methods that can be used to identify suitable partitions adds significant representation and processing power to XTT without adversely affecting the explainability or interpretability of XTT. XTT explanation model partitions can be initially created as part of XTT by an external process that creates the initial XTT. Alternatively, XTT partitions can be similarly pre-trained or properly initialized from a linked taxonomy or ontology that provides suitable partitioning information. Once created, the XTT explainable model can use gradient descent to fit or further refine the partitions if desired. XTT partitions can be locked by having their values ​​set as static values ​​that cannot be changed, or can be dynamically movable for each backward training pass using standard backpropagation and related techniques. Hierarchical partitions can model data at different levels of abstraction. Such abstraction may be required when generating explanations. In an exemplary embodiment, a ranking function may be used to determine how XTT selects, merges, or splits its overlapping or non-overlapping partitions. It may be further contemplated that the hierarchical nature of XTT partitions may be used to represent symbols and concepts at different levels of semantic and semiotic detail.

[0100] XTT can allow its explainable partition structure and values ​​to be altered using techniques such as multiple objective optimisation (MOO), genetic algorithms (GA), and Monte Carlo Simulation Method (MCSM), or by causal logic and simulation techniques. XTT partitions can be hierarchical rather than simply flat partitions. Flat partitions can be considered a subset of possible hierarchical partitions where there is only a single level. The use of hierarchies allows more expressive explanations to be generated that better model the expected nature of good explanations.

[0101] Explainable transformer for 3-tuple datasets<I、O、E> Explainable Transformer may be trained on a Transformer 200, where I may refer to the input, O may refer to the output, and E may refer to the explanation between the input I and the output O. As shown in FIG. 6, the input may include a combination of token embeddings 200, positional information of the input tokens 210, and optionally partitioning information 1500, 1501 or similar explanation generated data from an external explainable model, to train the Explainable Transformer with an explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture.

[0102] An explainable transformer can be trained without using an explainable architecture in the encoder and decoder layers, and without appending partitioning information 1500, 1501 or similar explanation-generated data from an external explainable model to the input embeddings in the encoder, and optionally to the output embeddings in the decoder.

[0103] The explainable transformer can have two more layers parallel to the encoder and decoder layers, as shown in Figure 7. The two more layers can be focused on building an explainable architecture from the input space, as shown in Figure 7.

[0104] The explainable transformer can include a parallel explainable encoder layer 1630 that takes two inputs: the output of the multi-head-attention component 215 or the output of the summation and normalization component 217 and the partitioning or explainable information of the input tokens 1507.

[0105] The explainable transformer may include a parallel explainable decoder layer 709 that takes the output 1600 of the parallel explainable encoder layer 1630. The parallel explainable encoder layer may include an explainable architecture component 701 and a summing and normalizing component 702. The parallel explainable decoder layer 709 may include an explainable architecture component 703 and a summing and normalizing component 704. In an exemplary embodiment, the output of the parallel explainable encoder layer is used as an input 1631 to a multi-head attention layer in the decoder layer of the XTT as shown in FIG. 7. In another exemplary embodiment, the output of the parallel explainable encoder layer 1631 is merged with the output 1632 of the encoder layer prior to being used as an input for the multi-head attention layer 240 in the decoder of the XTT. In another exemplary embodiment, the output of the parallel explainable encoder layer 1631 is only used as an input to the parallel explainable decoder. As shown in FIG. 7, the XTT outputs a prediction output 290 and an associated explanation 1620. The explanation 1620 may be utilized as a feedback loop 1610 to the decoder component.

[0106] The explanation and partition information can be predicted using an XTT-decoder architecture that uses historical explanation and partition information. In an exemplary embodiment, the explainable model is a 3-tuple vector<i、o、e> where i refers to the input, o refers to the output, and e refers to the generated explanation. A generated explanation e (where e={e1, e2, ..., e n}) is a historical explanation of the current explanation e i and the following explanation considering the context window k: (i+1) The xtt-decoder is then used as input to the xtt-decoder to pre-train a model to predict values ​​for downstream tasks. The xtt-decoder can then be fine-tuned on a separate set of explanations to classify or predict values ​​for downstream tasks.

[0107] For the same component in the explainable model, the difference or change in one or more explanations e may be recorded and optionally used as input to XTT to predict the next difference or change in explanation. In an exemplary embodiment, the XTT-decoder is used to predict the next change in explanation. The XTT-decoder compares the current explanation g with the historical explanation gradient. i and the following gradient g (i+1) A set of explanatory gradients to predict e g (where e g = {g1, g2, ..., g m}). The XTT-decoder can then be fine-tuned on a separate set of explanatory gradients to classify or predict values ​​for downstream tasks. Pre-training on XTT can also utilize embedded representations of the input and output data to consider both similarity and contrast factors during training. In an exemplary embodiment, for example, multi-modal embedding of images and text captions can be utilized to simultaneously minimize similar related images and text, while ensuring that contrast between unrelated images and text is maximized. Sparse embeddings within XTT can also provide further performance improvements, especially in conjunction with sparse explainable models.

[0108] The explanations, gradients of the explanations, or partitioning information of the explainable architecture x may be used as input 1507 to the parallel explainable encoder layer of XTT, as shown in Figure 7. In an exemplary embodiment, the input 1507 of the parallel explainable encoder layer 1630 processes explanations of an explainable model trained on the same dataset that will later be used to train the XTT architecture. In a further exemplary embodiment, the input 1507 to the parallel explainable encoder layer refers to explanations generated on a dataset similar to the dataset that will be used to train XTT.

[0109] The inputs of the explainable transformer-encoder can be identical to the inputs of the encoder layer of the Transformer (Vaswani et al., 2017). The explainable transformer-encoder can induct a black-box feedforward neural network 1900 using an explainable architecture x 2000 (where x ∈ {XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture, as shown in Figure 8(b).

[0110] In another example embodiment, the XTT can have its explainable model trained on a full encoder layer whose input consists of token embeddings 1901 combined with positional embeddings 1902 or the output of a previous encoder layer. It is further contemplated that a variety of different positional encoding methods can be used for the XTT, including but not limited to Direct-Sum, Lookup Table, and Product kernel type positional encoding methods.

[0111] In an exemplary embodiment, the inputs and outputs of the explainable transformer-decoder can be the same as the decoder-based transformer (Liu et al., 2018). The inputs of the explainable transformer-decoder explainable architecture 2400 can include the output of the summation & normalization component 2340, as shown in FIG. 9(b). The outputs of the explainable transformer-decoder can be the output of the black-box feedforward neural network component 2320 or the output of the summation and normalization component 2330, as shown in FIG. 9(b).

[0112] In an exemplary embodiment, one or more components in the XTT may be replaced by an explainable model in addition to the explainable feedforward network itself. In a further exemplary embodiment, an explainable Auto-Encoder-Decoder (XAED) may be utilized to replace some or all of the Transformer-Encoder and / or Transformer-Decoder.

[0113] In an alternative embodiment, the explainable model in the explainable transformer-decoder can induct an entire decoder layer whose input consists of the token embeddings 2360 combined with the position embeddings 2310 or the output of the previous encoder layer.

[0114] The output of the explainable architecture XTT, the XTT-encoder, and the XTT-decoder may include explanations and interpretabilities generated from an explainable architecture x (where x ∈ {XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture.

[0115] The explainable architectures in XTT, XTT-Encoder, and XTT-Decoder can use training datasets that have no black-box components as input, in which case new explainable models are trained.

[0116] XTT can include feature transformations of training data sets. XTT transformation functions can be a pipeline of transformations including, but not limited to, polynomial expansions, rotation transformations, dimensional and non-dimensional scaling, Fourier transforms, Walsh functions, state space and phase space transformations, Haar and non-Haar wavelets, generalized L2 functions, fractal-based transformations, Hadamard transforms, type-1 and type-2 fuzzy logic, knowledge graph networks, categorical encodings, topological transformations of Kolmogorov / Fréchet / Hausdorff / Tychonoff spaces, dissimilarity analysis, and data normalization / standardization. The transformation function pipeline can further include transformations that analyze sequences of data that are ordered according to the values ​​of one or more variables, including time-ordered data sequences. The transformation function pipeline can further include transformations obtained by gradient descent and other explainable architectures.

[0117] It is also contemplated that the explainable model data synthesis capabilities may be used to generate training dataset samples that are presented as training for the explainable architectures in the proposed explainable transformer architectures XTT, XTT-encoder, and XTT-decoder using a variety of techniques, such as causal GAN-based generation, genetic engineering, Monte Carlo simulation, Petri nets, reinforcement learning techniques, simulations using white-box models and their global models, and other methods that may be available.

[0118] Human knowledge injection (HKI) or system knowledge injection is another type of input to the XTT, XTT-encoder, and / or XTT-decoder architectures. The coefficients of the explainable architecture in the XTT, XTT-encoder, and / or XTT-decoder architectures can be modified to enforce certain rules to provide better and more efficient collaboration between human users and AI systems.

[0119] It is further contemplated that Human Knowledge Injection (HKI) can be used to even further modify and extend the scope of synthetic data. The synthetic and HKI methods can help XTT overcome the data distribution shift problem, allowing XTT to better generalize to a wider range of possible inputs in its input and training space, including Out of Distribution (OOD) data.

[0120] The explainable model within XTT allows for practical implementation of zero-shot learning or few-shot learning solutions. Few-shot learning can utilize hierarchical partitions and local models to adapt more quickly when facing sparse data availability conditions. Zero-shot learning can be achieved in a practical manner by the incorporation of rules and thus the modification of partition structures within the explainable model by the HKI. Such use of the HKI can allow XTT to be bootstrapped directly from a suitable rule or set of rules without training data at all. It is further contemplated that the use of inductive, deductive, and hypothetical logic can be utilized to increase the adaptability of XTT. It is further contemplated that the use of causal logic can be utilized to increase the adaptability of XTT while ensuring that there is an element of plausibility and model-based checking that can help thwart underfitting and / or overfitting problems.

[0121] Explainability can be achieved through the extraction of partition information of explainable model components in XTT and its variants XTT-encoder and XTT-decoder, and through the construction of feature contributions of features in the training dataset.

[0122] The coefficients c of the local model can be used as feature contributions to the feature input space. The partition information and feature contributions are outputs of the explainable architecture components in the XTT, XTT-transducer, XTT-encoder, and / or XTT-decoder architectures.

[0123] The feature contributions c can be coefficients that indicate the importance of each input feature to the input space. Hence, the feature contributions are part of the explainability by transforming the input space into an explainable input space.

[0124] When the XTT, XTT-encoder, and / or XTT-decoder architectures are connected to other machine learning systems, the input space can represent some form of abstract features or concepts. For example, the inputs to the XTT, XTT-encoder, and / or XTT-decoder architectures can be the outputs from a CNN network. In an exemplary embodiment where explainability is built into the encoder and decoder portions of the transformer architecture, the outputs are feature contributions to the input space features and feature contributions to the latent space. XTT allows for the complete elimination of unexplainable latent space to the extent that the input features themselves are explainable. As another option, the XTT, XTT-encoder, and / or XTT-decoder architecture may be adapted to apply at least one of the following to its output: Fourier transforms, integer transforms, real transforms, complex transforms, quaternion transforms, octonion transforms, Walsh functions, state space transforms, phase space transforms, Haar and non-Haar wavelets, generalized L2 functions, fractal-based transforms, Hadamard transforms, fuzzy logic, knowledge graph networks, categorical encoding, disparity analysis, normalization, standardization, multidimensional Bezier curves, recurrence relations, and causal operators.

[0125] In another exemplary embodiment, the XTT, XTT-encoder, and / or XTT-decoder architecture may be required to apply a suitable activation function and / or a suitable transformation to the predicted output. Exemplary activation functions may include sigmoid or SoftMax functions or suitable equivalents. Exemplary transformations may include transforming the predicted output using some form of structure, such as (i.) hierarchical trees or networks, (ii.) causal diagrams, (iii.) directed and undirected graphs, hypergraphs, or simplicial complexes, (iv.) multimedia structures, or (v.) a set of hyperlinked graphs, or a suitable structure.

[0126] In an alternative exemplary embodiment, the partitions may be constructed externally using an inductive process, such as the exemplary model induction method shown in Figure 15. In such an exemplary embodiment, the output would be the partition information of such a system along with the feature contributions of the XNN or INN architecture.

[0127] In an exemplary XAI model induction method, the modeling system can input data into a black box system and record the output corresponding to each input. In another embodiment, the modeling system can use a trivial predictor model, such as a simple mapping function, that returns a label or response for all or a portion of the input data set, which is equivalent to performing induction directly on the training data. In another embodiment, the modeling system can use a suitable automatic predictor construction method to create a predictor model directly from the training data. The addition of a black box or predictor model as an input to the induction method can help reduce or eliminate error and / or noise from the original input data set.

[0128] The data points may be initially obtained from all or a portion of the training data, optionally combined with synthetically generated data. The synthetically generated data may be generated by an external process such as a simulation system, a world model, etc. The input data may also be an empty set, in which case the induction method will work using perturbation data obtained from the input predictor model. An optional data quality assurance function may be applied to a subset of the input data points and / or the input data as a whole prior to performing the induction method. The induction method builds a global model by dynamically creating perturbation sample data points that are then injected into the input predictor model, which may be a black box, and the output results are then noted and combined with all other available data points from the training and / or synthetic data. It may be contemplated that the perturbation data may be utilized on its own without using the original training data. Exemplary embodiments of the perturbation method may utilize a combination of random noise addition, rotation perturbation, projection perturbation, k-anonymization, Generative Adversarial Networks (GANs), or any other suitable perturbation method. The resulting combined data points may form the input to the partitioning function of the induction method. The data points from the predictor model may be continuous, discrete, or categorical valued. The predictor model that outputs a classification result may have such a classification applicator (i.) on the discretized output via some suitable transformation function that assigns the appropriate class label, or (ii.) on the probabilistic output that may be used directly or via some suitable transformation function.

[0129] The next exemplary step can identify a hierarchy of "partitions." Each partition can cover a set or cluster of data points taken from the combined data available to the induction method. Each partition can define zero, one or more rules and can optionally include child partitions with their respective associated rules. The extracted partitions can incorporate overlapping and non-overlapping partitions. In the case of overlapping partitions, some priority function can be used to determine which partitions should be activated. In an alternative embodiment, if more than one partition is activated, some aggregation function can be used to combine or split the results from the multiple activated partitions. Various methods can be implemented to find rules or clusters of data points for rules, as long as each partition has enough data to fit a linear model and is not too complex for a linear model to work well. Nonlinear transformations including, but not limited to, polynomial expansions, rotations, dimensional and dimensionless scaling, state space and phase space transformations, integer / real / complex / quaternion / octonion transformations, Fourier transforms, Walsh functions, continuous data bucketing, Haar and non-Haar wavelets, generalized L2 functions, fractal-based transformations, Hadamard transforms, Type-1 and Type-2 fuzzy logic, knowledge graph networks, categorical encodings, topological transformations in Kolmogorov / Fréchet / Hausdorff / Tychonoff spaces, dissimilarity analysis, and data normalization / standardization, as well as conditional features may be applied to individual partitions prior to linear fitting to enhance model performance.

[0130] Partitions can be clusters that group data points optionally according to some rules and / or distance similarity functions. Each partition can represent a concept or discriminative category of data. A partition represented by exactly one rule has a linear model that outputs a predictive or classification value. Since the model can be modeled linearly, the coefficients of the linear model can be used to score the features by their importance. The underlying features can represent a combination of linear and nonlinear fits, since the induction method deals with both linear and nonlinear data and / or linear and nonlinear predictor models.

[0131] Exemplary embodiments may be illustrated in mathematical terms. With reference to the exemplary embodiment of FIG. 15, FIG. 15 may illustrate an exemplary model induction method. In an exemplary embodiment, X may denote an input data set 1502, and Predict(x) may denote a predictor model function with n outputs 1504. X may be a matrix with m dimensions. Each dimension (m) may denote an input feature of X, and each output may denote a number of classes (n). The input data set may be a combination of training data and synthetically generated data. Further, in an exemplary embodiment, the samples may be perturbed 1506 using a function such as Perturbate(X), where Perturbate(X) may be any perturbation function that generates sample perturbed data given some input training or testing data X, which may be a subset of the input data, some neighborhood of the input data, any suitable embedding of input / output / explanation data, or may more commonly be an empty set, in which case the perturbation function should query the predictor model using some form of random or non-random pattern. X p Let denote the combination of the perturbed input data set and the original input data set, and let Y denote the combination of X p = X ∪ Perturbate(X) and Y = Predict(X p) the output from the predictor model function 1508.

[0132] The partition hierarchy is Partition(X p , Y)=P, where P={P1, ..., P i , ..., P k}, where k is equal to the number of rules where each partition is defined by exactly one rule. The partitioning function can be a clustering algorithm such as k-means, Bayesian, connectivity-based, centroid-based, distribution-based, grid-based, density-based, fuzzy logic-based, entropy-based, mutual information (MI)-based methods, or any other logically suitable method. The partition function can also include ensemble methods that will result in multiple overlapping or non-overlapping partitions. In the case of overlapping partitions, an aggregation function may be used to combine or split the results from multiple partitions. The partition function can alternatively include an association-based algorithm, causal-based partitioning, or other logically suitable partitioning implementation.

[0133] R i (x) can denote each rule in the i-th partition. Each rule (R i (x)) is R i (x)=C1∧C2∧...∧C j ∧...∧C q where q denotes the number of conditions in the rule, and C j can be the jth condition on the input matrix x of the ith rule (i.e., the ith partition). p When partitions do not overlap, {X L1 , X L2 , ..., X Li , ..., XLk} union = X p So that X L But {x|x∈X p ∧R i (x) = (true)}, where k is equal to the number of rules. When the partitions overlap each other, {X L1 , X L2 , ..., X Li , ..., X Lk} union ≠ X p When partitions overlap each other, a ranking function needs to be applied to select the most relevant rules or some form of probability weighted combining method is used. In an alternative embodiment, when more than one partition is activated, some aggregation function is used to combine the results from multiple partitions.

[0134] A next exemplary step may include fitting a local model to each partition defined by Ri. L X p and R i (x) can be used to denote inputs that are found in both L x can be the input or feature data to apply to the i-th rule. The local samples can be filtered 1512. L can then be perturbed, X LP X L We can show the combination of Y and the locally perturbed dataset of Perturbate(XL). L =Predict(X LP ) and locally predict the perturbed data1516. Finally, X T X T =Transform(X LP ) such that LPThe transform function may be computed by finding 1518 a transform of X. The transform function may be a pipeline of transforms including, but not limited to, polynomial expansions, rotations, dimensional and non-dimensional scaling, Fourier transforms, integer / real / complex / quaternion / octonion transforms, Walsh functions, state space and phase space transforms, Haar and non-Haar wavelets, generalized L2 functions, fractal-based transforms, Hadamard transforms, type-1 and type-2 fuzzy logic, knowledge graph networks, categorical encoding, dissimilarity analysis, and data normalization / standardization. The transform function pipeline may further include transforms that analyze sequences of data that are ordered according to values ​​of one or more variables, including time-ordered data sequences. T can generate new features by the transformation function z, where z denotes the total number of features.

[0135] For each j (i.e., the number of outputs) in the range 1 to n, the system can compute 1520 a local model in a linear fashion. L is Y Lj is a matrix containing n vectors (one for each output), where Y denotes the jth output vector. ij =Fit(X T , Y Lj ), where Fit is a function that fits a linear model such as linear regression, logistic regression, kernel methods, etc. A linear model can be defined by ij =β0+β1x1+…+β i x i +…+β z x z The set of coefficients {β0..β z} 1522. A linear model or a combination of linear models can provide an XAI model with rules 1524. In an alternative embodiment, the fitting of the local models can be accomplished in one atomic fit function using parallel programming or gradient descent techniques or suitable equivalents. i is a polynomial (x 2Note that it could be a transformed feature such as a feature vector (e.g., x, y), an intersection (e.g., x>10 and y<2), a conditional feature (e.g., x>10 and y<2), a Fourier transform, etc. i Note that Y can be positive or negative, corresponding to a positive or negative contribution to the model for the associated transformed feature. Positive or negative contributions may be commonly referred to as excitatory and inhibitory influences. Compression or regularization techniques may similarly be applied to simplify the fit of the local model, which indirectly simplifies the resulting description as well. Finally, the rule is ij can be constructed from various combinations of: Rule1(x) = {Y 11 , Y 1,2 , ..., Y i,n}if R1(x) Rule2(x) = {Y 21 , Y 2,2 , ..., Y 2,n}if R2(x) Rule i (x)={Y i1 , Y i,2 , ..., Y i,n}if R i (x) Rule k (x)={Y k1 , Y k,2 , ..., Y k,n}if R k (x)

[0136] n p The exemplary XNN architecture shown in FIG. 4 has n partitions, n local models 525, a value output layer value layer = {r1, r2, ..., r n}426, switch output layer switch layer = {s1, s2, ..., s n}416, and value layer 426 and switch layer Conjunctive Layer Combining 416 layer = {v1, v2, ..., v n} 516. An activation function 520, such as Sigmoid, may be used on the output values ​​to generate the final output of the exemplary XNN architecture. It is contemplated that a sparse XNN / INN may be utilized to implement multiple parts of the partition structure in an energy efficient manner.

[0137] The explainable model hierarchical partitions may also be subjected to one or more iterative optimization steps 1523, which may optionally include merging and splitting the hierarchical partitions using any suitable aggregation, splitting, or optimization method. A suitable optimization method may attempt to find an all paths connected topological space within the predictor's computational data space, giving an optimal gauge fix that minimizes the overall number of partitions.

[0138] The XTT transducer uses a 6-tuple to replace multiple components in the XNN architecture.

number

number

number

number

number

[0139] XTT can be built on a trained conditional network layer 410. In an exemplary embodiment, the input Σ is set to the partition boundary 412 of the XNN architecture.

number

[0140] In an exemplary embodiment, an XSFT style XTT accepts local model indices and respective predictions and outputs partition information for each of the local models utilized within the XNN architecture along with the predictions. The tape is inserted into a start state s2700 as shown in FIG. 12. For each observation, the tape contains the index of the value layer element followed by the respective prediction. As shown in FIG. 12, the exemplary XTT contains two partitions. Matching of the respective index of any partition enters the state of such partition and returns the partitioning information for such partition, where the partitioning information replaces the index of the value layer.

[0141] In another exemplary embodiment, an explainable finite-state transducer accepts an index of a local model and associated prediction values ​​and / or query input values, and outputs prediction values ​​and explanations multiplexed together in some suitable output encoding.

[0142] XTT can infer multiple connected components, such as black-box feedforward neural networks 230, 270 and subsequent summation and normalization components 232, 272, using an explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture.

[0143] XTT can induct the encoder-decoder multi-head attention component 240, summation and normalization component 242, feedforward neural network component 270, and / or summation and normalization component 272 of the decoder layer with an explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture to create a white-box model. Such explainable models may be intentionally sparse and / or compressed to maintain bisimulation equivalence between the original components and their explainable model replacements.

[0144] An XTT architecture that recurses the Transformer architecture may be referred to as an inline XTT architecture, as shown in Figure 6. An XTT architecture that recurses the Transformer architecture and uses an additional parallel explainable encoder layer and a parallel decoder layer may be referred to as a parallel XTT architecture, as shown in Figure 7.

[0145] Explainable transformer for 3-tuple databases<I、O、E> where I can refer to the input, O can refer to the output, and E can refer to the explanation between the input I and the output O. As shown in Figure 6, the input can include a combination of token embeddings 200, token position information 210, and optionally partitioning information 1500, 1501 or similar explanation-generated data from an external explainable model to train the XTT with an explainable model, as shown in Figure 6. It is further contemplated that a grey-box XTT implementation can be achieved by using a black-box feedforward component instead, as shown in Vaswani et al. (2017).

[0146] The transformer can be trained only on the encoder part as shown in Fig. 8(a) by using self-supervised learning techniques such as masked language modeling (Devlin et al., 2018). The explainable transformer-encoder can recurse the black-box feedforward neural network component 1900 of the encoder layer by an explainable architecture x2000 (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture as shown in Fig. 8(b). The architecture of the XTT-encoder can consist of input embedding 1901, positional encoding 1902, multi-head attention component 1920, summation and normalization component 1930, explainable architecture 2000, and summation and normalization component 1910.

[0147] The explainable transformer-encoder can recurse a feedforward neural network component 1900 and a summation & normalization layer 1910 using an explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture.

[0148] The explainable transformer-encoder can induct the multi-head attention 1920, the summation and normalization component 1930, the feedforward neural network component 1900, and / or the summation and normalization component 1910 using an explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture.

[0149] The Transformer may be trained only on the decoder portion of the Transformer architecture, as shown in Figure 9(a) (Radford et al., 2018). XTT may induce the feedforward neural network component 2320 of the decoder layer by an explainable architecture x2400 (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture, as shown in Figure 9(b).

[0150] The explainable transformer-decoder can infer multiple components, such as a combination of a feedforward neural network component 2320 and a summation and normalization component 2330, using an explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture.

[0151] The explainable transformer-decoder can induct a multi-head attention 2350, a summation and normalization component 2340, a feedforward neural network component 2320, and / or a summation and normalization component 2330 with an explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture.

[0152] The explainable transformer-based architecture XTT, XTT-encoder, and XTT-decoder can generate multi-level explanations that can include partitioning information, model internal coefficients, and feature contributions of the input space, and such explanations can be output to an interpreter. The explanations can be in the form of text or numerical formats that can indicate the importance of input dimensions, or some form of appropriate transformation applied to the inputs and appropriate interactions, and such transformations can output the explanations in various output formats, including multimedia images, videos, 3D data, multi-dimensional data, and other suitable formats. The explanations can also include the results of applying interventions and / or counterfactual processes, for example, by utilizing bias-controlled swaps to determine bias in a given set of input dimensions. Any such biases detected directly by XTT may not even require the use of a dataset for such bias testing. It is further contemplated that such results can be generalized by application of suitable identification, assessment, recommendation, ranking, and resolution functions within an Identify-Assess-Recommend-Resolve (IAR) framework.

[0153] XTT can handle both structured and unstructured data, including hierarchically structured data in the form of trees, graphs, hypergraphs, and simplicial complexes. Related prior work on the Tree Transformer (Wang et al., 2019) constrains the attention head using a compositional attention component to introduce a tree-based structure of the input tokens. Unlike our proposed method, the Tree Transformer utilizes a black-box feed-forward neural network component and can therefore be classified as a grey-box method. Activation path tracing will not be possible to implement using the Tree Transformer, as is the case with XTT. Our proposed method introduces a white-box component into the transformer-based architecture to generate global and local explanations. The hierarchically structured partition system in the XTT explainable model is naturally suited to handle complex structured data that is only partially achieved in the Tree Transformer using compositional priors. The XTT attention model is capable of expressing the equivalent of a hierarchy of such priors at different levels of detail and scales, which is not possible to achieve using the Tree Transformer and related transformations. This can be possible through linear or nonlinear equations available in the rule-based model (XAI model) or logically equivalent methods such as XNN, INN, XSN, XMN, and other appropriately equivalent implementations. In the linear or nonlinear equations, a list of features can be extracted along with their corresponding coefficients that can indicate feature importance for that given model or set of rules. The coefficient matrix C of the explainable model can be directly analyzed to extract the most important features for the rules / partitions in the model defined by F(X).

[0154] In an alternative embodiment, C p,i Let I={θ0, ..., θ identify the ith coefficient of the pth partition / rule. i , ..., θ k-1}(where:

number

[0155] Normalization may be similarly applied to the resulting feature importances. In an exemplary embodiment, the F and M vectors may be used to generate appropriate reports and analyses of biases and their sources, and may also be inputs to formulate strategies for bias reduction, mitigation, or elimination by supervised, unsupervised, or semi-supervised means.

[0156] In an exemplary embodiment, XTT may include multiple input and output modules, one or more of which may include a normalization module configured to normalize an input before it is received by or after it is output from an input layer, a scaling module configured to scale the input before it is received by or after it is output from an input layer, and a standardization module configured to standardize the input before it is received by or after it is output from an input layer.

[0157] In an alternative exemplary embodiment, the F and M vectors may be used as inputs to a feature discovery and model discovery method, which utilizes this information as further input parameters when perfecting its strategy and / or to allow the feature discovery and model discovery method to suggest and optionally implement structural changes to the AI ​​model.

[0158] For example, in an exemplary embodiment, bias information may be used to focus a GA or MCSM system on specific parts of an XNN / XSN / XMN that exhibit bias, or on rules in an XAI model that may be causing bias, or on attention models of an XTT that exhibit bias, or on actions taken by an XRL agent that may be causing bias under some policy.

[0159] Local bias detection can be applied to a particular sample. The answer, together with its explanatory coefficients, can serve as the bias for localized bias detection. Referring to the example model, a particular input sample can have two inputs, in this case x and y. The feature importance vector I is a set of features {x, y, xy, x 2 , y 2}, and I={β1, β2+β 10 , β3+β5, β7, β8}. The vector I may be sorted in descending order such that the most salient features are placed at the beginning of the vector. It may be noted that given a particular input vector {x, y}, one or more rules may be triggered through a function f(x, y).

[0160] In this exemplary embodiment, let x≦10. This can trigger the function Sigmoid(β0+β1x+β2y+β3xy), which results in the following localized feature importance vector that serves as the bias as an explanation candidate, where E={β1x, β2y, β3xy}. It can be noted that different features can have different importance at global and local levels; for example, in this particular embodiment, globally, feature x can be the most important, however, after evaluating the function locally, it can be noted that y is the most important.

[0161] XTT can be used in conjunction with a suitable Distributed Explainable Architecture (DEA) to improve the performance of a prescribed model. A DEA is a set of m={x1, x2, ..., x n} and x i DEA with multiple explainable architectures, ∈{XAI, XNN, INN, XTT, XRL, XSN, XMN} m Alternatively, the number of explainable architectures 2610 in the explainable distributed framework may include logically equivalent or similar architectures, which are processed in parallel. n It is defined as:

[0162] Figure 11 shows the high-level architecture of the distributed training system. m shows the XTT architecture, and DEA n DEA further includes n models, which can be homogeneous or heterogeneous mixtures of explainable architectures. mTo train , the dataset is split into multiple subsets 2600 of data. It is further contemplated that suitable distributed training techniques, including but not limited to collective operation techniques such as All-Reduce, Horovod, and other suitable distributed learning techniques, may be applied to homogeneous and heterogeneous explainable model mixes within the DEA.

[0163] Models trained in the distributed framework DEA are aggregated during the training phase by computing the average (or weighted average) from the parallel models 2620. An aggregate model can be formed based directly on the model weights, rather than from the results of the individual models. The distributed explainable architecture DEA can be useful for large data sets where the training data cannot fit into the CPU / GPU memory of a single machine.

[0164] DEA can include hybrid models where the model of the architecture is a mix of x (where x ∈ {XAI, XNN, INN, XTT, XRL, XSN, XMN}) or logically equivalent or similar architectures. Exemplary embodiments can implement multiple different models. For example, one data part can implement XNN while another data part of the same system can implement XTT. The models can then be combined to create an aggregate model. The aggregate model can be equal to x (where x ∈ {XAI, XNN, INN, XTT, XRL, XSN, XMN}) or logically equivalent or similar architectures, or can be a hybrid model implementing multiple different models.

[0165] DEA can incorporate multiple independent models, and one model, once trained, can work independently without the need to rely on a fully distributed architecture that is optimized primarily for training.

[0166] The XTT architecture builds an explanation scaffold from the output produced by XTT's explainable architecture and uses it to present results to interpreters, helping them understand how the model arrived at such predictions. Interpreters can be end users or components within the XTT architecture.

[0167] Components of the XTT foundation of the explainable transformer architecture may require model interpretation to update their state or internal parts of the model.

[0168] There are three types of model interpretations: (i.) basic interpretations or answers, (ii.) explanatory interpretations or explanations, and (iii.) meta-explanatory interpretations or justifications.

[0169] The basic interpretation or answer may refer to a predicted output o that can be understood by the subcomponents.

[0170] Explanation The interpretation or explanation is a 2-tuple vector<o、w> The model explanation may refer to a predicted output o combined with a model explanation w for such a predicted value, which may be understood by the subcomponents, denoted by (x) and may include coefficients θ of an explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture that may be utilized to explain the feature importance of input features for a given observation.

[0171] The meta-explanation interpretation or justification is a 3-tuple vector<o、w、j> and may include the predicted output o, the model explanation w, and the model explanation justification j. The model justification j provides further information about the assumptions, processes, and decisions taken by the explanation system that were considered to generate the model explanation.

[0172] The model explanation and meta explanation interpretations can be utilized by an interpreter to update components within the XTT architecture.

[0173] In an exemplary embodiment, XTT can be utilized within an explanation and interpretation generation system (EIGS) to provide a suitable model for processing an input query and generate appropriate answers, explanations, and optional justifications for the input query. A typical EIGS explanation and interpretation pipeline is shown in FIG. 14. It may be further contemplated that XTT may be utilized to transform appropriate portions of the explanation scaffold within the EIGS, including, but not limited to, transforming presentation output, user-specific output personalization, context-specific output transformations, goal-specific output transformations, plan-specific output transformations, and action-specific output transformations. It is further contemplated that XTT may be utilized to transform EIGS output based on a combination of statistical, causal, taxonomic, ontological, domain, and semantic information.

[0174] In an exemplary embodiment, XTT can be utilized within an Explanation and Interpretation Generation System (EIGS) to provide a suitable model for a Filter to generate appropriate filtering of explanation scaffolds. It may be further contemplated that XTT may be utilized to filter appropriate portions of explanation scaffolds within the EIGS utilizing a combination of interpretation framing, protocol context, interpretation rules, interpretation procedures, interpretation scenarios, and conflict resolution information. The XTT attention model may be used to implement intra- and / or inter-filter consensus processes within the EIGS.

[0175] In an exemplary embodiment, XTT can be utilized within an explanation and interpretation generation system (EIGS) to provide suitable models for interpreters to generate appropriate interpretations and generate elements of interpretation scaffolds. XTT can be utilized to transform appropriate portions of interpretation scaffolds within the EIGS, including but not limited to transforming interpretation outputs, interpretation scenario-specific output transformations, context-specific transformations, and interpretation summary-specific output transformations. XTT can be utilized to transform EIGS interpretation outputs based on a combination of framing, protocols, interpretation rules, interpretation procedures, interpretation templates, interpretation summaries, scenario models, domains, interactive contexts, and conflict resolution information. XTT attention models can be used to implement intra- and / or inter-interpreter consensus processes within the EIGS. XTT can be utilized to transform EIGS interpretation outputs based on appropriate selection models and selection processes.

[0176] XTT can be trained on images without the need for a convolutional neural network (CNN) (Dosovitskiy et al., 2020; Chen et al., 2020). Transformers can be trained on image data without incorporating the 2D input structure of the image to autoregressively predict pixels of the input image instead of predicting language tokens. iGPT (Chen et al., 2020) used a low-resolution version of the ImageNet dataset to train the GPT-2 architecture to learn image representations in a transformer architecture. Vision Transformer (Dosovitskiy et al., 2020) splits the input image into multiple patches, flattens them, and uses them as input tokens to train the transformer architecture. Vision Transformer (Dosovitskiy et al., 2020) and iGPT (Chen et al., 2020) can be seen as black-box transformers. XTT is a transformer architecture that is not suitable for learning neural networks because its components, such as multilayer perceptrons (MLPs), are not explainable or interpretable. XTT introduces white-box explainable components into the transformer architecture, which allow for the generation of interpretable internal coefficients and explanations that can be matched with those of vision transformers (Dosovitskiy et al., 2020) or iGPTs (Chen et al., 2020). A variation of the self-attention mechanism can be used in the transformer.

[0177] The EQ Transformer (Mousavi et al., 2020) is a black-box model because the predicted seismic signal, P-phase output, and S-phase output are not explainable or interpretable. XTT introduces a white-box explainable component within the transformer architecture, allowing the generation of explanations such as interpretable internal coefficients and feature contributions to the input space that indicate the importance of each input feature. Thus, an exemplary XTT-based implementation of an earthquake detection system results in a white-box explainable system, unlike the EQ Transformer. XTT similarly allows further generalization of hierarchically structured explanations without the need to rely on multiple encoder-decoders as in the case of the EQ Transformer, relying solely on explainable models embedded within the XTT, such as XNN.

[0178] A self-attention mechanism or similar mechanism may be induced by an explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture to create an explainable self-attention mechanism. The input tokens of the self-attention mechanism and the black-box self-attention mechanism or similar mechanism may be required as inputs to induce the explainable architecture x. An explainable self-attention mechanism may generate multiple levels of explanations that may include partitioning information, internal coefficients of the model, and feature contributions of the input space, and such explanations may be output to an interpreter.

[0179] XTT can be trained using self-supervised techniques such as masking, cloze deletion, Pattern-Exploiting-Training (PET), game-theoretic approaches, and other suitable techniques. Typed mask symbols can represent classes / families of different symbols, concepts, or objects and can be distributed and encoded across the input data. The interaction of the typed mask with the rest of the input symbols / features represents the generalized training data used by XTT. Typed masks can be selected from a permutation of a subset of possible tasks and modalities. Thus, an untyped mask is the equivalent of a typed mask that combines all possible tasks and modalities into one mask symbol.

[0180] XTT can utilize previously learned representations or history states from all encoder or decoder layers to merge into a single vector v. The vector v can be used as feedback for any layer in XTT so that the initial layers of XTT can access higher level representations. Past work such as Feedback Transformer (Fan et al., 2020) is limited to self-attention modifications to accommodate higher level representations. XTT can utilize any internal coefficients of the white-box model to accommodate higher level representations due to the nature of the white-box model. It is further contemplated that such XTT can also impose causal constraints on the previously learned representations to avoid making causally improbable inferences or predictive paths that may be statistically valid but causally improbable. XTT can also utilize behavioral models and their condition, event, trigger, and action state representations to impose further environmentally plausible constraints on the previously learned representations to achieve an explainable but simultaneously causally and environmentally plausible feedback memory. Adding feedback memory to XTT makes it Turing Complete because it can exhibit conditional processing and simulate Turing machines, opening up new application areas such as automated code generation, refinement, and optimization.

[0181] In an exemplary embodiment, the XTT system can be used to analyze and parse an existing formal computer program written in a suitable formal language and, once loaded, further refine it using the XTT attention model. A human user can be able to indicate areas that the user needs to fill in by using a similar mechanism for the next sequence tokens used to generate natural language content. In the case of formal languages, the hierarchical partition structure in the explainable model can be used to directly model the structure or artifacts of the formal language program. It is further contemplated that a formal specification language can be used to specify the problem that needs to be solved using a formal language program, thus allowing the XTT system to automatically generate candidate code solutions for the posed problem. In another option, the XTT system can be used to transform code according to predefined stylistic standards, highlight inconsistencies or errors, suggest better alternatives as well as re-factorings and re-writes of the code, de-obfuscate obfuscated code, and similarly apply functional programming language concepts such as alpha-transformation, beta-reduction, and eta-reduction to the generated code. In another option, the XTT system may use Metamath, F to automatically analyze, generate, and autocomplete mathematical expressions, statements, and proofs from scratch or from an existing collection of proven and / or incomplete statements. *, HOL, Mizar, and others, along with an Automated Theorem Proving system. In another option, the XTT system can be used to analyze the DNA code in conjunction with proteins or other suitable molecular shapes to explain regulatory changes in DNA, gene expression, protein folding, and other related biochemical applications, while providing an explanation of the underlying assumptions between the input-output transformations applied by XTT, which are necessary for regulated and reliable applications involving drug discovery, medical applications, and pharmacologic applications.

[0182] In an exemplary embodiment, the end-to-end deep learning architecture consists of XTT, which is used for image and video compression. Video frames are divided into patches that are then used as input tokens to train XTT. Such contextualized tokens are generated by downsampling distinctive features required for optimal performance. The explanations generated by XTT's explainable architecture can form lossless explanation paths during the backmapping process. The paths can indicate the importance of filtered and contextualized tokens for a particular classification label. Redundancy optimization and adaptive token elimination techniques, such as those presented in (Goyal et al., 2020), can be utilized to further accelerate XTT performance.

[0183] XTT can be used in end-to-end automatic speech recognition (ASR) architectures. In an exemplary embodiment, XTT is used to translate speech audio waveforms into corresponding text. The pre-trained XTT contextual representations are connected to a linear layer and a SoftMax operation to predict the output. The predicted output is then connected to a decoder, such as a beam search decoder, to convert the output into corresponding text.

[0184] In an exemplary embodiment, the XTT is used in an end-to-end deep learning architecture to process speech. The XAED-encoder is used to convert speech into a latent discrete representation. The XTT is connected to the output of the XAED-encoder to predict the next output based on the previous output. The XAED-decoder is connected to the XTT to convert the predicted output into speech. To measure the performance of the exemplary embodiment, a phoneme error rate (PER) metric is used with a pre-trained ASR model to convert the generated audio to text.

[0185] In an exemplary embodiment, XTT is used for multiple object tracking. XTT consists of an encoder and two decoders d1 and d2 to generate bounding boxes as output. The keys and values ​​of the XTT-encoder and a query q1 are passed as inputs to d1. d1 can predict objects in the current video frame. The keys and values ​​of the XTT-encoder and a query q2 are passed as inputs to d2. d2 can predict objects in the current video frame using objects detected in the previous frame. The performance of d1 and d2 can be measured using metrics such as intersection over union.

[0186] XTT can be used to match satellite, aerial, or other types of bird-eye-view (BEV) images for digital surface model (DSM) or depth map (DM) generation. In an exemplary embodiment, the XTT-encoder is used to measure disparity between satellite image pairs utilizing an exemplary architecture similar to that shown in FIG. 10(a), where predictions are calculated using disparity regression instead of classification. The input tokens of the XTT-encoder represent the image pairs as:

number

[0187] In an exemplary embodiment, XTT is used for an audio-visual classification task to predict whether audio is present in a video. Audio is represented using an extracted spectrogram divided into frames using a sliding window approach. Visual frames are represented using a flattening layer of the CNN-XNN architecture as visual frame embeddings, using frames sampled by the sliding window approach used in the audio representation. The input for the XTT-encoder can consist of audio embeddings, and the input to the decoder can consist of visual embeddings. The order of the inputs can be reversed so that the visual embeddings become the input to the XTT-encoder and the audio embeddings become the input to the decoder. XTT is connected to a sigmoid layer to predict whether an input audio event class is present in the video embeddings. In a further exemplary embodiment, the CNN-XTT architecture is used in place of the CNN-XNN architecture. In a further exemplary embodiment, the final feedforward layer of the CNN architecture is used in place of the flattening layer of the CNN-XNN architecture. In a further exemplary embodiment, the CNN-XTT architecture is utilized to detect other input modalities, except audio, including objects in the audio-visual input.

[0188] XTT may be used for the generation and detection of synthetically generated multi-dimensional data. In an exemplary embodiment, XTT implementations may be used to detect and classify deepfake images from authentic images, or to detect the presence of adversarial attack information added to data to confuse machine learning systems. It is contemplated that suitable explainable models, such as Explainable Generative Adversarial Network (XGAN) and / or Explainable Generative Adversarial Imitation Learning (XGAIL) models, may be used to extend XTT's generation and detection capabilities. In another exemplary embodiment, XTT may be implemented within a camera processing system to automatically generate, insert, and blend precisely positioned, framed, scaled, lit, and rendered synthetic images of people and objects within a combination of real and computer-generated imageries. It is further contemplated that such synthetic image blending systems may be implemented within Augmented Reality (AR), Virtual Reality (VR), or similar metaverse systems. In further exemplary embodiments, the XTT-based system may be implemented in portable hardware ophthalmic devices utilized for AR / VR / Metaverse applications to perform real-time blending of synthetic images with real-life or metaverse imageries. It is further contemplated that such XTT-based systems may be implemented in medical hardware for use in surgery or analysis of medical images, in engineering applications for use in equipment and manufacturing inspection processes, and in medical implants that transmit data in a human-readable and interpretable format, such as in an appropriate bio-electrical human-machine interface, such as an electromechanical interface, an electrochemical bio-interface, or a bio-neuron-artificial AI system interface.In further exemplary embodiments, the XTT-based system may be implemented as part of a medically implanted device that bridges two or more biological neurons, receiving input signals from the biological neurons and outputting signals to the biological neurons in a suitably encoded format. It is contemplated that such applications may provide a practical solution for bridging damaged biological nervous system connections, for example, in paraplegic patients, or aid in the implantation and connection of artificial devices, such as cameras and other sensors, in advanced prosthetic devices. It is further contemplated that the simulator and data sample synthesis of the XGAIL system implemented as part of the XTT system will enable the XTT system to generate multimodal mixed outputs that are appropriately synchronized (e.g., temporally synchronized) according to the associated sequence ordering and environmental world context of the different modalities. It is further contemplated that such exemplary XTT system implementations may similarly be used to create novel data samples, imageries, digital and analog mixed media paintings, and 3D sculptures, using various learned styles that may be used to appropriately modify the generated data samples, for example, to mimic a particular artistic style. It is further contemplated that such generated data samples may be tagged with any secure traceable digital code, distributed ledger entry, or non-fungible token (NFT).

[0189] In an exemplary embodiment, the XTT implementation may be used to generate music using a combination of formal music notation and synthesis, sound sample mixing, text-to-speech generation, and general audio sample generation. It is further contemplated that the multimodal XTT implementation may be used to generate additional output in different modalities, for example, using video generation to create accompanying videos for music, pose estimation and deepfake generation models to create dance videos with realistic synthetically generated avatars with human-like movements, and similar applications. It is further contemplated that the multimodal XTT implementation may be used to analyze input in different modalities, for example, to correlate speech, pose estimation, text, and gesture identification to create avatars that move and behave realistically in a virtual environment.

[0190] XTT can be pre-fine-tuned as a further step after pre-training to further improve performance on downstream tasks.

[0191] The XTT-encoder can be used in an end-to-end deep learning architecture to predict annotations for video frames. In an exemplary embodiment, the deep learning consists of a CNN-XNN component and an XTT component. Video frames of the input dataset are used as input to the CNN-XNN architecture to predict object embeddings. The corresponding transcribed text of the video frames, input t (Here, input t = {t1, t2, ..., t n}) is used as input to the XTT-encoder to predict the contextualized word embedding. The explainable end-to-end deep learning model j (where t jThe CNN-XNN architecture can learn embeddings of contextual word representations for (which may refer to identifiable words in the transcribed text of a video frame), such that the CNN-XNN architecture can learn the semantic context of the predicted labels. In a further exemplary embodiment, the CNN-XTT architecture is used instead of the CNN-XNN architecture.

[0192] The XTT-encoder can be used in end-to-end deep learning architectures for dialogue state tracking (DST) tasks to track dialogue states and learn unknown slots instead of being predefined in an ontology. In an exemplary embodiment, the inputs to the XTT-encoder are [CLS] tokens, system dialogue token embeddings, and s (where d s = {s1, s2, ..., s n}), User interaction token embedding u (where d u = {u1, u2, ..., u m}), and in the input space, d u From d s The contextualized token representation [CLS] may then be connected to an explainable architecture x (where x ∈ {XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture to classify the context class of the interaction. The contextualized token representation of the remaining part of the input tokens may be connected to an explainable architecture x to predict the start and end positions of the slot values ​​by selecting the token with the maximum predicted probability for the start output and selecting the token with the maximum predicted probability for the end output. The explainable architecture of the XTT-encoder may be inducted from a black-box model or may be learned directly from the dataset.

[0193] In an exemplary embodiment, the XTT-encoder is used to identify entities within a given input sequence. The input sequence {s1, s2, ..., s n} is the contextualized representation {y1, y2, ..., y n}. Generate the contextualized representation {y1, y2, ..., y n} may be connected to an explainable model to predict two outputs: a start position of an entity and an end position of an entity. The contextualized token representation with the highest probability prediction for the start position may be selected for the start position of the entity representation. The contextualized token representation with the highest probability prediction for the end position may be selected for the end position of the entity representation.

[0194] In a further exemplary embodiment, an XTT-encoder is used to identify the start and end positions of an answer for an input question. The input to the XTT-encoder is a [CLS] token, a question input q (Here, input q = {q1, q2, ..., q n}) token embedding, answer input a (Here, input a = {a1, a2, ..., a m}) token embedding, and input a From input q The Contextualized Token Representation [CLS] then indicates that the answer is a token embedded input a The input is connected to an explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture to classify whether it is present in aThe contextualized representations of may be connected to an explainable model to predict two outputs: a start position of the answer and an end position of the answer. The contextualized token representation with the highest probability prediction for the start position may be selected for the start position of the answer. The contextualized token representation with the highest probability prediction for the end position may be selected for the end position of the answer.

[0195] In an exemplary embodiment, the XTT-encoder is used to link mentions of entities in text to corresponding entities in the knowledge base. The input to the XTT-encoder is the [CLS] token at the start of the input space, followed by the context input c (Here, input c = {t1, t2, ..., t n}) token embedding, mention input m (Here, input m = {t1, t2, ..., t m}) token embedding, entity input e (Here, input e = {t1, t2, ..., t k}) token embedding, and input e From input m and input c The contextualized token representation of [CLS] can then contain a [SEP] token separating the entity input m The mention of the entity input e The XTT-encoder is connected to an explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture to predict whether a mention will be linked to an entity. The XTT-encoder can assign mentions that are similar to an entity as negative predictions to learn an XTT that is semantically aware in its entity linking task.

[0196] The XTT-encoder may be combined with an XTT-decoder to predict an output based on the previous output. In an exemplary embodiment, the XTT-encoder is combined with an XTT-decoder to predict the next observed action in a video frame. The XTT-encoder performs a feature representation l for each frame, such that l = XTT-Encoder(V), in a mechanism similar to the work of Dosovitskiy et al. (2020). n}) as non-overlapping flattened image patches. It is further contemplated that the XTT-encoder process may be used in practical implementations of explainable video coding and video modeling systems. The XTT-decoder processes the feature representation l combined with the temporal location embedding to predict the action present in a video frame based on the immediately preceding video frame in a manner similar to that shown in FIG. 10(b).

[0197] In an exemplary embodiment, the XTT-encoder architecture can be used to induct an encoder-based transformer trained to classify medical imaging observations. Given input tokens {t1, t2, ..., t n} can refer to multiple portions of medical imaging data combined with a location embedding mechanism. The XTT-encoder can induct a black-box feedforward neural network component 1900 with an explainable model x2000 to build a multi-level explanation associated with the output of the black-box feedforward neural network component that can include partitioning information, the model's internal coefficients, and feature contributions of the previous layer 1930. The XTT-encoder can induct a black-box MHA component 1920 with an explainable model x to build a multi-level explanation associated with the output of the black-box feedforward neural network component that can include partitioning information, the model's internal coefficients, and the input tokens {t1, t2, ..., t nA multi-level explanation can be constructed in relation to the output of the black-box MHA component 1920, which can include the feature contributions of the}. The model's internal coefficients and feature contributions can indicate the importance of biases found in the input dimensions or the encoder-based transformer.

[0198] XTT can include an explainable model x that employs further inputs, such as, but not limited to, coefficients from a constituent attention module (Wang et al., 2019), to construct explanations for the hierarchical structures learned in the Transformer architecture.

[0199] In an exemplary embodiment, an XTT system implementation for medical imaging can use an explainable encoder-based transformer trained to classify pneumonia observations in a chest x-ray image dataset by utilizing classification tokens on an input image token sequence. The explainable architecture x can employ additional inputs such as, but not limited to, configuration priors for each image patch. The link probabilities and respective feature contributions at each encoder layer can be utilized to form an explainable parse tree for the input chest x-ray image patch using unsupervised parsing with a top-down greedy parsing approach.

[0200] The Explainable Transformer XTT architecture can be a component within an end-to-end explainable deep learning system or suitable machine learning system that incorporates XTT as part of the pipeline. The underlying architecture that draws the input tokens to the XTT architecture can be a Convolutional Neural Network (CNN), and the Explainable Transformer can be the final component of the architecture.

[0201] Exemplary embodiments may be implemented directly as hardware circuitry, which may be implemented using (i.) flexible architectures such as FPGAs, or (ii.) more static architectures such as ASICs, analog / digital electronic components, optoelectronic components, or optical processors, or (iii.) neuromorphic architectures, spintronics, or memristors, which may be more suitable for hardware implementation of connectionist models, or (iv.) quantum computing hardware.

[0202] In an exemplary embodiment, XTT may be implemented using an FPGA, which can be useful for applications that include the benefits of high throughput and hardware acceleration while maintaining the flexibility typically associated with software implementations.

[0203] In an exemplary embodiment, XTT may be implemented using an ASIC, which may be useful for applications involving mass production of similar products. Such an implementation may have high performance and throughput, perhaps at the cost of having less flexibility.

[0204] In an exemplary embodiment, the XTT may be implemented using discrete components, which may be useful for a small size or compact XTT. Cost and robustness considerations may make this type of implementation attractive, for example, when high power systems are involved or when a simple XTT is needed for basic control or safety monitoring.

[0205] In another exemplary embodiment, XTT may be implemented using neuromorphic hardware. Neuromorphic hardware similarly couples memory and processing in a manner similar to spintronics. A hardware neuromorphic XTT implementation may be highly efficient while having most of the same advantages and flexibility as that of a general-purpose computing architecture without actually requiring a general-purpose computing architecture. A neuromorphic XTT implementation may efficiently implement explainable models such as XNN and / or INN while allowing possible non-neural components of XTT to be efficiently implemented as well, for example by utilizing the neuro-symbolic architecture of the XNN conditional network. Interpretable models such as INN allow such architectures to be made understandable by presenting models that can often be further presented to people in the form of rules and / or workflows. This surface representation form may be edited and then seamlessly reconverted back to the neural connectionist format required for neuromorphic hardware. Such an exemplary embodiment provides a practical solution for editing, adding, and deleting relevant information, thereby altering the behavior of neuromorphic devices in a controlled, reliable, and replicable manner.

[0206] In another exemplary embodiment, XTT can be implemented on neuromorphic hardware using spiking neurons rather than the more common activation function-based neurons. Spiking neurons can increase power efficiency and processing throughput quite significantly, making some practical solutions possible that would otherwise be infeasible using non-spiking neurons.

[0207] In another exemplary embodiment, XTT may be implemented using spintronics or memristors. Spintronics bridges memory and computing components, resulting in a possibly more efficient hardware implementation since there does not need to be much, if any, transfer of memory values, weights, etc. during operation. Memristors offer similar advantages, with the advantage of being non-volatile. Spintronics and / or memristor implementations, if reprogrammable, may have similar properties to FPGA implementations. Non-reprogrammable spintronics or memristors may have similar properties to ASICs. XTT may be implemented efficiently using spintronics or memristors by converting all the different layers and neurons in any connectionist-based portion of the XTT architecture, along with any state machine or tabula-based portion of the XTT architecture, into spintronics or memristor circuits. Spintronics implementations may be particularly efficient in practice due to the conflation of memory and processing available in spintronic circuits. Memristor implementations may be particularly power efficient. Because their non-volatile weight storage allows for actively turning off part or all of a memristor-based system during operation, faster data access and the elimination or reduction of having computationally expensive data transfer requirements over a data bus can greatly improve the performance of such implementations.

[0208] In another exemplary embodiment, XTT may be implemented using a quantum processing system. It is contemplated that XTT implemented on a quantum processing system will have properties that are similar to the classical XTT model with the addition of quantum-specific extensions. For example, such extensions may enable the specification of quantum annealing effects and their correct interpretation. In another example, the extensions may enable the correct interpretation of multiple qubit states, qubit basis states, mixed states, ancillary bits, and other related quantum effects due to entanglement and / or decoherence. In another example, the extension can enable the introduction of quantum logic intrinsic operators and / or hardware logic gates into the XNN, such as quantum logic gates like CNOT (Controlled-NOT), CSWAP (Controlled-Swap or Fredkin gate), XX (Ising Coupling Gate XX), YY (Ising Coupling Gate YY), ZZ (Ising Coupling Gate ZZ) gates, Pauli gates, Hadamard gates, Toffoli gates, and other related quantum logic operations that can be combined in series or in parallel. Such quantum logic gates (or quantum gates) operate on a large number of qubits and act as classical logic gates in quantum analogy. The XX, YY, ZZ designation of the Ising gates relates to the Pauli X, Y, Z matrices, which are used to describe the interaction of a particle's spin with an external electromagnetic field. Taking these examples further, such quantum-specific extensions can be implemented in various parts of the XTT architecture, for example, by having quantum-extended versions of conditions, events, triggers, and actions.It is further contemplated that such quantum-enhanced versions of XTT may utilize quantum effects, for example, to perform multiple actions, evaluate multiple conditions, or evaluate systems of large constraints, in significantly fewer processing steps than is possible with respect to classical processing implementations. In an exemplary embodiment, XTT implemented on quantum computing hardware may utilize suitable quantum algorithms, such as algorithms based on the quantum Fourier transform, amplitude amplification, quantum walks, and the like. In an exemplary XTT embodiment on a quantum device, the Bernstein-Vazirani, Simon's algorithm, or the Deutsch-Jozsa algorithm may be utilized to predict and refine the boundary conditions of the XTT architecture, potentially accelerating the encoding-decoding aspects as well. In another exemplary quantum embodiment, Shor's algorithm, quantum phase estimation algorithms, Grover's algorithm, quantum counting, quantum Hamiltonian NAND trees, or the HHL algorithm may be used to accelerate the constraints, conditions, events, and triggers portions of the XTT architecture integrated with the Behavioral Model (BM). In another exemplary embodiment relating to quantum devices, hybrid solutions such as QAOA algorithms, VQE eigensolvers, CQE eigensolvers, and quantum matrix inversion may be utilized to accelerate some of the processes involved, for example, by using Gaussian estimation processes or linear equation system solvers that utilize quantum processing to give faster results both during XTT model training and when operating normally in the environment.

[0209] It may also be possible that quantum implementations of XTT can provide significant advantages in multitask and multimodal applications. For example, a Hilbert space can be defined for each modality and / or task (with appropriate biased definitions for each task / modality), and then a multitask and multimodal Hilbert space can be constructed that encompasses all individual tasks and modalities. The resulting multitask and multimodal Hilbert space can be used to represent all interactions between tasks and modalities and used to implement both training on a subset of modalities and / or tasks along with a quantum version of crossover learning. Crossover learning can also benefit from a quantum implementation—because the combination of sparse connections in the crossover learning structure and the superior search performance characteristics of quantum computing hardware may produce faster applications. It is further contemplated that a hybrid implementation can result in a practical implementation in which classical hardware is used to implement the individual tasks or modalities of the XTT-oriented, and quantum hardware is used to implement the crossover learning structure.

[0210] In an exemplary embodiment, XTT can be incorporated into a workflow system that contains both process data and event data, reading from and writing back to XTT, respectively. It is further contemplated that the combination of XTT and workflow can be further integrated into a Robotic Process Automation (RPA) system, a Decision Support System (DSS), or a Data Lake system.

[0211] XTT actions, objective metrics, conditions, constraints, actions, triggers, and events can utilize a combination of hypothetical, inductive, and deductive logic, along with causal logic. Using inductive logic, XTT can predict future behavior based on generalization rules and knowledge that are supposedly learned completely automatically. Using deductive logic, XTT can predict behavior according to a combination of one or more conditions or constraints. Using hypothetical logic, XTT can incorporate observed scenarios into a known set of possible states within XTT, or can explain currently observed behavior in a reasonably acceptable way. Hypothetical logic can also be useful in practical implementations of XTT-based diagnostic systems and can be used to aid in the diagnosis and troubleshooting of AI systems that use behavior monitoring and prediction data. Hypothetical logic can also be useful in creating XTT-based explanatory stories that fit observed states, while inductive and deductive logic can be generally useful in creating XTT-based explanatory stories along with prediction-like stories. Hypothetical logic can be useful in situations where the environment provides partial observability.

[0212] XTT can be used for causal explanations using counterfactual language representation architectures. In an exemplary embodiment, the XTT-encoder can be trained with an additional Treated Concept objective (Feder et al., 2020) and Controlled Concept (Feder et al., 2020) to predict the presence of possible confounding concepts. The predictions of the XTT-encoder in this exemplary embodiment are then compared to the predictions of an XTT-encoder that did not use counterfactual fine-tuning training using the average of the absolute differences between the prediction probabilities assigned to the output classes.

[0213] XTT can implement causal model specific functions to handle cause-and-effect, handle association, intervention, and counterfactual causal logic, perform plausibility checks with environmental model inputs, and augment training data with causally plausible isomorphism. In the exemplary embodiment shown in FIG. 19, XTT is implemented with explainable causal model functions. An input layer 400 (which may correspond to the input component 400 of FIG. 3 if the explainable system is an XNN) feeds a suitable feature interaction component 424 (which may correspond to the fit layer component 424 of FIG. 3 if the explainable system is an XNN), which then feeds an explainable system 3400, which may be a model component of XTT itself and / or one of its explainable model components. The input layer 400 in turn feeds a causal model component 3410. The feature interaction 424 and the explainable system 3400 have a two-way link to the causal model 3410. Causal models 3410 provide inputs to the causal processing pipelines 4100-4600. The causal processing pipeline starts with a causal scenario 4100, which can optionally receive environmental contextual inputs 4510 from an environmental model 4500. Such environmental context 4510 can be used to set plausibility constraints and appropriate boundaries that do not allow environmentally impossible conditions to be considered by the AI ​​system. The causal scenario 4100 feeds a causal selection component 4200, which selects the appropriate information to be considered for further processing by the pipeline. The causal transformation component 4300 receives the filtered information from 4200, applies suitable transformations, including association-based statistical transformations and symbolic transformations, and feeds to the intervention component 4400.The intervention component 4400 applies the causal intervention using suitable logic (such as the do() operator when using Perl's causal logic) and then feeds to the counterfactuals component 4600. The counterfactuals component 4600 applies the counterfactual causal logic to the current scenario being considered. Components 4300, 4400, 4600 can iteratively repeat the pipeline process, starting from any previous part of the pipeline as necessary. Once the pipeline is ready for its processing, the results are sent and stored back to the causal model 3410 via the causal features and causal feature interaction component 4700. It is further contemplated that an optional implementation would enable the exemplary embodiment to augment the training data by a process of causally and environmentally constrained isomorphism. In such an option, the output from 4700 is fed to the isomorphism component 4800. The environment model 4500 may also optionally send relevant environmental constraints and invariance information 4520 to the isomorphism component 4800, which applies possible isomorphism transformations to the known data and then applies causal constraints and filtering based on the information received from 4700 to ensure that only newly generated or synthesized data samples that are causally and environmentally likely are sent to the augmented causal features and interactions component 4900. Such augmented data samples are added to the causal model 3410 and made accessible to the explainable system 3400 for further retraining, adaptation, few-shot learning processes, and other suitable uses.

[0214] Continuing with the exemplary embodiment, it is further contemplated that various causal heuristics may be utilized to improve the performance of causal XTT. For example, two heuristics given in (Qi et al., 2020) may be incorporated whenever historical information is utilized by causal XTT, which essentially always inserts an unobserved confounder or noise variable between the historical information and the final answer.

[0215] FIG. 23 illustrates an implementation of an exemplary causal XTT system that automatically creates a draft causal model from a natural language document. Utilizing the XTT attention model, relevant original text snippets from a text document are identified 47010, for example, by identifying an antecedent-consequent link between “paying in time” and its result “our image depends” (because in this example, the company image depends on paying in time), along with a location link between the antecedent and its result. Through a suitable masking or generalization process or any suitable self-supervised process 47020, the original causally linked snippets 47010 are then generalized to match different phrases in the document, which are then presented to the exemplary XTT system as generalization targets 47030. For example, the generalization target for the causal link from “paying in time” resulting in “our image depends” may be “ <x>Pay on time * (pay * in <x>time)" is " <y>The image depends on <y>"image depends" where <x>and <y>are two different typed masks (i.e., semantic generalizations), * The " symbol is a morphological generalization (i.e., a syntactic generalization) that depends on the input language of the document. The generalization target 47030 then applies a suitable embedding scheme, e.g., "liquidity" and " <z>Payer ( <z>The causal concepts are clustered together using phrases related to "payer" to the causal concept (i.e., causal variables) of "liquidity" 47110, phrases related to "reputation" to the causal concept of "company perception" 47130, and phrases related to "negotiation" and "deal" to the causal concept of "buying a deal" 47120. The causal structure underlying the generalization target 47030 (which is then extracted from the original text snippet 47010) is used to assign the correct direction of the causal links between the causal concepts. In the example shown in FIG. 23, liquidity 47110 is the cause of both getting a deal 47120 and company perception 47130. It is further contemplated that an appropriate causal heuristic can guide the causal XTT to insert a confounding factor between 47110 and 47120 (representing the uncertainty inherent in the dealmaking position) and another confounding factor between 47110 and 47130 (representing the uncertainty inherent in company perceptions and its links to other non-financial causes).

[0216] In an exemplary embodiment, XTT can link neuro-symbolic conditional constraints to some or all of its current and / or previous history information and / or its previous history activation rates to constrain the rate of trigger activation. Neuro-symbolic constraints can be implemented in a variety of suitable exemplary implementations, including but not limited to, in the form of symbolic rules or systems of symbolic representations, polynomial representations, conditional and unconditional probability distributions, joint probability distributions, state space and phase space transformations, integer / real / complex / quaternion / octonion transforms, Fourier transforms, Walsh functions, Haar and non-Haar wavelets, generalized L2 functions, fractal-based transforms, Hadamard transforms, type-1 and type-2 fuzzy logic, and dissimilarity analysis. Neuro-symbolic conditional constraints can similarly be implemented in the form of data structures that reference different features and variables that are accessible to explainable models and any associated taxonomies, ontologies, and causal models. Neuro-symbolic constraints can similarly be implemented in the form of knowledge graph networks.

[0217] Named reference labels can be assigned to specific components within explainable models in the XTT architecture. Named reference labels can be descriptive in nature and can also include further metadata and links to external taxonomies, ontologies, and models. Named reference labels can consist of symbolic expressions and / or formulas in the form of Conjunctive Normal System (CNF) rules Boolean logic, first-order logic, second-order logic, propositional logic, predicate logic, modal logic, probabilistic logic, multi-valued logic, fuzzy logic, intuitionistic logic, non-monotonic logic, non-reflective logic, quantum logic, paraconsistent logic, or other suitable types of logical systems for the representation of logical or similar statements that provide an explanation of the set or sequence of decisions that led to the execution of the current component in the XTT, which we refer to as a path trace. An "activation path" may be defined as a data flow path followed by an AI model, starting from a particular input and ending with a particular output prediction. A path trace is a set or sequence of decisions, nodes, transitions, or applicable descriptive structures for a particular AI model that describes the activation path. Named references may be used in safety-related constraints to ensure easier and more reliable identification by humans, especially in potentially critical and stressful situations where human attention spans may be heavily taxed or severely limited.

[0218] In another exemplary embodiment, the same invariance of named reference labels under dynamic processes can be exploited by XTT to generate stable long-term descriptions of the dynamics occurring within an AI model without the need to record knowledge or retain explanations with each execution of the dynamic process. It is further contemplated that appropriate temporal transformations can be applied to describe visual dynamics occurring within 2D or 3D scenes, whether such scenes are directly observed or synthetically generated.

[0219] In an exemplary embodiment, XTT will use suitable computation and knowledge representation structures as the basis for its constraint and prediction logic implementation. Such suitable structures may be resource description framework (RDF) trees, RDF graphs, levigraphs, or other suitable forms of graph structures. It is further contemplated that hypergraph structures or simplicial complexes may be used in practical XTT implementations.

[0220] In an exemplary embodiment, XTT can implement an audit log hunctionality. An exemplary application of such XTT audit log information is in the creation of decision logs and path traces that clearly show the flow, interactions, and behavior of XTT, as well as its conditions, events, triggers, and actions, and overall dynamics. It is contemplated that path traces can be implemented in expert systems and rule-based systems as an annotated sequence of rules that were triggered and executed. It is further contemplated that path traces can be implemented in workflow systems as an annotated sequence of workflow nodes and paths executed by a workflow engine. Path traces can be used to show the exact sequence and behavior of XTT, and can be implemented and configured to display the nearest neighbors along the path that may be of interest to the user. It is further contemplated that XTT can utilize its own audit system logs, which may be stored in a system of record, DLT, database, or some other suitable system in a tamper-resistant and traceable manner. Such audit information may be stored in an independent system that is separate from the associated AI model or shared by the same audit system used by the associated AI model.

[0221] In an exemplary embodiment, XTT may be implemented on and verified in combination with a system based on one or more of Temporal Logic of Action, Abstract Machine Notation, PetriNet, computational tree logic, and other suitable implementations that can formally express modal logic, intuitionistic logic, and / or relational semantics, including but not limited to Kripke semantics and / or Alexandrov topologies.

[0222] In an exemplary embodiment, XTT can use anchor clauses. Many nodes and edges, as well as other events, triggers, constraints, and actions in XTT can be considered important or critical for a particular application domain. Such importance can be determined by the application or context of use, or by an external third party, such as a regulatory or legal authority or an industry standard making body, that imposes certain required constraints. These types of important or critical constraints can be referred to as anchor clauses, anchor variables, anchor nodes, anchor edges, anchor events, anchor triggers, anchor constraints, and anchor actions, respectively. Such anchor components are similar to anchor clauses in an explanatory structural model (ESM). Anchor clauses can be similarly assigned to named reference labels.

[0223] In an exemplary embodiment, quantization methods may be applied during XTT processing to increase performance and possibly reduce implementation size, especially for resource-constrained hardware. It is further contemplated that some form of instability reduction technique, such as using an Ordinary Differential Equation (ODE) solver such as the solver presented in Qin et al. (2020), may be applied to partially counteract any adverse effects of quantization on XTT accuracy and reduce instability during training.

[0224] In an exemplary embodiment, XTT can utilize a combination of model-free and model-based optimization methods. Model-free methods that rely on statistical relationships and suitable measures such as covariance, correlation, and mutual information and other suitable techniques can be used to generate the necessary structure within XTT without access to a model or a suitable set of priors. Model-based methods that rely on causal relationships, symbolic knowledge, or other suitable methods further improve XTT by adding richer and more expensive optimization steps due to the broader computational representation capabilities of model-based methods.

[0225] In an exemplary embodiment, XTT can be used as the basis or part of practical data privacy preserving AI system implementation. Data privacy can be intentionally or unintentionally violated by AI systems in a number of scenarios: (i.) personal data from training datasets is unintentionally incorporated into AI models, (ii.) personal data can be re-extracted or recreated by analyzing model answers, respectively, (iii.) certain uniquely identifiable groups of personal data can end up with higher risk identification, (iv.) model inversion and membership inference techniques that can relate model data by unique keys or signatures, (v.) other information sources such as public data sources that can be combined with private information can recreate or otherwise identify private information. The main data privacy preserving solutions for AI can be categorized into four categories: (i.) differential privacy, (ii.) private multi-party computation, (iii.) federated learning, (iv.) homomorphic encryption. An exemplary embodiment of an XTT-based system can enable practical implementation under all four categories.

[0226] In an exemplary privacy-preserving solution (i.), differential privacy, the introduction of noise into the training data or some other suitable obfuscation means may be used in the XTT architecture to generate a controllable amount of privacy through a noise factor or ratio. The noise level may be a variable that may be user-supplied or edited, and the noise level may be implemented as a constraint and / or objective. In a privacy-preserving solution (ii.), secure multi-party computation (SMPC) may be used to obtain the correct answer while hiding partial information about the data, and answers may be computed simultaneously using data from one or multiple sources. Exemplary embodiments of XTT and explainable models may extend the SMPC protocol to apply to explanation generation in addition to answer output. It is further contemplated that exemplary embodiments of XTT may be formally analyzed and tested for security and trust building without exposing any private information. Secure enclaves may similarly be used to decrypt data within a protected space within the hardware processor, limiting the possibility that other parts of the system may be able to access such data in the clear. An end-to-end hardware implementation of the combination of XTT and secure enclaves can be rather resilient against most forms of data attacks. In a privacy-preserving solution (iii.), federated learning, XTT can be distributed across various decentralized devices that only hold local data samples. The local data samples are not shared with other devices and thus limit, but do not completely eliminate, the privacy risks involved, and can be particularly suited to IoT or edge computing applications where messaging options are limited or constrained by the network topology, such as in mesh networks.In privacy-preserving solutions (iv.), homomorphic encryption or homomorphic computing may be used to enable computations on encrypted data without decrypting the data and also without optionally using the encrypted explainable model. In an exemplary embodiment of XTT using homomorphically encrypted data and homomorphically encrypted XNNs utilizing the CKKS protocol, a secret key and a public key are generated. The public key is used for encryption and may be shared, while the private key is used for decryption and must remain secret, e.g., in a secure hardware enclave or similar implementation solution.

[0227] The XTT system can utilize the generated structured explanations based on what-if, what-if-not, counterfactual, but-for, and conditional scenarios to generate scenario-based explanations corresponding to the explained strategies and outcomes of applying such scenarios to the XTT system inputs, with one or more hypothetical target outcomes being selected as part of the scenario modeling. Such scenario-based explanations are also known as constructive or comparative explanations depending on whether the target scenario outcomes are constructive or comparative in nature, respectively. In such scenario-based explanation generation applications, it is contemplated that the XTT system can associate constraints to the scenarios being examined, associate objective costs to the difficulty / achievability / severity of features, and associate actions as output targets to domain-specific and scenario-specific actions. In further exemplary embodiments, the XTT system can be implemented with XRL agents to provide further information regarding policies, costs, and rewards, and / or provide further training examples for the XRL agents as well. In an exemplary embodiment, an XAI model such as XTT can be used to predict the outcome of a loan application. A loan application may be accepted or rejected for a variety of reasons, such as total income, current occupation, age, total net worth, and other factors. A system user may want suggestions on how to change the outcome of a loan application. For example, in the case of a rejected loan application, a system user may want to know what combination of items in the inputs needs to be changed to change the application's status from rejected to accepted. It may be contemplated that only a subset of all input variables, features, and feature interactions may need to be changed. One or more costs may be associated with each variable, corresponding to some suitable metric for each type of cost.For example, bias protected classes such as gender or race may have very high costs associated or marked as impossible to reach unconditionally (i.e., wrong outcome states). In another example, high costs may be associated with policy actions that are irreversible. Other variables such as current income may have lower costs than the total net wealth variable. The cost of an action may be linear or nonlinear and may include dependent or independent variables that affect the total cost. Missing data may also have special costs associated with it, allowing the XTT system to deal with missing data appropriately, with or without corrective action. Explanatory data such as feature importances or coefficients provide an accurate measure of expected rewards or costs for a given action. Exogenous and endogenous variables and causal models may be used to estimate the total cost of an action, including any specific associations, interventions, or counterfactual rules that need to be applied, including in the case of data with partially missing values. The XTT system can be trained to learn suggested actions for a given user with a particular context, i.e., to suggest possible variable changes that result in a change in outcome while minimizing the total cost of the action. It can also be contemplated that the suggested actions that lead to a desired goal can include some form of sequence. It is further contemplated that nearest neighbor methods can be utilized in conjunction with such scenario-based XTT system explanations to provide (suitably anonymized) actual examples of applications that had the desired scenario outcomes (exemplars) or hypothetical averages (prototypes) of such applications. Such nearest neighbor methods can utilize the partition hierarchy in the explainable model to generate examples from the same partition or from nearby partitions or from partitions far away from the current partition.It is further contemplated that possible changes in variables may be ranked and prioritized by any suitable system and presented to a user or automated process in the form of an Identify-Assess-Recommend-Resolve (IAR) framework. In an exemplary embodiment, XTT may be implemented in conjunction with an XRL Agent to generate realistic environmental simulations and / or provide learning data sample experiences for the XRL Agent. It is further contemplated that such exemplary embodiments may be utilized in the creation of virtual reality simulations, augmented reality simulations, virtual collaborative spaces, and metaverses.

[0228] In an exemplary embodiment of an XTT system having an automated XAI system, optimization methods may similarly be used to generate scenario-based explanations, such as Multiple Objective Optimization (MOO), Pareto Front Method, Particle Swarm Optimization (PSO), Genetic Algorithm (GA), Bayesian Optimization, Evolutionary Strategies, Gradient Descent Techniques, and Monte Carlo Simulation (MCS). It is further contemplated that the encoding-decoding information for the XAED and / or XGAN system is fully or partially linked to the contribution values ​​and other data stored in the Explanation Structure Model (ESM). It is further contemplated that such an exemplary automated XAI system may utilize a formal language or some other suitable means to represent a computational graph for the state knowledge stored in the XTT architecture, along with more dynamic changes, such as feedforward networks in the XTT's encoder and / or decoder. This allows an automated XAI system to tailor the XTT system performance to one or more specific application domains or tasks, and provides a practical solution to the incorporation of meta-learning systems within XTT, a common, yet non-trivial use for XTT in reinforcement learning systems. Such meta-learning systems can also provide a practical solution to the initial incorporation of human process knowledge that is then refined during automated XAI model discovery.

[0229] An XTT, XTT-encoder, or XTT-decoder can process multiple tasks and multiple modalities while sharing the same parameters. In an exemplary embodiment of a multitask and multimodal XTT system, the explainable model includes an XTT-encoder or XTT-decoder that processes audio, images, and video using the same parameters. The input of the XTT-encoder or XTT-decoder is:

number

number

[0230] In a further exemplary embodiment, an XTT utilizing an explainable model such as an XNN, an INN, or an XTT can have multiple inputs corresponding to one or more tasks associated with one or more input features, and multiple outputs corresponding to the outputs for those tasks. The semantic notion of a task in such a context is only implemented by specifying some input features as task identifiers and task metadata input features, without requiring any modification in the XTT structure itself. The hierarchical partition structure in the explainable model forming part of the XTT, which can be a tree, graph, hypergraph, or simplicial complex structure, allows crossover between different knowledge learned for different tasks to occur efficiently. Such crossover occurs in the XTT prediction network depending on the feature interactions in the hierarchical partition structure. The XTT conditional network can be used to select, adjust, and multiplex the correct path trace through the partition structure linking the task, its associated inputs, and associated outputs. It is contemplated that a denser or sparser XTT can be utilized to implement parts of the partition structure in an energy-efficient manner. It is further contemplated that distributed XTT or DEA may be utilized to implement portions of the partition structure in a practical manner.

[0231] XTT may be implemented using sparse explainable models or DEA using sparse explainable models in some or all of its components, including the feedforward component, the query-value-key component, the attention model, and the output-related component. It is contemplated that such sparse models can result in significant power reduction and performance acceleration.

[0232] FIG. 20 shows an exemplary explainable system using multi-task (and multi-modal) crossover learning, which shows a practical solution for multi-task and multi-modal learning for explainable AI systems. Our novel formulation for crossover learning for explainable AI systems utilizes a crossover learning structure 45300, which is shown in more detail in FIG. 21. Continuing with FIG. 20, a crossover switch controller 45100 is used to determine which explainable subsystem 45200 gets 45150 task information and inputs sent to subsystem 45200, thus reducing the number of subsystems that need to run in parallel. The explainable subsystems can be implemented as DEA or any other suitable homogeneous or heterogeneous distributed explainable AI system. It is contemplated that a grey-box version of the multi-task and multi-modal system can be implemented by having one or more subsystems comprised of black boxes. Continuing with the exemplary embodiment, each explainable subsystem has a link to a crossover learning structure 45300, which in turn consists of various crossover subsystems 45350. Not all subsystems necessarily need to be connected to all other subsystems, and the crossover learning structure is designed to be implemented as a sparsely connected network (although dense fully connected networks may still be possible). The explainable outputs 45400 are generated in parallel, possibly ranked and prioritized until they reach an output selector 45500, which takes the interim explainable outputs 45400 together with control information 45110 from the crossover switch controller 45100 to finally select the system output 45600 to be output by the system.

[0233] FIG. 21 shows detailed components of an exemplary crossover structure implementation (i.e., detailed portion of crossover subsystem 45350 shown in FIG. 20) linking two explainable subsystems shown in FIG. 21 by representative nodes 45210 employed from each of the two subsystems, in this case, subsystem 1 node 1 (1.1) and subsystem 2 node 1 (2.1). In option (a.), a unidirectional link or directed graph structure is utilized to link the two nodes by having a crossover node 45353 linking subsystem 1 node 1 with subsystem 2 node 1 and another crossover node 45354 linking subsystem 2 node 1 with subsystem 1 node 1. An optional crossover noise node may be inserted in-line between each crossover connection node, i.e., a noise node 45351 may be inserted between subsystem 1 node 1 and crossover node 45353, and similarly, a noise node 45351 may be inserted between subsystem 2 node 1 and crossover node 45354. As another option, the crossover nodes themselves can transmit information to each other, as shown by exemplary links 45356 and 45357. In option (b.), a two-way link or undirected graph structure is utilized to link two nodes, in which case each crossover node becomes a two-way crossover node, as shown by component 45355. Noise node and inter-crossover node connections can be similarly added in option (b.) in a manner similar to that in option (a.).

[0234] The exemplary embodiment of crossover learning shown in Figures 20 and 21 can enable XTT to provide a practical solution for multi-task and multi-modal learning. Various distributed techniques can be used to optimize performance, including optimizing task order during training, different training schedules and optimizations for different modalities, and sharing the XTT-encoder across multiple tasks and modalities to optimize the parameter space.

[0235] In an exemplary embodiment, XTT may be integrated as a component after the final flattening layer of an exemplary CNN-based machine learning model. As shown in FIG. 16, the flattening layer may be used as an input to a transformer-encoder based architecture. With reference to FIG. 16 illustrating an exemplary medical application, the XTT-encoder may be fine-tuned by connecting the output of the final encoder layer to a classification layer to predict whether an X-ray image constitutes a normal unremarkable case or shows signs of bacterial or viral pneumonia. It is further contemplated that further refinements and explanations may be achieved by connecting additional explainable machine learning models to further analyze the explainable output. In a further exemplary embodiment, XTT or XNN may be connected to the X-ray prediction output of the system shown in FIG. 16 to determine whether the viral pneumonia was caused by some specific type of viral disease such as influenza virus, or coronavirus like COVID-19, adenovirus, etc. It is further contemplated that actionable explanations with risk assessment and recommended actions may be further generated in the resulting explanation and interpretation pipeline.

[0236] The explainable architecture in the XTT architecture can generate explanations for classification outputs. The explanations can be interpreted using base interpretations, explanatory interpretations, and meta-explanatory interpretations. A base interpretation can refer to a predicted output o that can be understood by a subcomponent. An explanatory interpretation can be a 2-tuple vector<o、w> The meta-explanatory interpretation is denoted by and refers to a predicted output o combined with a model explanation w for such a predicted value that can be understood by its subcomponents. A model explanation can consist of coefficients θ of an explainable architecture x (where x ∈ {XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture that can be utilized to explain the feature importance of input features for a given observation. A meta-explanatory interpretation is a 3-tuple vector<o、w、j> and includes the predicted output o, the model explanation w, and the model explanation justification j. The model justification j provides further information about the assumptions, processes, and decisions taken by the explanation system considered to generate the model explanation.

[0237] The fully connected layers of the black-box CNN may be replaced with the XTT 3212, as shown in FIG. 16, to construct the white-box CNN-XTT. The steps between 3202 and 3210 may remain the same for the black-box CNN and CNN-XTT architectures. First, the CNN layer may receive an input 3202. The input 3202 is then convoluted 3206, and different elements of the input are extracted and classified. The convoluted layer then pools or groups similar features based on the features. The pooled features may be convoluted and pooled again. In this exemplary embodiment, the CNN layer performs the convolution operation 3206 twice and the pooling operation 3208 twice, however, it may be contemplated that further iterations of these layers may be performed depending on the application. After the CNN layer performs the convolution and pools the convolved features, the features may be flattened 3210 for input to the next system. FIG. 16 illustrates the CNN-XTT architecture during a forward training pass 3200, which implements the XTT architecture to produce output 3214 and which can further generate explanations. The actual configuration of such steps can depend on the configuration of the particular CNN network. CNNs similar to the Tabular dataset utilize fully connected layers to learn patterns in the data. Other CNN layers are merely feature transformation steps that occur prior to the classification task. Similar concepts apply with respect to CNN-XTT, where such CNN transformations can be applied prior to the generic XTT.

[0238] Pooling layers are standard layers found in CNN networks. Pooling layers are typically placed after convolutional layers, which provide a summary of features in an image or some form of multidimensional data. A major problem with convolutional layers is that they are sensitive to the location of the input features. This means that small variations in the input data will result in different feature maps. One approach is to downsample the data, and the same data at a lower resolution is used for further processing. One method for downsampling in CNN networks is called pooling. Pooling utilizes an operator to reduce the size of the feature map. Two common pooling methods are max and average pooling. For example, max pooling can be used to calculate the maximum or largest value from the feature map.

[0239] CNN-XTT can similarly take full advantage of masked learning methods to better generalize their learning models, including the self-supervised models discussed above.

[0240] The reverse indexing mechanism (backmap) is something inherent to the CNN-XTT architecture, made possible by the white-box nature of CNN-XTT: if all layers (including the CNN layer) are interpretable, it is possible to apply reverse indexing using backmap to backtrack from the output all the way to the original input.

[0241] Referring now to FIG. 17, FIG. 17 can show an exemplary reverse indexing mechanism. The exemplary reverse indexing mechanism 3201 can be used to generate explanations in CNN-XTT. The reverse indexing process shown in FIG. 17 can perform steps that are the same as or similar to those of the forward pass indexing process shown in FIG. 16. However, in such a case, the operations can instead be performed backwards, starting with the output and ending with the input features. Furthermore, the pooling function can upsample the data, as opposed to the forward pass pooling, which downsamples the data. Similarly, the convolutional layers are calculated as deconvolution operations.

[0242] The output from the predictive network can serve as the basis for generating explanations in CNN-XTT. The predictive network can weight the input features by using an equation in which each feature is weighted by a coefficient. Each coefficient indicates the importance of the final convolved feature. To create a meaningful explanation, the convolved features, along with their importance, can be mapped back to the original input. Because the central layer of CNN-XTT is a sparsely connected layer (white box) rather than the fully connected dense layer (black box) of XNN, it is possible to apply a reverse indexing mechanism (backmap) that maps the output of the convolved features back to the original input. CNN-XTT is unique in its implementation of the reverse indexing mechanism that allows for fast output of explanations along with fast association of explanations with the answer and the exact activation pathways followed by CNN-XTT during processing.

[0243] The creation of the backmap may generally be implemented as a static process that is performed once the architecture of CNN-XTT is determined. If the CNN-XTT architecture itself (i.e., connections, layers, etc.) is updated, the backmap will need to be updated once for such architectural update. Possible embodiments may include dynamic creation and updating of the backmap when the architecture of CNN-XTT is dynamically updated. The once-per-architectural-update nature of the backmap creation allows for efficient implementation and room for further optimization based on the backmap itself. For example, the backmap may be used to efficiently identify segments and regions of interest occurring in the input multidimensional data, efficiently identify symbols and concepts, and similarly quickly optimize the process of creating summaries and lower resolution subsamples of the input multidimensional data.

[0244] The backmap can be treated as an external process or as a neural network that implements the inverse function. The neural network can be embedded within the CNN-XTT or maintained as a separate network specifically designed to implement the inverse indexing mechanism. The process can start by identifying feature importances determined by the coefficients of the XTT network and work backwards step by step. The output weights can be mapped back to the original input. Such steps can be applied sequentially depending on the type of layers found in the original CNN-XTT network. The mapping can be applied until the original input is reached, which will allow the conceptual feature importances to be mapped back to the original input.

[0245] CNN-XTT can use the backmap to visualize the activation pathways within CNN-XTT for a given query, and can similarly incorporate it as part of the justification. Backmaps can similarly be used to enhance visual question answering (Malinowski et al., 2016) by adding explanatory elements to the multimodal representation of the fused image, query text, and answer pair.

[0246] The reverse indexing mechanism can be useful to generate explanations in the form of heatmaps or in some form that takes into account the actual convolution filters when generating the explanations. The convolved features can be useful to detect strokes, edges, or patterns, which can then be visual, textural, audio, etc., and can accelerate the explanation generation process.

[0247] The CNN-XTT architecture can integrate kernel labeling methods, which associate human-readable labels with data formats such as images or 3D scans that are not textural in nature. Kernel labeling methods label data from any data format that allows a pattern recognition kernel to be defined with suitable keypoint description and activation maximization functions into incremental refinements of primitives, patterns, symbols, and concepts. Kernel primitives can be defined as matrices of a particular shape that are searched for in each filter that is analyzed. The kernel labeling method can first analyze the first convolution layer using human-defined kernel primitives, and proceed to slide through the convolution and pooling layers in the CNN architecture. It is further contemplated that different kernel types, such as linear, nonlinear polynomial, exponential, radial-basis-function, sigmoid, and other suitable kernels, can be utilized in conjunction with XTT. It is further contemplated that approximation kernels may be utilized in conjunction with XTT to reduce the amount of computation and / or resources required, for example, through quadrature methods (Nystrom), singular-value decomposition (SVD), random Fourier features, random binning features, and other suitable methods. It is further contemplated that temporally optimized variants of CNNs, such as Temporal Convolutional Networks (TCNs), may be used in conjunction with XTT.

[0248] XTT can also utilize various approximation techniques to improve prediction and / or training performance, including, but not limited to, approximating or completely eliminating dot products, sparse local attention models, using adaptive attention windows, using multi-dimensional attention matrix approximations (possibly using an explainable model for the approximation), intelligent weight sharing, and intelligent parameterization.

[0249] XTT can have a variety of applications, and input data can be in a variety of formats suitable for some form of token-based encoding, including, but not limited to, human-readable natural language, graphical or visual image data, audio, speech, tactile, video, time series, multispectral data, hierarchically ordered multimedia content, 3D data, and other suitable formats. It is contemplated that combinations of formats may be presented as well and processed by the XTT system using any suitable implementation method, such as multiplexing or other forms of input combining.

[0250] It is contemplated that the XTT output may present the generated explanations and interpretations using a variety of presentation formats and layouts, including, but not limited to, human-readable natural language, graphical or visual formats, audio, speech, tactile, video, time series, multispectral data, hierarchically ordered multimedia content, 3D data, and other suitable formats. Combinations of various formats may likewise be used by the XTT system in its output, using any suitable implementation method, such as multiplexing or other forms of output combination.

[0251] In an exemplary embodiment, the XTT system may be used to process sequence data. Sequence data may include a number of data points including feature data in various sequential formats, including but not limited to 2D data, 3D data, multi-dimensional data arrays, transactional data, time series, digitized samples, sensor data, image data, hyperspectral data, natural language text, video data, audio data, haptic data, LIDAR data, RADAR data, SONAR data, etc. Data points may have one or more associated labels that may indicate an output value or classification for a particular data point or a continuous or non-contiguous interval of data points. Data point sequences may originate from internal and / or external processes that may output a combination of synthetic data points, perturbed data, sampled data, or transformed data.

[0252] XTT can also allow an arbitrary length data point input sequence to be mapped to an arbitrary length data point output sequence in a multimodal and multitask manner. This is achieved by XTT by utilizing an explainable model (as opposed to an unexplainable transformer that uses an unexplainable latent space) for the mapping computation, which is done in a manner similar to Map-Reduce systems. The XTT crossover learning structure allows for multitask generalization that is domain and application independent.

[0253] In an exemplary embodiment, the XTT system may be used to analyze human behavior, including gait analysis, movement prediction, and emotional state prediction. Optionally, such an XTT system may be used to predict and analyze sports footage and athletic performance. As another option, such an XTT system may be used to detect and predict medical conditions, especially when combined with additional sensor data such as ECG heart rate data, blood pressure, oxygenation levels, etc.

[0254] In an exemplary embodiment, the XTT system utilizes a combination of causal models and multi-modal data to analyze financial stock trading patterns and predict market behavior, and can then perform automated actions such as buying, selling, or placing short or long positions in specific stocks.

[0255] In an exemplary embodiment, the XTT system is for Industry 4.0 applications, such as intelligent manufacturing, IoT-enabled applications, intelligent energy-efficient and environmentally friendly smart utility distribution and consumption networks, and related applications, incorporating behavioral models (BMs). Four major different types of functionality can be provided by such exemplary embodiments: (i.), passive prediction and analysis (ii.), reactive prediction and analysis (iii.), passive control and intervention, and (iv.) reactive control and intervention. In manufacturing applications, exemplary applications would be to (i.), analyze audio from machinery, analyze operating conditions, and predict failure modes in advance, (ii.), utilize boundary proximity conditions that can output warnings to end users, (iii.), utilize feedback thresholds to trigger feedback-based actions and interventions, and (iv.) utilize valid estimation and prediction of events to control processes in real time, including safety considerations.

[0256] In an exemplary embodiment, the XTT model is used to translate sentences from a source language to a target language. The input of the explainable encoder component is S e = {t1, t2, ..., t n } for a source language sentence S e The encoder component processes the input sequence to generate memory keys and values ​​for the decoder component. The memory keys and values ​​are passed to each explainable decoder layer in the encoder-decoder attention component. The explainable decoder component then calculates the sum of the S using linear layers and a softmax function or similar mechanism. e The predicted output of the softmax layer is used as an input to the decoder component for further prediction. In a further exemplary embodiment, an explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture is used instead of the linear layer to predict the output of the explainable decoder component.

[0257] In an exemplary embodiment, an n-layer XTT is used to translate a document from a source language to another target language. The input embeddings are combined with the positional embeddings and used as input to the explainable transformer XTT. The explainable architecture in the encoding layer inducts a black-box feedforward neural network component to predict the output of the black-box feedforward neural network 230 while generating global and local explanations. The explanations can be filtered using filters from the explanation framework to create interpretations used for the encoder-decoder multi-head attention component of the decoder. The XTT encoder-decoder can similarly use a dynamic attention span to improve adaptability to different situations, where the span window length is a parameter included in the output explanation and / or justification. The decoder's explanation architecture inducts a feedforward neural network component 270 to generate explanations for prediction of a linear layer of the target language vocabulary for translation, as shown in FIG. 6. The XTT explanations and / or XTT outputs can be filtered using filters from the EIGS system or a suitable explanation framework to create interpretations that are then presented to a suitable interpreter. An exemplary implementation of such a practical application pipeline is presented in Figure 14, where XTT may be used as a practical embodiment of components 904, 908, 912, and optionally 911, in an exemplary explanation and interpretation pipeline. It is further contemplated that XTT may be utilized to perform presentation-oriented transformations and enhancements of explanation and / or interpretation outputs, including, but not limited to, (i.) generation of human-readable explanations, (ii.) dialogue and conversation output, (iii.) question answering, (iv.) question and / or interpretation classification, (v.) frequently asked question (FAQ) type output, (vi.) entailment of answers and outputs relative to previously known context, (vii.) selection of output from multiple ranked selections.It is further noted that XTT offers significant advantages over non-Transformer architectures such as ELMo, both in terms of parallelization and overall throughput, and in terms of explainability.

[0258] In an exemplary embodiment, the n-layer XTT is used to detect whether the text content of an email is spam or not spam. The transformer-encoder architecture consists of a downstream classifier 1008 connected to the final encoder layer to predict whether an email is spam or not spam, as shown in FIG. 10(a). The classifier can be an explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture to generate explanations for a particular domain downstream task. In a further exemplary embodiment, the transformer-decoder architecture is used for a text classification task, as shown in FIG. 10(b). The downstream task classifier 1009 can be an explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture to generate explanations for a domain downstream task. The explainable architecture is inducted from the feedforward neural network component 1900. The generated global and local explanations are used in conjunction with the output of the XTT for downstream spam detection tasks using a machine learning classifier such as a feed-forward neural network to classify input text tokens as spam or non-spam. The downstream task classifier can similarly be inducted using an explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture to generate explanations for specific domain downstream tasks.

[0259] Finite State Transducers (FSTs) were one of the earliest models successfully applied to text processing, and XTT is a direct evolution and generalization of those earlier FSTs. FSTs, and by extension XTTs, have been very successful in Human Language Technology (HLT) applications such as morphological analysis and finite state parsing. The early motivation for FSTs for processing natural language was the realization that writing grammars for natural language are simply too difficult and laborious to implement. One of the main appeals of FSTs is their ability to be converted into rules and compiled back in FST form, while their expressiveness of their Non-Deterministic Finite Automaton (NDFA) origin allows them to handle complex input-output transformations. The XTT attention model, implemented using an explainable model, allows such rule extraction to be done with successive levels of detail approximation. A complete conversion of XTT's entire attention model to the derived rules may be possible, but without some form of compression, generalization, or approximation, the number of rules to be extracted may be so large that it may not result in a practical system. An advantage of XTT is that symbolic methods such as typed masking and grammar tree analysis, as well as information theory-based compression and analysis methods, can be used to generalize and compress such an initial set of rules into a more manageable format. The explainable models in XTT can be used to convert XTT attention models into a symbolic rule-based format by analyzing their feature interaction layers, which can then be converted into recursive relations (even if there are elements of recursion or feedback or previously learned representation references), and then finally into a human- and machine-readable format.

[0260] In another exemplary embodiment, XTT is used to process legal documents and correctly determine references and text sections applicable to different agencies and entities that relate to and are references in the legal document. The entity relationships, attributes, and constraints extracted by the XTT attention model, along with the textual linguistic information itself processed by XTT, are then utilized to double-check the document against a set of predefined rules stored, for example, in the XAI model and / or XNN and / or another XTT, for various legal issues including red flags, liability issues, omissions of clauses, highly aggressive requests by opposing parties or entities, priority issues, precedents, legend legal references, clause risk classification, clause impact classification, and a combination of decisions regarding the acceptability of certain clauses. It is further contemplated that XTT may be utilized to rank possible edits, including inclusions, deletions, and / or mitigation measures, to improve the quality of legal documents. It is further contemplated that XTT may optionally take into account appropriate context depending on which party to side with to ensure that the interests of such parties are represented according to different user-specified levels of comprehensiveness and aggressiveness.

[0261] 22 illustrates an exemplary XTT system implementation used for legal document processing using Explanation Structure Model (ESM) notation. In the example, the XTT attention model 46100 is used to identify possible liability-related clauses 46190 from a legal document corpus 46110 using a liability clause identification component 46150. XTT can similarly access historical case law information 46200 and a structured system of rules 46210 ("Lex" in the example) that represents the legal system for the relevant jurisdiction obtained by an external provider. The historical information 46200, legal system rules 46210, and liability-related clauses 46190 are used to assess the severity and consequences of each clause 46290 using a severity assessment component 46250. XTT similarly uses the user-supplied scenario information 46300 along with the extracted contracting party information 46310 in parallel to processing the severity assessment liability clauses 46290 using a liability strategy recommender component 46350 that recommends different liability minimization strategies 46400 depending on which contracting party is employed. Finally, XTT uses the ranking component from the explainable model to recommend precise edits 46450 to the legal document 46110 that can then be selected automatically or using a human-in-the-loop type of approval.

[0262] In an exemplary embodiment, an XTT system processing a document can utilize multimodal document layout and document understanding techniques to correctly segment, process, and link the text, images, and other multimodal content within the document.

[0263] XTT can similarly be used to track state changes within the dynamic systems it models, and to generate explanations that describe the dynamic processes identified within such systems.

[0264] In an exemplary embodiment, the XTT system can utilize Fast Weights in its explainable model component. Such an XTT system can be denoted as Fast XTT or F-XTT. Fast weights can provide a significant boost to the runtime performance of the resulting XTT system while allowing the resulting XTT system to quickly adapt to changing input dataset distributions and new previously unseen training data samples that were not part of the original training dataset. F-XTT may also be more efficient at processing sequence input data and other similar multi-dimensional input data. As understood in the art, fast weights can provide a temporal repository of certain past events through weight variables that can change more quickly than standard model weights, essentially providing a "short-term memory" for the system.

[0265] In another exemplary embodiment, XTT is used to control the flow of an interactive explanation and / or interpretation process, which is commonly used in applications such as dialogue processing, chatbots, call center management systems, case management systems, customer support systems, client relationship management (CRM) systems, conversational systems, question and answering systems, and similar applications where an element of interactivity with a user and / or another system is required. The XTT attention model is used in conjunction with its answer and explanation generation capabilities to provide a practical solution for efficient management of interaction flow and generation of related interactive output. It is further contemplated that an explainable transformer-decoder type solution is used to provide a practical solution in such an exemplary embodiment without requiring a full XTT architecture. The XTT-encoder can be used for related tasks such as named entity recognition (NER) and anaphora resolution. The XTT-encoder can be used to encode characters and words in an end-to-end deep learning architecture to detect the beginning and end of an entity. In an exemplary embodiment, an XTT-encoder is used to encode character-level information that is used as input to another XTT-encoder to generate word-level contextual embeddings. The word-level XTT-encoder can handle the combination of pre-trained word embeddings and extracted character features from the character-level XTT-encoder. A conditional random field layer (Yan et al., 2019) can be used to maximize P(y|s), where s is the sequence and y is the label sequence.Character-level and word-level XTT-encoders can use relative position encoding and unscaled attention (Yan et al., 2019) to optimize performance on NER tasks.

[0266] The XTT-encoder can be used to identify relationships between two sentences in downstream tasks, such as question-answers. The XTT-encoder can be pre-trained using next sentence prediction (NSP) (Devlin et al., 2018) to learn associations between two sentences using a monolingual corpus. The input tokens of the XTT-encoder for the question-answer downstream task are {[CLS], q1, q2, ..., q n , [SEP], a1, a2, ..., a m }, where [CLS] is a symbol token used before every input, [SEP] is a separator token between the question and the answer, and q is a set of q={q1, q2, ..., q n }, where a is a={a1, a2, ..., a m }. [CLS] Tokens are used as inputs to an explainable model in an end-to-end deep learning network for classification to determine whether an answer corresponds to the input question.

[0267] The XTT-decoder can be used to detect entities in input sequences without any knowledge of the entities and perform entity tracking by training the XTT-decoder on a target entity in an external process. j Add a token to the end of t j We use the contextualized token representation of , combine it with the entity representation and use it as input to an explainable model, which can yield class prediction probabilities. The XTT-decoder can be guided towards the target entity by reconstructing the input sequence. The input sequence can be reconstructed using a separator token [SEP] between the sentence and the target entity. [CLS] tokens can be added to the beginning of the input sequence or to the end of the input sequence. [CLS] tokens can be used as input to an explainable model x (where x ∈ {XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture to obtain class prediction probabilities.

[0268] It is further contemplated that XTT may be linked to knowledge graphs, RDF databases, key-value pair tables, or other suitable databases that provide facts in a symbolic way that can be seamlessly merged into the XTT explainable model. Such integration would similarly enable a combination of deductive, inductive, hypothetical, and causal logic to be utilized in XTT's feed-forward prediction path, combining symbolically constructed knowledge with a connectionist approach.

[0269] In an exemplary embodiment, the explainable system can be used in an explainable chatbot system to explain the decisions of a predictive machine learning model. The explainable chatbot system allows a user to interact with an explainable model x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture to request an explanation. The explainable chatbot system can consist of an explainable architecture x (where x∈{XAI, XNN, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture to classify the query intent and extract the requested entities. A dialogue management component can be utilized to manage state and context between the user and the explainable chatbot system. An XTT model can be trained to project the model explanation into user-readable text and present it to the user. The explainable chat system can enable interaction with the administrator of the system for identification of bias in human knowledge injection (HKI) or white-box models. The explanations generated for the user can include a base interpretation of the model, an explanatory interpretation of the model, and a meta-explanation interpretation of the model. Explanation Interpretation is a 2-tuple vector<o、w> A meta-explanation is a 3-tuple vector that can be used to explain the feature importance of input features.<o、w、j> and includes the predicted output o, the model explanation w, and the model explanation justification j. The model justification j provides further information about the assumptions, processes, and decisions taken by the explanation system that were taken into account to generate the model explanation.

[0270] XTT with an explainable model that has a hierarchical partition structure in the form of a graph or hypergraph where the attention model is generalized to Graph-XTT (GXTT) based on Graph-Attention (or Higher-Graph-Attention), and that can also handle graph-to-sequence, sequence-to-graph, and graph-to-graph transformation modes in addition to the standard XTT mode of sequence-to-sequence transformation. GXTT can utilize different graph, hypergraph, or simplicial complex representation formats, including directed, undirected graphs, topological networks, Levi graphs, or other suitable representation formats.

[0271] In another exemplary embodiment, XTT is used in reinforcement learning type applications or suitable applications requiring feedback control type mechanisms, such as in robotic systems or typical control engineering applications. XTT may be used to provide practical solutions for the selection of actions from an action policy and / or to implement an appropriate reward function as part of a reinforcement learning agent. It is further contemplated that XTT may be used in the training and learning of the action policy and / or reward function itself. It is further contemplated that XTT may be utilized to learn and discover possible actions that may be performed by a reinforcement learning agent utilizing the XTT.

[0272] It is further contemplated that XTT will be used in conjunction with XRL and / or as part of an XRL system. XRL agents may be used to generate explanations that include intervening or counterfactual elements that use XTT to provide necessary context.

[0273] The explainable transformer architecture can incorporate other machine learning systems to create an end-to-end explainable DL system that incorporates the XTT architecture as part of the pipeline. In an exemplary embodiment, the explainable autoencoder XAED is used to compress the language embeddings into a low-dimensional space. The compressed dimensional space, along with the generated explanation and partition information and positional encoding, is used as input to the explainable transformer architecture.

[0274] The Explainable Transformer XTT architecture can be a component in an end-to-end explainable system within an autonomous system. The autonomous system can be designed using a Behavioral Model Hierarchy (BMH). The BMH is a hierarchy of multiple behavioral models. BM where BMH BM = {BM1, ..., BM n }.

[0275] In an exemplary embodiment, the autonomous system implements the conditional constraint BMH for the XTT or explainable transformer-encoder architecture. c and the architecture is a Transformer-Encoder based architecture with n encoder layers, and the output of the final encoder layer is fed to an output SoftMax to predict the risk of collision with another vehicle. The conditional constraint c may be placed on the internal coefficients of the explainable architecture 3400x, where x∈{XAI, XNN, INN, XRL, XSN, XMN}, or a logically equivalent or similar architecture. The condition c may be based on other conditions in a hierarchical form. The condition may be in Conjunctive Normal Form (CNF) or Disjunctive Normal Form (DNF) or a suitable form of first-order logic to fit formal verification problem solvers such as Satisfiability module theories (SMT) and conflict-driven clause learning (CDCL) Satisfiability (SAT) solvers.

[0276] The behavior model BM may include an explainable system 3470, an event 3420, a trigger 3430, a terminal action 3440, a feedback action 3450, and a system component 3460. One or more conditions BM c One or more trigger BM t One or more events to activate BM 3430 e 3420. Triggers can be connected between multiple BMs in the BMH. The triggers can provide feedback actions 3450 to the explainable transformer XTT architecture or feedback to system components 3460, or can activate terminal actions 3440 in the behavior model. The behavior model hierarchy BMH of an autonomous system can be connected to multiple machine learning models BMH x (Here, BMH x = {x1, ..., x n }), where XTT is the BMH x In this case, the condition of the behavioral model hierarchy BMH is one of the models in the BMH t These may be fused together as conditional requirements for a particular trigger within.

[0277] The explainable transformer XTT architecture can be an explainable architecture component 3400 in an explainable system 3470 in a behavioral model framework, as shown in FIG. 18. The explainable system 3470 can include other components, such as a causal architecture 3410. The behavioral model BM can consist of an explainable architecture x (where x∈{XAI, XNN, XTT, INN, XRL, XSN, XMN}) or a logically equivalent or similar architecture. Input constraints can be included in the formal verification of the explainable architecture of the behavioral model BM. The formal specification constraints can be based on interpretable internal coefficients of the explainable architecture XTT. The formal specification constraints can be designed such that the formal verification does not allow bias in the explainable system. The formal specification constraints can be used to enable formal verification methods to be utilized with XTT and all XTT variants. It is further contemplated that the formal specification constraints can enable advanced simulation and behavioral analysis methods to be utilized with XTT and all XTT variants.

[0278] In an exemplary embodiment, an XTT-based system (or a subsystem controlled or monitored by an XTT-based system) may be monitored during its normal operating cycle for anomalies, variances, and deviations from normal operating states and statuses. It is further contemplated that such monitoring mechanisms may be accomplished by combining XTT with a behavioral model (BM) or similar control engineering solution that allows for the specification and enforcement of conditions, events, triggers, and actions. Monitoring the constraints and expressions underlying the conditions, events, triggers, and actions may be implemented in a variety of suitable exemplary implementations, including, but not limited to, in the form of symbolic rules or systems of symbolic expressions, polynomial expressions, conditional and unconditional probability distributions, joint probability distributions, state space and phase space transforms, integer / real / complex / quaternion / octonion transforms, Fourier transforms, Walsh functions, Haar and non-Haar wavelets, generalized L2 functions, fractal-based transforms, Hadamard transforms, type 1 and type 2 fuzzy logic, and difference analysis. Monitoring constraints and expressions may also be implemented in the form of data structures referencing different features and variables accessible to the explainable model and any associated taxonomies, ontologies, and causal models. Monitoring constraints and expressions may also be implemented in the form of knowledge graph networks, control charts, Nelson rules, Bode plots, Nyquist plots, and related methods for determining whether any measured behavior variable is out of control - i.e., providing a measure of unpredictability versus consistency.

[0279] In an exemplary embodiment, the XTT architecture can be used within the behavioral model BM to perform formal verification of an autonomous system to detect nearby pedestrians described using constraints and / or rules such as "if a pedestrian is near the autonomous system; and such pedestrian is coming from the left, perform a right turn." An automated constraint and / or rule extension system can extend the verification statement with further background rules adopted from the world context, for example by adding "where such turn is determined by the system speed, predicted pedestrian speed and right-hand obstacle boundaries." Such a verification problem can be solved by solving a problem that involves a distance between the autonomous system and the pedestrian. l <D<dist u (where dist l and dist u The parameters are the distance D between the autonomous system and the pedestrian (where D is the lower and upper bounds of such distance), and ang l <θ <ang u At the angle θ (where ang l and ang u are the lower and upper bounds on the angle of the predicted pedestrian movement vector), can be transformed into a solvable system of equations. Formal verification techniques show that such parameter constraints ensure that the predicted output for the right turn angle is within the bounds o lower <right turn <o upper Continuing with the example, such a formal verification solution may be implemented in parallel with a solution based on observational dynamic verification of actual observed behavior, which in this case is performed on the autonomous system itself. Continuing with the example, the autonomous system may also output a user-level description such as "Had to swerve strongly to the right to avoid the pedestrian who ran in front of the car". The adjectives in the description such as "strongly" may be determined by any suitable classification method, value lookup table, fuzzy logic system, AI model, or suitable text generation implementation. The autonomous system is implemented using an explainable transformer-encoder based architecture XTT that takes as input an input image. e The input image can be divided into patches, flattened, and used as input tokens by their corresponding positional encoding. Transformer-Encoder Based Architecture XTT e can be attached to a fine-tuning linear layer to predict control actions. The autonomous system can also output a more detailed description at a diagnostic level, such as "Car swerved strongly to the right (acceleration profile: angle=20°, speed delta=2.77m / s) to avoid pedestrian (detected object: class=pedestrian, confidence=95%; object vector (predicted):relative degree=90°, relative speed delta=2.22m / s)collision (predicted collision=91%; predicted injury risk=HIGH; predicted vehicle component=front headlamp)." The autonomous system can similarly utilize the diagnostic level description encoded in a suitable format, such as a machine-readable method for encoding and transmitting XAI knowledge, to generate multimedia output, such as animations, on a suitable dashboard display or head-up display system or augmented reality display.It is further contemplated that such information may similarly be transmitted over a suitable Intelligent Transport System (ITS) network, such as a vehicle to vehicle (V2V) network, a vehiqular ad hoc network (VANET), a mobile ad hoc network (MANET) using a suitable inter-vehicle communication (IVC) protocol, such as an IEEE 802.11p compatible protocol, or the like.

[0280] In an exemplary embodiment of the XTT system in an autonomous vehicle, the XTT can be used to predict driving lane shape, perform end-to-end object detection and behavior prediction, perform point cloud processing, and utilize gaze attention models in conjunction with monocular, compound eye, and multi-view image data to create an internal representation of the surrounding environment, while providing a contextual aware description to a human passenger or user, to create predictions of the dynamics of the environment.

[0281] Exemplary embodiments can provide a Transformer that is explainable. Exemplary embodiments can include an explainable transducer, an explainable transformer, an explainable transformer-encoder, and / or an explainable transformer-decoder. The explainable transformer architecture can explain the feature contribution of an input token to a predicted data representation. This can be used to understand the predicted output for a given component in the transformer. Using XNN / INN as the underlying architecture for the explainable encoder and decoder of the transformer, it is possible to build an end-to-end white-box machine learning system. Advances in white-box transformers include:

[0282] The explainable transformer architecture is as powerful as a standard black-box transformer.

[0283] The explainable Transformer architecture leverages and is fully compatible with all current deep learning libraries and architectures, allowing the explainable Transformer architecture to take advantage of all performance advances available for deep learning systems.

[0284] The explainable transformer architecture allows interpretable models to be created in a flexible way: it can be trained in one iteration without the need to have an external induction step as well as the possibility to train it in multiple phases or by incorporating induction for multiple parts of the model.

[0285] The explainable transformer architecture enables model fusion through partitioning.

[0286] The explainable transformer architecture can compute the encoding and explanation in a single feed-forward pass, without the need for further processing or perturbations.

[0287] The explainable transformer can be initialized, for example, by transferring knowledge from a black-box model or from an existing rule set that has been trained externally. Transfer learning can improve the speed and quality of the resulting interpretable model.

[0288] The Explainable Transformer architecture can incorporate other machine learning systems to create an end-to-end explainable DL system that incorporates XTT as part of the pipeline.

[0289] The explainable transformer architecture can be used to create interpretable automated ML models.

[0290] One aspect is an explainable machine learning system, the explainable machine learning system comprising a finite state transducer, FST, configured to enable the machine learning system to translate, map, and transform data from an input language to a second output language, the finite state transducer adapted to receive said data from the input language including one or more input features used for transformation or transforming said one or more input features, the finite state transducer further configured to provide translated, mapped, and / or transformed data as a second output language based on said one or more input features, and / or an explainable transformer having an attention-based architecture based on a combination of decoder and encoder components formed in association with said data from the input language, the finite state transducer and the explainable transformer configured to be combined to generate an explainable transducer-transformer, XTT, configured to translate, map, and / or transform said data from the input language.

[0291] Another aspect is a system for providing an explainable transducer transformer, the explainable transducer transformer including a conditional network comprising an input layer configured to receive an input and identify one or more input features, a conditional network configured to model the input features based on one or more partitions, each of the one or more partitions including a rule, an aggregation layer configured to aggregate the one or more rules into one or more aggregate partitions, and a switch output layer configured to selectively pool aggregate partitions from the aggregation layer with one or more partitions from the conditional layer, and a feature generation and transformation network comprising one or more transformation neurons configured to apply one or more transformations to the input features. the prediction network comprising: a fit layer configured to combine features transformed by the feature generation and transformation network to identify one or more features and one or more coefficients associated with at least one of the one or more partitions; a value output layer configured to analyze the one or more coefficients and output values ​​associated with the one or more features or at least one of the one or more partitions; an encoder layer and a decoder layer, at least one of the encoder layer and the decoder layer comprising an explainable architecture formed from inputs; and an output layer configured to generate outputs interpretable and explainable by at least one of a machine program or a human, where one or more execution paths through the partitions are identifiable by an external process.

[0292] Optionally or alternatively, the machine learning system according to claim 1, wherein the explainable transformer comprises an input layer configured to receive input associated with the data from an input language and identify the one or more input features; a conditional network configured to model the one or more input features based on one or more partitions, each of the one or more partitions comprising a rule; an aggregation layer configured to aggregate the one or more rules into one or more aggregate partitions; and a switch output layer configured to selectively pool aggregate partitions from the aggregation layer with one or more partitions from the conditional layer; and a feature generation and transformation network comprising one or more transformation neurons configured to apply one or more transformations to the one or more input features. a prediction network comprising a transformation network, a fit layer configured to combine features transformed by the feature generation and transformation network to identify one or more features and one or more coefficients associated with at least one of the one or more partitions, a value output layer configured to analyze the one or more coefficients and output a value associated with the one or more features or at least one of the one or more partitions, and a decoder and encoder component comprising at least one layer for encoding the input and at least one layer for decoding the input, and comprising an explainable architecture formed from the inputs, and an output layer configured to generate outputs that are interpretable and explainable by at least one of a machine program or a human, wherein one or more execution paths through the partitions are identifiable by an external process.Optionally, the machine learning system according to claim 1 or claim 2, wherein the XTT is configured to operate as an explainable transformer-encoder or explainable transformer-decoder, the system optionally further comprising conditional constraints on the internal coefficients of the XTT, the conditional constraints being optionally configured to be monitored during the nominal operating cycle of the XTT. Optionally, the explainable transducer comprises a white-box model component in a sub-layer of the decoder component and / or the encoder component. Optionally, an explainable architecture including an explainable neural network, XNN, is utilized within the XTT, the XNN optionally comprising an input layer, the input layer being configured to be input to a conditional network including a conditional layer, an aggregation layer, and a switch output layer, and a predictive network including a feature generation and transformation layer, a fit layer, and a predictive output layer, and a selection and ranking layer configured to multiply the output of the switch output layer and the output of the predictive output layer to generate a ranked or scored output, the XNN optionally comprising fast weights. Optionally, an explainable architecture comprising an interpretable neural network, INN, is utilized within XTT, and / or an explainable architecture comprising an explainable reinforcement learning, XRL system is utilized within XTT, and / or XTT is configured to support wide learning models. Optionally, the explainable architecture is configured to generate an explainable or interpretable output comprising an answer, generate optional model explanations of the answers, and optionally generate justifications for the answers and / or model explanations.Optionally, the XTT comprises partitions configured to process the one or more input features, the partitions being clusters that optionally group data points according to rules and / or distance similarity functions, the partitions being locked partitions, where the value of the locked partition is set as a static value, or movable partitions that are dynamically movable for each backward training pass of the XTT, the partitions being initially created as part of the XTT by an external process that initially creates the XTT, or the partitions being pre-trained or suitably initialized from a linked taxonomy or ontology that provides partitioning information, the XTT being optionally pre-fine-tuned after the partitions are pre-trained, once the partitions are created the XTT is configured to use gradient descent to fit or further refine the partitions, the partition structure is optionally arranged within a graph or hypergraph, and the system is optionally configured to process graph-to-sequence, sequence-to-graph, and graph-to-graph transformation modes. Optionally, the XTT is configured to implement a ranking function to determine how the XTT selects, merges, or splits partitions, the partitions being optionally arranged according to a hierarchical nature that can be used to represent symbols at different levels of semantic and semiotic detail. Optionally, the partitions are modifiable using one or more of a multi-objective optimization technique, a genetic algorithm, a Monte Carlo simulation method, and / or a causal logic and simulation technique. Optionally, the partitions comprise a priority function configured to determine which of the two or more overlapping partitions and the overlapping partitions should be activated, two or more non-overlapping partitions, and / or an aggregation function configured to combine or split results from the activated partitions, the partitions being optionally implemented using a sparse XNN or INN.Optionally, each partition includes one or both of the data to be fitted with a linear model and data to which at least one of the following has been applied prior to fitting the linear model to the data: polynomial expansion, rotation, dimensional scaling, non-dimensional scaling, state space transformation, phase space transformation, integer transformation, real transformation, complex transformation, quaternion transformation, octonion transformation, Fourier transform, Walsh function, continuous data bucketing, Haar wavelet, non-Haar wavelet, generalized L2 function, fractal based transformation, Hadamard transform, type 1 or type 2 fuzzy logic knowledge graph network, categorical encoding, topological transformation in Kolmogorov space, Fréchet space, Hausdorff space, or Tikhonov space, dissimilarity analysis, normalization, standardization, and conditional features. Optionally, the partitions are arranged in a hierarchical structure according to a partitioning function, the partitioning function comprising a clustering algorithm including one or more from a k-means function, a Bayes function, a connectivity-based partitioning function, a centroid-based partitioning function, a distribution-based partitioning function, a grid-based partitioning function, a density-based partitioning function, a fuzzy logic-based partitioning function, an entropy-based function, or a mutual information-based method, the partitioning function optionally comprising an ensemble method that generates multiple overlapping and / or non-overlapping partitions. Optionally, the XTT is configured to perform one or more iterative optimization steps on one or more of the partitions, the one or more iterative optimization steps optionally comprising merging and splitting of partitions using suitable aggregation, splitting, or optimization methods.

[0293] Optionally, the XTT comprises an XTT-decoder architecture configured to predict model explanations and information associated with the partitions, the XTT-decoder architecture is pre-trained using the generated model explanations to predict the next model explanation, the XTT-decoder architecture is optionally further pre-trained on a set of explanation gradients to predict the next explanation gradients to predict differences or changes in the model explanations, the XTT-decoder architecture is optionally pre-trained on an embedding representation of the input and output data to take into account both similarity and contrast factors during training, the embedding representation is optionally a sparse embedding.

[0294] Optionally, the explainable transformer is trained by appending partitioning information from an external explainable model to the input embeddings of the encoder component of the explainable transformer and optionally to the output embeddings of the decoder component of the explainable transformer; or the explainable transformer comprises two further layers in parallel to the encoder and decoder components, the two further layers configured to build an explainable architecture from the input space of the explainable transformer; or the explainable transformer comprises parallel explainable encoder layers comprising a multi-head attention component or an add-and-normalization component, the parallel explainable encoder layers configured to receive as input the output of the multi-head attention component or the output of the add-and-normalization component and the partitioning or explainable information. , the explainable transformer optionally further includes a parallel explainable decoder layer configured to receive as input an output of the parallel explainable encoder layer, the parallel explainable encoder layer optionally including an explainable architecture optionally configured to receive as input one or more model explanations, one or more gradients of the one or more model explanations, or information associated with one or more partitions, the parallel explainable decoder optionally including an explainable architecture and an add-and-normalize component, the parallel explainable encoder layer optionally configured to send an output of the parallel explainable encoder layer to a multi-head attention layer in the decoder component, the parallel explainable encoder layer optionally configured to merge its output with an output of the encoder component of the explainable transformer before sending the output to the multi-head attention layer in the decoder component.

[0295] Optionally, the decoder component of the explainable transformer is configured to receive as input the output of the XTT as part of a feedback loop. Optionally, the encoder and / or decoder components of the explainable transformer are configured to operate as an explainable auto-encoder-decoder, XAED. Optionally, the XTT comprises one or more transformations of the training dataset, the one or more transformations comprising: the one or more feature transforms optionally including one or more of polynomial expansions, rotation transformations, dimensional scaling, dimensionless scaling, Fourier transforms, Walsh functions, state space transformations, phase space transformations, Haar wavelets, non-Haar wavelets, generalized L2 functions, fractal based transformations, Hadamard transforms, Type 1 fuzzy logic graph networks, Type 2 fuzzy logic graph networks, categorical encoding, topological transformations in Kolmogorov spaces, Fréchet spaces, Hausdorff spaces, or Tikhonov spaces, dissimilarity analysis, data normalization, data standardization, the one or more feature transforms optionally arranged as a pipeline of transforms, the pipeline further optionally including transforms configured to analyze sequences of data ordered according to values ​​of one or more variables, the sequences of data optionally including time-ordered data sequences, and / or transforms obtained by gradient descent and other explainable architectures. Optionally, the XTT is configured to receive training dataset samples to train the XTT, which are generated using one or more techniques from among causal GAN-based generation, genetic engineering, Monte Carlo simulation, Petri nets, reinforcement learning techniques, simulation using white-box models and related global models, and / or any other methods that may be available. Optionally, the XTT is further configured to receive human knowledge injection, HKI, or system knowledge injection as input, and the coefficients of the explainable architecture in the XTT are modifiable to enforce certain rules to provide more effective collaboration between human users and machine learning systems, and the XTT is optionally further configured to use HKI or system knowledge injection for zero-shot or few-shot learning. Optionally, the XTT is configured to implement one or more of inductive, deductive, hypothetical, and causal logic to increase the adaptability of the XTT, and the XTT is optionally further configured to utilize a combination of model-free and model-based optimization methods.Optionally, the XTT is configured to allow complete removal of unexplained latent space from the output of the XTT to the extent that said one or more input features of the XTT are explainable. Optionally, the XTT is configured to apply to the output of the XTT one or more of the following: Fourier transform, integer transform, real transform, complex transform, quaternion transform, octonion transform, Walsh function, state space transform, phase space transform, Haar wavelet, non-Haar wavelet, generalized L″ function, fractal based transform, Hadamard transform, fuzzy logic graph network, categorical encoding, dissimilarity analysis, normalization, standardization, multidimensional Bezier curve, recurrence relation, and causal operator; or the XTT is configured to apply to the output of the XTT an activation or transformation function, the activation or transformation function comprising one or more of the following: a sigmoid function, a SoftMax function, a hierarchical tree or network, a causal diagram, a directed graph, an undirected graph, a hypergraph, a simplicial complex, a multimedia structure, or a set of hyperlinked graphs. Optionally, XTT is configured to process both structured and unstructured data, and to process hierarchically structured data, optionally constructed in the form of trees, graphs, hypergraphs, and / or simplicial complexes. Optionally, XTT further comprises a normalization module configured to normalize the multiple inputs before being received by or after being output from an input layer of XTT, a scaling module configured to scale the inputs before being received by or after being output from the input layer, and a standardization module configured to standardize the inputs before being received by or after being output from the input layer. Optionally, normalizing the inputs includes generating appropriate reports and analyses of biases and sources of bias, and formulating strategies for bias reduction, mitigation, or elimination by supervised, unsupervised, or semi-supervised means.Optionally, the XTT is distributed across a distributed explainable architecture, DEA, where the DEA is comprised of a homogeneous or heterogeneous mixture of explainable architectures, where the DEA is configured to split the dataset into multiple subsets of data to train each of the explainable architectures in the DEA, where each explainable architecture of the DEA is trainable using distributed training techniques applied to the homogeneous and heterogeneous explainable model mix in the DEA, optionally including bulk manipulation techniques. Optionally, the DEA can include one or both of the multiple independent models, where the models in the DEA are a mix of Explainable Artificial Intelligence (XAI), Interpretable Neural Nets (INN), Explainable Neural Nets (XNN), Explainable Spiking Nets (XSN) and Explainable Memory Nets (XMN), and / or Explainable Reinforcement Learning (XRL), where a given independent model, once trained, can be configured to work independently without relying on the DEA, where the DEA is optimized for training.Optionally, the XTT is usable within an explanation and interpretation generation system, EIGS, and configured to provide a model for processing an input query and generating appropriate answers, explanations, and optional justifications for the input query, and the XTT is optionally configured to transform one or more parts of the EIGS, where transforming one or more parts of the EIGS includes one or more of transforming presentation output, user-specific output personalization, context-specific output transformation, goal-specific output transformation, plan-specific output transformation, and action-specific transformation, and the XTT is configured to provide a suitable model for a filter to generate appropriate filtering of explanation scaffolding or utilize a combination of interpretation framing, protocol context, interpretation rules, interpretation procedures, interpretation scenarios, and conflict resolution information. Optionally usable within the EIGS to filter one or more portions of the EIGS using the XTT, or to perform an intra- and / or inter-filter consensus process within the EIGS, or to provide a suitable model for the interpreter to generate an appropriate interpretation and generate elements of the interpretation scaffold, or to transform an appropriate portion of the interpretation scaffold within the EIGS, or to transform the interpretation output of the EIGS based on a combination of framing, protocols, interpretation rules, interpretation procedures, interpretation templates, interpretation summaries, scenario models, domains, interactive contexts, and conflict resolution information, or to perform an intra- and / or inter-interpreter consensus process within the EIGS, or to transform the EIGS interpretation output based on an appropriate selection model and selection process. Optionally, the XTT comprises an explainable self-attention mechanism configured to generate multi-level explanations, optionally including partitioning information, internal coefficients of the XTT, and feature contributions of the input space of the XTT, and the explanations are available as output to the interpreter. Optionally, the XTT is trainable using self-supervised techniques.Optionally, XTT is configured to use previously learned representations or history states from all decoder or encoder components to merge into a single vector, which can be used as feedback for any layer within XTT; XTT is optionally further configured to use any internal coefficients of the white-box model to merge into a single vector; XTT is optionally further configured to impose causal constraints on the previously learned representations to avoid making causally improbable inferences or creating predictive paths that may be statistically valid but causally improbable; and XTT is optionally further configured to utilize behavioral models and associated condition, event, trigger, and action state representations to impose further environmentally improbable constraints on the previously learned representations to achieve an explainable, causally and environmentally improbable feedback memory.

[0296] Optionally, the system is configured to analyze and parse existing formal computer programs written in a suitable formal language and, once loaded, further refine it using the XTT attention model, although the hierarchical partition structure in XTT's explainable model can optionally be used to directly model the structure of formal language programs; or to transform code according to pre-defined stylistic standards, highlight inconsistencies or errors, suggest better alternatives and refactorings and rewrites of code, de-obfuscate obfuscated code, and apply functional programming language concepts such as alpha-transformations, beta-reduction, and eta-reduction to the generated code; or to be used in conjunction with an automated theorem prover in conjunction with a suitable mathematical formal language system to automatically analyze, generate, and auto-complete mathematical expressions, statements, and proofs from scratch or by existing collections of proven and / or incomplete statements; or to provide explanations of underlying assumptions between the input-output transformations applied by XTT while providing explanations of underlying assumptions between the input-output transformations applied by XTT, in order to provide an understanding of the nature of theorem proving in a variety of applications, including DNA gene expression, protein folding, and other related biochemical applications. or configured to analyze DNA code in conjunction with proteins or other suitable molecular shapes to account for regulatory changes in a genetically engineered application, or optionally configured for use in an end-to-end automatic speech recognition architecture to translate speech audio waveforms into corresponding text, or configured for use in an end-to-end deep learning architecture to process speech, where the machine learning system optionally further comprises an XAED encoder configured to convert speech to a latent discrete representation, or configured for use in multiple object tracking, or configured to collate satellite, aerial, or other types of bird's-eye view imagery for digital surface model or depth map generation, or configured for use in an end-to-end deep learning architecture to process speech, where the machine learning system optionally further comprises an XAED encoder configured to convert speech to a latent discrete representation ...and configured to process imagery of multi-view input data, or configured to be used for audio-visual classification tasks such as predicting whether audio is present in a video, or configured to be used for generating and detecting synthetically generated multi-dimensional data, including one or both of detecting and classifying deepfake images from authentic images, or detecting the presence of adversarial attack information added to the data to confuse machine learning systems, where XTT comprises an explainable model, or configured to automatically generate, insert, and blend precisely positioned, framed, scaled, lit, and rendered synthetic images of people and objects within a combination of real and computer-generated imagery, or configured to automatically generate, insert, and blend precisely positioned, framed, scaled, lit, and rendered synthetic images of people and objects within a combination of real and computer-generated imagery, where the machine learning system is implemented within a camera processing system, or , configured to automatically generate, insert, and blend precisely positioned, framed, scaled, lit, and rendered composite images of people and objects within a combination of real imageries and computer-generated imageries, where the machine learning system is implemented within medical hardware for use within surgery or medical image analysis; or configured to automatically generate, insert, and blend precisely positioned, framed, scaled, lit, and rendered composite images of people and objects within a combination of real imageries and computer-generated imageries, where the machine learning system is implemented within engineering applications for use in equipment inspection processes and manufacturing inspection processes; orand configured to be implemented within a medical implant that transmits data in a human readable and interpretable format, or configured to receive input signals from biological neurons and output signals to biological neurons in a suitably encoded format, where XTT may be implemented as part of a medical implantable device that bridges two or more biological neurons to provide a practical solution for bridging damaged biological nervous system connections or to assist in the implantation and connection of artificial devices in advanced prosthetic devices, or implemented as part of an XGAIL system simulator as part of XTT to generate a multimodal mixed output that is suitably combined according to the associated sequence ordering and environmental world context of the different modalities. the machine learning system is optionally further configured to tag the generated data with a secure traceable digital code, distributed ledger entry or non-fungible token; or to generate music using a combination of formal music notation and synthesis, sound sample mixing, text-to-speech generation, and general audio sample generation; and The machine learning system is optionally further configured to perform multi-modal XTT to analyze inputs in different modalities, or configured to predict annotations for video frames, where XTT is implemented as an XTT encoder in an end-to-end deep learning architecture, or configured to track dialogue states and learn unobserved slots instead of being pre-defined in an ontology, or configured to identify entities in an input sequence, or configured to identify start and end locations of answers to input questions, or configured to link mentions of entities in text to corresponding entities in a knowledge base, or configured to predict outputs based on previous outputs, or configured to be incorporated into a workflow system, and optionally further configured to be integrated into a robotic process automation system, a decision support system, or a data lake system, or configured to use causal explanations, where the causal explanations are based on counterfactuals. or configured to use a real language representation architecture, or to implement causal model specific functionality for dealing with cause-and-effect, dealing with association, intervention, and counterfactual causal logic, performing plausibility checking with environmental model inputs, augmenting training data by causally plausible isomorphism, or configured to automatically create draft causal models from natural language documents, where XTT specifies a causal XTT system, or configured to use computation and knowledge representation structures as a basis for constraint and predictive logic implementation, where the computation and knowledge representation structures optionally include Resource Description Frameworks, RDF, trees, RDF graphs, Levi-graphs, hypergraph structures, or simplicial complexes, or configured to implement audit logging functionality, where implementing the audit logging functionality includes creating decision logs and path traces showing the flow, interactions, and behavior of XTT, as well as its associated conditions, events, triggers, and actions, and overall dynamics, where the path traces are represented as annotated sequences of triggered and executed rules.Optionally implemented on expert systems and rule-based systems, or path tracing is optionally implemented on workflow systems as an annotated sequence of workflow nodes and paths executed by a workflow engine, where path tracing can be used to show the exact sequence and behavior of XTT, and is optionally configured to display nearest neighbors along paths of interest to users of the system, where XTT is optionally further configured to utilize its own audit system log and store said log in a tamper-resistant and traceable manner, or configured to be implemented on and verified in combination with a combination of systems based on one or more of the following: temporal logic of actions, abstract machine notation, Petri nets, computation tree logic, or implementations that can formally show modal logic, intuitionistic logic, and / or relational semantics, or configured to utilize anchor clauses, where the anchor clauses specify nodes, edges, events, triggers, constraints, or actions that are of critical importance, or configured to apply quantization techniques to enhance performance, wherein the system is optionally further configured to apply instability reduction techniques to partially counteract any adverse effects of quantization on XTT accuracy and reduce instability during training; or configured to enable practical implementation of data privacy protection solutions; or configured to utilize the generated structured explanations based on what-if, what-if-not, counterfactual, but-for, and conditional scenarios to generate explained strategies and scenario-based explanations that correspond to outcomes of applying such scenarios to XTT system inputs; or configured to use exogenous and endogenous variables and causal models to estimate a total cost of an action, wherein estimating the total cost optionally includes applying any specific association, intervention, or counterfactual rule, and optionally includes estimating the total cost for data with partially missing values; and the system is configured to provide a practical example of an application,Optionally further configured to utilize nearest neighbor techniques to provide actual examples of the application having a desired scenario outcome or hypothetical average of the application; or configured to utilize one or more of dot product approximation or outright elimination, sparse local attention models, adaptive attention windows, multi-dimensional attention matrix approximation, intelligent weight sharing, and intelligent parameterization to enhance prediction and / or training performance; or configured to analyze human behavior including gait analysis, movement prediction, and emotional state prediction; or configured to predict and analyze sports footage and athletic performance; or configured to detect and predict medical conditions; or configured to analyze financial stock trading patterns and predict market behavior and subsequently perform automated actions such as buying, selling, or placing short or long positions on specific stocks; or configured to perform a variety of tasks related to Industry 4.0 applications, including, but not limited to, the following: or configured to operate within a legal document (a legal application), or configured to translate sentences from a source language to a target language, or configured to perform presentation-oriented transformation and enhancement of explanation and / or interpretation output, or configured to process legal documents and correctly determine references and text sections that relate to and are references in legal documents and are applicable to different institutions and entities, or configured to control the flow of interactive explanation and / or interpretation processes commonly used in applications such as dialogue processing, chatbots, call center management systems, case management systems, customer support systems, client relationship management systems, conversational systems, and question and answer systems, or configured to be used in reinforcement learning type applications requiring feedback control type mechanisms.

[0297] Optionally, the system is implemented as a hardware circuit using a flexible architecture, optionally comprising an FPGA, or a static architecture, optionally comprising an ASIC, or a discrete component, or a neuromorphic architecture, optionally using spintronics or memristors, or a spiking neuron, or a quantum computing hardware, the quantum computing hardware being configured to enable correct interpretation of multiple qubit states, qubit basis states, mixed states, ancillary bits, and other quantum effects due to entanglement and / or decoherence, or the incorporation of quantum logic into the XNN, optionally including quantum logic gates. The XTT may be configured to allow for the introduction of intrinsic operators and / or hardware logic gates, or may be configured to perform multiple actions, evaluate multiple conditions, or evaluate highly constrained systems by utilizing quantum effects, where the quantum computing hardware may be optionally configured to utilize quantum algorithms or hybrid solutions, or may be configured to define Hilbert spaces for multiple modalities and / or tasks, where a multimodal or multitask Hilbert space may be used to represent all interactions between tasks and modalities, with optional extensions that may be used to perform both training on a subset of modalities and / or tasks along with a quantum version of crossover learning. Optionally, the XTT may be configured to link neurosymbolic constraints to some or all of the current and / or past history states associated with the XTT, and / or to past history rates of activation associated with the XTT.Optionally, named reference labels are assigned to specific components within the explainable architecture, the named reference labels optionally include metadata and can optionally consist of symbolic and / or stylized representations, the named reference labels are optionally usable within safety-related constraints, and the machine learning system is optionally configured to exploit the immutability of the named reference labels to generate stable long-term descriptions of dynamics occurring within the system. Optionally, the machine learning system is at least partially defined by an identity assessment recommendation framework. Optionally, the machine learning system further comprises an automated XAI system, the system configured to generate scenario-based explanations. Optionally, to process multiple tasks and multiple modalities sharing the same parameters, the XTT utilizes an explainable model, configured to receive multiple inputs corresponding to one or more tasks associated with one or more input features, and generate multiple outputs corresponding to outputs for the tasks, the explainable model is optionally defined by a hierarchical crossover structure consisting of optionally multiple crossover subsystems configured to enable crossover between different knowledge learned for different tasks to occur efficiently, the hierarchical crossover structure comprising one or more one-way link nodes and / or one or more two-way link nodes, and optionally crossover noise nodes and / or links between the crossover nodes, the explainable model is optionally a sparse explainable model or a DEA.Optionally, the system further comprises a convolutional neural network, CNN, coupled to an input of the XTT to define a CNN-XTT architecture, the CNN-XTT configured to use a backmap to visualize activation pathways within the CNN-XTT for a query and optionally incorporate the backmap as part of the justification, the CNN-XTT optionally configured to integrate a kernel labeling method with associated human-readable labels having a data format that is not inherently textual, the kernel type associated with the kernel labeling method optionally being an approximation kernel, and the CNN optionally being a temporally optimized CNN. Optionally, the system is configured to output data in a format or layout that is one or more of human readable natural language, graphical or visual format, audio, speech, tactile, video, time series, multispectral data, hierarchically ordered multimedia content, and 3D data, and the output data is optionally in a sequential format or layout that is one or more of 2D data, 3D data, multidimensional data arrays, transactional data, time series, digitized samples, sensor data, image data, hyperspectral data, natural language text, video data, audio data, tactile data, LIDAR data, RADAR data, and SONAR data.Optionally, the explainable transducer transformer is configured to perform multiple tasks or modalities using the same set of parameters, the multiple inputs corresponding to one or more tasks associated with one or more input features and multiple outputs corresponding to each task, the explainable transducer transformer further comprises a hierarchical partitioning structure and a crossover structure subsystem configured to crossover learned knowledge for different tasks, the predictive network is configured to identify one or more feature interactions within the hierarchical partitioning structure, the conditional network is configured to select, adjust, and multiplex path traces through the hierarchical partitioning structure to link tasks with associated inputs and outputs, the sparse explainable model is implemented within at least one of the conditional network and the predictive network, and the crossover structure subsystem implements at least one of a one-way link mode, a two-way link mode, crossover noise nodes, and crossover inter-node links. Optionally, the convolutional layer is configured to apply one or more convolutional layers, a back-map or reverse indexing mechanism to generate explanations, and a kernel labeling method configured to associate human readable labels to non-text data using progressive refinement of kernels, patterns, symbols, and concepts, where one or more kernel types including at least one of a linear function, a nonlinear polynomial function, an exponential function, a radial basis function, or a sigmoid kernel are implemented using at least one of an orthogonal method, a singular value decomposition, a random Fourier transform, or random binning features to reduce the amount of computation and / or resources required, and further comprising one or more temporal convolutional networks.Optionally, the explanations include scenario-based explanations associated with one or more what-if, what-if-not, counterfactual, but-for, or conditional scenarios to generate explained policies and scenario-based explanations according to the actions and decisions of the explainable agent, and / or the explainable agent is trained to learn proposed actions for a given user with a particular context that result in a change in the decision outcome and minimize a total cost of the action, the total cost being an aggregation of one or more costs associated with each variable based on a metric for each type of cost, and / or the scenario-based explanations involve the use of nearest neighbor methods, identify-assess-recommend-solve (IAR) frameworks, many-objective optimization (MOO), Pareto Front Method, particle swarm optimization (PSO), genetic algorithms (GA), Bayesian optimization, evolutionary strategies, gradient descent techniques, and Monte Carlo Simulation (MCS). Optionally, the encoder layer is a parallel explainable encoder layer configured to receive an output of the multi-head attention component, an output of the summation-and-normalization component, and at least one of the input explainable information or the partitioning information; and the decoder layer is a parallel explainable decoder layer configured to receive an output from the parallel explainable decoder layers, the parallel explainable encoder layer comprising a summation-and-normalization component and a multi-head attention component, the output of the parallel explainable encoder layer being merged with the output of the parallel explainable decoder layer. Optionally, the difference and / or change between the current explanation and previously presented explanations is used as input to an explainable transducer transformer to predict subsequent changes in the explanation; and the decoder. The sparse embedding representation of the input and / or output is trained on a set of explanatory gradients to predict the next gradient given a context window including the current explanation and historical explanatory gradients, and identifies one or more similarity and contrast elements. Optionally, the one or more explanations, explanatory gradients, or partitioning information of the explainable architecture are used as inputs to an encoder layer, and the explainable autoencoder-decoder comprises an encoder layer and a decoder layer, and the output includes one or more training dataset samples configured to be used as training inputs to one or more layers of an explainable transducer transformer. Optionally, coefficients of one or more explainable architectures in the explainable transducer transformer are modified by human knowledge injection using zero-shot learning or few-shot learning, and the one or more layers of the explainable transducer transformer implement at least one of inductive logic, deductive logic, hypothetical logic, and causal logic, and the explainable transducer transformer is configured to eliminate one or more unexplainable potential spaces. Optionally, the output is further subjected to one or more of the following transformations: Fourier transform, integer transform, real transform, complex transform, quaternion transform, octonion transform, Walsh functions, state space transformation, phase space transformation, Haar and non-Haar wavelets, generalized L2 functions, fractal-based transform, Hadamard transform, fuzzy logic, knowledge graph networks, categorical encoding, dissimilarity analysis, normalization, standardization, multidimensional Bezier curves, recurrence relations, and causal operators. Optionally, the system further includes at least one activation function and / or transformation function, the transformation function configured to transform the output using at least one of a hierarchical tree, a causal diagram, a directed or undirected graph, a hypergraph, or a set of simplicial complexes, a multimedia structure, and a hyperlinked graph.Optionally, each partition is configured to fit linear models and apply nonlinear transformations including at least one of polynomial expansions, rotations, dimensional and non-dimensional scaling, state space and phase space transformations, integer / real / complex / quaternion / octonion transformations, Fourier transforms, Walsh functions, continuous data bucketing, Haar and non-Haar wavelets, generalized L2 functions, fractal-based transformations, Hadamard transforms, type-1 and type-2 fuzzy logic, knowledge graph networks, categorical encoding, topological transformations in Kolmogorov / Fréchet / Hausdorff / Tychonoff spaces, dissimilarity analysis, and data normalization / standardization. Optionally, each partition includes clusters configured to group a plurality of data points according to a rule or distance similarity function, each partition representing a concept or category of data. Optionally, the partitions are formed based on a clustering algorithm including one or more of k-means, Bayesian, connectivity-based, centroid-based, distribution-based, grid-based, density-based, fuzzy logic-based, entropy, or mutual information (MI)-based methods, the clustering algorithm further including an ensemble method resulting in multiple overlapping or non-overlapping partitions, an association-based algorithm, or a causality-based partitioning function. Optionally, the input includes structured and unstructured data, or hierarchically structured data including one or more of trees, graphs, hypergraphs, and simplicial complexes, the explainable transducer transformer further includes a knowledge representation structure including at least one of a resource description framework (RDF) tree, an RDF graph, or a Levi-graph, and the audit log is configured to identify one or more of decision logs and path traces.Optionally, the explainable transducer transformer is configured to identify cause-and-effect, associations, interventions, and counterfactual causal logic, and is further configured to perform plausibility checks using the environmental model input, and further implements at least one causal model configured to augment the training data using causally plausible isomorphism, the at least one causal model being automatically generated using an attention model, and configured to identify causal links between preceding and subsequent generalizations to the generalization target, extract a causal structure, and create an initial causal model, and further includes one or more neuro-symbolic constraints that identify historical activation rates, where the activation rates may be constrained, and further includes named reference labels assigned to one or more components or layers within the explainable model, each named reference label including at least one description or metadata and linking to models providing external taxonomies, ontologies, and safety-related constraints. Optionally, the explainable transducer transformer further comprises one or more input and output modules comprising a normalization module configured to normalize the input before or after the input layer, a scaling module configured to scale the input before or after the input layer, and a standardization module configured to standardize the input before or after the input layer. Optionally, the explainable transducer transformer is implemented on a distributed explainable architecture including one or more explainable layers or models, and the one or more independent models are configured to activate independently from the one or more explainable layers or models. Optionally, the explainable transducer transformer is configured to be trained using self-supervised techniques and validated using one or more of action-time logic, abstract machine notation, Petri nets, computational tree logic, intuitionistic logic, and / or relational semantics.Optionally, the explainable transducer transformer is configured to analyze and parse existing formal computer programs, the computer programs being modeled with a hierarchical partition structure in one or more explainable models, and a formal specification language is used to specify a problem to be solved using the computer programs, and the explainable transducer transformer is configured to generate candidate code solutions for the problem to be solved, or the explainable transducer transformer is configured to transform code according to pre-defined stylistic standards, highlight inconsistencies or errors, suggest code alternatives and refactorings or rewrites, de-obfuscate obfuscated code, and apply one or more of an α-transformation, a β-reduction, and an η-reduction to the generated code, or The Explainable Transducer Transformer may be implemented in conjunction with an automated theorem proving system to automatically analyze, generate, and autocomplete mathematical expressions, statements, and proofs using an existing collection of proven and / or incomplete statements, or the Explainable Transducer Transformer may further comprise an end-to-end automatic speech recognition architecture to translate speech audio waveforms into corresponding text or into latent discrete representations, or the Explainable Transducer Transformer may further comprise a system for matching satellite, aerial, or bird's-eye view imagery for multiple object tracking, or for digital surface model or depth map generation, or for processing imageries from monocular, compound eye, and multi-view input data, or for audio visualization classification tasks to predict whether an audio clip is present in a video or not.Optionally, at least one layer is implemented on hardware comprising at least one of flexible architecture or field programmable gate arrays, static architecture or application specific integrated circuits, analog or digital electronic components, optoelectronic components, optical processors, neuromorphic architectures, spintronics or memristors, discrete computing components, spiking neurons, robotic hardware, autonomous vehicles, industrial control hardware, or quantum computing hardware, and the implementation of the at least one layer on the hardware is based on at least one of application of quantum or hardware directed compression techniques on the hardware, and the at least one layer comprises a sparse explainable neural network architecture, and the explainable model is one or more of workflows, process flows, fast weights, robotic process automation (RPA), decision support systems (DSS), data lakes,. Perform Root Cause Analysis (RCA), Goal-Plan-Action (GPA) systems, process descriptions, state transition charts, Petri networks, electronic circuits, logic gates, optical circuits, digital-analog hybrid circuits, biomechanical interfaces, bioelectrical interfaces, and quantum circuits. Optionally, the explainable transducer transformer further comprises at least one explainable model configured to form an explanatory structural model (ESM) comprising the explainable model, a statistical structural model modeling a plurality of statistical relations, a structural causal model (CSM) modeling a plurality of causal relations, and a symbolic structural model modeling a plurality of symbolic and logical relations formed as one or more rules and / or symbolic logic, where the one or more statistical, causal, symbolic, or logical relations are modeled as anchor components, and further comprises implementing an explanation interpretation generation system (EIGS) and / or explanation filter interpretations configured to output an explanation output template (EOT), and an explainable self-attention mechanism configured to generate a multi-level explanation including at least one of partitioning information, internal coefficients of one or more explainable models, and feature contributions of an input space, where the explanation is used as an input to the interpreter.Optionally, the explanation further includes at least one of basic interpretations, explanatory interpretations, and meta-explanatory interpretations, and neuro-symbolic conditional constraints having rates of activation to constrain the rate of trigger activation with respect to the explainable model, the neuro-symbolic conditional constraints being implemented as at least one of a system of symbolic rules or symbolic representations, polynomial representations, conditional and unconditional probability distributions, joint probability distributions, state space and phase space transformations, integer / real / complex / quaternion / octonion transforms, Fourier transforms, Walsh functions, Haar and non-Haar wavelets, generalized L2 functions, fractal-based transforms, Hadamard transforms, type-1 and type-2 fuzzy logic, difference analysis, and knowledge graph networks. Optionally, the explainable transducer transformer further comprises an explanation component configured to receive the model output to generate an explanation using an explanatory scaffold, the explanatory scaffold comprising an explanatory model component, the model output including the answer, the model explanation, a model fusion and linking component configured to store metadata and information associated with one or more links between one or more systems and databases, a hypothesis and causal component, the hypothesis and causal component configured to model at least cause-and-effect relationships by forming one or more structural equation models, structural causal models, and / or causal directed acyclic graphs, and a scenario, interaction, and presentation component. Optionally, the output includes the answer, the model explanation, and a justification for the answer and / or the model explanation, the justification indicating at least one assumption, process, or decision used by the explainable neural network in arriving at the answer or the model explanation.Optionally, the output includes at least one of a human-readable natural language format, a graphical or visual format, audio, speech, haptic, video, time series, multispectral data, hierarchically ordered multimedia content, and 3D data, and the explainable transducer transformer is configured to process the sequential format, as well as the sequence data including the output value or classification or one or more associated labels indicating a data point or a continuous or non-contiguous interval of the data points, and the input sequence is mapped to the output sequence in a multi-model and multi-task manner. Optionally, the at least one explainable neural network is configured to implement a wide learning model. Optionally, the conditional layer is configured to receive one or more partitions based on an external partitioning process. Optionally, the one or more partitions are pre-trained or initialized from a linked taxonomy or ontology. Optionally, gradient descent is applied to further refine the partitions, and one or more of the partitions are locked at static values, dynamically adjustable using backward training techniques, many-objective optimization, genetic algorithms, Monte Carlo simulation techniques, or causal logic and simulation techniques, or are hierarchical. Optionally, the explainable transducer transformer further includes a ranking function configured to select, merge or split one or more overlapping or non-overlapping partitions, where the one or more partitions are configured to represent symbols and concepts at at least two different levels of semantic and semiotic detail.

[0298] Aspects of any of the embodiments described above may be combined with aspects and / or options of any of the other embodiments described to form further embodiments without losing the desired effect.

[0299] The above description and accompanying figures illustrate the principles, preferred embodiments, and modes of operation of the present invention. However, the present invention should not be construed as limited to the particular embodiments discussed above. Further variations of the above-discussed embodiments will be apparent to those skilled in the art (for example, features associated with a particular configuration of the present invention may instead be associated with any other configuration of the present invention, as desired).

[0300] Therefore, the above described embodiments should be considered as illustrative rather than restrictive, and accordingly, it should be recognized that modifications to these embodiments may be made by those skilled in the art without departing from the scope of the invention as defined by the appended claims.< / z> < / z> < / y> < / x> < / y> < / y> < / x> < / x>

Claims

1. 1. An explainable machine learning system, comprising: The machine learning system comprises a finite state transducer, FST, configured to enable translating, mapping, and converting data from an input language into a second output language; the finite state transducer is adapted to receive the data from the input language including one or more input features used for transformation or transforming the one or more input features, the finite state transducer being further configured to provide translated, mapped and / or transformed data based on the one or more input features as the second output language; and an explainable transformer having an attention-based architecture based on a combination of decoder and encoder components formed in relation to the data from the input language; The finite state transducer and the explainable transformer are configured to be combined to generate an explainable transducer transformer, XTT, configured to translate, map, and / or transform the data from the input language.

2. The explainable transformer is an input layer configured to receive an input associated with the data from the input language and to identify the one or more input features; a conditional network comprising: a conditional layer configured to model the one or more input features based on one or more partitions, each of the one or more partitions including a rule; an aggregation layer configured to aggregate the one or more rules into one or more aggregate partitions; and a switch output layer configured to selectively pool the aggregate partitions from the aggregation layer with the one or more partitions from the conditional layer; a prediction network comprising: a feature generation and transformation network comprising one or more transformation neurons configured to apply one or more transformations to the one or more input features; a fit layer configured to combine features transformed by the feature generation and transformation network to identify one or more coefficients associated with at least one of the one or more features and one or more partitions; and a value output layer configured to analyze the one or more coefficients and to output a value associated with the one or more features or at least one of the one or more partitions; the decoder and encoder component comprises at least one layer for encoding the input and at least one layer for decoding the input, and comprises an interpretable architecture formed from the input; 10. The machine learning system of claim 1, comprising an output layer configured to generate output that is interpretable and explainable by at least one of a machine program or a human, where one or more execution paths through the partitions are identifiable by an external process.

3. 3. The machine learning system of claim 1 or 2, wherein the XTT is configured to operate as an explainable transformer encoder or an explainable transformer decoder, the system optionally further including conditional constraints on internal coefficients of the XTT, the conditional constraints optionally configured to be monitored during a nominal operating cycle of the XTT.

4. An explainable architecture is utilized within the XTT, including the explainable neural network, XNN, which: An input layer, a conditional network including a conditional layer, an aggregation layer, and a switch output layer; and A prediction network that includes a feature generation and transformation layer, a fit layer, and a prediction output layer. an input layer configured to receive the a selection and ranking layer configured to multiply the output of the switch output layer and the output of the prediction output layer to generate a ranked or scored output; The machine learning system of claim 1 , wherein the XNN optionally includes fast weights.

5. An explainable architecture comprising an interpretable neural network, INN, is utilized within the XTT; and / or Explainable architecture comprising an explainable reinforcement learning, XRL system is utilized within the XTT; and / or The machine learning system of claim 1 , wherein the XTT is configured to support wide learning models.

6. The explainable architecture comprises: Generate explainable or interpretable output of answers, generating a model explanation of said answer options; Optionally generating justification for the answer and / or the model explanation. The machine learning system of claim 2 , configured to:

7. The XTT comprises partitions configured to process the one or more input features, the partitions being clusters that arbitrarily group data points according to rules and / or distance similarity functions, the partitions being either locked partitions, where the value of the locked partition is set as a static value, or movable partitions that are dynamically movable for each backward training pass of the XTT, the partitions being initially created as part of the XTT by an external process that initially creates the XTT, or 7. The machine learning system of claim 1, wherein the partitions are pre-trained or suitably initialized from a linked taxonomy or ontology that provides partitioning information, the XTT is optionally pre-fine-tuned after the partitions are pre-trained, and once the partitions are created, the XTT is configured to use gradient descent to fit or further refine the partitions, and partition structures are optionally placed within a graph or hypergraph, and the system is optionally configured to handle graph-to-sequence, sequence-to-graph, and graph-to-graph transformation modes.

8. 8. The machine learning system of claim 7, wherein the XTT is configured to implement a ranking function to determine how the XTT selects, merges, or splits partitions, the partitions being optionally arranged according to a hierarchical property that can be used to represent symbols at different levels of semantic and semiotic detail.

9. 9. The machine learning system of claim 7 or 8, wherein the partitions are modifiable using one or more of the following: multi-objective optimization techniques, genetic algorithms, Monte Carlo simulation methods, and / or causal logic and simulation techniques.

10. The partitions may include two or more overlapping partitions and a priority function configured to determine which of the overlapping partitions should be activated; two or more non-overlapping partitions; and / or 10. The machine learning system of claim 7, further comprising an aggregation function configured to combine or split results from multiple activated partitions, said partitions being optionally implemented using sparse XNN or INN.

11. 11. The machine learning system of claim 7, wherein each partition includes one or both of the data to be fitted with a linear model and data to which at least one of the following has been applied prior to fitting a linear model to the data: polynomial expansion, rotation, dimensional scaling, non-dimensional scaling, state space transformation, phase space transformation, integer transformation, real transformation, complex transformation, quaternion transformation, octonion transformation, Fourier transform, Walsh function, continuous data bucketing, Haar wavelet, non-Haar wavelet, generalized L2 function, fractal-based transformation, Hadamard transform, Type-1 or Type-2 fuzzy logic knowledge graph network, categorical encoding, topological transformation in Kolmogorov space, Frechet space, Hausdorff space, or Tikhonov space, dissimilarity analysis, normalization, standardization, and conditional features.

12. 12. The machine learning system of claim 7, wherein the partitions are arranged in a hierarchical structure according to a partitioning function, the partitioning function comprising a clustering algorithm comprising one or more from: a k-means function, a Bayes function, a connectivity-based partitioning function, a centroid-based partitioning function, a distribution-based partitioning function, a grid-based partitioning function, a density-based partitioning function, a fuzzy logic-based partitioning function, an entropy-based function, or a mutual information-based method, and the partition function comprises an ensemble method that generates a plurality of optionally overlapping and / or non-overlapping partitions.

13. 13. The machine learning system of claim 7, wherein the XTT is configured to perform one or more iterative optimization steps on one or more of the partitions, the one or more iterative optimization steps optionally including merging and splitting the partitions using a suitable aggregation, partitioning, or optimization method.

14. 7. The machine learning system of claim 6, wherein the XTT comprises an XTT-decoder architecture configured to predict the model explanation and information associated with the partition, the XTT-decoder architecture being pre-trained using generated model explanations to predict next model explanations, the XTT-decoder architecture being optionally further pre-trained on a set of explanatory gradients to predict next explanatory gradients to predict differences or changes in the model explanations, the XTT-decoder architecture being optionally pre-trained on embedding representations of input and output data to take into account both similar and contrast factors during training, the embedding representations being optionally sparse embeddings.

Images