Explainable adaptable artificial intelligence networks

Abstract

Computer-implemented methods and systems make a generative AI system more explainable. A programmed computer system grows a generative AI system by adding one or more explainable network elements to the generative AI system. Each explainable network element can be trained to discriminate two or more explainable sets of training data items for the generative AI system. After adding the one or more explainable network elements, training of the generative AI system can be updated with the one or more explainable network elements added. Then the programmed computer system can determined whether continued growth of the generative AI system is required.

Description

PATENT Docket No.230458PCT IN THE UNITED STATES RECEIVING OFFICE PATENT APPLICATION FOR EXPLAINABLE ADAPTABLE ARTIFICIAL INTELLIGENCE NETWORKS Inventors: James K. Baker and Heather Baker Nielsen PRIORITY CLAIM

[0001] The present application claims priority to, and incorporates herein by reference, each of the following U.S. provisional patent applications: (1) Serial No.63 / 529,563, filed July 28, 2023, titled “Explainable Adaptable Artificial Intelligence Networks”; and (2) Serial No. 63 / 537,671, filed September 11, 2023, titled “Explainable Adaptable Artificial Intelligence Networks.” RELATED APPLICATIONS

[0002] The present application is related to: U.S. provisional patent application Serial No. 63 / 481,697, filed January 26, 2023, titled “Training Dynamic Hybrid AI Networks”; international application No. PCT / US24 / 12671, filed January 24, 2024, titled “Training Dynamic Hybrid AI Networks”; U.S. provisional patent application Serial No.63 / 468,145, filed May 22, 2023, titled “Training Human-Guided Hybrid AI Networks”; and international application No. PCT / US24 / 30324, filed May 21, 2024, titled “Training Human-Guided AI Networks.” BACKGROUND

[0003] Deep neural networks have had remarkable success in recent years. However, some fundamental problems remain, such as sensitivity to small adversarial perturbations in the data and the difficulty of interpreting the inner nodes in a large network. The sensitivity to small adversarial perturbations can cause a deep neural network classifier to make mistakes that no sensible entity would make. The difficulty of holistically interpreting inner nodes in context may make it impossible to fully trust the decisions and actions of an AI system based on such a network. The dangers posed by these problems can become very serious as society becomes increasingly dependent on AI systems using deep neural networks.

[0004] Although deep learning using large, deep neural networks is one of the most successful techniques in artificial intelligence, the size and complexity of a large, deep network can make it very difficult to understand its inner workings and to detect and 1 318976815.3Docket No.230458PCT diagnose any problems. Furthermore, the design of neural networks and the training techniques make a large neural network vulnerable to making mistakes that no sensible person would make. In general, deep neural networks are trained by a process called gradient descent in which, for each training data item, a computer system applies the chain rule of calculus to back propagate the derivative of an objective such as a divergence measure that penalizes errors. Adversarial attacks may make use of gradient descent to find small adversarial perturbations that cause a deep neural network classifier to make a mistake. Designing a network to be trained by gradient descent makes the network vulnerable to adversarial attacks based on gradient descent and to other sources of small perturbations.

[0005] The mistakes caused by such small perturbations are examples of the fact that the system lacks sensibility. That is, the system may make a mistake that no sensible person would make. More generally, deep neural networks lack common sense. Furthermore, the complexity of large neural networks makes it difficult if not impossible for humans to comprehend the details of the training process, much less to help contribute common sense. As AI systems become more and more capable and take on more and more tasks, the lack of common sense will become an increasing danger. Once AI systems take over the task of designing the next generation of AI systems without human understanding and control, it will become increasingly difficult to introduce sensibility and common sense. As AI systems control more aspects of human life, the consequences of mistakes could become catastrophic.

[0006] The difficulty of understanding inner nodes of a neural network is mainly caused by the large size and depth of the network together with training routines that give no guidance to get inner nodes to represent concepts that can be expressed in human language. In large part, the lack of sensibility and the lack of holistic interpretability are a consequence of the methods of training deep neural networks. SUMMARY

[0007] In one general aspect, the invention presents the concept of a dynamic hybrid network, which is a generalization of the concept of a neural network. Methods of hybrid training provide alternatives to training the network solely by gradient descent. The architecture of a hybrid network includes new elements called units and cells as well as neural network nodes. The training techniques for dynamic hybrid networks support training architectures that are robust against disturbances in the input data. The system supports several methods of training elements such as piecewise constant activation functions, 318976815.3 2Docket No.230458PCT including linear threshold functions. The training supports incremental growth of the network and continuing training during deployment. The configuration of a hybrid network is dynamic and may be changed and customized after receiving a specific input data item. Techniques are included to train the system to avoid classification errors that violate sensibility, including mistakes caused by adversarial attacks. Hybrid models and training techniques also contribute to the interpretability of inner elements in the context of surrounding elements and the rest of the network. The system supports supervision of the training process by a cooperative effort of a human team and one or more AI systems trained in the supervision of the training of a hybrid network.

[0008] In another general aspect, the present invention is directed to computer-implemented systems and methods for adaptively tuning a large language model (LLM) with human input. The method can comprise the steps of: (a) generating a plurality of training pairs for a first image generator, where each of the plurality of training pairs comprise (i) an image and (ii) a corresponding description of the image, where generating the plurality of training pairs comprises generating the plurality of training pair with a second image generator; (b) training, by a programmed computer system, the first image generator with the plurality of training pairs; (c) generating, by the LLM, based on a first prompt, from a human, received by the LLM, a first detailed description of an image to be generated by the first image generator; (d) generating, by the first image generator, a first image based on the first detailed description of an image generated by the LLM; (e) receiving, from a human, by the programmed computer system, a first edit to the first detailed description based on a review by the human of the first image; and (f) training, by the programmed computer system, the LLM with the first edit as training data for the LLM. Steps (c) to (f) could be repeated multiple time to train the LLM.

[0009] In another general aspect, the present invention is directed to computer-implemented systems and methods for generating a textual work. In various embodiments, the method can comprise the step of generating, by a computer system that comprises a LLM, an outline for a textual work based on a topic prompt received by the LLM, where the outline comprises N sub-topics, where N > 2. The method can also comprise iteratively, by the computer system, for each of the n = 1, …, N sub-topics: generating, by the LLM, a passage of text for the nth sub-topic; soliciting user feedback from a user of the passage of text for the nth sub-topic; updating, by the computer system, the passage of text for the nth sub-topic based on user feedback, if any; and adaptively training the LLM with the user feedback, if any. 318976815.3 3Docket No.230458PCT

[0010] In another general aspect, the present invention is directed to computer-implemented systems and methods for training a target node in a neural network to be more interpretable. In various embodiments, the method comprises the step of adding, by a programmed computer system, an additional node to the neural network, where adding the additional node comprises initializing the additional node to have same connections and weights as the target node, and where the additional node is to be associated with a first specified set of data items. The method also comprises the step of training, by the programmed computer system, the neural network, with the additional node added, where training the neural network comprises imposing a regularization on the additional node to train the additional node to have an activation value for each data item in the first specified set that is in better agreement with the data item being a member of the first set. The method also comprises the step of creating, by the programmed computer system, at least three new test neural networks, where: at least one of the three new test neural networks comprises the target node but not the additional node; at least one of the three new test neural networks comprises the additional node but not the target node; and at least one of the three new test neural networks comprises both the target node and the additional node. The method also comprises the step of computing, by the programmed computer system, a regression on a measured performance of each new test neural network in the at least three new test neural networks as a function of whether each new neural network comprises (i) the target node but not the additional node; (ii) the additional node but not the target node; and (iii) both the target node and the additional node. The method also comprises the step of creating, by the programmed computer system, a new neural network based on the regression, where creating the new neural network comprises deciding whether to include in the new neural network, based on the regression, (i) the target node but not the additional node; (ii) the additional node but not the target node; and (iii) both the target node and the additional node. And the method also comprises the step of training, by the programmed computer system, the new neural network.

[0011] In another general aspect, the present invention is directed to computer-implemented systems and methods for improving interpretability of a neural network. The neural network can comprise attention block output node, where the attention block output node computes a weighted correlation between two n-tuples. In various embodiments, the method comprises the step of replacing, by a programmed computer system, the attention block output node with a multi-node unit, where the multi-node unit comprises: first and second product nodes, where the first product node computes a multiplication of values in a first n-tuple and the 318976815.3 4Docket No.230458PCT second product node computes a multiplication of values in a second n-tuple; and a summation node computes a weighted sum of outputs from the first and second product nodes. The method also comprises the step of training, by the programmed computer system, the neural network, with the multi-node unit.

[0012] Another general aspect of the present invention related to chained sequences of mappings. Assume that a chained sequence has a plurality of n= 1, …, N mappings, such that, for n < N, an output form of representation of the nth mapping is an input form of representation for the (n+1)th mapping; an n > 2 mapping has an output form of representation that is the same as an output form of representation of the 1st mapping; and an n < N mapping has in input form of representation that is the same as an output form of representation of the Nth mapping. In various embodiments, the method comprises the step of training, through machine learning, by a programmed computer system, an autoencoder set, that comprises one or more autoencoders, with an objective for the autoencoder set of generating an instance of the output form of representation of the Nth mapping for an input that is an instance of the input form of representation of the 1st mapping. The autoencoder set comprises at least one latent space that comprises text and training the autoencoder set can comprise receiving an edit, from a human, of text for the at least one latent space that comprises text and training the autoencoder set with the edit from the human.

[0013] In another general aspect, the present invention is directed to computer-implemented systems and method for detecting text generation by an LLM. According to various embodiments, the method can comprise the step of training, through machine learning, by a programmed computer system, a LLM to generate textual passages with low probability linguistic units. The method also comprises the step of training, through machine learning, by the programmed computer system, a detector to detect low probability linguistic units in input text. The method also comprises the step of, after training the detector, detecting with the detector text generated by the LLM that comprises one or more low probability linguistic units.

[0014] In another general aspect, the present invention is directed to computer-implemented systems and method for training a set of one or more nodes as named-set discriminators and for training and using associated confidence estimators. In various embodiments, the method comprises selecting, by a programmed computer system, a pair of known sets of data items to be associated with a selected node of a neural network as pair of sets to be discriminated. The method also comprises the step of creating, by the programmed computer system, a new node 318976815.3 5Docket No.230458PCT for the neural network, where the new node is to discriminate the pair of sets to be discriminated, and where creating the new node for the neural network comprises connecting the new node to other nodes in the neural network and training the new node differently from the selected node. The method also comprises training, by the programmed computer system, a confidence score network for each of the selected node and the new node. The method also comprises the step of generating, by the programmed computer system, a single output value from output values of both the selected node and the new node, where generating the single output value comprises generating the single output value according to a combining rule for the selected node and new nodes, where the combining rule is selected based on confidence scores from the confidence score networks. The method also comprises the step of training, by the programmed computer system, the network to compute single output value.

[0015] In another general aspect, the present invention is directed to computer-implemented systems and methods for making a generative AI system, such as LLM, more explainable. A method according to such an embodiment can comprise the step of (a) growing, by a programmed computer system, a generative AI system by adding one or more explainable network elements to the generative AI system. The generative AI system can comprise one or more machine learning networks trained such that the generative AI system generates textual passages in response to prompting and context; and each of the one or more explainable network elements can be trained to discriminate two or more explainable sets of training data items for the generative AI system. The method can further comprise, at step (b), after adding the one or more explainable network elements, performing updated training, by the programmed computer system, of the generative AI system with the one or more explainable network elements added. The method can further comprise, at step (c), after steps (a) and (b), determining, by the programmed computer system, whether continued growth of the generative AI system is required. Upon a determination at step (c) that continued growth of the generative AI system is required, the programmed computer system can repeat steps (a) through (c). Upon a determination at step (c) that continued growth of the generative AI system is not required, the generative AI system can be deployed to generate textual passages (e.g., inference, i.e., using the trained generative AI model to make predictions or generate outputs based on new, unseen prompting and context).

[0016] These and other benefits of that can be realized through embodiments of the present invention will be apparent from the description that follows. 318976815.3 6Docket No.230458PCT DRAWINGS

[0017] Various embodiments of the present invention are described in conjunction with the following figures.

[0018] Figure 1 is a flow chart of an illustrative embodiment of the invention.

[0019] Figure 2 is a flow chart of an illustrative embodiment of processes for enhancing elementary sensibility in an aspect of the invention.

[0020] Figure 3A is an illustrative diagram of a hybrid unit in an illustrative embodiment of the invention.

[0021] Figure 3B is an illustrative diagram of an aspect of the invention called active defense.

[0022] Figure 3C is an illustrative diagram of a substitute derivative function used in an aspect of the invention.

[0023] Figure 4 is an illustrative diagram of a hierarchy of levels of techniques for improving sensibility.

[0024] Figure 5 is an illustrative diagram of embodiments of aspects of hybrid training organized by stages of the training process.

[0025] Figure 6 is a flow chart of an illustrative embodiment of constrained optimization in training.

[0026] Figure 7 is a flow chart of an illustrative embodiment of hidden state space modeling in an aspect of the invention.

[0027] Figure 8 is a flow chart of an illustrative embodiment of the operation of sensible classification with a trained hybrid network and rapid matching.

[0028] Figure 9 is an illustrative diagram of a type of autoencoder used in an aspect of the invention.

[0029] Figure 10 is a diagram of an illustrative embodiment of a robust template model used in an aspect of the invention.

[0030] Figure 11 comprises flow charts for illustrative embodiments for training data exclusion and data delegation in aspects of the invention.

[0031] Figure 12 is a flow chart of an illustrative embodiment for training alignment models in an aspect of the invention.

[0032] Figure 13 is a flow chart of an illustrative embodiment of an aspect of the invention 318976815.3 7Docket No.230458PCT called “conditional hybrid training.”

[0033] Figure 14 is a diagram of an illustrative embodiment of an aspect of the invention for transformation or translation of data spaces.

[0034] Figure 15 is a flow chart of an illustrative embodiment of an aspect of the invention using regression on counts in histogram bins.

[0035] Figure 16 is an illustrative diagram of a hybrid network of units and cells.

[0036] Figure 17 is an illustrative diagram of a multi-processor computer system such as might be used to implement various aspects of the invention.

[0037] Figure 18 is a flow chart of an illustrative embodiment of back propagation of data examples in an aspect of the invention.

[0038] Figure 19 is a flow chart of an illustrative embodiment of parallel or serial computations in a network of cells connected by data communication links.

[0039] Figure 20 is a flow chart of an illustrative embodiment of empirical training.

[0040] Figure 21 is a diagram of illustrative embodiments of aspects of the invention in which an artificial intelligence system comprising one or more hybrid networks implemented on computer system 1700 cooperates with a team of one or more humans on joint tasks.

[0041] Figure 21A is a diagram of a multi-layer, feed-forward neural network.

[0042] Figure 22 is a flow chart of an illustrative embodiment of the training and use of a system for image generation with human guidance.

[0043] Figure 22A is a block diagram of a system used in the method of Figure 22 according to various embodiments of the present invention.

[0044] Figure 23 is a flow chart of an illustrative embodiment of the process of building and training of an interactive, human-guided writer’s assistant.

[0045] Figure 24 is a flow chart of an illustrative embodiment of a process for training a selected node to be more interpretable.

[0046] Figure 25 is a diagram and a flow chart of an illustrative embodiment of a process of replacing an attention block output node with a multi-node unit and of training the nodes in the unit to be interpretable.

[0047] Figure 26 is a flow chart of an illustrative embodiment of a process herein called “round robin training.”

[0048] Figure 27 is a flow chart of an illustrative embodiment of a process for increasing the 318976815.3 8Docket No.230458PCT security of a text generation system.

[0049] Figure 28 is a flow chart of an illustrative embodiment of a process for training a set of one or more nodes as named-set discriminators and for training and using associated confidence estimators.

[0050] Figure 29 is a flow chart of an illustrative embodiment of targeted systematic growth of a network to improve performance and interpretability.

[0051] Figure 30 is a system diagram of a distributed system comprising a plurality of autonomous modular cooperative subsystems.

[0052] Figure 31 is a flow chart of an illustrative embodiment of a process of training a system comprising one or more autonomous modular cooperative subsystems, such as illustrated in Figure 30. In preferred embodiments, computer system 1700 may grow the system during initial training and may continue the training and growth during the use of the system by end users. During the training, computer system 1700 may grow the system with the goal of making it easier for a human user to understand and control.

[0053] Figure 32 is a flow chart of an illustrative embodiment of a process by which computer system 1700 may efficiently train a large language model with an arbitrarily large number of trainable parameters comprising transformer models and stochastic models.

[0054] Figure 33 is system diagram of an illustrative embodiment of an aspect of the invention in which computer system 1700 uses diverse types of models cooperatively to efficiently train and rapidly incrementally grow one or more machine learning systems while improving performance, interpretability and control.

[0055] Figure 34 is a flow chart of an illustrative embodiment of an aspect of the invention related to user control and to computer system 1700 tracking data and resources used during the training and use of a system.

[0056] Figure 35 is a flowchart of an illustrative embodiment of a number of optional processes that computer system 1700 may use in some embodiments in system such as illustrated in Figures 30 and 33 and / or in processes such as illustrated in Figures 31, 32, 36, 37, 38 and 39.

[0057] Figure 36 is a flow chart of an illustrative embodiment of cooperative process using diverse machine leaning systems such as illustrated in Figure 33 in which the generative system is a transformer-based large language model.

[0058] Figure 37 is a flow chart of an illustrative embodiment of a process for building a 318976815.3 9Docket No.230458PCT large system for text generation based on a hierarchy of ensembles of conditional probability models and joint optimization combining networks. In some embodiments, computer system 1700 may implement the process illustrated in Figure 37 on a distributed computer system with a plurality of local computers.

[0059] Figure 38 is a flow chart of an illustrative embodiment of an aspect of the invention by which computer system 1700 may expand the state space of a hidden Markov process modeling sequences of text.

[0060] Figure 39 is a flow chart of an illustrative embodiment of a process for incrementally building and training an arbitrarily large, distributed AI system from components that each fit within specified limits on memory and / or on the amount of computation.

[0061] Figure 40 is a flow chart of an illustrative embodiment of text generation that may use a system comprising a stochastic process model.

[0062] Figure 41is a flow chart of an illustrative embodiment of an aspect of the invention in which may incrementally grow a neural network, or a hybrid network making one or more duplicates of a component to improve the performance of the network or making the network easier to understand and control.

[0063] Figure 42 is a flow chart of an illustrative embodiment of computer system 1700 selecting a node to split based on tests of one or more criteria for potential improvements from various reasons and methods of splitting a node.

[0064] Figure 43 is a flow chart of an illustrative embodiment of an aspect of the invention in which computer system 1700 may manage the training, saving and loading of certain types of conditional probability models.

[0065] Figure 44 is a diagram of an illustrative embodiment of an aspect of the invention in which computer system 1700 may use a combining network, data dependent relation regularization links, and selective back propagation for decorrelation of errors for jointly optimizing the performance of a set of networks and training them to be diverse from each other.

[0066] Figure 45 is a flow chart of an illustrative embodiment in which computer system 1700 may generate text using a combination of transformer language models and stochastic models, with cooperation among the AI language models as well as explicit cooperative interaction between the human author, and the AI system, as the writer’s assistant.

[0067] Figure 46 is a flow chart of an illustrative embodiment of an aspect of the invention in 318976815.3 10Docket No.230458PCT which, in some embodiments, computer system 1700 may efficiently train a large neural network by first training a smaller neural network.

[0068] Figure 47 is a flow chart of an illustrative embodiment of a process by which computer system 1700 may train a large language model.

[0069] Figure 48 is a flow chart of an illustrative embodiment of a process by which computer system 1700 may generate text using a pretrained large language model.

[0070] Figure 49 is a flow chart of an illustrative embodiment of an aspect of the invention in which computer system 1700 trains a large language model comprising a hidden Markov process model.

[0071] Figure 50 is a flow chart of an illustrative embodiment of an aspect of the invention in which computer system 1700 incrementally increases the size of a transformer by increasing the number of attention heads in a specified attention layer.

[0072] Figure 51 is a flow chart of an illustrative embodiment of an aspect of the invention that uses fictitious play to train guardrails for a generative AI system and to train a system to detect guard rail violations.

[0073] Figure 52 is a flow chart of an illustrative embodiment of the invention in which computer system 1700 trains a translation system using a multi-path chain of one-way translations in which each link in the chain translates from a source language to a target language.

[0074] Figure 53 is a flow chart of an illustrative embodiment of an aspect of the invention in which computer system 1700 uses a multi-path chain of paired language translations to compute a robust composite translation.

[0075] Figure 54 is a flowchart of an illustrative embodiment of an aspect of the invention in which computer system 1700 may add nodes with linear threshold activation functions to a neural network or hybrid network and train the nodes using methods other than gradient descent.

[0076] Figure 55 is a flow chart of an illustrative embodiment of an aspect of the invention, in which, in some embodiments, computer system 1700 may develop, grow, and train an explainable large language model generative A.I. system.

[0077] Figure 56 is a flow chart of an illustrative embodiment of the process of using an explainable large language model text generation system in an interactive deployment.

[0078] Figure 57 depicts an example of a transformer network according to various 318976815.3 11Docket No.230458PCT embodiments of the present invention.

[0079] Figure 58 is a diagram of an explainable large language model text generation system according to various embodiments of the present invention.

[0080] The processes illustrated in the figures may be implemented in a multi-processor computer system 1700, such as shown in Figure 17. In preferred embodiments, the training and development of the system being developed may be supervised by a cooperative effort of a human team of knowledge engineers and AI systems, herein called the hybrid network learning management system (HNLMS). The AI systems in the HNLMS may also be implemented on a computer system such as computer system 1700. DESCRIPTION

[0081] The following paragraphs provide definitions for discussion of the figures.

[0082] Neural network: A directed graph comprising a set of nodes and a set of directed connections between ordered pairs of nodes. Typically, each connection has an associated learned parameter, called its weight. Typically, computer system 1700 multiplies the output of the source node of the connection by the weight of the connection to compute a value to supply as an input value to the destination node of the connection. Figure 21A shows a feed- forward neural network with multiple hidden layers.

[0083] Most discussions in this disclosure may refer to non-recurrent neural networks for which the graph is a directed acyclic graph. However, computer system 1700 may make multiple copies of a recurrent neural network in which all connection that would create a recurrence are redirected to the next copy of the network. By this means, computer system 1700 may model a recurrent neural network as a large “unrolled” network of non-recurrent copies of the base network so, for practical purposes, there is no loss in generality in assuming that the graph of a neural network is a directed acyclic graph.

[0084] Computer system 1700 may also use this unrolling mechanism with a hybrid network. In addition, hybrid networks provide additional ways to train models of recurrent processes. For example, computer system 1700 may model a recurrent process using a hidden state space model in the cells of the hybrid network. In a hybrid network, cells may be connected using bidirectional data communication links. The network of data communication links may contain cycles.

[0085] Node: A node in a neural network. In a hybrid network, the elements are called units and cells rather than nodes, except for internal neural nodes within a unit. A node within a 318976815.3 12Docket No.230458PCT unit may receive connections from nodes in other units and may source connections to nodes in other units.

[0086] Unit: A unit is a generalization of a neural network node. A unit may have multiple output values as well as multiple connections for each output value. A unit may comprise multiple nodes and subunits. A unit may also comprise special purpose elements called “cells” that are linked by data communication links rather than by network connections. A unit may comprise a single neural node or may comprise a single cell.

[0087] Cell: An element in a hybrid network that may store and transmit values of specified variables. Computer system 1700 may store and execute program code associated with a cell upon receiving data as input to the network or transmitted from other cells. A cell may be associated with program code that computer system 1700 may execute when computing the activation and response of the network for a specified input data item.

[0088] Hybrid network: A network of units and connections, rather than neural nodes and connections. A hybrid network may also comprise cells and data communication links. Computer system 1700 may change and customize the configuration of a dynamic hybrid network after receiving a data item to be classified.

[0089] Components of a neural node: A typical node in a neural network comprises two component operations: an affine summation and an activation function.

[0090] Affine sum: In the affine sum operation of a neural node, computer system 1700 computes a weighted sum of incoming values from connections into the node plus a node- specific bias term.

[0091] Activation function: In a typical neural node, computer system 1700 computes a specified function of the affine sum. The function is called the “activation function” of the node. The value of the activation function for a data item d, is called “the activation” of the node for data item d. The output value of the node is the output of the activation function for data item d. Examples of activation functions include but are not limited to, sigmoid, softmax, Tanh, and ReLU (Rectified Linear Unit) activation functions.

[0092] Implicit error: A determination that computer system 1700 may make that an interior node with a standard discriminator activation function (defined in block 203 of Figure 2) has made an error on a specific data item when computer system 1700 compares the activation of the node relative to a specified threshold with the sign of the back propagated derivative of an objective function. 318976815.3 13Docket No.230458PCT

[0093] Known set: A known set is a set of data items for which computer system 1700 can determine for any specific data item, to a specified degree of accuracy, whether the data item is in the known set. For example, the set of training data items for any output category in a classification system is a known set. Any set of items that computer system 1700 may detect, to a specified degree of accuracy, based on an output value of a node, cell, unit, or network being within a specified interval is a known set.

[0094] Named set: A named set is a known set for which computer system 1700 has a name that may be easily understood by a human. Generally, the set of data items for any output category is a named set. In some embodiments, a human may supply a name for an unnamed known set.

[0095] Network repository: A repository of previously trained nodes, cells, units, and networks that may be implemented by computer system 1700. In some embodiments, computer system 1700 may place a trained network or a partially trained network into a network repository. In some embodiments, computer system 1700 may place the subnetwork that activates a selected node, cell, or unit into a network repository. In some embodiments, computer system 1700 may mutually share some or all the contents of its network repository with other computer systems.

[0096] Knowledge engineering: The development of tools for analyzing data and computing useful functions and properties of the data in a specified domain in order to facilitate the development of machine learning systems to classify data items in the domain.

[0097] Hybrid Network Learning Management System (HNLMS): A system comprising a cooperation of a team of one or more humans with one or more AI systems. The human team and AI systems guide the training of hybrid networks to improve the sensibility and holistic interpretability as well as the performance of the networks being trained.

[0098] Detector: A node, unit, or cell with an output value that computer system 1700 characterizes as attempting to have values in a specified interval for data items in a target acceptance set and values not in the specified interval for data items not in the acceptance set. In some embodiments, the specified interval is the set of values above a specified threshold value. In some embodiments, the target acceptance set is known to computer system 1700, for example for an output node of a classifier for supervised training data. The actual set of data items in the specified interval may be called the “empirical acceptance set.” Where the meaning is clear, either the target acceptance set or the empirical acceptance set may simply 318976815.3 14Docket No.230458PCT be called “the acceptance set.” In some embodiments, the target acceptance set of a network element is not explicitly specified and is not a known set. In some embodiments in which the acceptance set of a detector is not explicitly known, computer system 1700 may tentatively empirically associate the output values with a known set.

[0099] Discriminator: A node, unit, or cell with an output value that computer system 1700 characterizes as attempting to have values in a first specified interval for data items in a first target acceptance set and a second specified interval for data items in a second target acceptance set. In some embodiments, computer system 1700 may have no target interval for data items not in either acceptance set. In some embodiments, a unit may have additional output values to characterize data items that are not in either target acceptance set.

[0100] Recall: In a data retrieval task or a detection task, the fraction of the number of correct retrievals or detections of target data items made from a specified set of data items by a machine learning system divided by the total number of target data items in the specified set of data items.

[0101] Precision: In a data retrieval task or a detection task, the fraction of the number of correct retrievals or detections of target data items made from a specified set of data items by a machine learning system divided by the total number of data items in the specified set of data items that are detected or accepted by the machine learning system, including false or incorrect items.

[0102] Association: The association of a specified known or named set with the set of data corresponding to a specified detector node, unit, or cell or to an interval of the activation function of a node is the determination that the specified detection satisfies a specified criterion for recall and / or precision with respect to the specified known or named set.

[0103] Knowledge Sharing Links: A knowledge-sharing link is a link between an ordered pair of nodes, a reference node and a receiving node. The nodes may both be nodes in the same network, or the nodes may be in two separate networks. Only the receiving network needs to be in a network currently being trained. If the nodes are in separate networks, it must be possible to activate both nodes on the same data item. For example, the two nodes may share a global or local input data space. In some embodiments, computer system 1700 may compute a mapping from one data space to the other. During training of the network comprising the receiving node, for specified data items, computer system 1700 may impose a regularization penalty if the activations of the two nodes fail to satisfy a specified relationship. 318976815.3 15Docket No.230458PCT

[0104] The Relation of a Knowledge Sharing Link: A common example relation of a knowledge-sharing link is the “is-equal-to” relation. For the is-equal-to relation between actreference(data) and actreceive(data), computer system 1700 may impose the regularization penalty,where ^^ is a hyperparameter controlled, for example, by the HNLMS. The hyperparameter ^^ is called the “strength” of the knowledge-sharing link. The HNLMS may also specify that the regularization only be imposed for specified data items. A knowledge-sharing link is not a connection. For example, in a non-recurrent network, a link may go from a reference node in a higher layer to a receiving node in a lower layer, which is not allowed for a connection in a non-recurrent network. Other common knowledge sharing relations include, is-less-than, is- greater-than, and is-not-equal-to. By convention, in the asymmetric relations, the reference node is the first argument.

[0105] The inequality relations, is-greater-than and is-less-than, are useful, for example, in sharing knowledge between two nodes in which one node is associated with a known set that is a subset of a known set associated with the other node. For example, the set of horses is a subset of the set of equines, which is a subset of the set of mammals, which is a subset of the set of animals, which is a subset of the set of living things. In some embodiments, computer system 1700 may impose a knowledge-sharing link that the activation of a node associated with a superset should be greater than or equal to the activation of a node associated with a subset. For the is-greater-than relation between actreference(data) and actreceive(data), computer system 1700 may impose the regularization penalty:where ^^ is a hyperparameter controlled, for example, by the HNLMS. For example, in phonetic recognition, the activation of a node associated with the set of vowels should be should greater than or equal to the activation of a node associated with the set of high front vowels. In some embodiments, computer system 1700 may limit the enforcement of the regularization to data in a specified interval in the reference node.

[0106] In some embodiments, computer system 1700 may limit the maximum regularization penalty for the is-not-equal-to relation. For example, for the is-not-equal-to relation,computer system 1700 may impose the regularization penalty:318976815.3 16Docket No.230458PCT with maximum penalty ^^, where ^^ and ^^ are hyperparameters controlled, for example, by the HNLMS.

[0107] In some embodiments, computer system 1700 may impose an is-equal-to knowledge- sharing link or an is-not-equal-to knowledge-sharing link in both directions between a pair of nodes.

[0108] The use of an is-equal-to knowledge-sharing link in both directions is also called “soft-tying” of the pair of nodes. The use of an is-not-equal-to knowledge-sharing link in one or both directions is also called “counter-tying” of the pair of nodes. In some embodiments, soft-tying and counter-tying links may be bi-directional, although the counter-tying links are asymmetrical.

[0109] In some embodiments, computer system 1700 may use is-equal-to soft-tying and / or is-not-equal-to counter-tying regularization on the weight parameters of one or more of the corresponding connections into a pair of homologous nodes. However, because the values of weight parameters are not data dependent, the knowledge sharing links between weights are also not data dependent.

[0110] Flat activation interval: An interval in an activation function that satisfies a specified flatness criterion, such as a limit on the magnitude for the derivative of the function within the interval or a limit on the difference between the maximum and minimum values of the function within the interval. The extreme case of a flat activation interval is an interval in which the function has a constant value throughout the interval.

[0111] Data exclusion: A process of excluding data in the training or deployment of a unit in a hybrid network based on a specified criterion.

[0112] Data switch: An element of a network that may selectively pass an activation or other incoming variable to only a specified subset of one or more destinations. In some embodiments, the specified subset may be the empty set.

[0113] Local data space: An n-tuple of variables in a hybrid network that are the input variables for a specified set of units and / or nodes. The variables of a local data space may be in an inner layer of the network. A local data space may also be called a “local input space” or a “local feature space.” A local data space may be an encoding of a larger set of variables.

[0114] Decision element: A specified interval in the range of a computable variable f(d) dependent on the input data d to a network, where a value of f(d) being in the specified interval is interpreted as the variable indicating that the data item d is in a specified set 318976815.3 17Docket No.230458PCT (detection) or that the data item is not in a specified set (rejection).

[0115] Decision element group: A set of one or more detection decision elements for which the specified target detection sets are disjoint. Computer system 1700 may interpret a discriminator as a decision element group comprising two intervals, each a decision element detector for one of the discriminator alternatives. Computer system 1700 may interpret a softmax set as a decision element group with each node in the softmax set as a detector for a target set disjoint from the others.

[0116] Holistic interpretation: A human understandable explanation of a node or unit in relation to other nodes and units and the whole system. Many of the techniques for improving sensibility also contribute to holistic interpretability and vice versa. For example, association of a node or unit with a named set is directly an aspect of holistic interpretability that also facilitates improving sensibility.

[0117] Substitute derivative function: A specified function of the input to the activation function that computer system 1700 uses for one or more specified data item in place of the actual derivative of the activation function. The HNLMS may specify the same substitute derivative function for a selected node for all data items or may specify different substitute activations for different data items. The HNLMS may change the specified substitute derivative functions during the training.

[0118] Template model: A specified computation designed to assign higher values for data items in a specific target set than for data items not in the target set while satisfying specified criteria for elementary sensibility. In an illustrative embodiment, the template model comprises input from a local or global data space, a specified norm in the data space, a specified central point for the target set in the data space, and an output value that is a function of the distance from the central point to an input data item as measured by the norm. A template model may be represented in a node, unit, or cell. Without loss of generality, in illustrative embodiments, computer system 1700 may represent a template model as a dedicated cell since a cell paired with a specified node or unit can represent the same computation as the node or unit comprising the computation of the cell.

[0119] Robust template model: A template model designed to satisfy specified sensibility criteria.

[0120] Having now provided various definitions, embodiments of the present invention are further described below. Figure 1 is a flow chart of an illustrative embodiment of an aspect of 318976815.3 18Docket No.230458PCT the invention. In the embodiment illustrated in Figure 1, computer system 1700 builds and trains a hybrid network. In terms of equivalent computations, the class of hybrid networks includes the class of neural networks as a strict subset.

[0121] In preferred embodiments, the process of building and training the hybrid network is a process of continual growth and improvement of the systems being built and trained with a plurality of training methods. In blocks 101 to 107, computer system 1700 modifies and grows the systems being developed before deployment. In blocks 108 to 114, computer system 1700 continues the growth and training during and after deployment. In various aspects of the invention, computer system 1700 may use a variety of processes to improve the sensibility of a system being developed. For the purpose of discussion, the processes of improvement are divided into two levels. Each level is associated with different criteria for assessing sensibility. Generally, the second level of sensibility involves more complex criteria for sensibility. In some embodiments, computer system 1700 may use a specific process for improvement in a level of sensibility other than the level in which that specific process has been discussed.

[0122] In block 101, computer system 1700 selects one or more base machine learning systems. In some embodiments, computer system 1700 may select a base machine learning system that is not represented as a network and use incremental growth to build a hybrid network. In some embodiments, computer system 1700 may select a partially trained or fully trained conventional neural network as a base system. A conventional feed-forward neural network is described below in connection with Figure 21A. In some embodiments, computer system 1700 may select a hybrid network as a base network. In the processes illustrated in Figure 1 and other figures, computer system 1700 may make modifications and additions to the base systems in a continual training process.

[0123] In some embodiments, computer system 1700 may co-train a plurality of networks. In some embodiments, computer system 1700 may co-train a diverse set of homologous networks comprising a diverse set of sensible hybrid networks and a diverse set of canary networks and, optionally, a diverse set of networks optimized for classification accuracy without regard to sensibility as explained in association with Figure 21 and block 516 of Figure 5.

[0124] In some embodiments, computer system 1700 may select a single base network.

[0125] If the base network is a conventional neural network, computer system 1700 may modify and grow the network to become a hybrid network. In some embodiments, computer 318976815.3 19Docket No.230458PCT system 1700 may select an empty network as the starting network, growing a sensible hybrid network from scratch. In some embodiments, computer system 1700 may grow a sensible hybrid network from scratch using a non-network or a network base system as a reference system for knowledge sharing and / or imitation learning. Imitation learning is described in U.S. Patent Nos.11,410,050 and 11,531,900, both titled “Imitation training for machine learning systems with synthetic data generators,” and published PCT application WO / 2021 / 194516, titled “Data-dependent node-to-node knowledge sharing by regularization in deep learning,” all of which are incorporated herein by reference in their entirety.

[0126] In some embodiments, computer system 1700 may use one or more reference networks as a reference for known or named sets.

[0127] In some embodiments, computer system 1700 may use human consultation to associate a name with a known set. In some embodiments, when computer system 1700 associates a name with a known set, computer system 1700 may then train one or more detectors for the known set to better match detection of the named set. Computer system 1700 may use a named-set detector in a reference system to train a detector in the current system by knowledge sharing and / or imitation learning. In imitation learning, an element in the system being trained is trained with a local training target to match the output of a specified element in the reference system. In some embodiments, computer system 1700 may use a unidirectional or a bidirectional transformation between a data space in the current network and a reference network in order to apply knowledge sharing and / or imitation learning. Human consultation is discussed further in association with block 414 of Figure 4. Unidirectional and bidirectional transformation of data spaces is discussed in association with Figure 14.

[0128] In block 102, in some embodiments, computer system 1700 optionally obtains and / or builds and trains one or more systems that are smaller or simpler than the current base system. For example, in some embodiments, computer system 1700 may specify a simpler system to facilitate potential human guidance and consultation, as discussed in association with block 414 of Figure 4. In some embodiments, a human consultant may specify experimental changes to the system. In some embodiments, specifying experimental changes in a simpler system may take less time and effort than in a more complex system. In some embodiments, computer system 1700 may follow specified design rules to make a simpler system easier for a human consultant to understand and control.

[0129] In some embodiments, computer system 1700 may specify a simpler network to 318976815.3 20Docket No.230458PCT reduce the amount of computation required for training. In some embodiments, computer system 1700 may specify a simpler network for which it is easier to design and train sensibility. In some embodiments, computer system 1700 may specify a simpler network for better holistic interpretability.

[0130] In some embodiments, computer system 1700 may work with one or more simpler systems in parallel with the current base system. In some embodiments, computer system 1700 may temporarily replace the current base system with a simpler system.

[0131] In some embodiments, these simpler systems may also be designed to generalize better from limited amounts of training data. The goal of these simpler systems is not to match the classification accuracy of the base systems selected in block 101. Rather the main goal is to be less vulnerable to making non-sensible mistakes. The vulnerability of a classifier system to non-sensible mistakes tends to be proportional to the number of input variables, so it is easier for computer system 1700 to make a simpler system with fewer input variables less vulnerable. In some embodiments, computer system 1700 may use one or more smaller, simpler systems to accelerate the training and use of a larger system.

[0132] In an image recognition task, an example of a smaller and simpler system is for computer system 1700 to preprocess the image to obtain a lower resolution image. In a speech recognition task, an example of a smaller and simpler system is for computer system 1700 to use fewer spectral frequencies and / or to compute fewer spectral frames per second of speech. In some embodiments, computer system 1700 may reduce the average number of spectral frames per second by using a variable frame rate. For example, if several successive spectral frames differ by less than a specified amount, computer system 1700 may replace the multiple frames with a single frame.

[0133] In a smaller, simpler system, computer system 1700 may use fewer categories in a classification task. More generally, computer system 1700 may use fewer, larger sets at each level of an ontology. In some embodiments, a larger set in the simpler system may be the union of sets in the ontology of the less simple system.

[0134] On the other hand, in the case of image recognition, in some embodiments, computer system 1700 may make use of the availability of the higher resolution image in analysis of an input data item for the smaller, simpler base system. For example, in the alignment of a data item to a mereology graph, as discussed in association with Figure 12, computer system 1700 may use the higher resolution image to verify a tentative alignment of a region in the low- resolution image with a specified part in the mereology. In this example, the system 318976815.3 21Docket No.230458PCT analyzing the higher resolution image is a “simpler” system in the sense of block 102 of Figure 1.

[0135] In block 103, computer system 1700 begins or resumes the process of continual growth and improvement of the current network (i.e., the base system(s) selected at block 101, optionally in combination with the simpler system selected at block 102 if one is selected at block 102). In block 101 and / or block 102, computer system 1700 may have replaced a previous base network with a new base network based on the validation testing in block 106 or block 111.

[0136] In some embodiments, computer system 1700 may use incremental growth (504 of Figure 5) to improve classification performance and sensibility by training without using any back propagation, neither back propagation of derivatives (506 of Figure 5) nor back propagation of labeled data examples (510 of Figure 5). For example, in some embodiments, computer system 1700 may use constrained optimization (524 of Figure 5 and Figure 6) to train each new node incrementally added to the network without use of back propagation. As long as there are any remaining errors on the training data, computer system 1700 may use incremental growth combined with constrained optimization to reduce the number of errors.

[0137] In some embodiments, computer system 1700 may add elements to a network as part of various embodiments of hybrid training, such as data delegation (Figure 11 and block 518 of Figure 5), splitting one or more nodes (519 of Figure 5), adding additional output values to an element, training distinct sets in a discrimination (523 of Figure 5), adding a local autoencoder to the network or simply adding one or more elements for some other purpose.

[0138] In some embodiments, computer system 1700 may add a local autoencoder to a network to support improved sensibility (Figures 2, 9, and 10), as a local data space (Figure 3C and blocks 411 of Figure 4 and Figure 9), or as a data generator (514) of Figure 5.

[0139] In some embodiments, computer system 1700 may create a plurality of networks from the original base network and may continue to grow and improve each of the plurality of networks. For example, computer system 1700 may use one or more simpler systems specified in block 102 in addition to one or more current base systems. As another example, computer system 1700 may develop one or more canary networks in parallel with the development of the current base network. Canary networks are designed to be vulnerable to adversarial attacks and other perturbations to the input as a means of detecting and diagnosing such disturbances. Canary networks are discussed in association with block 415 of Figure 4. 318976815.3 22Docket No.230458PCT

[0140] In some embodiments, in block 103, computer system 1700 may build a hybrid network from scratch.

[0141] In blocks 104 and 105, computer system 1700 makes modifications to the base network to improve the network’s sensibility in a hierarchy of two levels of sensibility and a plurality of training methods and techniques for improving sensibility. Each level of sensibility has different criteria. Computer system 1700 may use different processes, models, and system designs for improvement in each level. Illustrative processes and models for each level of sensibility are discussed in more detail in association with Figures 2, 4, 5, and other figures. However, in some embodiments, computer system 1700 may also use an improvement process or model in a level other than the level with which it is discussed.

[0142] In some embodiments, in block 104, computer system 1700 may improve elementary sensibility (Figure 2 and block 405 of Figure 4), do active flattening (block 406 of Figure 4), perform hybrid training (Figure 5 and block 407 of Figure 4), find the best location in the network for a piece of knowledge (block 408 of Figure 4), and / or do data selective training (block 409 of Figure 4). In some embodiments, computer system 1700 may use randomization training (block 520 of Figure 5) and randomized activation (418 of Figure 4) in blocks 104, 105, 106, 109, and / or 110 of Figure 1 to improve sensibility, robustness, and / or classification performance.

[0143] An aspect of preferred embodiments of the invention is a hybrid network learning management system (HNLMS), which comprises a cooperative association of a team of human experts and one or more AI systems to develop tools and models to help computer system 1700 improve the sensibility, classification performance, and holistic interpretability of the system being developed. In some embodiments, the HNLMS may guide the training of the system being developed and may judge its sensibility.

[0144] An illustrative criterion for first level sensibility of a detector or discriminator is that, for any data item in an empirical acceptance set and in the interior of the target acceptance set, a change in the input with an ^^^^^ ^^ ^^ ^^ < ^^, for a specified ^^, should not cause the data item to no longer be in the empirical acceptance set. In other words, a nearly imperceptible change in an input data item should not cause the system to make a mistake that it did not make before the change. Any successful ^^^adversarial attack violates first level sensibility. Techniques for designing ^^^adversarial attacks are well known to those skilled in the art of developing deep neural networks. 318976815.3 23Docket No.230458PCT

[0145] An important subset of first level sensibility is called “elementary sensibility.” Elementary sensibility (405 of Figure 4) has criteria that can be checked for each node, unit, or internal variable. For elementary sensibility, computer system 1700 makes changes in the base system to improve elementary sensibility of each node or unit.

[0146] Informally, the levels of sensibility differ in the degree to which the HNLMS participates in the development done by computer system 1700 of techniques in each level and / or in judging the sensibility of the developed system.

[0147] Level one techniques require the least participation by the HNLMS during the development. The sensibility of a system modified by level one sensibility improvements are also the easiest for computer system 1700 to evaluate objectively, with the HNLMS mainly controlling hyperparameters in the sensibility criteria.

[0148] In block 105, in some embodiments, computer system 1700 may modify the current network to improve level two sensibility. In level two techniques, computer system 1700 may utilize more guidance from the HNLMS during development and during evaluation (414 of Figure 4).

[0149] As discussed in association with Figure 4, in block 105 of Figure 1, in some embodiments computer system 1700 may analyze and improve decision boundaries (block 410 of Figure 4) and / or create and train local normed spaces (Figure 9 and block 411 of Figure 4).

[0150] In some embodiments, in block 105 of Figure 1, computer system 1700 may compute attributes and other variables the computer system 1700 may store in cells, as discussed in association with block 412 of Figure 4.

[0151] In block 105 of Figure 1, computer system 1700 may also build and train hidden state space models under direction from, for example, the HNLMS, as discussed in association with block 413 of Figure 4 and Figure 7. In some embodiments, computer system 1700 may also use the hidden state space models in active classification as discussed in association with block 109 of Figure 1 and blocks 403, 416, and 417 of Figure 4.

[0152] Computer system 1700 may specify and / or change the states of a hidden state space model and / or associated learned parameters or hyperparameters under control of, for example, the HNLMS. In some embodiments, computer system 1700 may specify and / or change the states of a hidden state space model and / or associated learned parameters or hyperparameters based on human consultation, as discussed in association with block 414 of 318976815.3 24Docket No.230458PCT Figure 4.

[0153] In block 105 of Figure 1, computer system 1700 may use human consultation in verifying the sensibility of discriminator and / or classifier decision boundaries, as discussed in association with block 414 of Figure 4.

[0154] To improve sensibility, in blocks 105 and 106 of Figure 1, computer system 1700 may analyze decision boundaries (410 of Figure 4), construct local normed spaces (411 of Figure 4), computed attributes and cell variables (412 of Figure 4), construct and train hidden state space models (Figure 7 and block 413 of Figure 4), construct and train active defense structures, optionally with data switches (416 of Figure 4 and 803 of Figure 8), perform active alignment (Figures 12 and 19 and block 417 of Figure 4), train with randomized activation (418 of Figure 4), construct and train robust template models (Figure 10 and block 419 of Figure 4) and / or use hybrid conditional training (Figure 13 and block 512 of Figure 5).

[0155] In addition to improving sensibility, some of the illustrative processes and models may improve the holistic interpretability of nodes and units in the system. Some of the illustrative processes and models may improve the performance on an assigned classification or regression task. In one aspect of the invention, computer system 1700 may reformulate a regression task as a classification task. Without loss of generality, in this disclosure, the term “classifier” is used to refer to a system for which the task may be either a classification task or a regression task.

[0156] The phrase “neural network,” as described above, is used to refer to a directed network comprising a set of nodes and a set of directed connections between ordered pairs of nodes. The phrase “neural network” refers to the commonly accepted concept that is well known to those skilled in the art of training and using neural networks.

[0157] The phrase “hybrid network,” as described above, refers to a generalization of a neural network comprising more complex elements, herein called “units.” A unit may have multiple output values and may comprise multiple internal nodes and connections, as illustrated in Figure 3A. A unit may also comprise special elements herein called “cells.” On the other hand, in a hybrid network, a unit may consist of only a single neural node, so any conventional neural network is also a simple hybrid network.

[0158] The modifications to the base network made by computer system 1700 in blocks 104 and 105 may comprise changing the activation functions of one or more selected nodes. The modifications may include converting one or more nodes to a more complex structure called 318976815.3 25Docket No.230458PCT a “unit.” The modification may comprise adding nodes and units to the network. In some embodiments, the modifications may comprise creating and adding one or more cells to the network. Computer system 1700 may add a cell to a unit or may add a cell to the network external to any unit.

[0159] A cell in a hybrid network is distinct from a node. A cell may comprise the values of one or more variables computable by computer system 1700. For example, computer system 1700 may save in a cell the output value of a selected element of the network for the current input data item and / or from the output value of a selected element of the network for a previous data item. Each cell may comprise or be associated with an arbitrary stored program to be executed on computer system 1700. For example, computer system 1700 may execute a serial computation associated with a cell to compute a logical inference or a probabilistic inference. A cell may comprise one or more incoming data communication links and / or one or more outgoing data communication links. Data links are distinct from neural network connections. A data link only transmits data and does not have an associated “weight” parameter. A data link may be unidirectional or bidirectional.

[0160] The data transmitted by computer system 1700 on a data link from a first cell to a second cell may comprise any value that computer system 1700 may compute from the values stored in the first cell.

[0161] Computer system 1700 may also transmit data via a data link from a neural node to a cell. For example, the data transmitted from a neural node to a cell may be the input to or the output from the activation function of the neural node. In some embodiments, the data transmitted from a neural node to a cell may be the value of a back propagated derivative computed by computer system 1700 during computation of a gradient by back propagation. In some embodiments, the back propagated derivative may be from a substitute local derivative (509 of Figure 5).

[0162] Computer system 1700 may also transmit data via a data link from a cell to a neural node. The data transmitted by computer system 1700 on a data link from a cell to a neural node may be any value that computer system 1700 may compute from the values stored in the cell. In some embodiments, computer system 1700 may use the received data value as an additional input connection to the receiving node with a connection weight of 1.0. In preferred embodiments, computer system 1700 does not back propagate derivatives along a data link from a cell. However, if desired, in some embodiments, computer system 1700 may achieve a similar effect by creating a second node to receive data from a cell and then 318976815.3 26Docket No.230458PCT connecting the second node to the first node by a neural network connection through which computer system 1700 may back propagate derivatives.

[0163] The details of the processes used in each level (blocks 104 and 105) are discussed in more detail in association with Figures 2, 4, 5, and other figures.

[0164] In block 121, in some embodiments, computer system 1700 may train the network for participation in joint human + AI activities, in which one or more humans play a sufficient role to contribute some amount of common sense. An example of a joint + AI activity is the HNLMS. Figure 21 discusses additional joint activities, including the production of creative works. Figure 21 also discusses joint educational activities.

[0165] In block 106, computer system 1700 trains the modified network and tests the trained network on validation data that has been set aside from the training data. Illustrative embodiments of the process of training a hybrid network, called “hybrid training,” are discussed in association with Figure 5 and other figures.

[0166] In some embodiments, in block 106, computer system 1700 may perform histogram analysis (Figure 15 and block 507 of Figure 5), back propagate derivatives (506 of Figure 5), create a low dimension local data space and build low dimension models (517 of Figure 5), perform data delegation and data exclusion (block 420 of Figure 4, block 518 of Figure 5, and Figures 10 and 11), determine local targets (508 of Figure 5), use substitute derivative functions (509 of Figure 5), back propagate labeled data (Figure 18 and block 510 of Figure 5), imitate another network (511 of Figure 5), perform conditional hybrid training (Figure 13 and bock 512 of Figure 5), perform empirical training (521 of Figure 5), generate more data, optionally with human guidance (514 of Figure 5), build homologous networks (Figure 21 and 516 of Figure 5), do randomized training (520 of Figure 5), empirically compute weights of individual items to estimate their reliability (522 of Figure 5), create distinct sets to better represent combinations of known sets (523 of Figure 5), and / or use constrained optimization (Figure 6 and block 524 of Figure 5)

[0167] If the validation test done by computer system 1700 in block 106 meets a specified acceptance criterion, then computer system 1700 replaces the previous base network with the network as modified by computer system 1700 in blocks 104, and 105. In some embodiments, computer system 1700 may save the new base network or selected subnetworks in a network repository.

[0168] In some embodiments, computer system 1700 may compare the performance of the 318976815.3 27Docket No.230458PCT current base system on validation data with the performance of a simpler system. In some embodiments, computer system 1700 may compare the performance of the current system on data from an adversarial attack on validation data to the performance of one or more canary systems. In some embodiments, based on analysis of these comparative results, computer system 1700 may make experimental changes in the current system and retest, preferably on new validation data. In some embodiments, computer system 1700 may request human consultation, as discussed in association with block 414 of Figure 4.

[0169] In block 107, computer system 1700 checks a stopping criterion for the modifications and training being done in the loop from block 101 to block 107. If the stopping criterion is met, computer system 1700 proceeds to block 108. Otherwise, computer system 1700 returns to block 101 to continue modifying the current base network.

[0170] In block 108, computer system 1700 receives an item to be classified. In some embodiments, the phrase “an item to be classified” may include an item for which a regression value is to be computed.

[0171] In block 109, in some embodiments, computer system 1700 may perform a process herein called “active classification,” or “active sensible classification.” In preferred embodiments, during active classification, computer system 1700 may make changes to the network and / or may do additional computations other than neural network activation after receiving a data item to be classified. Computer system 1700 may customize these additional computations to the received data item.

[0172] Active sensible classification comprises the computation of the activation values of the neural nodes in the network, a process which is called “inference” in neural networks. However, in illustrative embodiments, “active sensible classification” may comprise additional processes that are distinct from neural network inference.

[0173] In block 109, computer system 1700 may perform diagnosis and defense against the specific data item received in block 108. For example, computer system 1700 may classify the received item using diverse unprotected canary networks and diverse robust networks to analyze the patterns of difference in the responses, as discussed in association with block 415 of Figure 4.

[0174] In active classification with a hybrid network in block 109, computer system 1700 may do serial computations in the cells after the item to be classified has been received. This ability enables additional capabilities for a hybrid network. 318976815.3 28Docket No.230458PCT

[0175] For example, in active sensible classification, computer system 1700 may make changes in the hybrid network, after the item to be classified has been received, as an active defense (416 of Figure 4 and 803 of Figure 8), which enables computer system 1700 to make the network sensible for the specific item received. In some embodiments, computer system 1700 may build the hybrid network to have data switches that effectively reconnects the hybrid network in a configuration that is specifically designed to avoid a non-sensible response for the specific data item received in block 108.

[0176] In some embodiments, in block 109, computer system 1700 may compute an alignment of the data item to be classified to a model and / or to other data examples (Figures 12 and 19 and block 417 of Figure 4). In some embodiments, computer system 1700 may use cells in the network to store information used in computing the alignment. In some embodiments, computer system 1700 may use a hidden state space model (Figure 7 and block 413 of Figure 4) in computing the alignment. In some embodiments, computer system 1700 may retrieve example alignments or other information from a repository in computing an alignment for the data item to be classified. In some embodiments, computer system 1700 may store, for future use, information computed in aligning the data item to be classified.

[0177] As another example, computer system 1700 may use a set of cells to model a hidden stochastic process, as discussed in association with Figure 7 and block 413 of Figure 4. With a set of cells in a hybrid network, computer system 1700 may do a recurrent computation even though the network of neural nodes is a non-recurrent network.

[0178] In block 110, in some embodiments, computer system 1700 may continue training after a machine learning system is deployed. In some embodiments, computer system 1700 may continue to acquire new data while a system is deployed. In some embodiments, computer system 1700 may acquire data from other systems that have been deployed. In some embodiments, computer system 1700 may continue to train a deployed system using data acquired during the development and training of new systems.

[0179] In some embodiments, in block 110, computer system 1700 may continue modifying and growing the network to improve classification performance, sensibility, and / or holistic interpretability.

[0180] In block 110, in some embodiments, computer system 1700 may compute incremental training using the item received for classification in block 108. Since the item was received for classification, unlike for training data, the correct classification might not be known. In 318976815.3 29Docket No.230458PCT this situation, in some embodiments, computer system 1700 may do semi-supervised training, that is, after classifying the received item, computer system 1700 may do incremental training on the item as if it were training data labeled with the classification computed during the classification.

[0181] However, as is well known to those skilled in the art of semi-supervised training, although semi-supervised training often works well, sometimes it may fail catastrophically.

[0182] In preferred embodiments, in block 110, computer system 1700 may perform extra processes to improve the reliability of semi-supervised training. For example, computer system 1700, may use the data switches mentioned in association with block 109 to construct a virtual ensemble not only to improve the performance of the classification in general but more specifically to detect and diagnose that the classification of the received item may be unreliable. If computer system 1700 detects that the classification of an item may be unreliable, computer system 1700 may skip that item in semi-supervised training.

[0183] In some situations, during deployment, computer system 1700 may know the correct classification from the interaction with the end-user, who may correct errors made by the system. In some cases, computer system 1700 may not know the correct answer but, from the reaction of the user may know that the computed classification is incorrect or unreliable.

[0184] In block 111, in some embodiments, computer system 1700, may perform iterative training using the accumulated data acquired from multiple passes through the loop from block 108 to 112. In some embodiments, computer system 1700 may then validate the performance of the trained system on a set of labeled validation data that computer system 1700 has set aside from the set of training data. If the validation test satisfies a specified acceptance criterion, computer system 1700 may replace the current base network with the newly validated network.

[0185] In block 112, computer system 1700 checks a criterion for stopping or pausing the process of blocks 108 to 112. If the stopping criterion is satisfied, computer system 1700 proceeds to block 114. Otherwise, computer system 1700 returns to block 108 to process more items to be classified.

[0186] In block 113, computer system 1700 may determine whether to add more data to the training data and may determine how much data to select in a specific region. In some embodiments, computer system 1700 may begin training with a selected sample of the data and gradually add more training data as the system grows. In some embodiments, in which 318976815.3 30Docket No.230458PCT there is a large amount of data, the data might not be uniformly distributed among regions of interest. In some embodiments, computer system 1700 may selectively add sample data in a region in which the current sampling is sparse. In preferred embodiments, compute system may keep track of the relative frequency of sampling and properly adjust any estimates of a priori or a posteriori probability.

[0187] As an example, in some embodiments, computer system 1700 may use selective sampling in histogram analysis, which is discussed in association with Figure 15. In some embodiments, computer system 1700 may use selective sampling in any procedure that splits the data, for example: (1) data switching of activation intervals (209 and 211 of Figure 2), (2) interval dependent training (406, 407, 409, 410, and 416 of Figure 4), (3) node splitting (519 of Figure 5), and (4) histogram analysis (Figure 15 and 507 of Figure 5).

[0188] In some embodiments, computer system 1700 may use selective sampling in other situations that use additional data, such as, (5) back propagation of data (Figure 18 and block 510 of Figure 5), (6) adjusting data delegation and exclusion norms (block 420 of Figure 4, block 518 of Figure 5, and Figures 10 and 11), (7) generation of data with human guidance (514 of Figure 5), and (8) randomized training and diagnosis (520 of Figure 5).

[0189] In block 114, computer system 1700 checks whether to resume training and growth of the current base network as modified in blocks 103 to 110 as validated in blocks 106 and 111. If so, computer system 1700 returns to block 102. Otherwise, computer system 1700 proceeds to block 115.

[0190] In block 115, computer system 1700 checks a stopping criterion. If the stopping criterion is satisfied, computer system 1700 exits the process illustrated in Figure 1. Otherwise, computer system 1700 returns to block 101.

[0191] In some embodiments, if additional training data has been acquired, in block 101, computer system 1700 may resume the training of the current updated base systems. In some embodiments, computer system 1700 may select one or more new base systems.

[0192] Figure 2 is a flow chart of illustrative embodiments of processes for enhancing elementary sensibility in an aspect of the invention. As shown in blocks 401 and 405 of Figure 4, elementary sensibility is one aspect of level one sensibility. As shown in Figure 2, there are multiple aspects to elementary sensibility.

[0193] In block 201 of Figure 2, computer system 1700 may modify a regression-type output to be represented as a sensible classification-type output. A regression-type output is a 318976815.3 31Docket No.230458PCT continuous-valued output value from a network or from a unit in which the output value is a parametric function of the input values and in which the parameters are trained to optimize a specified measure of fit between the output of the parametric function and the target values in a set of training data. The regression-type could be, for example, a linear regression, a logistic regression, or some other suitable type of regression.

[0194] In some embodiments, in block 201, computer system 1700 may replace the continuous valued output by a piecewise constant function. In the typical case in which the parametric continuous-valued function is monotonic, computer system 1700 may replace the parametric function with a step function.

[0195] In some embodiments, in block 201, computer system 1700 may replace the parametric function with a vector of one or more finite discrete-valued variables. The vector of discrete variables may be called a vector embedding of the values of the continuous-valued function. Computer system 1700 may compute the vector embedding as the bottleneck layer of an autoencoder. In some embodiments, computer system 1700 may impose a sparsity constraint or regularization on the bottleneck layer. In some embodiments, computer system 1700 may use a hybrid parametrically controlled autoencoder with some specified features, as discussed in association with Figure 9. In some embodiments, computer system 1700 may use such a discrete-valued vector embedding for multiple regression of two or more continuous-valued variables. In some embodiments, computer system 1700 may use such a discrete-valued vector embedding for multiple regression of a continuous-valued data space.

[0196] With either the piecewise constant function or the vector embedding, computer system 1700 may train a neural network or a hybrid network to imitate the continuous-valued function or the continuous-valued vector to any desired degree of precision, since computer system 1700 may use the continuous-valued function to compute the target value for an unlimited number of examples of input values, making available an unlimited quantity of training data.

[0197] However, in some embodiments, computer system 1700 may limit the number of intervals in the piecewise constant function or the discrete vector space of the embedding in order to better satisfy the criteria for sensibility.

[0198] In block 202, computer system 1700 may replace one or more unbounded variables with bounded variables. For example, computer system 1700 may replace one or more unbounded activation functions with bounded activation functions. In some embodiments, computer system 1700 may simply impose as constraints a minimum value and a maximum 318976815.3 32Docket No.230458PCT value for the output of the activation function. In some embodiments, computer system 1700 may replace the activation function with a new activation function that approaches limiting values asymptotically, which computer system 1700 may change to a step function later in the training. In some embodiments, computer system 1700 may limit global or local data space values. In some embodiments, computer system 1700 may limit the values stored in and / or transmitted by a cell. In some embodiments, computer system 1700 may limit the value of variables in a local data space.

[0199] In some embodiments, with a trained or partially trained network, computer system 1700 may use the minimum and maximum values observed for the activation of a node in the training data to set the limits for the bounded activation function of the node, perhaps allowing some extra margin for the values that might be needed for new data.

[0200] In some embodiments, computer system 1700 may implement a semi-automated process with a controlled amount of human consultation to specify or verify the limits, as discussed in association with block 414 of Figure 4. In some embodiments, computer system 1700 may use empirical training (521 of Figure 5) to determine the limits.

[0201] In some embodiments, computer system 1700 may replace a node or unit that has an unbounded activation function with one or two detectors or with a discriminator, as discussed in association with blocks 211, 212, and 213.

[0202] In block 203, computer system 1700 may replace the activation function of each of one or more nodes that have non-monotonic activation functions with a monotonic activation function or a modified monotonic function. For example, computer system 1700 may specify an activation function that is monotonic on a specified interior interval rather than monotonic over the full domain of the activation function.

[0203] In some embodiments, computer system 1700 may specify a non-monotonic activation function that is monotonic within a specified interior interval but that computer system 1700 modifies outside the specified interval. For example, for an activation function that computer system 1700 characterizes as a discriminator between a set S1 and a set S2, computer system 1700 may specify an activation function that has a maximum value for the activation value corresponding to the mode in the probability distribution for set S2 and a minimum value for the activation value corresponding to the mode in the probability distribution for set S1. Computer system 1700 may specify an activation function that is monotonic in the interval between the minimum value and the maximum value. 318976815.3 33Docket No.230458PCT

[0204] However, if, for example, the mode of either set S1 or set S2 is at an interior point of the data space, computer system 1700 may specify an activation function that has a local maximum for S2 and a local minimum for S1. In some embodiments, computer system 1700 may specify an activation function that outside the monotonic interval between the minimum and the maximum is equal to or asymptotic toward a specified out-of-domain background value, such as used in data exclusion (204 of Figure 2, 518 of Figure 5, and Figure 11). In some embodiments, computer system 1700 may specify an activation function that is monotonic between the background value and the value of the minimum or maximum. An activation function that is monotonic on the interval between a unique minimum value and a unique maximum value and monotonic outside that interval is herein called a “standard discriminator function.” In a standard discriminator function, either the minimum value or the maximum value may occur at an end point (or the limit at infinity), so the monotonic interval may be the whole domain or a half-open interval.

[0205] In some embodiments, in block 203, computer system 1700 may convert the activation for any node that computer system 1700 characterizes as a discriminator to become a standard discriminator function.

[0206] For a node with a standard discriminator function and specified threshold value T between the minimum value and the maximum value, computer system 1700 may determine if the node has made an implicit error on a specific data item.

[0207] Implicit error: In some embodiments, for a node with a standard discriminator activation function f(x) and a specified discrimination threshold T, where x(d) is a function of the input data d, then computer system 1700 may designate that the node has made an implicit error, for an activation value x(d) in the interval between the minimum and the maximum, if the sign of (x(d))*(f’(x(d) – T)) is the same as the sign of the back propagated derivative of an error measurement objective function that is to be minimized. In some embodiments, computer system 1700 may reverse the sign test for an activation outside the interval between the minimum and the maximum. In some embodiments, computer system 1700 may apply no test for data that has been delegated or excluded. Computer system 1700s reverses the sign test if the derivative is of an objective function to be maximized.

[0208] In some embodiments, computer system 1700 may determine that the node has made a close call on the data item d if the magnitude of |T – act(d)| is less than a specified multiple of the magnitude of the back propagated derivative, where “act(d)” represents the activation value of a node for a datum d. A close call may be either a close call with an implicit error or 318976815.3 34Docket No.230458PCT a close call with an implicitly correct answer.

[0209] In some embodiments, computer system 1700 may add regularization penalties, such as knowledge sharing regularizations, soft-tying, and counter-tying to the back propagated derivatives in determining whether a node with a standard discrimination activation function has made an implicit error. Soft-tying is described in U.S. Patent No.10,839,294, titled “Soft- tying nodes of a neural network,” and counter-tying is described in U.S. Patent No. 11,151,455, titled “Counter-tying nodes of a nodal network,” both of which are incorporated herein by reference in their entirety. Data-dependent node-to-node knowledge sharing by regularization is described in published PCT application WO / 2021 / 194516 A1, titled “Data- dependent node-to-node knowledge sharing by regularization in deep learning,” which is also incorporated herein by reference in its entirety.

[0210] In some embodiments, computer system 1700 may determine that a node has made an explicit error if the node is being trained to a known set and the data item d has activation x(d) that is on the wrong side of the discrimination threshold T. In some embodiments, when such an explicit error criterion is known, computer system 1700 may use the explicit error criterion rather than the implicit error criterion.

[0211] In some embodiments, computer system 1700 may ignore relatively small deviations from monotonicity, such as the dip in a Gaussian error linear unit (GELU). The GELU activation function is well known to those skilled in the art of neural networks. In some embodiments, computer system 1700 may use a replacement activation function that is monotonic except specified dips such as in the GELU function. In some embodiments, for a detector unit, computer system 1700 may use a center-surround function, in which the function has a dip in value for activations close to but not in the acceptance region. Computer system 1700 may make the function value in this dip less than the function value for activations further from the acceptance region as well as from the values in the acceptance region.

[0212] In some embodiments, computer system 1700 may partition the domain of a node with non-monotonic activation function into alternating intervals of monotonically increasing and monotonically decreasing values. In some embodiments, computer system 1700 may create a new node for each interval.

[0213] In some embodiments, computer system 1700 may create a node for each pair of a monotonically increasing interval followed by a monotonically decreasing interval to create one or more nodes with unimodal activation functions. In some embodiments, computer 318976815.3 35Docket No.230458PCT system 1700 may replace a node with a unimodal activation function with a robust template unit, such as illustrated in Figure 10.

[0214] In some embodiments, computer system 1700 may replace an activation function with a plurality of local maxima with a plurality of robust template units.

[0215] In some embodiments, computer system 1700 may partition the domain of a discriminator node into a first interval in which a local minimum in the activation function represents detection of a first target set and a second interval in which a local maximum in the activation represents detection of a second target function. In some embodiments, computer system 1700 may create a first interval for a local maximum and a second interval for a local minimum. In some embodiments, computer system 1700 may replace the discriminator node with a unit comprising a detector for the first target set, a detector for the second target set, and an element that computes a discrimination score from the two detector scores. In some embodiments, for each of the target sets, computer system 1700 may train a template model as a detector of the target set.

[0216] In some embodiments, computer system 1700 may create a unit in which a node with a non-monotonic activation function is replaced by a unit with multiple monotonic or unimodal activation functions, separating the computation of the affine sum of the inputs from the computation of the activation functions, with a data switch in between. In some embodiments, computer system 1700 may switch any incoming data item to the monotonic or unimodal activation function corresponding to the interval for the incoming data item. Such a structure within a unit is illustrated in Figure 3A.

[0217] In some embodiments, computer system 1700 may replace the node with the non- monotonic activation function with a set of nodes with the activation function of each node being constant outside a specified interval and monotonic or unimodal within the interval. In some embodiments, computer system 1700 may initialize the incoming connections to each node to copy the incoming connections of the node being replaced. In some embodiments, computer system 1700 may then train the weights on the new connections separately from the weights of the connections to the original node. In some embodiments, computer system 1700 may tie or soft tie one or more of the weights on corresponding connections.

[0218] In block 204, in some embodiments, computer system 1700 may implement data exclusion and / or data delegation for detector elements and discriminator elements. In some embodiments, computer system 1700 may implement data trimming, limiting the detection region, and / or data exclusion. In some embodiments, computer system 1700 may adjust the 318976815.3 36Docket No.230458PCT limits for data delegation, data exclusion and / or trimming based on empirical training (521 of Figure 5).

[0219] In some embodiments, computer system 1700 may use data delegation to improve the performance of an element by limiting the training to a proper subset of the training data.

[0220] In elementary statistical analysis, a data item may be dropped from the training data as being an outlier. In robust statistics, a substantial fraction of the data may be dropped from the training the sufficient statistics of the parameters in a parametric probability distribution. Typically, in training a neural network, for every training data item, a feed forward computation is performed that computes the activation of every node in the network and a back propagation of derivatives is computed to update every connection into every node.

[0221] However, in a large neural network or a large hybrid network, the situation is more complicated. The input that one node received from another node for a specified data item may change as the weights in the network are updated during training. Whether a data item is an outlier for the first node may change.

[0222] In some embodiments of the invention, computer system 1700 may build redundancy into the network such that having delegated a data item that is no longer an outlier of the first node does not necessarily degrade performance.

[0223] In block 205, in some embodiments, computer system 1700 may replace the activation function of one or more selected nodes with an activation function for which the change in the value of the activation function in one or more selected intervals is less than in the activation function being replaced. In some embodiments, computer system 1700 may make such a change in an activation function to continue training a selected node by back propagation of derivatives, but later in the training may change the activation function to a piecewise constant function, as described in association with block 206.

[0224] In block 206, in some embodiments, computer system 1700 may change the activation function of one or more selected nodes to piecewise constant functions. Preferably, computer system 1700 specifies a piecewise constant function that satisfies a specified criterion for approximation of the selected function being replaced. For example, for each constant interval in the piecewise constant function, computer system 1700 may set the value of the piecewise constant function to the value of the selected function averaged over the interval. In preferred embodiments, computer system 1700 may replace a monotonic activation function, or a monotonic interval in any function, with a monotonic step function. 318976815.3 37Docket No.230458PCT

[0225] In some embodiments, computer system 1700 may make the value of the piecewise constant function in a specified interval a hyperparameter, which computer system 1700 may change during the training. In some embodiments, computer system 1700 may make the value of the piecewise constant activation function a learned parameter, which computer system 1700 may train using hybrid training methods such as discussed in association with Figure 5. For example, computer system 1700 may train such a parameter using empirical training.

[0226] In block 207, in some embodiments, computer system 1700 may specify a substitute derivative function for a node. An illustrative example of a substitute derivative function is shown in Figure 3C.

[0227] In block 208, computer system 1700 may replace a selected node with a plurality of nodes. One example was discussed in association with block 203. Computer system 1700 may replace a node that has a non-monotonic activation function with a set of nodes with one node for each monotonic interval in the non-monotonic activation function.

[0228] As another example, in block 208, computer system 1700 may replace a node with two or more nodes or with a unit comprising two or more nodes. For example, if an interval of the activation function of a specified node is associated with a known set, computer system 1700 may create a unit with a two or more output values, and a node trained to detect data items in the known set and second node trained to detect data items not in the known set.

[0229] In some embodiments, computer system 1700 may replace a node that discriminates between two known sets with two new nodes or add two new nodes, with one new node trained to detect one of the known sets and the second node trained to detect the second known set.

[0230] In some embodiments, in each of the cases in which computer system 1700 creates two new detector nodes, computer system 1700 may create a unit comprising the two new detector nodes and comprising one or both of two new nodes. Computer system 1700 may create one additional node to detect data items that are not in either of the two known sets and a second additional node to directly detect data items that are in the intersection of the two known sets.

[0231] Note that a node that is directly trained on the task of detecting data items that are in the intersection of the two sets or in the intersection of their complements will not necessarily agree with detections of the individual detectors since generally each of the detectors will 318976815.3 38Docket No.230458PCT have a non-zero error rate and the errors may be different under the different objectives. In addition, in some embodiments, computer system 1700 may train the new detectors with a different trade-off between precision and recall than used for the known set detectors. In any case, the two new detectors provide separate outputs to the unit to indicate directly to nodes and units in higher layers of the hybrid network whether a data item near the decision boundary of a discrimination of the two detectors is an equally good match for both detectors, herein called a “BOTH” detector, or an equally poor match to both detectors, herein called a “NEITHER” detector. Computer system 1700 may use the indication of BOTH or NEITHER, as a useful distinction for a higher-level node or unit receiving connections from the discriminator unit. This information is not available from the output of a single node discriminator.

[0232] As another example, in block 208, computer system 1700 may replace a node with two or more nodes or with a unit comprising two nodes, where one of the new nodes is trained to detect a known set and the second new node is trained to detect a distinct known set. In some embodiments, computer system 1700 may add a third node comprising incoming connections from the two detector nodes and, optionally, additional incoming connections. Computer system 1700 may train the third node as a discriminator of the two known sets. For example, the activation of the third node may comprise the difference between the scores of the two detector nodes or a smoothed monotonic function of the difference between the scores of the two nodes. The two detector nodes may be newly created nodes that computer system 1700 may initialize from two intervals of the node being replaced. Computer system 1700 may further train the unit or the three-node discriminator to discriminate the two known sets.

[0233] In block 208, computer system 1700 may also replace a node having a monotonic activation function and one or more feature-like intervals. A “feature-like” interval is an interval in which the maximum value in the interval is larger than the minimum value in the interval and for which, for example, the HNLMS has determined that replacing the interval with a constant would degrade performance by more than a specified amount. The feature- like interval may comprise the entire range of the node, in which case the node may be called a “feature” node.

[0234] In some embodiments, computer system 1700 may treat the extreme values near the ends of the feature-like intervals and / or the values beyond the extremes of the feature like interval as detectors. 318976815.3 39Docket No.230458PCT

[0235] In this case, in some embodiments, computer system 1700, controlled by, for example, the HNLMS may choose one or more of several options for the treatment of the feature-like interval: (1) Computer system 1700 may replace the feature-like interval with a unit comprising one or more of the following detectors, which preferably are sensible in the sense described in association with block 212 of Figure 2: a. A sensible detector for each extreme of the feature-like interval b. A sensible detector to detect that a data item is in a “boundary region” in which is it is not clear which, if either of the extreme detectors has correctly made a detection or a rejection; c. Two sensible detectors to distinguish two reasons for uncertainty about the extreme detectors: i. Both detectors have scores above a specified value ii. Neither detector has a score above a specified value d. Two or more sensible detectors to detect clusters within the boundary region. (2) Computer system 1700 may replace the node with multiple nodes, splitting the feature-like interval. a. Computer system 1700 may create two or more sensible detectors to detect clusters in a detection in a specified interval in the activation function. (3) Computer system 1700 may replace the node with multiple step functions with different constant intervals, such as discussed in association with block 211 of Figure 2, block 416 of Figure 4, and block 803 of Figure 8.

[0236] As another example, in block 208, in some embodiments, computer system 1700 may replace a single node with a plurality of nodes for redundancy. In this example, computer system 1700 may initialize each of the plurality of new nodes to have identical connections and identical weights on their connections as the single node being replaced. Computer system 1700 may then train the network, including the plurality of new nodes allowing the weights of connections incoming to each node copy and the weights of connections outgoing from each node copy to drift away from each other. In some embodiments, computer system 1700 may impose regularization, such as counter-tying or an is-not-equal-to regularization link, to make the node activations and the weights train to be diverse.

[0237] In block 209, computer system 1700 may replace a single activation function with a plurality of activation functions and a data switch, such as data switch 325 in Figure 3B, to 318976815.3 40Docket No.230458PCT select which activation function is to be used for a specific data item. In some embodiments, computer system 1700, may create a node for each activation function and a data switch, such as 342 in Figure 3, to select between the two nodes. The HNLMS, for example, may specify that computer system 1700 make such a replacement for any of several reasons: (1) To assign a new node or activation function to detect an associated known set with one or more new nodes to imitate the original node for data that is not in the associated known set. (2) To delegate one or more problematic data items. The HNLMS or computer system 1700 may delegate a specified data away from a first node or activation function by controlling a data switch such that activation from input of the specified data item is blocked from activating the first node or activation function. In some embodiments, computer system 1700 or the HNLMS may control the data switch to send the data item to a specified second node. In some embodiments, computer system 1700 or the HNLMS may create a new node to receive the data item. (3) To exclude data based on elementary sensibility criteria: a. Computer system 1700 may base the exclusion on the distance from a specified central point as measured by a specified norm defined on a local data space. The HNLMS, for example, may specify some features for hybrid parametrically controlled autoencoder to create the local data space. (4) For active defense, as discussed in association with block 416 of Figure 4 and block 803 of Figure 8.

[0238] In block 210, computer system 1700 may add extra nodes or units to the network to improve classification performance.

[0239] In some embodiments, computer system 1700 may add an error prediction node and an error correction node to fix one or more explicit or implicit errors. In some embodiments, computer system 1700 may interpret the activation of a first node in a specified interval as acceptance or rejection of a received data item as belonging to a specified known set. In some embodiments, computer system 1700 may train a second node to predict whether the first node has made a false positive error and may train a third node to predict whether the first node has made a false negative error. In some embodiments, computer system 1700 may create an additional node or cell, called an error correction element, which substitutes a change in the output of the first node when one of the error prediction nodes predicts an error on the received data item. In some embodiments, computer system 1700 may add the outputs 318976815.3 41Docket No.230458PCT of the error prediction nodes as additional output values to the unit comprising the first node. Error prediction nodes are also called judgment nodes and are described in published U.S. patent application Pub. No.2022 / 0335296, titled “Deep learning with judgment,” which is incorporated herein by reference in its entirety.

[0240] In some embodiments, computer system 1700 may determine that a node has made an explicit error if the activation of the node is in an interval that computer system 1700 interprets as an acceptance or rejection of the received data item being in a known set for a received data item for which computer system 1700 knows the acceptance or rejection to be false.

[0241] In some embodiments, computer system 1700 may add one or more nodes to receive data delegation of one or more data items on which a node or unit has makes an explicit or implicit error.

[0242] In some embodiments, computer system 1700 may add one or more nodes to represent clusters in a known or named set. In some embodiments, computer system 1700 may add one or more nodes to detect clusters in a specified target set. In some embodiments, computer system 1700 may determine the need to model clusters from the analysis of multiple local maxima in smoothed histogram function, as discussed in block 1509 of Figure 15.

[0243] In some embodiments, computer system 1700 may add one or more nodes to represent clusters in the complement of a detected set. The complement of a detected set may be more diverse than the detected set. In some embodiments, computer system 1700 may represent the complement set by a plurality of clusters to represent diversity in the data.

[0244] In some embodiments, computer system 1700 may add one or more nodes to support continual, lifelong learning. For example, computer system 1700 may add one or more nodes to detect and / or discriminate new data that the system encounters during continued use.

[0245] Computer system 1700 may add extra nodes in active defense after an item to be classified has been received. Active defense is discussed in association with block 416 of Figure 4 and block 803 of Figure 8.

[0246] In block 211, in some embodiments, computer system 1700 may partition the domain of an activation function into intervals. In some embodiments, computer system 1700 may replace the activation function with an activation function that satisfies a specified criterion for flatness on each of a specified set of the intervals. For example, the computer system 1700 318976815.3 42Docket No.230458PCT may specify that the difference between the maximum value and the minimum value of the activation function is less than a specified value. In some embodiments, computer system 1700 may specify that the activation function be constant in a selected interval. In some embodiments, computer system 1700 may select all the intervals in the partition of the activation function to be subject to the requirement to satisfy specified criteria for flatness. In some embodiments, computer system 1700 may specify that the activation function be piecewise constant.

[0247] In block 211, in some embodiments, computer system 1700 may create two or more partitions of an activation function. In some embodiments, computer system 1700 may define the partitions such that the end points of some or all the intervals in one partition are offset from the end points in one or more other partitions. In some embodiments, for each partition, computer system 1700 may specify an activation function that satisfies interval flatness conditions for that partition. In some embodiments, computer system 1700 may create a hybrid node with multiple activation functions, with an activation function for each partition and a data switch such as 325 in Figure 3B. In some embodiments, computer system 1700 may create multiple nodes, with each node having a different one of the plurality of activation functions, with a data switch such as 342 or 362 of Figure 3B.

[0248] In some embodiments, computer system 1700 may control the data switch 325, 342, or 362 based on the relative position of the value of the input to the data switch for a data item compared to the beginning and end points of the associated interval in the respective partition. In some embodiments, computer system 1700 may control the data switch as an active defense, as discussed in association with block 416 of Figure 4 and block 803 of Figure 8.

[0249] In block 212, in some embodiments, computer system 1700 may replace a detector with a more sensible detector. In some embodiments, in block 212, computer system 1700 may replace a selected detector with a piecewise constant function, preferably with exclusion of some data, both of which properties contribute to greater sensibility.

[0250] Computer system 1700 may have replaced an activation function with a piecewise constant function in block 206 or block 212. A piecewise constant function facilitates computer system 1700 making a network more sensible. However, a piecewise constant activation function requires special training techniques, such as a substitute derivative function (207 of Figure 2 and 509 of Figure 5), hybrid training (407 of Figure 4), selective training (409 of Figure 4), back propagation of data (510 of Figure 5), imitation (511 of 318976815.3 43Docket No.230458PCT Figure 5), and / or hybrid conditional training (Figure 13 and block 512 of Figure 5).

[0251] Exclusion of data is discussed in association with Figure 11.

[0252] However, for a detector node, in some embodiments, computer system 1700 may take a different approach.

[0253] In some embodiments, computer system 1700 in block 203 may replace a non- monotonic bounded activation function from block 202 with a bounded monotonic activation function. However, in some embodiments, for a detector node, computer system 1700 may determine that a non-monotonic activation function with a single mode may be a more realistic model for a set of target data items.

[0254] In some embodiments, in block 212, computer system 1700 may compute a histogram of the input to the activation function. In some embodiments, computer system 1700 may compute a smoothed function approximation to the histogram. In some embodiments, if there is a single local maximum in the smoothed histogram function, or if one local maximum is larger than the others by at least a specified criterion, computer system 1700 may model the data as a unimodal probability distribution.

[0255] In some embodiments, if there is a plurality of local maxima in the smoothed histogram function, computer system 1700 may tentatively split the domain of the activation function into intervals with a new node for each interval and a data switch based on the selected intervals distributing each data item to the corresponding new node. In some embodiments, computer system 1700 may train a unimodal parametric probability distribution for the original node and for each of the plurality of new nodes, using statistical training techniques such as maximum likelihood estimation. In some embodiments, computer system 1700 may train a parametric template model, such as illustrated in Figure 10. In some embodiments, the parametric template model may comprise parameters comparable to the parameters of a parametric probability model. In some embodiments, the parametric template model may comprise additional parameters or hyperparameters, such as limits on one or more exclusion norms. In some embodiments, computer system 1700 may estimate template parameters using statistical training methods such as maximum likelihood. In some embodiments, computer system 1700 may train template parameters using empirical training (block 521 of Figure 5). In some embodiments, computer system 1700 may train some of the parameters of a template using gradient descent. In some embodiments, some of the parameters may be specified as hyperparameters controlled, for example, by the HNLMS. In some embodiments, computer system 1700 may specify a local data space of the input values 318976815.3 44Docket No.230458PCT for a detector template. In some embodiments, computer system 1700 may compute a weighted norm in the local data space.

[0256] In some embodiments, computer system 1700 may then test the comparative performance of the single node system with the performance of the multi-node system. In some embodiments, computer system 1700 may evaluate the performance of the single node and multi-node systems based on measurements of precision and recall in detection of a specified target set, preferably evaluated on data that has been set aside from the training data. In some embodiments, computer system 1700 may evaluate the performance based on a divergence or other measure of accuracy of the system or subsystem comprising the selected element or its replacement.

[0257] In some embodiments, computer system 1700 may repeat the process of dividing the domain of a detector if one or more of the detectors in the multi-node version has multiple modes in its smoothed histogram function.

[0258] In some embodiments, computer system 1700 may impose data exclusion limits on the input values and output value of a parametric probability model or of a template model, as illustrated by annuli 1002, 1003, 1004, and 1010 in Figure 10. In some embodiments, computer system 1700 may use a “center-surround” detection score with a lower score for a data item close to but outside the acceptance distance than for a data further from the central point.

[0259] In some embodiments, computer system 1700 may use a flatter function for data within an acceptance norm, such as a super-Gauss trimmed to one standard deviation or less, while using a substitute derivative function such as anfor training, as discussed in association with block 207 of Figure 2. In some embodiments, computer system 1700 may use a constant acceptance score while using a substitute derivative function for training.

[0260] In block 213, in some embodiments, computer system 1700 may create sensible discriminators. For example, computer system 1700 may replace a discriminator with two sensible detectors and a combining node with connection weights and an activation function by which computer system 1700 may compute some approximation to the difference or the ratio of the two detection scores.

[0261] In block 214, computer system 1700 may train a node or a cell to imitate one or more known sets. In some embodiments, computer system 1700 may train the node to have activations values specified to be above or specified to below a specified threshold value for 318976815.3 45Docket No.230458PCT data items in a known set and to have activation values on the opposite side of the specified threshold for data items that are not in the known set. In some embodiments, for two or more known sets, computer system 1700 may train a node or cell to have values specified to be above or below the specified threshold for one or more of the known sets and on the opposite side of the specified threshold for one or more other known sets.

[0262] In block 215, computer system 1700 may convert a node to a cell. The cell may have multiple output values. The cell may store one or more values. In some embodiments, computer system 1700 may pass a value to be stored by the cell from the activation value of a node. In some embodiments, computer system 1700 may pass to a specific cell a value from another cell to be stored in the specific cell. In some embodiments, computer system 1700 may pass a value that represents an attribute of a node stored in a cell associated with the node. Attributes are discussed in association with block 412 of Figure 4. In some embodiments, computer system 1700 may store in a cell a value inferred from a state-space probability estimate computed by computer system 1700 with a set of cells representing a hidden state space. Hidden state spaces are discussed in association with Figure 7.

[0263] Figure 3A is an illustrative diagram comprising an illustrative example of a unit 301. Figure 3A further comprises some external elements including cells 313 and 314, nodes 316, 317, and 318, and a hybrid parametrically controlled autoencoder bottleneck layer 319. Figure 3A further comprises elements internal to unit 301, including cell 312, node 315, components of a hybrid network node (302, 303, 304, 305, 306, and 307), and components of a template model. A unit may comprise an unlimited number of nodes, cells, template models, and other units.

[0264] In Figure 3A, the illustrative unit also comprises a robust template model comprising input variable norm cells 309, 310, and 311, bias cell 320, and template summation cell 308. Each of the cells 309, 310, and 311 computes a single-variable norm of the formwhere ^^^may be a learned parameter or a hyperparameter specified, for example, by the HNLMS.

[0265] The norm value p is a hyperparameter specified, for example, by the HNLMS. In some embodiments, computer system 1700 may estimate the ^^^values by empirical training (521 of Figure 5). For a network not optimized for sensibility, typical values for p are 1 or 2. For a flatter response and greater sensibility, a larger value of p is preferred. In some embodiments, computer system 1700 may change the value of p during the training, as specified by the system design and / or the HNLMS, for example. 318976815.3 46Docket No.230458PCT

[0266] In the template summation cell 308, computer system 1700 may compute. ^^( ^^) = ^^ ∗ (∑^^^^^^^( ^^) )^ / ^, where S is a scaling hyperparameter set, for example, by the HNLMS. In some embodiments, the output of cell 308 may be -g(x) or exp(-g(x)). The values ^^^may be learned parameters or may be hyperparameters specified, for example, by the HNLMS. In some embodiments, all the ^^^are set to 1.0. In some embodiments, computer system 1700 may train the values ^^^and the bias 320 by empirical training (521 of Figure 5). In some embodiments, computer system 1700 may train the values ^^^and the bias 320 by maximum likehood for a parametric probability distribution model. In Figure 3A, the weights for the input connections for the inputs to the template are written asrather than as the more traditional wkto avoid confusion with normal node connection weights wkfor the connections into element 302. A more detailed illustrative diagram of a template is shown in Figure 10, in which it is indicated that the wk values (corresponding to the ^^^values in Figure 3A) may be estimated as the reciprocal of an estimated measure of spread ^^^= ^^^^.

[0267] Internal elements of unit 301 further comprise the internal components of a hybrid network node, including multiple activation functions 305,306, and 307, a data switch 304, an element 302 that computes a weighted sum of input values and a bias 303, with the input values comprising the output value of node 316 multiplied by connection weight w1, the output value of node 317 multiplied by connection weight w2, the output value of node 318 multiplied by connection weight wk, and bias 303. The solid arrows indicate directional connections like the connections between nodes in a neural network. The dash-dot arrows indicate data communication links between cells and between cell 312 and node 315. Data communication links may be unidirectional or bidirectional, such as the link between cell 314 and cell 312.

[0268] In preferred embodiments, computer system 1700 may also impose data exclusion limits (Figure 11 and block 518 of Figure 5) on the template summation variable 308 and / or the input variables 309, 310, 311. For example, in some embodiments, computer system 1700 may impose a data exclusion limit on the template with output 321 by substituting a specified background score for the output if one or more of the variables 309, 310, 311, or 308 exceeds a specified limit. In some embodiments, computer system 1700 may impose norm-based data exclusion, substituting a specified background value for the output 321 if, for a specified norm in the data space 319, the norm of the difference between the data item and a specified central data point for the template exceeds a specified limit. In some embodiments, computer 318976815.3 47Docket No.230458PCT system 1700 may impose data exclusion limits both during training and during deployment. A hybrid network template unit with data exclusion limits is illustrated in Figure 10.

[0269] Each of the input variables 309, 310, and 311, may have an incoming connection from a node or cell or, as shown in the illustration, the input variables may receive incoming connections from the bottleneck layer of a conventional autoencoder or of a hybrid parametrically controlled autoencoder 319.

[0270] Figure 3B illustrates three embodiments of data switching that computer system 1700 may use in active defense (block 416 of Figure 4 and block 803 of Figure 8).

[0271] Element 322 is an illustrative embodiment of a hybrid element comprising two activation functions 323 and 324 with outgoing connections to one or more nodes such as 327. Element 322 further comprises data switch 325, which selectively forwards the result of summation element 326 to one of the activation functions 323 or 324. In illustrative embodiments of active defense (block 416 of Figure 4 and block 803 of Figure 8), computer system 1700 may control data switch 325 to choose between the activation functions 323 and 324 to decrease the vulnerability of 322 to data that may cause non-sensible mistakes.

[0272] In some embodiments an element may have more than two activation functions. In some embodiments, computer system 1700 may include a probabilistic component in its control of data switch 325 in which the probabilistic component may choose among two or more activation functions that all satisfy a specified sensibility criterion. In some embodiments, computer system 1700 may change the selection probabilities in data switch 325, depending on the value of the data item being switched.

[0273] Element 331 is an illustrative embodiment of a unit comprising a summation element 334, an activation function 333, and a data switch 332. In the illustrative embodiment of element 331, in some embodiments, computer system 1700 may control data switch 332 as part of a less direct method of active defense than the illustrative example of 322.

[0274] In some embodiments, computer system 1700 may control data switch 332 to control data delegation. Data delegation is discussed in association with block 518 of Figure 5 and Figure 11.

[0275] Element 342 is a pure data switch that switches data stream 341 between node 343 and node 344. Like the other examples in Figure 3A, computer system 1700 may use data switch 342 in active defense (block 416 of Figure 4 and block 803 of Figure 8) or to control data delegation. The difference is that computer system 1700 may directly control data switch 318976815.3 48Docket No.230458PCT 342 without data switch 342 being tied to a specific node.

[0276] In some embodiments, computer system 1700 may use data switch 342 merely to control data flow. For example, computer system 1700 may use data switch 342 to control data distribution in a distributed computing system. As another example, computer system 1700 may use data switch 342 to select a specific member of an ensemble to classify a specified data item.

[0277] Because data switch 342 is not internal to an element, computer system 1700 may use the illustrative embodiment represented by data switch 342 in a conventional neural network or in a component of a hybrid network in which the component is specified to only contain conventional neural network nodes.

[0278] Figure 3C is a diagram of an illustrative example of a substitute derivative of an activation function. In some embodiments, computer system 1700 may use as a substitute derivative the derivative of a function that differs from the actual activation function in a selected node in the network. In the illustrative example of Figure 3C, the substitute derivative is the function represented by the bold dash-double-dot segments 361, 362, and 363, which is the derivative of the function represented by the plain dash-double-dot segments 364, 365, and 366.

[0279] In the illustrative example in Figure 3C, the actual activation function is a piecewise constant function, represented by the segments 351, 352, 352, 354, 355, and 356. In some embodiments, computer system 1700 may use such an activation function for a node that is discriminating a known set S1 associated with interval 352 from known set S2 associated with interval 355. In some embodiments, computer system 1700 may use a step function, as represented by intervals 353 and 354 to represent the lack of a firm decision between 352 and 355. In some embodiments computer system 1700 may use more steps for the middle region. In some embodiments computer system 1700 may use a single intermediate step or may jump with a single discontinuity directly from 352 to 355.

[0280] Although the illustrative example is a piecewise constant activation function, in some embodiments, computer system 1700 may use a substitute derivative function for any activation function.

[0281] In some embodiments, computer system 1700 may use a piecewise constant activation function as the activation function of a detector node. For example, in some embodiments, computer system 1700 may represent a detector node using an activation function with only 318976815.3 49Docket No.230458PCT the three segments 354, 355, and 356. As another example, for a feature variable with ordinal values, computer system 1700 may use a pure step function, such as segments 352, 353, 354, and 355. In any of these cases, in some embodiments, computer system 1700 may use a substitute derivative function.

[0282] Figure 4 is an illustrative diagram of the hierarchy of the levels of sensibility and of active sensible classification. For the purpose of discussion, the dashed blocks 401, 402, and 403 in Figure 4 place each illustrative technique into the dashed block that best fits the technique. However, the grouping is not absolute. Many techniques may be useful for more than one dashed block.

[0283] Dashed block 401 comprises illustrative examples of models and processes related to first level sensibility. First level sensibility is the first line of defense against non-sensible mistakes in a hybrid network. In some embodiments, computer system 1700 may be able to definitively test whether a system satisfies first level sensibility.

[0284] Dashed block 402 comprises illustrative examples of models and processes related to second level sensibility.

[0285] Dashed block 403 comprises illustrative examples of models and processed related to active classification, including classification during deployment and continual, lifelong learning.

[0286] In block 405, computer system 1700 may use a set of relatively simple first level sensibility techniques, discussed in association with Figure 2, called “elementary sensibility” techniques. These elementary sensibility techniques may be based on simple criteria based on the properties of (1) the dimensionality of the number of variables, and (2) the derivatives of the output function with respect to the input. In some embodiments, computer system 1700 may evaluate these properties in their relationship to the degree of vulnerability of an element to making non-sensible mistakes. In some embodiments, computer system 1700 may test for violations of elementary sensibility by using simulated adversarial attacks.

[0287] A classifier system violates sensibility if a trivial change in the input may change a correct classification to an incorrect classification. In some instances, the small change may be imperceptible or easily ignored by a human observer, or by any sensible animal.

[0288] In image recognition, for example, a change is easily ignored by, or imperceptible to, a human observer of a digital image if the change in each color component of a pixel is comparable to or less than the quantization level. The maximum of the magnitude of the 318976815.3 50Docket No.230458PCT change in any one input variable is called the ^^^norm of the vector of changes. For a change in the input with ^^^≤ ^^, the maximum change in a function ^^( ^^^, ^^ଶ, … , ^^ே) with continuous derivatives is roughly^^ ≤ ^^ ^^. If N, the number of input variables, is large, a small change in the ^^^norm may produce a large change in the output. This property of multivariate functions in high-dimension spaces is the main source of non- sensible mistakes by classifier networks.

[0289] Unfortunately, for a classifier system, the number of input variables is a fixed, specified number. Furthermore, in many classification tasks, including image recognition, N may be very large. On the other hand, the number of input variables to an individual element may be specified, for example, by the system design and / or by the HNLMS and may be much smaller than the number of input variables to the overall system.

[0290] In elementary sensibility, computer system 1700 focuses on assuring that each element satisfies specified criteria of sensibility.

[0291] An illustrative example of criteria for sensibility for a single element: 1) The derivative of an output of should be less than a specified magnitude, with the possible exception of data items within a specified distance of a decision boundary. 2) For any interval of an activation function that represents detection, the difference between the maximum output value and the minimum output value should be less than a specified magnitude. 3) The difference between the maximum output value and the minimum output value for all data in a “remote region” should be less than a specified value: a) Where a “remote region” is a specified region where the minimum distance from any point in the region to any point in one or more specified detection regions is greater than a specified criterion. b) A detection region may be specified by an interval in an activation function or by a norm with respect to a specified point in a template detector.

[0292] In block 405, computer system 1700 may modify activation functions in nodes, add elements to the network, add several kinds of special models, and / or make various other changes to the network to better satisfy several elementary criteria for sensibility that computer system 1700 may check automatically. Illustrative examples of the modifications made by computer system 1700 in block 405 are discussed in association with Figure 2.

[0293] For example, in some embodiments, in block 202 of Figure 2, computer system 1700 318976815.3 51Docket No.230458PCT may change an unbounded activation function to a bounded activation function to better satisfy criterion (3) above.

[0294] In some embodiments, in block 204 of Figure 2, one of the reasons that computer system 1700 may exclude data is to better satisfy criterion (3) above.

[0295] In some embodiments, computer system 1700 may change an activation function to have flatter intervals in block 205 of Figure 2 and / or piecewise constant intervals in block 206 of Figure 2 to better satisfy criteria (1) and (2) above. In some embodiments, computer system 1700 may use a substitute derivative function to accelerate the training process especially after applying the changes made by computer system 1700 in blocks 205 and 206, which might otherwise slow down or halt training in back propagation through a modified element.

[0296] In some embodiments, computer system 1700 may make changes in blocks 203, 208, 209, 212, and 213 to better meet elementary sensibility criteria such as the illustrative example above.

[0297] In block 406, computer system 1700 may select one or more of several methods to improve sensibility of a node with an activation function that includes one or more intervals that fail a criterion for flatness, that is, in which the change in the value of the activation function within an interval exceeds a specified limit.

[0298] In some embodiments, computer system 1700 may first partition the domain of the activation function of a node into intervals. The HNLMS, for example, may specify rules for dividing an activation function into intervals. For example, in some embodiments, computer system 1700 may attempt to find one or more intervals that satisfy a specified criterion for flatness. Computer system 1700 may then divide the domain into alternating flat and non-flat intervals. In some embodiments, computer system 1700 may divide the domain arbitrarily into intervals.

[0299] Computer system 1700 may then select a non-flat interval, which in some embodiments may be the entire domain of the activation function.

[0300] In some embodiments, computer system 1700 may partition the selected interval into subintervals. Computer system 1700 may then create a unit with a separate activation function for each subinterval, with the input to the activation function of the original node being used as a data switch. This structure with a data switch selecting an activation from among a plurality of activation functions was shown in Figures 3A and 3B. In some 318976815.3 52Docket No.230458PCT embodiments, computer system 1700 may use such a structure to partition an activation function into alternating monotonically increasing and monotonically decreasing intervals. In some embodiments, block 406, computer system 1700 may use the same structure in a two- tiered arrangement, first dividing the domain of the original activation function into alternating flat and non-flat intervals, then dividing each non-flat interval into a plurality of subintervals. Computer system 1700 may use other embodiments to achieve a similar result.

[0301] Once a non-flat interval has been divided into subintervals, computer system 1700 may approximate the activation function in a subinterval with a function that satisfies a flatness criterion. In some cases, computer system 1700 may approximate the activation function on a subinterval with a constant.

[0302] In some embodiments, computer system 1700 may make a separate copy of the subnetwork of the selected node and train the subnetwork for each subinterval separately. In some embodiments, computer system 1700 may use knowledge sharing links with is-equal-to relations to regularize the copies of the subnetwork to have activation values similar to those of the original subnetwork. In some embodiments, computer system 1700 may use knowledge sharing links with is-not-equal-to relations to create diversity among a plurality of copies of the subnetwork.

[0303] In some embodiments, possibly under guidance of the HNLMS, computer system 1700 may analyze the selected node as a discriminator. For example, if the selected node is the output node of the network or of a unit with an explicit objective, then computer system 1700 may interpret the node as discriminating data items for one target set from data items of a different target set. In some embodiments, if the node has been associated with two known sets, then computer system 1700 may characterize the node as discriminating between those two known sets. In some embodiments in which the selected node is trained by back propagated derivatives, computer system 1700 may interpret the selected node as discriminating between data items with a negative back propagated derivative from data items with a positive back propagated derivative.

[0304] If the selected node does not have a bounded monotonic activation function, in some embodiments, computer system 1700 may modify the node in the steps 201, 202, and 203 in Figure 2 to obtain a node with a bounded monotonic activation function. With a bounded monotonic activation function, the data items that are correctly discriminated will have activations at the extremes of the domain of the activation function where the activation function is relative flat because the activation function is bounded. That is, the non-flat 318976815.3 53Docket No.230458PCT intervals will be in the middle region of the domain of the activation function. In other words, the data items in a non-flat region are data items that are not yet correctly discriminated at the current state of training. Training each subinterval separately may enable computer system 1700 to successfully discriminate many of the data items in each subinterval.

[0305] In some embodiments, under guidance of the HNLMS, for example, computer system 1700 may take advantage of this opportunity to improve classification performance. For example, in some embodiments, computer system 1700 may train a subinterval with the original non-flat activation function until a stopping criterion is met before changing the activation function for the subinterval to be flatter while approximating the original activation function.

[0306] In some embodiments, computer system 1700 may partition the domain of the activation function of a selected node in a plurality of different ways. For example, computer system 1700 may first do one partition of the domain into intervals and then do a second partition of the domain in which, except for the open-ended intervals at the extremes, each interval boundary in the second partition is positioned at the center of an interval in the first partition. In some embodiments, computer system 1700 may create more than two ways of partitioning the domain into intervals. In some embodiments, computer system 1700 may also partition each non-flat interval into subintervals in multiple ways. Two confusable data items that are in the same subinterval in one partition may be in separate subintervals in another partition. Thus, the units with different partitions may be diverse with respect to which pairs of confusable data pairs become distinguishable.

[0307] In some embodiments, computer system 1700 may use this diversity to improve the classification performance even more than achieved with a single partition. In some embodiments, computer system 1700 may test each partition and choose the one with the best performance. In some embodiments, computer system 1700 may use the set of networks with diverse partitions like an ensemble.

[0308] In some embodiments, computer system 1700 may use the set of networks with diverse partitions for diagnosis and detection, as explained in association with block 415 of Figure 4. In some embodiments, computer system 1700 may use the set of networks with diverse partitions for active defense, as explained in association with block 416 of Figure 4 and block 803 of Figure 8.

[0309] In some embodiments, in block 406, computer system 1700 may use a different method for making a non-flat interval sensible in place of or, in addition to, the partition into 318976815.3 54Docket No.230458PCT subintervals. In some embodiments, computer system 1700 may verify that the outgoing connections from a non-flat node or interval are only connected into robust template models. In some embodiments, computer system 1700 may impose data exclusion limits on a node or unit receiving a connection from a non-flat node or interval.

[0310] In block 407, in some embodiments, computer system 1700 may perform hybrid training. That is, computer system 1700 may use multiple training techniques, not just training by gradient descent computed by back propagation of derivatives. Many example hybrid training techniques are discussed in association with Figure 5.

[0311] In block 408, in some embodiments, computer system 1700, in coordination with the HNLMS, may find the best locations in the network to integrate a selected “piece of knowledge.” The selected piece of knowledge may be from an external source, or it may be knowledge represented in the cells and / or nodes of the network or in a companion network. In some embodiments, the piece of knowledge may be in a network in a network repository.

[0312] An example of a “piece of knowledge” is the knowledge of which data items are members of a known set. By definition, a set of data items is a known set only if there is a way for computer system 1700 to determine whether a specified data item is in the set. Although any subset of the training data items is a known set, preferably, in block 408, computer system 1700 may be able to determine whether a data item not in the training data is in the known set. For example, any set that is defined as the set of data that is accepted by a specified detector node or unit is a known set and computer system 1700 may determine whether a specified data item is in the known set by computing the activation of the subnetwork of the detector and observing the output of the detector. In some embodiments, the “piece of knowledge” may relate to the two sets distinguished by a discriminator. Without loss of generality, some illustrative examples may be discussed with respect to a detector element. However, in some embodiments, computer system 1700 may use essentially the same process with a discriminator element.

[0313] In some embodiments, in block 408, for a specified piece of knowledge, computer system 1700 may test selected candidate locations in the network to see whether integrating the piece of knowledge in a selected network location may improve classification performance, sensibility, and / or holistic interpretability.

[0314] If the piece of knowledge is the detection of a known set, computer system 1700 may integrate the piece of knowledge in any of several ways. In some embodiments, computer system 1700 may connect the detector to one or more nodes or units in a candidate location. 318976815.3 55Docket No.230458PCT

[0315] In some embodiments, computer system 1700 may create a new node or unit in the current base network and train the new node or unit to imitate the detector. In imitation training, the new node or unit is trained to match the output of detector for all specified data items. The specified data items do not need to be labeled. The specified data items do not even need to be real data items. They may be generated or synthetic data items. Computer system 1700 may train the new node to match the output of the detector for synthetically generated data. Computer system 1700 is not limited to using the existing subnetwork of the candidate location with the new node. With the unlimited amount of potential training data for imitation, in some embodiments, computer system 1700 may train a completely new subsystem.

[0316] In some embodiments, computer system 1700 may test the performance, sensibility, and / or holistic interpretability of each selected candidate location. Computer system 1700, under guidance from the HNLMS, for example, may then select a set of one or more of the candidate locations and integrate the piece of knowledge in those locations.

[0317] In some embodiments, computer system 1700 may screen potential candidate locations. For example, in some embodiments, computer system 1700 may compute the correlation of the output of a detector with the back propagated derivative of a global or local objective of a potential candidate node. This correlation indicates the amount that an incremental training update would improve the objective, averaged over the set of data on which the correlation is measured. A high magnitude of correlation would indicate a good candidate location. If a potential candidate node back propagates data examples rather than derivatives, computer system 1700 may compute the degree of agreement between the back propagate data examples and the detected and rejected sets of the detector. In some embodiments, computer system 1700 may limit the measure of agreement to the recall or to the precision, based on analysis of the needs of a candidate location as estimated by computer system 1700 under guidance of the HNLMS, for example.

[0318] In block 409, in some embodiments, computer system 1700 may selectively train only a subset of the elements in a network being trained and / or selectively train an element only on a specified subset of the data. Computer system 1700 may use selective training to accelerate or better control hybrid training, which might be applied to any level of sensibility. In Figure 4, selective training is somewhat arbitrarily placed in block 401.

[0319] In some embodiments, computer system 1700 may selectively train an element discriminating two associated known sets only on data items in the union of the two known 318976815.3 56Docket No.230458PCT sets.

[0320] In some embodiments, computer system 1700 may selectively train a decision element only on data that is close to a decision boundary. In some embodiments, computer system 1700 may change the selection of training data items as the position of the decision boundary changes during training.

[0321] In some embodiments, computer system 1700 may apply selective training by selecting a subset of the elements to be trained for one or more specified data items.

[0322] The selectiveness of training a subset of the elements complements two characteristics of hybrid learning for sensibility. The first characteristic of hybrid training in some embodiments is that computer system 1700 will continually modify the network during training and, in some embodiments, during deployment. In preferred embodiments, when computer system 1700 has modified a trained network, computer system 1700 may temporarily focus training on the modified elements and the other elements most effected by the modified elements.

[0323] The second characteristic of hybrid training is that the learning process may be actively controlled by, for example, the HNLMS. Either the AI systems in the HNLMS or the human team, for example, may direct computer system 1700 to focus on training particular elements. Furthermore, the HNLMS may actively monitor the training process and focus the training on elements that most need improvement.

[0324] In an illustrative embodiment, computer system 1700 may maintain a list of elements actively being trained.

[0325] Computer system 1700 may add an element to the list actively being train or add a data item to the list of data items for an element because of an error or a close call on an explicit or implicit local or global target. In some embodiments, the error or close call may be on data for which the element previously made no error or close call. In some embodiments, the error or the close call may be on new real data or on new generated or simulated data. The error or close call may be on a data item that has modified by a simulated attack or other disturbance.

[0326] In some embodiments, computer system 1700 may drop an element from the list based on a specified criterion.

[0327] In some embodiments, computer system 1700 may add an element that has been newly created or that has been modified to the list being actively trained. 318976815.3 57Docket No.230458PCT

[0328] In some embodiments, for a new element or a modified element, computer system 1700, under direction from, for example, the HNLMS, may temporarily suspend training of elements that have connections to the new or modified element. In other embodiments, computer system 1700 may activate training of elements with connections from a new or modified element.

[0329] In block 410, computer system 1700 may test the sensibility of decision boundaries and, if necessary, computer system 1700 may modify the network to move the position of the decision boundary to improve the sensibility of decisions. For the discussion of block 410, a “decision boundary” is the set of points in a local or global data space at which the activation of a discriminator of two target sets is at a specified threshold. Preferably, each target set is a known set. The discriminator may be a node or unit trained as a discriminator or may be a new node or unit created by computer system 1700 by combining the scores of two trained detectors, one for each of two target sets.

[0330] For block 410, the desired objective is to have any data point in a selected normed local or global data space that is on or near the decision boundary be reasonable to a human observer as a data example that is on the boundary. The human observer may agree that a data point is reasonable because (1) it is a reasonably good match to both target sets. In some embodiments, the human observe may agree that a data point is reasonably on the boundary because (2) it is such a poor match to either target set that it should not be accepted as an example of either. For the purpose of block 410, in some embodiments, data points that complete a smooth surface connecting data points that satisfy reasonableness condition (1) to data points that satisfy reasonableness condition (2) may also be considered reasonable.

[0331] In some embodiments, in block 410, computer system 1700 may build and train a conventional neural network with outputs that are differentiable with respect to the input values from a global or local data space to imitate a hybrid network discriminator for which computer system 1700 is testing and improving the decision boundary. Computer system 1700 may train a neural network or a hybrid network to imitate another network using generated or simulated data as well as unlabeled real data. Using as much data as necessary, computer system 1700 may train the imitating network up to the limits of the capability of the imitating network using as much unlabeled or generated data as necessary. In some embodiments, computer system 1700 may train an imitation neural network that has a node corresponding to each node in hybrid network being trained, with each node in the neural network being trained to imitate the corresponding node in the hybrid network as well as 318976815.3 58Docket No.230458PCT possible. In preferred embodiments, computer system 1700 at least uses the same local or global input space as the discriminator being imitated and trains a node in the neural network to imitate the discriminator as well as possible. The imitation is not expected to be perfect. For example, an imitation neural network with differentiable activation functions can at best only approximate the activation of a node with a discontinuous activation function, and vice versa.

[0332] In some embodiments, computer system 1700 may find a data point on the decision boundary of the conventional neural network with differentiable outputs by back propagating to the input data value d an objective to minimize | ^^ ^^ ^^( ^^( ^^) − ^^)|, where T is a discrimination threshold for the decision boundary and act(x(d)) is the activation of the discriminator node for the data item d.

[0333] Each point on the decision boundary of the imitation neural network will have the value zero for this objective. With many different random starts, computer system 1700 may find a plurality of points on the decision boundary of the neural network that is imitating the hybrid network. In some embodiments, computer system 1700 may locally estimate a tangent hyperplane to the decision boundary of the imitating neural network by fitting a multivariate linear regression model to example points on the decision boundary. In some embodiments, computer system 1700 may then compute an orthogonal line to the estimated decision boundary. In some embodiments, computer system 1700 may then search along this orthogonal line, for example by using a binary search, to find a point in the data space that is on the decision boundary of the hybrid network.

[0334] In some embodiments, computer system 1700 may test for reasonableness by testing for consistency. That is, computer system 1700 may train a diverse set of networks. Then, computer system 1700 may measure how much the position of the decision boundary changes from one network to another. If there is significant variation among the networks, computer system 1700 may use that as a diagnostic that at least some of the networks are not finding a reasonable decision boundary.

[0335] In some embodiments, computer system 1700 may train a “BOTH” detector and / or a “NEITHER” detector for data points on or near the decision boundary of the hybrid network and / or the imitating neural network. Computer system 1700 may train a BOTH and / or a NEITHER detector as described in association with block 208 of Figure 2. In some embodiments, computer system 1700 may assign as a unit output value a constant background score for all data items detected by the NEITHER detector. 318976815.3 59Docket No.230458PCT

[0336] In some embodiments, computer system 1700 may train a discriminator between the sets “BOTH” and “NEITHER” as well as a detector for each of the sets. In some embodiments, if the discriminator variable associated with the decision boundary comprises input from a detector for each alternative, computer system 1700 may use both detectors being above a specified detection threshold as an initial indication that a data item is in the “BOTH” sets. In some embodiments, computer system 1700 may use both detectors being below a specified detection threshold as an initial indication that a data item is in the “NEITHER” set. In some embodiments, computer system 1700 may train a detector for each set being discriminated if the discriminator element does not already comprise such detectors or input from such detectors.

[0337] In some embodiments, computer system 1700 may use additional indications to distinguish the “BOTH” set from the “NEITHER” set. For example, in some embodiments, computer system 1700 may compute a histogram for data from the union of the two sets on or near the decision boundary. Computer system 1700 may then determine whether the histogram appears to be unimodal or bimodal, as discussed in association with block 1509 of Figure 15. In some embodiments, computer system 1700 may compute such a histogram for data projected to a line orthogonal to a hyperplane to the estimated decision boundary. In some embodiments, computer system 1700 may compute such projections to orthogonal lines for a plurality of such orthogonal lines.

[0338] As a second example, computer system 1700 may compute the magnitude of the derivative of the discrimination score along the line orthogonal to the decision boundary through the point on the decision boundary for the data input being evaluated. A low magnitude for this derivative is an indication that the data point is in the “NEITHER” set. A high magnitude is an indication that the data point is in the “BOTH” set.

[0339] In some embodiments, computer system 1700 and the HNLMS may create one or more new features to discriminate among the data items detected by the BOTH detector. For example, in some embodiments, computer system 1700 may create a new feature by standard training of a discriminator node to discriminate the two sets. In some embodiments, computer system 1700 may train additional new nodes in the subnetwork for the new discriminator node. As another example, computer system 1700 may train a new discriminator using constrained optimization (524 of Figure 5).

[0340] In some embodiments, computer system 1700 may use knowledge of a mereology to refine a decision boundary. In an illustrative embodiment, computer system 1700 may 318976815.3 60Docket No.230458PCT compute the alignment of the parts of an image to the parts represented in a mereology of the object in the image or of an object hypothesized to be in the image. Alignment of parts to a specified mereology is discussed further in association with block 413 and 415 of Figure 4 and Figures 7, 12, 13, and 19.

[0341] For example, in some embodiments, computer system 1700 may sample a pair of data items near the decision boundary, one from each of two known sets with mereologies comprising one or more shared components. In some embodiments, a pair of data items from the same category or named set may share identical mereologies. With any shared mereology components, computer system 1700 may than align each of the data items to its mereology and store the alignment information in cells in units that detect specified parts of each image, thus at least partially aligning the two images with each other. Even if the mereologies are not identical, computer system 1700 may then create and train detectors and / or feature variables that discriminate one or more pairs of two aligned parts from each other.

[0342] In some embodiments, computer system 1700 may project a set of selected data items to a line that computer system 1700 has computed as orthogonal to the estimated decision boundary of the imitation neural network and / or to the estimated decision boundary of the hybrid network. In some embodiments, computer system 1700 may limit the selected data items to be within a specified distance of the orthogonal line. In some embodiments, computer system 1700 may generate additional data items for each of the two sets being discriminated. In some embodiments, computer system 1700 may generate additional data items by random perturbations and / or adversarial attacks on each selected data item. In some embodiments, preferably, computer system 1700 may augment each selected data items with the same number of generated items. In some embodiments, computer system 1700 may generate additional data items using a pair of generators, one generator trained to generate examples for one of the known sets being discriminated and a second generator trained to generate examples of the second known set. In general, computer system 1700 may use any method to create a proportional number of additional examples of each known set in the vicinity of the decision boundary.

[0343] In some embodiments, computer system 1700 may then estimate a probability density function for each of the two sets being discriminated. In some embodiments, computer system 1700 may compute a histogram of the counts of data items as a function of the position of the projection of each selected data item to the line orthogonal to the decision boundary. In some embodiments, computer system 1700 may estimate a regression function 318976815.3 61Docket No.230458PCT for the difference or the ratio of the two estimated density functions. In some embodiments, computer system 1700 may estimate a Bayes minimum error dividing point for the two estimated probability density functions or of the smoothed estimates obtained from the regression estimates or a smoothed approximation to the histogram counts. In some embodiments, computer system 1700 may use this estimated Bayes minimum error point as a point on an updated decision boundary.

[0344] In block 411, in some embodiments, computer system 1700 may create local normed spaces. In some embodiments, computer system 1700 may create a local normed space using a neural network autoencoder or a hybrid network parametrically controlled autoencoder with specified features (Figure 9). In block 411, the specified features may be engineered features specified and / or computed by, for example, the HNLMS. As is well known to those skilled in the art of neural networks, an autoencoder is a network trained, for a specified set of data examples, to encode each input data item with a restricted encoding, called the “bottleneck” layer of the autoencoder, such as a vector with a specified limited number of dimensions, and then, for the specified set of training data, to produce an output for each data example that matches the input as well as possible. For a local autoencoder, computer system 1700 or the HNLMS, for example, may specify a set of nodes as the input data space. For example, the input space of the autoencoder may be the set of nodes connected into a node or unit, such as a detector node or unit or a discriminator node or unit. As another example, the input space may be the union of the elements that are connected into a pair of detectors, or to a classifier or to a set of more than two detectors. The input space may be the union of the input variables to the union of the elements connected to a decision element group.

[0345] Hybrid parametrically controlled autoencoders with specified features are discussed in association with Figure 9.

[0346] In some embodiments, computer system 1700 may introduce a local normed space to limit the effective dimensionality of the input to one or more detectors and / or discriminators in order to facilitate improving the sensibility of the detectors and / or discriminators. For example, in some embodiments, local normed spaces are used by computer system 1700 in block 410.

[0347] In block 412, in some embodiments, computer system 1700 may manipulate data and perform sequential computation in ways that cannot be represented in a conventional neural network. In some embodiments, each cell has a local memory. In some embodiments, computer system 1700 may perform sequential computations associated with a cell before, 318976815.3 62Docket No.230458PCT during, and / or after the computation of the activations of the units and nodes.

[0348] For example, in some embodiments, computer system 1700 may use the cells to compute attributes and features using the cells, as described in the following paragraphs.

[0349] In some embodiments, computer system 1700 may use the cells to implement special purpose code developed specifically for the domain in which the hybrid network is to be deployed. Such special purpose code may represent a process called “knowledge engineering.” In some embodiments, computer system 1700 may use the cells to make logical inferences (2102 of Figure 21). In some embodiments, computer system 1700 may use the cells to represent a probability network, such as a hidden Markov process or a dynamic Bayesian network for probabilistic inference (2102 of Figure 21). In some embodiments, computer system 1700 may use the cells to represent a cellular automaton. These uses of the cells to do sequential computations after receiving a data item to be classified are discussed in association with Figures 19 and 21.

[0350] In some embodiments, computer system 1700 may perform sequential computations specified by knowledge engineering on data stored in a cell or in input or output data. For example, if computer system 1700 has generated text, images, or video, in some embodiments, computer system 1700 may compare the proposed generated output to the training data to verify that the proposed output is not close enough to any item data to violate copyright.

[0351] As another example, in some embodiments, computer system 1700 performs logic or set theory computations on the input, the output, and or data computed within the network. For example, in a text generator, in some embodiments, computer system 1700 may test the output for logical consistency. For example, computer system 1700 may have program code representing such syllogisms as “If A implies B is true, and A is true, then B is true” and “If A is true and B contradicts A, then B is not true.” In some embodiments, computer system 1700 may have logic based on ontologies, such as “If A is a kind of B and there is an example of A that has a property C, then there is an example of B that has property C.”

[0352] As a specific example of violating the use of ontological logic, a state-of-the-art text generator repeatedly asserted that “A perceptron cannot represent the XOR function” while also acknowledging that “An elementary perceptron can represent the XOR function” and even supplying an algorithm to train an elementary perceptron to represent the XOR function. This behavior is neither logical nor sensible. 318976815.3 63Docket No.230458PCT

[0353] The statement “A perceptron cannot represent the XOR function” is false, but is widely quoted on the web. The text generator was trained on text from the web but also could quote verbatim from the out-of-print book in which Frank Rosenblatt introduced perceptrons and proved that even an elementary perceptron could be trained to represent any Boolean function, which includes the XOR function. Without explicit logical analysis, it is difficult to get a neural network with trillions of learned parameters to forget something even if it logically contradicts something else that it has learned. In various embodiments, computer system 1700 may overcome this difficulty by explicitly applying logical reasoning in the cells in a computation separate from and / or overriding the computations in the nodes.

[0354] In some embodiments, computer system 1700 may store, as a variable in a cell, a known value, called an “attribute,” associated with a specified element. In some embodiments, computer system 1700 may determine whether to store an attribute associated with an element based on the activation value of the element for the current data item. For example, in some embodiments, for a detector or discriminator element, computer system 1700 may only store an attribute if the activation value is in a specified interval, such as a detection acceptance interval.

[0355] An example of an attribute is the position within an image of a node in a convolutional network. Another example of an attribute is the orientation of a detected object, such as the angle of rotation of a line segment. Other attributes of an object are the size, the color, and the texture. In a model based on a hierarchical knowledge structure such as a mereology or an ontology, an element may have attributes inherited from other elements in the hybrid network. In some embodiments, cells may be programmed to communicate attributes through the data communication links among the cells and between cells and nodes. In some embodiments, computer system 1700 may control the communication of attributes dependent on node activation values and attribute values of the current data item.

[0356] In some embodiments, computer system 1700 may implement software to compute attributes or features specified by, for example, the human team of the HNLMS. In some embodiments, computer system 1700 may store the value of a human specified feature in a cell within a specified unit. An example of a human specified feature is the estimated frequency of a formant in speech analysis. The estimation of formant frequencies is well known to those skilled in the art of speech signal processing. Another example of a human specified feature is explicit detection of edges in an image by a high pass filter. Although convolutional neural networks can detect edges in an image, the edge detection in a 318976815.3 64Docket No.230458PCT convolutional neural network is mixed in with all the other activations of the nodes of the network. In some embodiments, computer system 1700 may explicitly label detected edges as edges. The detection of edges in images is well known to those skilled in the art of digital signal processing of images. In some embodiments, computer system 1700 may use detected edges in a mereology. In some embodiments, computer system 1700 may use detected edges in aligning an image to a model or to another image.

[0357] In some embodiments, computer system 1700 may design and train a new feature specifically to improve the discrimination of two known sets. In some embodiments, computer system 1700 may use such a feature as a specified feature in a hybrid parametrically controlled autoencoder with specified features with the bottleneck layer including the new feature among the variables in a local normed space. In some embodiments, computer system 1700, as part of HNLMS, may develop a new feature to discriminate real or generated data item examples near a decision boundary between two known sets, as described in association with block 410 of Figure 4. For example, computer system 1700 may create and train a new feature to discriminate between data items from the two known sets that are detected by a BOTH detector such as described in association with block 410.

[0358] In some embodiments, computer system 1700 may automatically create a new feature by training a new discriminator node on the task of improving the discrimination of an existing discriminator node or unit for a specified pair of target sets. In some embodiments, computer system 1700 may train the new feature node or unit on a selected set of data. In some embodiments, computer system 1700 may select the errors and close calls of the existing discriminator as the training data for the new feature. In some embodiments, computer system 1700 may select the data items near the decision boundary of the existing discriminator as the training data for the new feature.

[0359] In some embodiments, computer system 1700 may create and train one or more candidate new features and then test the performance of the system with one or more selected candidate new features added to the specified features in a hybrid parametrically controlled autoencoder with specified features. In some embodiments, computer system 1700 may test the comparative sensibility of the system with a selection of new features as well as the classification performance. For example, in some embodiments, computer system 1700 may implement one or more simulated adversarial attacks on the system and measure the rate of success of the adversarial attacks. 318976815.3 65Docket No.230458PCT

[0360] For example, in some embodiments, computer system 1700 may sample a pair of data items near the decision boundary, one from each known set. Computer system 1700 may than align each of the data items to a mereology and store the alignment information in cells in units that detect specified parts of each image, thus aligning the two images with each other. Computer system 1700 may then create and train detectors and / or feature variables that discriminate two aligned parts from each other.

[0361] In some embodiments, computer system 1700 may use an attribute as a feature. In some embodiments, a node may have a known potential attribute that is realized for a specified data item if the activation of the node is in a specified interval when the specified data item is used as an input to the global or to a local data space. For example, a node may have a potential position attribute that is activated when the value of the activation of the node is above a specified threshold.

[0362] For example, in a convolutional network designed for image recognition, typically each low-level node receives activation connections only for a small number of pixels located at and close to a specified position in the image. Similarly, in a speech recognition system a node receives a sequence of input vectors, each from a limited interval of time. In addition, a node in a speech recognition system may receive values for only a single frequency or a limited range of frequencies. The position of the inputs received by a node in a convolutional image recognition is a constant that does not vary from one input data item to another. However, in some embodiments, computer system 1700 may store in a position attribute cell the position of a detector node that is activated above a specified detection threshold as an attribute for the current data item. Similarly, in a speech recognition system, computer system 1700 may store in a time-frequency attribute cell the time and frequency position of a detector node that is activated above a specified detection threshold.

[0363] In some embodiments, when one or more nodes associated with an attribute cell are activated above a specified minimum threshold, computer system 1700 may set the attribute value in the cell to be the known attribute of the associated cell with highest activation level. Such an attribute is not explicitly represented in a node activation and therefore is not available to higher level nodes through the network connections. However, based on the design of the system or as specified by the HNLMS, for example, computer system 1700 may store the attribute in a cell and create data links from that cell to other cells and / or other nodes in the network. In a network that represents a mereology, in a higher-level node or cell, computer system 1700 may match two or more attributes, such as the position of related parts 318976815.3 66Docket No.230458PCT in the mereology of an object to a trained model for the relative values of the attribute in an image of a specified object. In some embodiments, computer system 1700 may scale the position values of components of an object based on the size of the object as seen in an image.

[0364] In some embodiments, computer system 1700 may use cells to hold state information in state space modeling (Figure 7 and block 413 of Figure 4). In some embodiments, computer system 1700 may do state space analysis for a data item thereby changing the behavior of the system after the data item has been received for classification.

[0365] In some embodiments, computer system 1700 may use cells in computing active alignment (Figure 12 and block 417 of Figure 4) of a data item, changing the behavior of the system after the data item has been received for classification.

[0366] Changing the behavior of the system after a data item has been received may help computer system 1700 make the system more robust against adversarial attacks and other disturbances that may cause non-sensible errors.

[0367] In using cells for active alignment and / or other analyses related to mereology and other human knowledge representations, computer system 1700 may make the system easier to understand and may facilitate interaction with the HNLMS and other human consultation.

[0368] For example, in some embodiments, as part of the HNLMS, computer system 1700 may train models of attribute combinations while training the weights and biases of the connections of the network.

[0369] In block 413, computer system 1700 may build one or more hidden state space models.

[0370] In a classification task in which the input data variables can be organized by position in time and / or space, computer system 1700 may add cells to the network connected into a structure that represents the geometry of the relative locations of the input variables. More generally, computer system 1700 may build a structure among cells in the network to represent any adjacency graph among the input variables. In some embodiments, at a higher layer of the hybrid network, computer system 1700 may construct an adjacency graph among sets of cells in the higher layer. In each cell, in each layer, computer system 1700 may store the value of one or more hidden variables. In some embodiments, the cells at a higher layer may have the same adjacency graph as in lower layers, but with different or additional hidden variables. 318976815.3 67Docket No.230458PCT

[0371] In some embodiments, in block 413, computer system 1700 may implement probabilistic inference or dynamic Bayesian networks in the cells of the network (2102 in Figure 21).

[0372] Hidden state space models are explained in association with Figure 7.

[0373] In block 414, computer system 1700 may manage the option of human consultation in many aspects of the invention. In preferred embodiments, computer system 1700 may manage the human consultation to maximize the amount of improvement per the amount of human time and labor required. In some embodiments, computer system 1700 may semi- automate a process that would otherwise require human knowledge engineering by humans with expert knowledge and an amount of labor that could grow with the size and complexity of the network. Additional aspect of communication between computer system 1700 and one or more humans are discussed in association with Figure 21.

[0374] There are multiple examples of aspects of the invention in which computer system 1700 may manage human consultation to be efficient and effective. In some embodiments, computer system 1700 may provide information to human team members of the HNLMS and / or to users of the system such that a human may initiate a process of human consultation.

[0375] An example in which either computer system 1700 or a human may initiate human consultation is the naming of known sets. Computer system 1700 may ask a human to supply a human understandable name for a known set for which computer system 1700 may provide examples. In preferred embodiments, computer system 1700 may manage the efficiency of this process by only asking for names for known sets that are associated with elements that play vital roles in a hybrid network that is already trained to a degree that satisfies a specified criterion. In some embodiments, a human may volunteer a name for any known set or for any variable at any time at the discretion of the human volunteering the name. For example, a human may volunteer a name if the human consultant believes that a name will enable computer system 1700 to guide the training to learning concepts that will generalize better to new data. A human may also volunteer a name wherever the human believes the supplied name will efficiently improve the holistic interpretability of the hybrid network.

[0376] In some embodiments, in associating a set with an element being actively trained, computer system 1700 may give preference to associating the element with a named set to associating the element with an unnamed known set. This preference may help meet the expectation of the human that the naming of the set will help improve the generalization 318976815.3 68Docket No.230458PCT performance of the network. This preference will also increase the holistic interpretability of any element associated with a named set.

[0377] Either computer system 1700 or a member of the human team in the HNLMS, for example, may initiate human consultation in defining the initial state space for a hidden state space model such as discussed in association with Figure 7. In some embodiments, computer system 1700 may largely automate future changes in the state space. However, either computer system 1700 or a human may initiate further human consultation whenever it appears that the consultation will be efficient, worthwhile, and effective.

[0378] In preferred embodiments, computer system 1700 may make available data and displays that will aid humans in following and understanding the training process and system being trained. For example, in histogram analysis in Figure 15 and block 507 of Figure 5, computer system 1700 may generate plots of the histograms.

[0379] In some embodiments, computer system 1700 may provide data from any comparative evaluation that makes a significant improvement or, alternately, that shows a degradation in performance that exceeds a specified criterion.

[0380] Humans may supply mereologies and other human knowledge representations and / or provide oversight on the selection of human knowledge representations from publicly available sources by computer system 1700.

[0381] Humans may provide oversight on any change in the hybrid network that changes the trade-off between classification performance and sensibility by more than a specified amount.

[0382] In some embodiments, for changes that improve both classification and sensibility, computer system 1700 may provide data to keep humans informed although no consultation may be needed.

[0383] In some embodiments, humans may provide guidance in decisions of when to use alternatives to back propagation of derivatives in hybrid training. Preferably, to reduce human labor, this human guidance would apply a single decision to a substantial portion of the hybrid network such as one or more complete layers rather than to individual elements. In some embodiments, computer system 1700 may enable a human to intervene on a single element if the element is critically important in overall performance based on a specified criterion. This enablement may include computer system 1700 gathering data and presenting it in a fashion that enables efficient and effective human understanding. In some embodiments, computer system 1700 may enable a human to intervene on a single element if 318976815.3 69Docket No.230458PCT the element is critical to one or more data items that are critically important based on a specified criterion.

[0384] In some embodiments of continual lifelong learning, computer system 1700 may continually test performance of new versions of the system on old tasks and prepare a report for humans on any degradation in performance on old tasks.

[0385] In some embodiments, computer system 1700 may seek human consultation to verify the sensibility of a decision boundary in a discriminator. If the human consultant does not agree that the supplied example of data items on or near the decision boundary are appropriately characterized as being near the boundary, it is an indication that the system fails to satisfy second level sensibility and that computer system 1700 should take remedial action. In some embodiments, computer system 1700 may take remedial action by delegating and / or excluding data items. For example, computer system 1700 may identify additional data items to delegate by empirically training data weights and delegating data items with negative weights, as discussed in association with blocks 1123-1126 of Figure 11. If the human consultation indicates that one of the alternatives is a poor match, then computer system 1700 may take remedial action by data exclusion.

[0386] In preferred embodiments, computer system 1700 may seek this form of human consultation for only a fraction of examples that is less than a specified criterion for amount of consultation.

[0387] In block 415, in some embodiments, computer system 1700 may perform diagnosis and detection of instances that violate sensibility. In some embodiments, computer system 1700 may use a tool called “canary” networks. A canary network is a network designed and trained to be vulnerable to changing its classification output due to an adversarial attack and other small change in the input. In some embodiments, computer system 1700 may train a diverse set of canary networks and a diverse set of robust networks. In some embodiments, computer system 1700 may train multiple networks with the same architectures or similar architectures to be diverse by using counter-tying. The use of counter-tying to increase diversity in a set of networks is described in Patent No.11,151,455, titled “Counter-tying nodes of a nodal network,” which is incorporated herein by reference in its entirety.

[0388] Given a classification task, computer system 1700 may create a canary network by training a conventional neural network on the classification task, avoiding any of the methods used to make a neural network resistant against adversarial attacks. For example, in some embodiments, computer system 1700 could avoid training the canary neural network with 318976815.3 70Docket No.230458PCT either random perturbations or simulated adversarial attacks. In some preferred embodiments, computer system 1700 could also avoid any of the steps to improve the sensibility of the network discussed in association with Figures 1, 2, 3, 4, 5, and other figures. Further, in some embodiments, computer system 1700 may do the reverse of some of the recommended steps in association with those figures. For example, in some embodiments, instead of replacing unbounded activations with bounded functions, computer system 1700 could replace bounded activation functions, if any, with unbounded activation functions. In some embodiments, computer system 1700 could increase the slope and / or the length of a non-flat interval of an activation function. Preferably, computer system 1700 would select changes that would increase the vulnerability of the canary network to changes in the input while minimizing the impact of the changes on classification performance. In some embodiments, computer system 1700 may retrain the canary networks to get the best performance it can on clean data while allowing it to fail on perturbed data.

[0389] In some embodiments, computer system 1700 may create one or more robust networks by the methods recommended in association with Figures 1, 2, 3, 4, 5, and other figures.

[0390] From one or more examples of a canary network and one or more examples of a robust network, in some embodiments, computer system 1700 may create an arbitrarily large set of diverse networks by continuing or resuming training of multiple copies of a base network with counter-tying between selected pairs of corresponding nodes in any two copies of the same base network. In some embodiments, computer system 1700 may counter-tie a pair of nodes by creating a bi-directional pair of knowledge sharing links with the is-not- equal-to relation. By selecting different subsets of the nodes in different pairs of networks, and / or selecting different subsets of the set of training data on which to enforce the regularization of the link, computer system 1700 may create a wide variety of differences among the pairs of networks in the set of diverse networks.

[0391] Once computer system 1700 has trained a diverse set of canary networks and a diverse set of robust networks, computer system 1700 may use the diverse networks to diagnose any data item that is presented for classification. Any adversarial attack or other disturbance to the input data will be more likely to change the answer for a canary network than for a robust network.

[0392] In some embodiments, computer system 1700 may test the null hypothesis that there is no difference between the response of the canary networks and the robust networks. 318976815.3 71Docket No.230458PCT Computer system 1700 may continue testing with new selections of one or more canary networks and one or more robust networks until the null hypothesis is rejected or a stopping criterion is reached.

[0393] In other embodiments, computer system 1700 may test the differences between the response of the canary networks and the robust networks in other ways. In some embodiments, computer system 1700, to further confirm that the normal input has been disturbed, may perform an untargeted reverse adversarial attack. That is, computer system 1700 may simulate an adversarial attack of the data item presented to be recognized and present the data as changed by the simulated adversarial attack to one or more canary networks. Preferably, in an untargeted attack, computer system 1700 may simulate a form of adversarial attack that attempts to get the canary network to lower the score of the current answer without targeting any one new answer over any other. If, in multiple simulated untargeted attacks, one new answer occurs a plurality of times, that is an indication that the plurality answer is easily accessible by small changes in the input. If the plurality answer from the untargeted simulated attacks agrees with the plurality answer of the robust networks, that is strong evidence that the data item presented was changed by an adversarial attack or other disturbance, and that the plurality answer is the correct answer for the original, unperturbed input.

[0394] In block 416, in some embodiments, computer system 1700 may implement an active defense against non-sensible mistakes. In some embodiments of the active defense, computer system 1700 may control one or more units with data switches such as shown in Figure 3B. In some embodiments, active defense is used in association with block 803 of Figure 8.

[0395] In some embodiments, to implement the active defense, computer system 1700 may train two or more activation functions for a node with the discontinuities and the intervals with high magnitude derivatives offset from each other, separated by intervals with zero derivatives and / or intervals in which the difference between the maximum value and the minimum value is less than a specified value and the magnitude of the derivative is less than a specified value.

[0396] In some embodiments, computer system 1700 may implement one or more data- dependent data switches. In some embodiments, computer system 1700 may specify a set of activation functions and data-dependent data switches such that for an input data value d, under control of computer system 1700, the data switch presents d as input to an activation function for which the input is in a relatively flat interval and is no closer than a specified 318976815.3 72Docket No.230458PCT amount to the closest end of the flat interval. In other words, computer system 1700 may be able to control the data switch such that small changes in the input do not cause a change in the output by more than a specified amount. In some embodiments, all the relatively flat intervals in all the activation function have constant values, so for any small change to the input to the element there is no change in the output.

[0397] In block 417, in some embodiments, computer system 1700 may perform data item specific active alignment. That is, computer system 1700 may compute an alignment of a data item after that data item has been received for classification. In some embodiments, computer system 1700 may be doing a local classification within a hybrid network and the classification may be to a specified set of known sets rather than to final classification categories.

[0398] In active alignment specific to a data item, in some embodiments, computer system 1700 may compute values for variables in a set of cells that specify the alignment of the cells with a human knowledge representation such as a mereology. In an image recognition task, each of the alignment cells may be associated with a specific position in an image that has been received for classification. Thus, in such an embodiment, computer system 1700 is computing an alignment between the received image and the mereology model.

[0399] In some embodiments, computer system 1700 may have trained an augmented mereology model that also models the relative positions of parts in the mereology.

[0400] The process of training mereology alignment models in discussed in association with Figure 12.

[0401] In some embodiments, computer system 1700 may align a data item with a type of human knowledge representation other than a mereology. For example, in a task involving words, such as speech recognition, handwriting recognition, translation, or text understanding, computer system 1700 may align observed words or hypothesized words with a parse in a specified grammar. In some embodiments, computer system 1700 may align words with a semantic net.

[0402] In some embodiments of alignment of images or video, computer system 1700 may first create a lower resolution representation of the image of video in order to do a fast preliminary analysis that may speed up the analysis of the original image or video. Computing a low-resolution representation of a high-resolution image or video is well known to those skilled in the art of image processing. 318976815.3 73Docket No.230458PCT

[0403] In some embodiments, computer system 1700 may perform classification of a low- resolution image or video. In some embodiments, computer system 1700 may use the classification of the low-resolution data item to construct a list of the best scoring categories or known sets. In some embodiments, computer system 1700 may use the list of best scoring categories or named sets to partially restrict the possible classifications for the higher resolution data item. In some embodiments, computer system 1700 may add to the list of candidates if the fit of the alignment to the mereology is worse than a specified criterion. In some embodiments, computer system 1700 may have determined a specified criterion for each target category or named set based on measurements of degree of fit in previous alignment of instances of the category or named set.

[0404] In some embodiments, computer system 1700, may align the low-resolution image or video with cells in a simpler hybrid network trained on low-resolution images or videos. In preferred embodiments, computer system 1700 may design the mereology alignment cells in the low-resolution model to be homologous to a specified subset of the mereology alignment cells in the high-resolution model. In such an embodiment, computer system 1700 may use the alignment of the low-resolution image to initialize a rough alignment of the high- resolution image. In some embodiments, computer system 1700 may then refine the alignment of the high-resolution image by filling in the alignment for cells that have not yet been aligned. In some embodiments, computer system 1700 may iteratively improve the alignment, changing the alignment of one or more of the cells to fit better with the alignment of cells that are close in the mereology adjacency graph to the cell that is being changed. In some embodiments, computer system 1700 may stop the alignment computation if no changes are made during an iteration of incremental improvements or if computer system 1700 detects a repeating cycle. In some embodiments, computer system 1700 may stop the iterative alignment process if some other specified stopping criterion is met. For example, in some embodiments, computer system 1700 may stop the alignment process if the only changes still being made are so small that they are less than a criterion that computer system 1700 has previously trained to detect changes that are so small that they do not change a classification more than for a specified small error rate.

[0405] In some embodiments, computer system 1700 may update the mereology alignment model. In some embodiments, computer system 1700 may save the data and the analysis in a repository.

[0406] In block 418, computer system 1700 may implement randomized activation during 318976815.3 74Docket No.230458PCT training and inference, including inference during deployment. In randomized activation, the activation value of one or more elements in a hybrid network may be different on repeated presentation of the same input data item.

[0407] In some embodiments, in block 418, computer system 1700 may use one or more of six types of randomizations or noise: (1) additive noise to the output of one or more elements and / or other variables, (2) simulated errors in one or more elements, (3) probabilistic switching of the destination of a data switch, (4) probabilistic switching of the interval of a partitioned activation function, (5) randomized dropout, and / or (6) simulated adversarial attacks on the network input and / or on one or more local data spaces. In preferred embodiments, computer system 1700 may use the same types of randomizations or noise in randomized training and diagnosis (520 of Figure 5). In some embodiments, computer system 1700 may use higher degrees of randomization and / or noise during training than in inference during deployment.

[0408] In some embodiments, in block 418, computer system 1700 may implement any of the six types of randomizations or noise using the techniques explained in association with block 520 of Figure 5. In some embodiments, in generating the randomizations and / or noise during inference during deployment, computer system 1700 may use control hyperparameters that generate less variation than used in training and / or diagnose.

[0409] In some embodiments, in block 418, for a data item received to be classified, for the network or for a selected unit, computer system 1700 may generate randomizations and noise a plurality of times. In some embodiments, computer system 1700 may combine the plurality of sets of output values across the plurality of randomization like the output values from a virtual ensemble. In such embodiments, computer system 1700 may empirically train the hyperparameters that control the randomizations.

[0410] In some embodiments, computer system 1700 may use randomized activation to help create a diverse set of canary networks and / or a diverse set of robust networks (415 of Figure 4). In some embodiments, computer system 1700 may use counter tying and / or is-not-equal- to knowledge sharing regularization links to further increase the diversity. In some embodiments, computer system 1700 may use soft-tying and / or is-equal knowledge sharing regularization links to moderate the differences among the set of diverse networks so that corresponding elements in each network stay in correspondence other than the differences in randomized activation.

[0411] In some embodiments, during deployment, computer system 1700, after receiving a 318976815.3 75Docket No.230458PCT data item to be classified, computer system 1700 may randomly select a subset of the diverse canary networks and a subset of the robust networks, using a selection probability distribution that computer system 1700 does not specify until after receiving the data item to be classified.

[0412] In block 419, in some embodiments, computer system 1700 may construct and train robust template models, such as illustrated in Figure 10.

[0413] In block 420, in some embodiments, computer system 1700 may substitute in an activation function f(x) a constant background score for values of x less a specified threshold T1 and / or for values of x greater than a specified value T2.

[0414] In some embodiments, in block 420, computer system 1700 in a robust template model may substitute a specified constant background score for the output value of the template model if the value of input value ^^^satisfies | ^^^− ^^^| > ^^^for a specified value ^^^for more than a specified number of the k input values. In some embodiments, computer system 1700 may substitute background score for the output value of the template model if ^^ ^^ ^^ ^^ + ∑^^^^^^(| ^^^− ^^^|) exceeds a specified value. Robust template models are discussed in association with Figure 10.

[0415] In block 421, in some embodiments, computer system 1700 may build and train one or more generators and / or classifiers jointly with a team of one or more humans, as described in association with Figure 21. In some embodiments illustrated in this figure, the human participation in the training and development may be more extensive than described in Figures 1 to 5. In the joint development of Figure 21, one or more humans may directly control the training process. In generators discussed in association with Figure 21, computer system 1700 may implement an interface that allows one or more humans to directly control details of the generation. In the joint development process, rather than minimizing the amount of human labor as in semi-automated knowledge engineering, greater human participation may be used to associate names with more known but unnamed sets and with unnamed features. The additional names make the networks easy to interpret which, in turn, enables more human guidance during training. Additional named features also enable more human control of generators. In some embodiments, in block 412, computer system 1700 may implement logical and / or probabilistic inference in the cells of the network, as discussed in association with block 2102 of Figure 21.

[0416] In some embodiments, joint development and human guidance may be used with cooperative generators, such as used to generate additional training data in block 514 of Figure 5. 318976815.3 76Docket No.230458PCT

[0417] In block 422, in some embodiments, computer system 1700 may train an adversarial generator and a real versus non-real discriminator. In some embodiments, computer system 1700 may also train one or more cooperative generators. In some embodiments, computer system 1700 may train generators such as described in blocks 2109, 2110, and 2111 of Figure 21. In some embodiments, computer system 1700 may use variable resolution game theory- based training of the real versus non-real discriminator and the adversarial and cooperative generators, as described in international application PCT / US23 / 64296, titled “Generation and discrimination training as a variable resolution game,” which is incorporated herein by reference in its entirety, for generation and discrimination training / resolution game.

[0418] Figure 5 is a diagram of illustrative embodiments of aspects of hybrid training. In Figure 5, the topics are grouped by the phase of the training process, shown by the dashed blocks: 501 for initial training, 502 for the main hybrid training phase, 503 for lifelong learning and continued training during deployment (i.e., the model being deployed to perform the task it is trained to perform). However, many of the concepts and techniques apply to more than one phase.

[0419] Figure 5 is not a flow chart. No sequential ordering of the blocks is implied. Computer system 1700 may apply the concepts and techniques in the respective blocks in any order, other than the rough grouping into phases represented by dotted blocks 501, 502, and 503. In some embodiments, computer system 1700 may impose some constraints on the order of application of the techniques’ prerequisites in some of the details. In some embodiments, all of the concepts and techniques work together and computer system 1700 may co-develop them.

[0420] In some embodiments, to begin the training, in block 504, computer system 1700 may select a base network and incrementally make changes to the network to improve it, as discussed in association with blocks 101 and 103 of Figure 1 and other blocks in Figures 1 and 2. The selected base network may be either a neural network or a hybrid network.

[0421] In some embodiments, in block 504, computer system 1700 repeatedly seeks opportunities to improve sensibility, holistic interpretability, classification performance and / or cost / performance. In some embodiments, computer system 1700 may repeatedly test the system on validation data that has been set aside from the training data.

[0422] In some embodiments, computer system 1700 may grow a network from scratch. In some embodiments, in block 505, computer system 1700 may grow a neural network from 318976815.3 77Docket No.230458PCT scratch and later convert the neural network to a hybrid network. In some embodiments, computer system 1700, may directly grow a hybrid network from scratch.

[0423] In block 506, in some embodiments, computer system 1700 may train the connection weights and node biases of one or more elements using back propagation of derivatives by gradient descent. Back propagation by gradient descent is the standard method for training neural networks. However, for a sensible network, it is essential that not all training be done by gradient descent.

[0424] However, in some embodiments, even after the initial training, the training may be partially based on gradient descent. However, in preferred embodiments, training is not based on gradient descent alone. In preferred embodiments, computer system 1700 uses a hybrid of training methods to improve the sensibility of the network being built and trained.

[0425] In block 507, in some embodiments, computer system 1700 performs histogram analysis. Block 507 is grouped with initial training for two reasons: (1) histogram analysis is an elementary technique that does not require other techniques as a prerequisite, and (2) histogram analysis is a broadly useful technique that may serve as a preliminary step for other techniques. On the other hand, histogram analysis may also be used during main training (502) and / or continued training (503). For example, computer system 1700 may use histogram analysis to facilitate human consultation (414 of Figure 4) in any situation in which computer system 1700 uses human consultation. Histogram analysis is discussed further in association with Figure 15, blocks 202 and 204 of Figure 2, and block 405 of Figure 4.

[0426] Computer system 1700, in block 507, may compute a histogram of one, two, or more variables. A variable may be a continuous valued real number or may be a discrete variable with values from a specified finite set. A specified finite set may represent a finite number of classification categories, a collection of known sets, or a collection of possible states for a hidden state space model.

[0427] For a neural network node, computer system 1700 may compute a histogram using as variables the value of the affine sum or the value of the output of the activation function of the node. Computer system 1700 may also use as a histogram variable the value received from any one of the connections into the node.

[0428] For a hybrid network, computer system 1700 may also use as a histogram variable a value supplied by a cell.

[0429] Computer system 1700 may also use as a histogram variable the value of a back 318976815.3 78Docket No.230458PCT propagated derivative. The derivative may be the derivative of the classification objective or of some other specified function. In a hybrid network, computer system 1700 may use as a histogram variable a derivative of a local target, such as described in association with block 508 of Figure 5. Computer system 1700 may also use as a histogram variable a local substitute derivative function, such as described in association with block 509 of Figure 5.

[0430] In some embodiments, computer system 1700 may perform regression on histogram counts to test a set of known sets to determine if any of the known sets satisfy specified criteria to be associated with a specified variable. For example, in some embodiments, computer system 1700 may tentatively associate a variable with a known set if the magnitude of the regression coefficient is greater than a specified value. Use of regression on histogram counts is discussed further in association with Figure 15.

[0431] In some embodiments, computer system 1700 may select an interval of a specified variable to be used to represent detection of a known set, with the selection based on histogram counts.

[0432] In some embodiments, computer system 1700 may select a variable to be used as a discriminator between two known sets. In some embodiments, computer system 1700 may use histogram counts to determine initial threshold values to be used with the discriminator. In some embodiments, computer system 1700 may perform comparative performance tests to empirically adjust the threshold values used with a discriminator. In some embodiments, computer system 1700 may perform such comparative performance tests using data that has been set aside from training data. In some embodiments, computer system 1700 may continue to empirically adjust threshold values using data that is gathered from one or more deployed systems.

[0433] In testing a selected variable as a detector of a known set or as a discriminator of two known sets, computer system 1700 may find more than one known set for which the variable meets a specified criterion as a detector or as a discriminator. In such a case, in some embodiments, computer system 1700 may make multiple copies of the variable and of the subnetwork that leads to the variable. Computer system 1700 may then train each copy and its subnetwork on a distinct one of the detector and / or discriminator tasks.

[0434] In some embodiments, computer system 1700 may use histogram counts to determine bounds for a data dependent variable. The variable may be an output value of a node, cell, or unit. In some embodiments, computer system 1700 may limit the maximum and / or the minimum value for a specified variable in order to better assure the sensibility of nodes or 318976815.3 79Docket No.230458PCT units that directly or indirectly receive an input value that is a function of the variable. In some embodiments, computer system 1700 may limit the minimum and / or the maximum value for a variable based on the most extreme values observed for the variable on a specified set of data, such as the set of training data. In some embodiments, computer system 1700 may set the limiting values for a variable to be the most extreme plus a specified margin. In some embodiments, for some variables, the margin may be zero or negative, reducing the observed range. In some embodiments, computer system 1700 may later adjust the limits for a variable.

[0435] In some embodiments, computer system 1700 may use histogram counts to determine the bounds to use in a new activation function when computer system 1700 is replacing an unbounded activation function in block 202 of Figure 2.

[0436] In some embodiments, computer system 1700 may use the histogram counts to determine the initial values to use for the ^^^parameters in a template model. In some embodiments, computer system 1700 may estimate the ^^^parameters as the mean of a set of data items, or as the median, or as the mode. In some embodiments, computer system 1700 may make any of these estimates from the histogram.

[0437] In some embodiments, if a selected variable is associated with one or more known sets, computer system 1700 may limit the data selected for the histogram to data in the union of the associated known sets. Computer system 1700 may then set exclusion limits for the selected variable initially based on the histogram, as explained in association with Figure 15.

[0438] In some embodiments, computer system 1700 may use histogram counts in setting decision thresholds, as described in block 1506 of Figure 15.

[0439] In some embodiments, computer system 1700 may compute a joint histogram for two or more variables. In some embodiments, computer system 1700 may use fewer, longer bin intervals for each variable in a multi-variable joint histogram than in a single variable histogram.

[0440] In some embodiments, computer system 1700 may do additional low-dimension analysis (block 517 of Figure 5) if computer system 1700 detects significant correlation or significant clustering in a low-dimension histogram.

[0441] In block 508, in some embodiments, computer system 1700 may determine implicit local targets for a node.

[0442] In some embodiments, computer system 1700 may determine implicit or explicit local targets from association of one or more intervals of a nodes activation function with a known 318976815.3 80Docket No.230458PCT set. For example, computer system 1700 may set a designated point in the interval as a target for a data item in the known set.

[0443] In some embodiments, computer system 1700 may determine an implicit local target based on the sign of a back propagated derivative for a data item. For example, computer system 1700 or the HNLMS may specify a pair of values, such as {0, 1} or {-1, 1}, with the lower value being the target for any data item with a negative back propagated derivative and the higher value being the target for any data item with a positive back propagated derivative. In some embodiments, computer system 1700 may use the lower bound of the activation function as the lower value and the upper bound of the activation function as the higher value.

[0444] In some embodiments, computer system 1700 may use one or more intermediate values as targets for data items with a back propagated derivative less than a specified absolute value.

[0445] In some embodiments, computer system 1700 may use the determination of the presence or absence of an implicit error to convert a back propagation of a derivative to a back propagation of data (Figures 18 and 6 and block 510 of Figure 5).

[0446] In some embodiments, computer system 1700 may make the determination of whether a node has made an implicit error in determining the degree to which the activation of the node in a specified interval agrees with membership in a known or named set. In some embodiments, computer system 1700 may use one or more outputs corrected to fix an implicit error in determining whether to associate the node with the known or named set. When computer system 1700 makes a new association or changes an existing association, in some embodiments, computer system 1700 may thereafter use the new or modified association to determine explicit errors for the node.

[0447] The use of implicit errors for training error prediction nodes is discussed in association with block 203 of Figure 2.

[0448] In block 509, in some embodiments, computer system 1700 may create a substitute derivative function for a node. In some embodiments, computer system 1700 may use a substitute derivative function to enable or accelerate the training of a node with one or more intervals with relatively low magnitude derivatives, as illustrated in Figure 3C. In some embodiments, the computer system 1700 may select a base substitute derivative function that computer system 1700 then multiplies by a back propagated derivative value or by the sign of 318976815.3 81Docket No.230458PCT a back propagated derivative value.

[0449] In some embodiments, computer system 1700 may use a continuous substitute derivative function activation during part of the training, such as early training until a criterion is met, and a discontinuous substitute derivative function once the criterion is met. In preferred embodiments of this type of substitute derivative function, computer system 1700 may multiply a base substitute derivative function by a back propagated derivative value or by the sign of a back propagated derivative value. In some embodiments, computer system 1700 may multiply the base substitute derivative function by the back propagated derivative or by the sign of the back propagated derivative only for an input value x, for which T1 < x < T2 for specified threshold values T1 and T2. In some embodiments, computer system 1700 may set a constant background score and use that background score for all data with a value outside a specified interval, regardless of the sign or magnitude of a back propagated derivative. In some embodiments, computer system 1700, as controlled by the HNLMS, may customize the criterion for a change of substitute derivative function to an individual node. The HNLMS, for example, may compute a customized criterion for a node based on measurements collected during the training of the node.

[0450] In some embodiments, computer system 1700 may design the substitute derivative function to drive activations away from points of discontinuity or high magnitude derivatives in the activation function toward centers of relatively flat intervals, as illustrated by function 361, 362, and 363 in Figure 3C. In some embodiments, computer system 1700 may delay the use of such a substitute activation function until after a training criterion is met.

[0451] In some embodiments, computer system 1700 may use a substitute derivative function in which the value of the substitute derivative function is always positive for input values less than a specified threshold T1 and / or is always negative for input values greater than a specified threshold T2. In some embodiments, computer system 1700 may use such a substitute derivative function for a node that multiplies a base substitute derivative function by a back propagated value for x in the interval T1 < x < T2, as illustrated by intervals 353 and 354 of Figure 3C.

[0452] In block 510, in some embodiments, for a specific node, computer system 1700 may back propagate labeled data examples rather than derivatives. In some embodiments, computer system 1700 may continue back propagating derivatives on pre-existing incoming connections while back propagating labeled data examples to one or more new elements.

[0453] In some embodiments, for a selected element with a standard discriminator activation 318976815.3 82Docket No.230458PCT function, computer system 1700 may determine, for each data item in a specified set, whether the element has made an implicit error. In some embodiments, computer system 1700 may use this information to back propagate data items with labels with the implicit errors corrected.

[0454] In some embodiments, computer system 1700 may back propagate these labeled data items to one or more new elements while optionally continuing to back propagate derivatives to its pre-existing incoming connections.

[0455] By correcting implicit errors, computer system 1700 may be able to train the new elements with information that is not available through regular back propagation. An illustrative example of such a training procedure is described below and illustrated in Figure 18. In the description, set S1 is the set associated with lower values x < X1 in the input of the activation function, and the set S2 is the set associated with the higher values x > X2, where X1 < X2. Either S1 or S2 may be associated with the maximum output value of the activation function, with the interior interval of the activation function being correspondingly either monotonically increasing or monotonically decreasing.

[0456] In some embodiments, computer system 1700 may use the following process, which is illustrated in Figure 18: (1801) Obtain a training data item. (1802) Compute the activation of the network; call the activation value x. (1803) Back propagate derivatives. (1804) Associate X1 with the set S1, X2 with the set S2. (1805) If x < X1, select the label S1 and go to (1809). (1806) If x > X2, select the label S2 and go to (1809). (1807) If the back propagated derivative to the discriminator element is negative, then select S1 and go to (1809). (1808) Select S2; (1809) Back propagate the current data item with label S1 or S2 to one or more of a. A pair of detector elements; b. A linear separator (Figure 6); c. A subnetwork with its output directly trained by the labels S1 and S2; d. To an error predictor, back propagate the label and whether there was an implicit error. (1810) Save corrected labels for S1 and S2 for all training data. 318976815.3 83Docket No.230458PCT (1811) Repeat steps (1801) to (1811) until a stopping criterion is met. (1812) Train the new elements using the corrected labels for S1 and S2.

[0457] In some embodiments, computer system 1700 may delay the saving of corrected labels for S1 and S2 until the training of the selected standard discriminator element has stabilized enough so the corrected sets S1 and S2 are no longer changing by more than a specified criterion.

[0458] In some embodiments, computer system 1700 may create a new unit to discriminate S1 and S2 with corrected labels using constrained optimization, as discussed in association with block 524 of Figure 5. From the solution to the constrained optimization, computer 1700 may create a linear threshold function as a new element. In some embodiments, computer system 1700 may freeze a copy of the subnetwork so that the performance of the linear threshold function will not degrade as the network is changed by further training. In some embodiments, if further training causes the selected standard discriminator element to make new errors, computer system 1700 may train another linear threshold function. If computer system 1700 drops the selected standard discriminator element from the network when or before the training is stopped, there will be no path to back propagate non-zero derivatives of a specified function of the output through the one or more linear threshold functions that computer system 1700 has used to replace the selected standard discriminator element.

[0459] In some embodiments, computer system 1700 may create one or more new units, each comprising a pair of detectors trained on the sets S1 and S2 and an associated discriminator node. In some embodiments, computer system 1700, may specify the associated discriminator to compute the difference of the outputs of the two detectors, or a similar combining function, without requiring any training of the connection weights. In some embodiments, the computer system 1700 may use a piecewise constant function as the activation function of the discriminator. In some embodiments, computer system 1700 make the activation function be a standard discriminator function. In some embodiments, computer system 1700 may train two or more of the new units to make their S1 and S2 detectors be diverse. In some embodiments, computer system 1700 may also train a diverse set of canary networks as S1 and S2 detectors and / or a diverse set of discriminators.

[0460] In some embodiments, computer system 1700 may connect one or more of the new elements created in block 1809 to elements in higher layers of the network up to and including the output of the network. In some embodiments, computer system 1700 may train the higher subnetwork by backpropagation of derivatives of an output objective without back 318976815.3 84Docket No.230458PCT propagating derivatives to or through the new elements. Furthermore, in some embodiments, computer system 1700 may drop the original standard discriminator element once the new elements have been trained. In such an embodiment, the activation of the network on a new data item through one or more of the new elements has no corresponding back propagation of derivatives, increasing the protection against adversarial attacks.

[0461] If the selected standard discriminator element still makes implicit errors when the training of the network has converged, then computer system 1700 may improve the performance of the network by replacing the standard discriminator element by one or more of the new elements trained to the corrected sets S1 and S2. In addition, because the new elements are trained with data explicitly labeled as S1 or S2, they may be easier to interpret than typical inner nodes of a deep network.

[0462] In block 511, in some embodiments, computer system 1700 may train a second network partially or approximately to imitate a semi-homologous first network, where every specified node in the second network is associated with a node in the first network to imitate. In some embodiments, computer system 1700 may use the output activation value of a node in the first network as a target for the activation value of one or more specified nodes in the second network. In some embodiments, computer system 1700 will use is-equal-to knowledge sharing links to train the specified nodes in the second network to better agree with the associated nodes in the first network.

[0463] In some embodiments, the design of the first network may be less sensible than the design of the second network. In some embodiments, the first network may be a neural network and the second network may be a hybrid network. On the other hand, in some embodiments, the second network may be less sensible than the first network. For example, the first network may be a hybrid network trained to be sensible and the second network may be a canary network (415 of Figure 4). In each of these situations, computer system 1700 may relax the imitation when the activation in the first network is near a discontinuity or a point of high magnitude derivative of the activation function of the node in the first network.

[0464] In some embodiments, the imitation may be limited to specified data items. For example, in some embodiments, the second network may be a new member of an ensemble that is being trained to be diverse on a specified subset of the data but to agree on a disjoint specified subset, and, in some embodiments, to be neutral on a third subset.

[0465] In block 512, in some embodiments, computer system 1700 implements conditional hybrid training. In conditional hybrid training, computer system 1700 may customize a 318976815.3 85Docket No.230458PCT hybrid training technique, such as applying the technique only for selected data items and / or only on selected units or nodes.

[0466] For example, in block 512, in some embodiments, computer system 1700 may implement conditional flattening. In some embodiments, computer system 1700 may implement conditional flattening customized to each selected data item, using a data switch such as 325 in Figure 3B. In some embodiments, after an amount of training specified by, for example, the HNLMS, computer system 1700 may begin with a partially trained selected node with an activation function y = act1(x) partitioned into disjoint intervals, such that act1(x) is non-flat for one or more of the intervals. Computer system 1700 may copy act1(x) as act1A(x) (323 in Figure 3B), perhaps making some of the intervals less flat. Computer system 1700 may then copy act1(x) as act1B(x), making some or all the intervals flatter. In some embodiments, computer system 1700 may make act1B(x) (324 in Figure 3B) a piecewise constant function. Computer system 1700 may then add a data switch 325 of Figure 3B to make the unit 322 of Figure 3B.

[0467] In some embodiments, computer system 1700 may conditionally apply any of the techniques discussed in association with blocks 508, 509, 510, 511, and / or 512 of Figure 5.

[0468] In some embodiments, computer system 1700 may apply any of the training techniques 513, 514, and / or 516 as on-going continual training after a system has been deployed. In some embodiments, computer system 1700 may apply one or more of these techniques during main training before deployment.

[0469] Hybrid conditional training is discussed further in association with Figure 13.

[0470] In block 513, computer system 1700 may apply continual learning during deployment, that is computer system 1700 may actively update the learned parameters using data acquired during operational use. In some embodiments, computer system 1700 may continue to add elements to the network.

[0471] In some embodiments, computer system 1700 may continue to test the performance on previous training and validation data. In some embodiments, computer system 1700 may apply is-equal-to knowledge sharing links from an earlier version of the network to specified nodes in a revised version of the network to maintain performance on specified data items.

[0472] In preferred embodiments, computer system 1700 may repeatedly test the performance of the system on data that has been set aside for validation testing. Preferably, computer system 1700 will add new data to the validation data on a specified schedule. 318976815.3 86Docket No.230458PCT

[0473] In some embodiments, computer system 1700 may train a new template model to match new data using one-shot or few-shot learning.

[0474] For example, in some embodiments, computer system 1700 may set the ^^^values in a new template such as illustrated in Figure 10 to the values in a single example or to the mean of the values in a plurality of examples. In some embodiments, computer system 1700 may set the ^^^values to a value specified by a hyperparameter. In some embodiments, computer system 1700 may tune the hyperparameter to a specified trade-off between precision and recall. Such a template is called a one-shot or few-shot template. In some embodiments, computer system 1700 may continue to train a one-shot or few-shot template as additional data is acquired.

[0475] In some embodiments, computer system 1700 may compute an alignment between the current data item and a mereology model or other model of human knowledge represented as graphical structure. Training alignment models is discussed in association with Figure 12 and block 417 of Figure 4.

[0476] Continual learning during deployment is discussed further in association with Figure 8.

[0477] In block 514, computer system 1700 may generate additional data examples. For example, in some embodiments, computer system 1700 may use a mixture of generators model as described in U.S. Patent No.11,354,578, tiled “Mixture of generator models,” which is incorporated herein by reference in its entirety. As another example, computer system 1700 may use a stochastic categorical autoencoder (SCAN) as described in U.S. Patent Nos.10,679,129 and 11,461,661, both titled “Stochastic categorical autoencoder network” and both of which are incorporated herein by reference in their entirety. In some embodiments, computer system 1700 may develop a SCAN with a parametrically controlled hybrid autoencoder, as illustrated in Figure 9. In some embodiments, computer system 1700 may train a mixture of generators system or a SCAN with back propagation from a joint objective to produce data classified as real by a real versus synthetic discriminator.

[0478] In some embodiments, computer system 1700 may generate additional data examples as a joint human + AI creative activity as described in Figure 21 and block 421 of Figure 4.

[0479] In some embodiments, computer system 1700 may generate data from some other form of cooperative generator, where the phrase “cooperative generator” is used in contrast to a generative adversarial generator (GAN). Unlike a GAN, computer system 1700 may train a 318976815.3 87Docket No.230458PCT cooperative generator on examples of real data. In some embodiments, computer system 1700 may train the generator to generate realistic data by using one or more real versus synthetic discriminators. In some embodiments, computer system 1700 may train a real versus synthetic discriminator as the discriminator in a GAN and then use that discriminator with one or more cooperative generators. In some embodiments, computer system 1700 may co- train the real versus synthetic discriminator as hybrid network, co-trained with one or more hybrid classifier networks and sharing known sets and human knowledge representations such as mereologies. In some embodiments, computer system 1700 may use unidirectional knowledge sharing links in either or both directions between classifier hybrid networks and the real versus synthetic discriminator. In some embodiments, computer system 1700 may also share human knowledge representation with one or more cooperative generators.

[0480] In some embodiments, computer system 1700 may generate additional data examples using a conventional autoencoder with a stochastic bottleneck layer or a parametrically controlled autoencoder (Figure 9) with a stochastic layer.

[0481] In block 516, computer system 1700 may co-train a set of partially or fully homologous networks. In a set of partially homologous networks, each specified node in a network is homologous in network structure to a corresponding node in one or more other networks. In a set of fully homologous networks, each node in each network is homologous in network structure to each corresponding node in each homologous network.

[0482] Computer system 1700 may use co-training of homologous networks during initial training (501 of Figure 5) and / or during main training (502 of Figure 5) as well as during continued training (503 of Figure 5).

[0483] In some embodiments, computer system 1700 may use co-training of homologous networks to reduce the amount of computation required to train a plurality of networks. For example, in some embodiments, computer system 1700 may use standard training on a single network or a selected subset of the set of networks. Computer system 1700 may then train the rest of networks by using is-equal-to knowledge sharing links on a specified subset of the nodes using a high value for the strength hyperparameter ^^. In some embodiments, computer system 1700 may also use is-not-equal-to knowledge sharing links for selected nodes and / or for selected data items to train the networks to be diverse.

[0484] In some embodiments, the activation functions in a specified set of nodes in one or more of the homologous networks may have a different activation function than the homologous nodes in other networks. For example, one network may have a continuous 318976815.3 88Docket No.230458PCT activation function for a node and a second network may have a piecewise constant activation function for the homologous node.

[0485] In some embodiments, computer system 1700 may create diversity by counter-tying a selected set of nodes in a specified pair of the set of networks. In some embodiments, computer system 1700 may create diversity by having one or more non-homologous nodes in each network.

[0486] In some embodiments, computer system 1700 may obtain one or more pretrained networks, such as conventional neural networks that have not been trained for sensibility. In some embodiments, computer system 1700 may then train homologous conventional or hybrid networks using is-equal-to knowledge sharing links in addition to or in place of gradient descent training. In some embodiments, computer system 1700 may decrease the strength hyperparameter ^^ during later stages of training the homologous conventional or hybrid networks.

[0487] As another example, computer system 1700 may use co-training to share knowledge among a set of distributed systems. For example, in continued learning during deployment of a distributed set of homologous networks, one specific distributed network may encounter a new data item that causes a misclassification. In some embodiments, computer system 1700 may train the specific distributed network so that it correctly classifies the new data item. In preferred embodiments, computer system 1700 may limit the changes in the specific distributed network to a selected set of nodes. In some embodiments, computer system 1700 may then use knowledge sharing links to train other networks to imitate the selected nodes of the specific distributed network.

[0488] Although the corresponding nodes in a set of homologous networks are homologous in network structure, computer system 1700 may co-train a set of diverse homologous networks by applying the is-equal-to regularization link only on a selected subset of the data and applying an is-not-equal-to knowledge sharing link on a selected subset of the data.

[0489] For example, computer system 1700 may co-train one or more robust networks and one or more canary network by not enforcing the is-equal-to knowledge sharing link between a robust node and a canary node when the activation of the robust node for a data item is closer than a specified value to a discontinuity of the activation function in the robust network.

[0490] In co-training a set of diverse homologous networks, in some embodiments, computer 318976815.3 89Docket No.230458PCT system 1700 may select a subset of the nodes and / or a subset of the data on which not to enforce the is-equal-to knowledge sharing link. In some embodiments, computer system 1700 may select a subset of the nodes and / or a subset of the data and enforce an is-not-equal-to knowledge-sharing link on the selected nodes and selected data. In some embodiments, computer system 1700 may select a different subset of the data for each selected node.

[0491] In some embodiments, computer system 1700 may train a set of homologous networks with is-equal-to and / or is-not-equal-to knowledge sharing links on unlabeled data.

[0492] In block 517 of Figure 5, computer system 1700 may perform analysis of two or more variables. Each variable may be the output value of a node, cell, or unit, or may be the input to an activation function or one of the input values to a node or to a template. The set of two or more variables may be a subset of the variables of a local data space.

[0493] In some embodiments, in block 517, computer system 1700 may compute the correlation of all pairs of variables in a specified set of variables. In some embodiments, computer system 1700 may compute the covariance matrix of a set of variables. In some embodiments, the specified set of variables may be the set of values of the incoming connections to an element. In some embodiments, the set of variables may be the union of the sets of values of incoming values for a specified set of elements. In some embodiments, the specified set of elements may be two or more detectors for disjoint sets. In some embodiments, computer system 1700 may compute the correlation or covariance evaluated only over a specified subset of the training data. For example, in some embodiments, computer system 1700 may compute the correlation or covariance only over data that is to be discriminated by a specified element. For example, for a discriminator of two known sets, in some embodiments, computer system 1700 may compute the correlation or covariance only for data in the union of the two known sets. In some embodiments, computer system 1700 may compute the correlation or covariance only over data that is to be classified by a specified unit or subnetwork.

[0494] In some embodiments, the set of elements may be two detectors whose outputs are the input to a combining node. In some embodiments, the combining node may be a discriminator. In some embodiments, the computer system 1700 may train the combining node to approximate some logic function of its inputs, such as (A AND B), (A OR B), (A = B), (A ≠ B), or (A implies B).

[0495] In some embodiments, computer system 1700 may multiply the set of variables by a matrix to remove one or more of the pairwise correlations. In some embodiments, computer 318976815.3 90Docket No.230458PCT system 1700 may specify a linear order of the variables and may multiply the variables by a matrix to remove the correlations of the pairs variables that are adjacent in the linear order. For example, in a frequency spectrum, computer system 1700 may multiply the spectrum by a matrix to remove the pairwise correlation of spectral amplitudes at adjacent frequencies.

[0496] In some embodiments, computer system 1700 may multiply the set of variables by the inverse of the estimated covariance matrix.

[0497] In some embodiments, computer system 1700 may replace the original variables with the set variables obtained by multiplication by a decorrelation matrix or by the estimated inverse covariance matrix. In some embodiments, computer system 1700 may copy the set of nodes that receive the original variables and connect the transformed variables to the new nodes while leaving in place the original nodes with untransformed variables. In some embodiments, computer system 1700 may temporarily create two networks, one without a specified variable transform and the second with the specified transform. In some embodiments, computer system 1700 may compare the performance of the two networks and select the one with better performance. In some embodiments, computer system 1700 may keep both networks as members of an ensemble.

[0498] In some embodiments, in block 517, computer system 1700 may perform cluster analysis of a specified set of data using a specified set of variables. In some embodiments, computer system 1700 may perform cluster analysis using a set of variables for which computer system 1700 has detected clustering of the data in a histogram analysis done by computer system 1700 in block 507 of Figure 5.

[0499] In some embodiments, in block 517, computer system 1700 may train a discriminator or a classifier in the data space for which computer system 1700 has detected a non-linear decision boundary between two or more known sets. In some embodiments, computer system 1700 may detect such a non-linear decision boundary by a multi-variable histogram analysis, such as discussed in association with Figure 15 and block 507 of Figure 5.

[0500] In block 518, in some embodiments, computer system 1700 may determine control parameters for excluding or delegating data from training and / or inference for a selected element. Exclusion and delegation of data are discussed in association with Figure 11.

[0501] In block 519, computer system 1700, may add new nodes and / or new connections to the network, as discussed in association with block 208 of Figure 2. In some embodiments, computer system 1700 may create new nodes to implement node splitting, in which a node is 318976815.3 91Docket No.230458PCT replaced by a set of two or more nodes.

[0502] In block 519, in some embodiments, computer system 1700 may make one or more copies of an element and then train the copies to be different from the original element and from in each other. In some embodiments, computer system 1700 may train each copy on a different set of data or may train each copy with data weighting with different weights. In some embodiments, computer system 1700 may implement the distribution of data with a data switch. In some embodiments, computer system 1700 may implement different data weights by a numerical multiplier in the learned parameter update. In some embodiments, computer system 1700 may implement data selection and weighting by specifying data dependent probabilities in a probabilistic data switch. Data weighting is described in U.S. Patent No.11,010,671, titled “Iterative training of a nodal network with data influence weights,” which is incorporated herein by reference in its entirety.

[0503] In some embodiments, computer system 1700 may split a node in order to create a node to receive data delegation as described in association with Figure 11.

[0504] In block 520, in some embodiments, computer system 1700 may use randomized training and diagnosis. In some embodiments, computer system 1700 may use randomized training to make the system more robust against both external noise, such as noise in the input data, and internal noise, such as noise and / or errors made by individual elements in the network. In some embodiments, computer system 1700 may use randomized training to support randomized activation (418 of Figure 4) to improve sensibility. In some embodiments, computer system 1700 may use randomized training and randomized activation to improve classification performance, for example, by training and using a virtual randomized ensemble. In some embodiments, in block 520, computer system 1700 may use one or more types of randomizations and or noise to better understand the interdependencies of elements in the network and to diagnose possible vulnerabilities.

[0505] In some embodiments, in block 520, computer system 1700 may use one or more of six types of randomization or noise: (1) additive noise to the output of one or more elements and / or other variables, (2) simulated errors in one or more elements, (3) probabilistic switching of the destination of a data switch, (4) probabilistic switching of the interval of a partitioned activation function, (5) randomized dropout, and / or (6) simulated adversarial attacks on the network input and / or on one or more local data spaces. In some embodiments, computer system 1700 may use higher degrees of randomization and / or noise during training than in inference during deployment. 318976815.3 92Docket No.230458PCT

[0506] In block 520, for noise type (1) above, in some embodiments, computer system 1700 may apply a technique herein called “additive noisy activation” to one or more variables during the computation of the activation of a hybrid network when presented with a specified input data item to the network global input space or to any selected local data space. In some embodiments, computer system 1700 may apply noisy activation to the output value of one or more nodes, units, or cells. The underlying variable to which noise is being added is called the “underlying activation variable.” The random variable specifying the amount to add to a specified underlying activation variable during a specific activation computation is called the “additive random noise variable.”

[0507] In some embodiments, computer system 1700 may use noisy activation during training, in a diagnostic procedure, and / or during inference for classification. Computer system 1700 may use noisy additive activation during initial training (dotted block 501 of Figure 5) and / or during main training (dotted block 502 of Figure 5). When a data item d is received for training or for classification, computer system 1700 determines the value of each additive random noise variable as a new random sample.

[0508] The probability distribution for an additive random noise variable for a specified noisy activation variable may be any type of probability distribution. For example, it may be a Gaussian distribution, a trimmed Gaussian distribution, or a uniform distribution.

[0509] The type of probability distribution may be specified, for example, by the system design, by the HNLMS, or may be selected by computer system 1700 through empirical testing of two or more specified choices for the distribution. In some embodiments, computer system 1700 may use a different type of probability distribution for different noisy activation variables.

[0510] Without loss of generality, the mean of an additive random noise variable may be set to zero, since any non-zero mean is merely equivalent to a change in the underlying activation variable.

[0511] For each additive random noise variable, computer system 1700 may specify one or more variables or hyperparameters to control the degree of spread of the population of random samples. For example, for a Gaussian distribution, computer system 1700 may specify the standard deviation. For a uniform distribution, computer system 1700 may specify the length of the interval, centered around zero. For a trimmed Gaussian distribution, computer system 1700 may specify the standard deviation and the number of standard 318976815.3 93Docket No.230458PCT deviations at which to trim.

[0512] In some embodiments, computer system 1700 may empirically estimate the value of one or more spread parameters for one or more additive random noise variables by empirical training, as discussed in association with block 521 of Figure 5.

[0513] In some embodiments, for error simulation type (2), for an element associated with one or more known sets computer system 1700 may simulate an error on a data item in a known set by randomly selecting a substitute activation value in an interval not associated with the known set. For a data item that is not in an interval associated with a named set, computer system 1700 may randomly select a substitute activation value in an interval that is associated with a known set that is distinct from the named set.

[0514] In some embodiments, for randomization type (3), activation interval switching, or type (4), data switch destination switching, compute system 1700 may generate a discrete valued random variable to select the activation interval or the destination of the data switch. The probability distribution for the discrete valued random variable may be specified by parameters or hyperparameters that are specified, for example, by the HNLMS or that computer system 1700 may determine by empirical training (521 of Figure 5).

[0515] In some embodiments, for randomization type (5), dropout, computer system 1700 may determine whether to do the dropout of a selected element for a specific data item at random with a probability specified by a hyperparameter. In some embodiments, the activation value to use in the case of dropout may be specified as zero or may be specified by a hyperparameter. In some embodiments, an element may have an element-specific substitute activation value in the case of dropout.

[0516] In some embodiments, for randomization type (6), computer system 1700 may randomly select whether to use a simulated adversarial attack on a specified element for a specified data item with a probability specified by a hyperparameter. In some embodiments, the system design and / or the HNLMS, for example, may specify a plurality of methods of adversarial attack. In such embodiments, computer system 1700 may randomly select which method of adversarial attack to use for a specific element for a specific data item.

[0517] In some embodiments, in block 520, computer system 1700 may use randomization and noise to understand and diagnose the interactions among elements in the network. For example, computer system 1700 may add noise and / or change the output of a first designated element to discover and / or evaluate the effect of those changes in the output of the first 318976815.3 94Docket No.230458PCT designated element on a second designated element. In some embodiments, the first designated element does not need to be directly connected to the second designated element. The second designated element may be any element in the network, directly or indirectly affected by the change in the output of the first designated element.

[0518] In some embodiments, in block 520, computer system 1700 may determine the amplitude of an additive noise, the probability of one or more of the other changes, and / or the strength of a simulated adversarial attack based on values of a set of hyperparameters. In some embodiments, computer system 1700 may use a separate randomization hyperparameter for each noise or randomization type for each element in the network.

[0519] In some embodiments, in block 520, computer system 1700 may use a greater degree of noise and randomization during training than during inference during deployment. In some embodiments, in block 520, computer system 1700 may estimate the best values for the randomization hyperparameters during training by using empirical training of the randomization hyperparameters, as discussed in association with block 521 of Figure 5.

[0520] In some embodiments, as a diagnostic procedure, computer system 1700 may select to study the effects of the randomization and noise of other variables on a specified set of significant elements or variables. For example, in some embodiments, computer system 1700 may select to study the effects of randomization of inner variables on the output nodes of the network. In some embodiments, computer system 1700 may select to study the effects of randomization of other variables on the output values of one or more units. In some embodiments, computer system 1700 may select to study the effect of randomization of other variables on the values of one or more variables in one or more local data spaces.

[0521] In some embodiments, in studying the effects on the specified set of significant variables, computer system 1700 may compute the effect of multiple randomizations, randomly varying the value of each of the randomization hyperparameters over a specified range of values. In some embodiments, computer system 1700 may measure the effect of noisy activation or every ordered pair comprising a noisy variable and an influenced variable.

[0522] For efficiency, in some embodiments, rather than analyzing every ordered pair of selected significant variables and noisy variables, computer system 1700 may first select a significant variable on which to measure influence and then a set of noisy activation variables that is specific to that affected significant variable, as described below. In some embodiments, computer system 1700 may reverse the order, first selecting a noisy variable and then a set of significant variables affected by the selected noisy variable, as described in a 318976815.3 95Docket No.230458PCT later paragraph.

[0523] In some embodiments, as a diagnostic procedure, computer system 1700 may select one of a set of significant variables on which to measure the influence of noisy activation. In some embodiments, computer system 1700 may choose each significant variable in turn. Computer system 1700 may then compute multiple randomizations and compute a regression correlation of the change in the chosen significant variable with respect to the degree of change in one or more of the variables changed in the randomization. In some embodiments, computer system 1700 may use a greater degree of randomization and noise in the diagnostic procedure than in the training.

[0524] In some embodiments, for a specified significant variable, computer system 1700 may select one or more noisy variables for which the effect of the randomization of the noisy variables on the specified significant variable is greater than a specified criterion. In some embodiments, computer system 1700 may use a specified criterion that preferentially selects noisy variables that are less directly connected to the significant variable than noisy variables that are more directly connected to the significant variable. In some embodiments, computer system 1700 may make additional changes to further increase the sensibility and robustness of one or more of the selected noisy variables.

[0525] In some embodiments, in diagnosing an error or close call of one of the significant variables, computer system 1700 may check the associated noisy variables to determine whether an error or perturbation in one of the associated noisy variables may have caused or significantly contributed to the error or close call of the significant variable. If so, computer system 1700 may take corrective action to improve the accuracy and / or the robustness of the noisy variable.

[0526] In some embodiments, computer system 1700 may select one of more candidate noisy variables and compute the effect of randomization and noise in the noisy variable on other variables in the network. In some embodiments, computer system 1700 may select a set of one or more other variables that are significantly affected by the selected candidate noisy variable based on a specified criterion. In some embodiments, computer system 1700 may add the selected candidate noisy variable and the selected significantly affected variable to the set of associated pairs of significant variables and noisy variables.

[0527] In some embodiments, computer system 1700 may use the relationship of a noisy variable and one or more of the associated significant variables to aid in the interpretation of the noisy variable. In some embodiments, computer system 1700 may use the relationship of 318976815.3 96Docket No.230458PCT a significant variable and one or more noisy variables to aid the interpretation of the significant variable.

[0528] For example, computer system 1700 may determine whether the set of data items with activation values in a specified interval of the activation in one member of a pair of variables approximates to specified degree a defined equality or inequality relationship with the set of data items with activation values for a specified interval in the other member of the pair. If so, in some embodiments, computer system 1700 may create a knowledge sharing link in one or both directions between the specified activation intervals.

[0529] In some embodiments, if an interval in a significant variable or a noisy variable is associated with a known or named set, computer system 1700 may check to determine whether the known or named set may be associated with a paired influence or significant variable.

[0530] In some embodiments, computer system 1700 may use the pairing of significant variables and noisy variables to diagnose the causes and potential cures to an error or close call on an individual data item. For example, computer system 1700 may attempt to determine the changes that computer system 1700 might be able to make in the network design and / or in the learned parameters of one or more of the noisy variables in order to correct the error or close call of a significant variable on the individual data item. In some embodiments, computer system 1700 may generate simulated adversarial attacks and / or random perturbations in the network input space and / or in a local data space to create one or more examples of errors or close calls by a significant variable.

[0531] In block 521, in some embodiments, computer system 1700 may empirically estimate the best value for one or more hyperparameters. In some embodiments, computer system 1700 may empirically estimate the value of one or more learned parameters. In some embodiments, computer system 1700 may use empirical estimation of a learned parameter as an alternative to training by gradient descent and / or as an alternative to training by back propagation of data. In some embodiments, computer system 1700 may alternate empirical estimation of a learned parameter with one or more other methods of training the learned parameter. In some embodiments, computer system 1700 may alternate between training a learned parameter by empirical estimation and / or by another training methods and further alternating with the parameter being a hyperparameter controlled by, for example, the HNLMS. In some embodiments, computer system 1700 may empirically estimate the performance of a hyperparameter as information supplied to the HNLMS for controlling the 318976815.3 97Docket No.230458PCT hyperparameter.

[0532] As mentioned in the discussion of block 520 of Figure 5, computer system 1700 may empirically estimate the value of one or more spread parameters for one or more additive random noise variables. Another example of parameters that computer system 1700 may empirically estimate are the end points of an acceptance or rejection interval in a detector or discriminator node or unit. As another example, computer system 1700 may empirically estimate the background score for any detector or discriminator variable. More generally, computer system 1700 may empirically estimate the value for any constant value interval of a variable. Furthermore, computer system 1700 may empirically estimate the maximum and minimum value for any specified relatively flat interval. As another example, in some embodiments, computer system 1700 may empirically estimate the norm or other limit to the acceptance region of a template model. In some embodiments, computer system 1700 may empirically estimate the norm for data exclusion for a detector or discriminator element. In some embodiments, computer system 1700 may estimate one or more norms for data exclusion for a robust template model.

[0533] In some embodiments, computer system 1700 may simultaneously empirically estimate multiple parameters. For example, in some embodiments, computer system 1700 may empirically estimate the spread parameters for one or more or all spread parameters for additive random noise variables. In some embodiments, computer system 1700 may empirically estimate one or more or all the parameters associated with one or more of the constant or relatively flat intervals.

[0534] In some embodiments, computer system 1700 may empirically estimate one or more parameters that characterize the position and orientation of a decision boundary.

[0535] In some embodiments, computer system 1700 may simultaneously evaluate a plurality of quantifiable objectives or a specified combination of multiple quantifiable objectives.

[0536] Without limitation, illustrative examples of quantifiable objectives that computer system 1700 may use in empirical learning for a classification task include: (1) classification performance, (2) sensibility, and (3) holistic interpretability.

[0537] Without limitation, illustrative examples of quantifiable objectives that computer system 1700 may use in empirical learning of a generative task include: (1) recall in generating examples of a named set, (2) precision in generating examples of a named set, (3) for either a cooperative or adversarial generator, performance against one or more previously 318976815.3 98Docket No.230458PCT trained real vs synthetic discriminators, (4) performance on new data of a classifier trained with supplementary data produced by a generator, (5) sensibility of a classifier trained with supplementary data produced by a generator.

[0538] In some embodiments, computer system 1700 may compute a function of two or more quantifiable objectives as a new quantifiable objective. For example, computer system 1700 may compute a weighted average of classification performance, sensibility, and holistic interpretability that represents a trade-off among the objectives.

[0539] In some embodiments, computer system 1700 may evaluate classification performance by running multiple trials with noisy activation and / or random noise added to the input variables.

[0540] In some embodiments, computer system 1700 may evaluate sensibility by running multiple trials using simulated adversarial attacks and / or with noisy activation.

[0541] In an illustrative embodiment, computer system 1700 may simultaneously empirically optimize multiple parameters and / or hyperparameters, as discussed in association with Figure 20.

[0542] In some embodiments, during training or during continual learning after deployment, computer system 1700 may repeat the empirical optimization of one or more parameters based on a criterion controlling the frequency of repetitions. In some embodiments, computer system 1700 may repeat the empirical estimation more frequently based on observations of the operation of the system. For example, computer system 1700 may repeat empirical estimation if measures of one or more quantifiable objectives degrade over the course of continued use or training. In some embodiments, computer system 1700 may repeat empirical estimation if continued training on new data examples has changed the values of learned parameters by more than a specified criterion.

[0543] In block 522, in some embodiments, computer system 1700 may replace a selected node with a set of three or more nodes. More specifically, computer system 1700 may replace a node with a unit or a set of nodes comprising (1) a first new node created from the selected node and copies of the connections into the selected node that have positive weights, (2) a second new node created from the selected node and copies of the connections into the selected node that have negative weights, and (3) a third new node with connections from the first and second new nodes and copies of the outgoing connections of the selected node. In some embodiments, computer system 1700 may copy connections into the selected node with 318976815.3 99Docket No.230458PCT weights with magnitudes less than a specified value to both the first and second new nodes.

[0544] In some embodiments, computer system 1700 may create more new nodes and divide the incoming connections into more sets.

[0545] In some embodiments, computer system 1700 may interpret each of the source nodes sending a connection into the selected mode as a detector for data items that produce higher activation values. Thus, computer system 1700 may interpret an incoming connection with a positive weight as evidence for a set of data items in which the source node of the connection has a high activation value. In some embodiments, computer system 1700 may interpret a node with a mixture of negative weights and positive weights as discriminating between the set of data items detected by a consensus of the source nodes with positive weights from the set of data detected by a consensus of the source nodes with negative weights.

[0546] In continued training in which the signs of the incoming connections do not change very much, computer system 1700 training back propagation from the selected node will tend to make the source nodes learn toward better matching this interpretation.

[0547] In some embodiments, computer system 1700 may create a new unit comprising the new nodes. Each of a pair of the new nodes may have a subset of the incoming connections of the original node and an outgoing connection to a third node. In some embodiments, computer system 1700, may select only connections with weights greater than a specified threshold T1 for the first node in a pair. Computer system 1700 may select only connections with weights less than a threshold T2 as incoming connections to the second node in the pair. In some embodiments, T1 ≤ 0 ≤ T2. In some embodiments, computer system 1700 may reverse the signs of weights on all incoming connections to the second node in the pair. In those embodiments, for the second node in the pair, computer system 1700 may replace the activation function the original node with an activation function equal to a constant minus the original activation function. In some embodiments, computer system 1700 may restrict the magnitudes of T1 and T2 to be less than a specified amount. In such embodiments, most of the incoming weights to each of the pair of new nodes will be positive. In some embodiments T1 = T2 = 0.

[0548] In some embodiments, computer system 1700 may interpret each of the nodes in the new pair as a detector with higher values of the activation function representing detection.

[0549] The third new node may have an activation function that represents some form of difference, such as 318976815.3 100Docket No.230458PCT ^^( ^^, ^^) = ^^ − ^^ ^^ ^^ ^^( ^^, ^^) = (exp( ^^) − exp( ^^)) / (exp( ^^) + exp( ^^)).

[0550] In some embodiments, computer system 1700 may associate the third node as a discriminator between two sets, which the discriminator models as disjoint.

[0551] In some embodiments, in continued training, computer system 1700 may add or remove an incoming connection if the updated weight of the connection crosses one of the thresholds T1 or T2.

[0552] In some embodiments, for two known sets A and B, computer system 1700 may associate one of the pair of new nodes with the set of data in A and not in B and associate the other node in the pair of new nodes with the set of data in B and not in A. In some embodiments, computer system 1700 may train an additional node associated with intersection of A and B and / or train an additional node associated with the set of data not in A and not in B.

[0553] If the original node was associated as a detector of a known set, in some embodiments, computer system 1700 may tentatively associate the first node in the pair as a detector of the known set and the second node of the pair as a detector of a subset of the complement of the known set.

[0554] If the original node is a discriminator of two known sets, in some embodiments, computer system 1700 may associate each of the nodes in the pair of nodes as a detector of one of the known sets. In this association, each detector has incoming connections with mostly positive weights.

[0555] In some embodiments, computer system 1700 may train the nodes with weight decay. That is, at each weight update, computer system 1700 may multiply the revised weight by a specified constant r < 1. The process of weight decay is well known to those skilled in the art of training neural networks. In some embodiments, computer system 1700 may prune a connection if the magnitude of the weight is less than a specified magnitude and has been so for a specified number of iterative updates.

[0556] In some embodiments, computer system 1700 may replace one or more of the new detector nodes with a template model.

[0557] In block 523, in some embodiments, computer system 1700 may select a set of two or more decision elements. In some embodiments, for each selected decision element, computer system 1700 may create a new decision element initialized to duplicate the selected decision element. In some embodiments, computer system 1700 may connect each duplicate element 318976815.3 101Docket No.230458PCT with connections duplicating the incoming connections of the selected decision elements and initialize the connection weights to be the same.

[0558] In some embodiments, computer system 1700 may then form a decision element group comprising the duplicates of the selected decision elements. In some embodiments, computer system 1700 may add one or more decision elements representing the set intersection of target sets and complements of target set of the original selected decision elements. In some embodiments, computer system 1700 may then form a softmax relationship on the expanded set of duplicate detectors. Computer system 1700 may then train the system, associating the expanded set of duplicate detectors with disjoint sets.

[0559] In some embodiments, computer system 1700 may replace one or more of the disjoint set detectors with a template model and continue training with the softmax relationship.

[0560] In block 524, in some embodiments, computer system 1700 may use constrained optimization to train the weights of a linear threshold function, as discussed in association with Figure 6. Having trained the weights of the linear threshold function, computer system 1700 may then back propagate to the nodes connected into the node of the linear threshold using back propagation of derivatives, back propagation of labeled data examples, or both or neither. With incremental growth (103 of Figure 1 and 504 of Figure 5), in some embodiments, computer system 1700 may build and train an entire network without using any back propagation.

[0561] Figure 6 is a flow chart of an illustrative embodiment of constrained optimization in training.

[0562] In block 601, computer system 1700 obtains or selects a network.

[0563] In block 602, in some embodiments, computer system 1700 may convert activations and / or make other modifications to the selected network, such as discussed in association with Figure 2.

[0564] In block 603, in some embodiments, computer system 1700 selects a discrimination task. For example, computer system 1700 may select an element comprising a standard discriminator activation function. In some embodiments, computer system 1700 may select the target set of a detector element or a known set and specify the discrimination task as discriminating between the selected set and its complement. In some embodiments, computer system 1700 may select the task of discriminating two known sets.

[0565] In block 604, in some embodiments, computer system 1700 may select a set of data 318976815.3 102Docket No.230458PCT items with target values for the task selected in block 603. For example, in some embodiments, computer system 1700 may select only data items for which the selected node makes an implicit error. In some embodiments, computer system 1700 may select data items on which the selected node has a close call. In some embodiments, computer system 1700 may avoid selecting a data item that is beyond a specified exclusion limit. In some embodiments, computer system 1700 may avoid selecting a data item that has been delegated from the selected node. In some embodiments, computer system 1700 may avoid selecting a data item that is classified correctly by the network despite being an error for the selected node.

[0566] In block 605, in some embodiments, computer system 1700 may determine whether the implicit targets for the node are linearly separable by finding weights that minimize T2 – T1, subject to the constraints that the input to the activation function is less than or equal to T2 for any data item with the lower value target and the input to the activation function is greater than or equal to T1 for any data item with the higher value target. For example, if the input to the activation function is a weighted affine sum of the values from the incoming connections to the node, computer system 1700 may find the optimum weights by linear programming. In some embodiments, computer system 1700 may select a non-linear objective function to optimize in block 605. In such a case, computer system 1700 may find the weights by non-linear programming with linear constraints. Linear and non-linear programming subject to linear constraints are well known to those skilled in the art of mathematical programming.

[0567] In some embodiments, computer system 1700 may use incremental growth (103 of Figure 1 and 504 of Figure 5) to build a hybrid network without any back propagation, neither back propagation of derivatives (612 of Figure 6) nor back propagation of data examples (613 of Figure 6 and 510 of Figure 5). For example, in some embodiments, computer system 1700 may repeatedly drop targets (607 of Figure 6).

[0568] In some embodiments, in block 605, computer system 1700 may create a new element, with an activation function such as a linear threshold function or other monotonic function with the weights and discrimination threshold computed in block 605.

[0569] In block 606, computer system 1700 checks whether the minimum for T2 – T1 is less than or equal to 0. If so, the selected data items are linearly separable. In this case, computer system 1700 proceeds to block 609. Otherwise, computer system 1700 proceeds to block 607.

[0570] In block 607, in some embodiments, computer system 1700 may determine whether to 318976815.3 103Docket No.230458PCT drop some of the selected targets and, if so, which ones. In some embodiments, computer system 1700 may choose to proceed without dropping any of the selected targets.

[0571] In some embodiments, the decision of whether to drop selected data items for a node may involve a cost / performance trade-off. In some embodiments, computer system 1700 may make the decision based on fixed criteria specified by the system design. In some embodiments, the HNLMS may do a cost / performance analysis for the specific situation of the selected node or unit. In some embodiments, computer system 1700 and the HNLMS, for example, may test the cost performance trade-off, preferably on data that has been set aside from the training data.

[0572] In block 608, computer system 1700 decides whether to repeat the constrained optimization after dropping some of the target data items. If so, computer system 1700 returns to block 605. Otherwise, computer system 1700 proceeds to block 609. In some embodiments, computer system 1700 may repeatedly drop target data items until there is a reduction in the number of errors. Unless there are two identical data items in which one is an error and one is not, as long as there are remaining errors, computer system 1700 can eventually reduce the number of errors because a set of two non-identical data items is always linearly separable.

[0573] In block 609, in some embodiments, computer system 1700 may check the performance of the selected unit on data items that were not selected in block 604, if any. Since the weights of incoming connections may have changed, the performance of the selected element on these non-selected data items may have changed.

[0574] In block 610, in some embodiments, computer system 1700 may determine whether to select additional data items for the element selected or created in block 603.

[0575] In some embodiments, the decision of whether to select additional data items for a node may involve a cost / performance trade-off. In some embodiments, computer system 1700 may make the decision based on fixed criteria specified by the system design. In some embodiments, the HNLMS may do a cost / performance analysis for the specified situation of the selected node or unit. In some embodiments, for example, computer system 1700 and the HNLMS may test the cost performance trade-off, preferably on data that has been set aside from the training data.

[0576] In block 611, computer system 1700 selects whether to back propagate data examples, derivatives, or both or neither. If computer system 1700 decides to back propagate data 318976815.3 104Docket No.230458PCT examples, it proceeds to block 613. If computer system 1700 decides to back propagate derivatives, it proceeds to block 611. If compute system 1700 decides to back propagate both, it may proceed in parallel to both block 612 and block 613. If computer system 1700 decides to propagate neither, computer system 1700 proceeds directly to block 614. Computer system 1700 may choose to back propagate neither, for example, if computer system 1700 determines to make and freeze a copy of the subnetwork of the new linear threshold function. If, other than linear threshold functions, every discriminator trained on the task selected in block 603 is eventually dropped from the network having been replaced by one of more linear threshold functions with frozen subnetworks, as suggested in block 510 of Figure 5, the final trained network will have no paths by which to back propagated derivatives for the selected discrimination task to the input variables, which prevents an adversary from using back propagation of a gradient to compute an adversarial attack. In some embodiments, computer system 1700 may use this strategy for multiple discrimination tasks without limit.

[0577] In block 613, in some embodiments, computer system 1700 may back propagate data examples. In some embodiments, computer system 1700 may back propagate only errors and close calls. In some embodiments, computer system 1700 may use a criterion for a data item being a close call for the purpose of back propagation that accepts more data items as close calls than the criterion for being a selected data item in block 604.

[0578] In block 612, in some embodiments, computer system 1700 may back propagate derivatives using a substitute derivative function such as illustrated in Figure 3C.

[0579] In block 614, in some embodiments, computer system 1700 may determine whether to select additional discrimination tasks based on a specified stopping criterion.

[0580] Figure 7 is a flow chart of an illustrative embodiment of an aspect of hidden state space modeling in an aspect of the invention. Note that the meaning of the word “hidden” in the phrase “hidden state space model” is very different from the phrases “hidden layer” or “hidden node” in discussions of a layered neural network. In discussions of a layered neural network, all the layers except the output layer and their nodes may be referred to as “hidden.” The input values are also not considered to be “hidden.” However, the values of the state variables in a hidden state space model are hidden more deeply. In a hidden state space model, the activations of all the nodes are considered observable values. In some embodiments, some of the values stored in cells may also be considered as observables values. However, in a hidden state space model in a hybrid network, the state variables are not considered to be observable values, although estimates of their values may be stored in 318976815.3 105Docket No.230458PCT cells.

[0581] In some embodiments, computer system 1700 may model the hidden state variables as unobserved random variables. In some embodiments, computer system 1700 may model the observable variables as random variables whose values are conditional on the unobserved hidden state variables. From the values of the observed variables, computer system 1700 may be able to make estimates of hidden variables by applying Bayes’ rule.

[0582] In block 701, in some embodiments, computer system 1700 may specify a space of cells comprising hidden state variables. For example, for an image, in some embodiments, computer system 1700 may formulate a two-dimensional rectangular grid of cells. A hidden state variable may then represent an interpretation of a local region in the image. Alternately, in some embodiments, computer system 1700 may formulate a two-dimensional hexagonal tiling or other tiling of the plane. In some embodiments, the hidden state space may represent a conditional random field.

[0583] For data represented as a sequence, in some embodiments, computer system 1700 may formulate a one-dimensional sequence of cells. A hidden state space variable may then represent the state of a time-varying process at a specified time. In some embodiments, the hidden state space may represent a hidden Markov process.

[0584] In some embodiments, computer system 1700 may specify an adjacency graph, that is, a graph in which each cell is connected to its neighboring cells, such as the four neighbors (or eight neighbors if corner neighbors are counted) in a rectangular grid or the six neighbors in a hexagonal grid. In a sequence of cells, computer system 1700 may connect each cell with the preceding cell and the following cell in the sequence of cells.

[0585] In some embodiments, computer system 1700 may represent the relationship of adjoining parts in a mereology as an adjacency graph. In some embodiments, computer system 1700 may determine the mapping from elements in a mereology to cells in a hybrid network specifically for each input data item by a process of alignment (Figure 12).

[0586] In block 702, in some embodiments, computer system 1700 may specify one or more hidden state variables. In some embodiments, a hidden state variable may be a variable with values selected from a finite set. In some embodiments, a hidden state variable may be a continuous-valued variable.

[0587] In some embodiments, computer system 1700 may represent a hidden state by an n- tuple of variables. 318976815.3 106Docket No.230458PCT

[0588] In block 703, in some embodiments, computer system 1700 may obtain a model of the relationship between the hidden state variables and the observable variables. In some embodiments, the relationship may represent an arbitrary numerical relationship. In some embodiments, the model may represent the conditional probability of the observed variables in and around the grid point for a hidden state cell, conditioned on the value of the hidden state variables. In some embodiments, the model may represent relationships of state variables in adjacent cells in an adjacency graph. For example, the graph may be the adjacency graph of the parts in a mereology model of a hypothesized object being detected.

[0589] In block 704, in some embodiments, computer system 1700 may obtain a model of co-occurrence of specified state pairs in adjacent cells. For example, computer system 1700 may represent the probability of a specific hidden state variable as a probability conditioned on the value of the hidden state variable in an adjacent position in the adjacency graph.

[0590] In some embodiments, computer system 1700 may train an abstract model of the degree of association of state values in cells that are in adjacent positions in the adjacency graph with learned parameters that are not necessarily trained to model conditional probabilities. In some embodiments, computer system 1700 may train a directional learned parameter between the state values in an ordered pair of adjacent cells. In some embodiments, computer system 1700 may train a degree of association parameter in each direction. In some embodiments, computer system 1700 may train a non-directional degree of association between the learned parameters for an unordered pair of adjacent cells.

[0591] In block 705, in some embodiments, computer system 1700 may select one or more paths in the state space for evaluation. For example, in a layer in a convolutional neural network, computer system 1700 may select a path of cells corresponding to a path of grid points in an image. In a model of a sequence, computer system 1700 may select a forward sequence or a backward sequence. More generally, in some embodiments, computer system 1700 may choose an arbitrary path through an adjacency graph.

[0592] In block 706, in some embodiments, computer system 1700 may compute the probability of a state given the observed context. In some embodiments, computer system 1700 may update learned parameters of an abstract model of the degree of association of ordered or unordered pairs of state values of cells that are adjacent in an adjacency graph.

[0593] In block 707, in some embodiments, computer system 1700 may update the model for observed variables given the estimated distribution of hidden state space variables. 318976815.3 107Docket No.230458PCT

[0594] In block 708, in some embodiments, computer system 1700 may update the model for the conditional probability model for state values in adjacent cells or for an abstract model of the directional or non-directional association of state values in adjacent cells.

[0595] In block 709, in some embodiments, computer system 1700 determines whether to select a new path through the graph, based on specified criteria. If so, computer system 1700 returns to block 705. Otherwise, computer system 1700 proceeds to block 710.

[0596] In block 710, in some embodiments, computer system 1700 determines, based on a specified criterion, whether to train a different model for the observed variables and of association of state values in adjacent cells. If so, computer system 1700 returns to block 703. Otherwise, computer system 1700 proceeds to block 711.

[0597] In block 711, in some embodiments, computer system 1700 may determine whether to perform an analysis of a different state space formulation. If so, computer system 1700 returns to block 701. Otherwise, computer system 1700 is done with the process illustrated in Figure 7.

[0598] Figure 8 is a flow chart of an illustrative embodiment of the operation of sensible classification with a trained hybrid network and rapid matching. The illustrative embodiment comprises defenses against potential disturbances in the data. The illustrative embodiment also comprises methods to reduce the amount of computation required for a classification. The illustrative embodiment also provides for continual training while using rapid matching and continual training during inference in an aspect of the invention.

[0599] In block 801, computer system 1700 obtains a trained system.

[0600] In block 802, computer system 1700 receives a data item to be classified.

[0601] In block 803, in some embodiments, computer system 1700 may implement an active defense against perturbed data using sensibility data switching, as discussed in association with block 416. In an active defense, the network comprises one or more data switches by which computer system 1700 selects among a plurality of activation functions or among a plurality of nodes such that the selected activation for the data item received in block 802 is in a relatively flat region and is not near the boundary of the region.

[0602] In block 804, in some embodiments, computer system 1700 may perform a fast preliminary classification. In some embodiments, computer system 1700 may compute a preliminary classification using a lower resolution image or other simplified representation of the data item received for classification. In some embodiments, computer system 1700 may 318976815.3 108Docket No.230458PCT use simpler models in place of the full hybrid network or in place of some of the units.

[0603] In some embodiments, computer system 1700 may perform a table lookup of a precomputed classification for a low-bit representation of the input to a unit.

[0604] In some embodiments, computer system 1700 may perform bottom-up component detection. In some embodiments, computer system 1700 may perform the bottom-up component detection using a simplified network. In bottom-up component detection, computer system 1700 may first perform classification and detection of smaller units, such as smaller objects or parts of an object in an image or short sound segments in speech or other audio. In bottom-up component detection, computer system 1700 may then classify a selected subset of larger units depending on the identities of the best scoring smaller units.

[0605] In some embodiments, computer system 1700 may do hypothesis pruning of some larger units based on their scores relative to the best scoring units at a stage in the bottom-up component detection.

[0606] In some embodiments, computer system 1700 may create a short list of the best scoring alternative classification for one or more units or for the full classification network. In some embodiments, computer system 1700 may then skip some computations for hypotheses that are not on the computed short list. In some embodiments, computer system 1700 may substitute a specified back-off score for a hypothesis that is not on a short list.

[0607] In some embodiments, computer system 1700 may coordinate bottom-up component detection with alignment with an adjacency graph, as described in association with block 805.

[0608] In block 805, in some embodiments, computer system 1700 may do a fast classification based on an alignment with an adjacency graph. Training based on alignment of adjacency graphs is discussed in association with Figure 12.

[0609] As an example of alignment as a preliminary to classification by the full hybrid network, computer system 1700 may detect some of the parts in the periphery of an object. Computer system 1700 may then align the detected parts and other elements in the periphery with a mereology of the object. Computer system 1700 may then align and classify parts in the interior of the mereology. In some embodiments, computer system 1700 may coordinate this alignment based fast classification with bottom-up component detection, as discussed in association with block 804.

[0610] In block 806, computer system 1700 may do other sequential processing in the cells. For example, computer system 1700 may compute a hidden state space model, as discussed 318976815.3 109Docket No.230458PCT in association with Figure 7. As another example, computer system 1700 may trace out line segments, curves, and / or contours by sequentially connecting a chain of pairwise associations or similarities of adjacent elements. Computer system 1700 may use this sequential processing for tasks such as: (1) determining if two local regions are connected, (2) finding the contour around an object, (3) finding the boundary separating two regions, or (4) solving a maze.

[0611] In block 807, in some embodiments, computer system 1700 may perform checks on the preliminary results.

[0612] In some embodiments, computer system 1700 may verify the classification results against results obtained by other means. For example, computer system 1700 may compare the results from the current preliminary match against the results obtained from other preliminary matches.

[0613] In some embodiments, in an image recognition task, if the current preliminary match uses a low-resolution representation of an image, computer system 1700 may compare the results of the current preliminary match with the results of classification using a higher resolution image. In some embodiments, computer system 1700 may accelerate the classification of the higher resolution image by pruning the computation based on the preliminary match results.

[0614] In some embodiments, computer system 1700 may verify the preliminary results against a higher resolution image at critical points in the mereologies of the short list of best candidate classifications of the preliminary match. For example, computer system 1700 may verify the classification of parts along the periphery of the aligned mereology.

[0615] In some embodiments, computer system 1700 may compute a back propagation from the output activation of each candidate classification on the short list of the preliminary match. In some embodiments, computer system 1700 may compute this back propagation using a network other than the network used in the preliminary match and / or may compute the back propagation from a higher resolution image. In some embodiments, computer system 1700 may then check each node in the network to see if the node has made an error relative to an implicit local target, such as described in association with block 508 of Figure 5. In some embodiments, computer system 1700 may augment the short list of answers from the preliminary match by adding candidate answers obtained by changing the activations of selected nodes that have activations close to a threshold that would change an error or close call on an implicit local target. 318976815.3 110Docket No.230458PCT

[0616] In some embodiments, computer system 1700 may verify the results of the preliminary match against the results obtained from classification using a different source of knowledge or a different source of input data. For example, in classification of speech or other audio, computer system 1700 may verify the preliminary results against classification using different signal processing of the audio signal. As another example, in speech recognition or hand writing recognition, computer system 1700 may compare the results obtained from recognizing phonemes or letters with the results obtained using a word sequence language model.

[0617] In some embodiments, computer system 1700 may verify the results of the preliminary match by using a parametric generator. In some embodiments, computer system 1700 may adjust the parameters of the parametric generator to fit the observed input data subject to constraints of the parameters of the generator being consistent with one of the choices on the short list of candidate answers from the preliminary match. In some embodiments, computer system 1700 may select the answer for which the output of the parametric generator best matches the input data to the classifier. In some embodiments, computer system 1700 may compare the output of the parametric generator to the input in order to prune the short list of candidate answers or to add to the short list.

[0618] In some embodiments, computer system 1700 may add additional answers to the short list from prior experience of errors among confusable output categories. For example, the HNLMS may maintain a confusion matrix of errors made by previous version of the network being developed or by other systems trained for the same classification task.

[0619] In some embodiments, computer system 1700 may use abductive reasoning to evaluate each candidate answer on the short list. For example, in some embodiments, computer system 1700 may apply abductive reasoning to explain potential causes for a candidate answer to have a poor score. As a specific example, if a candidate word in a speech recognition task matches well except for one phoneme based on formant tracking, computer system 1700 may check the hypothesis that the identification of the formants in the formant tracking may be errorful because two formants that are close in frequency may form a single peak in the frequency spectrum.

[0620] In block 808, in some embodiments, computer system 1700 may determine whether to do additional preliminary classification. If not, computer system 1700 proceeds to block 809. If so, computer system 1700 returns to block 803 to do an additional preliminary classification. In some embodiments, computer system 1700 may do a more complex 318976815.3 111Docket No.230458PCT classification based on a previous preliminary classification. In some embodiments, computer system 1700 may do a new preliminary classification designed to be different and diverse from previous preliminary classifications.

[0621] In block 809, in some embodiments, computer system 1700 may conduct tests to detect whether the data item received in block 802 has been disturbed by an adversarial attack or other disturbance that might change the classification. In some embodiments, computer system 1700 may check the network to verify that the nodes and activation functions satisfy the rules for elementary, first-level sensibility as discussed in association with Figure 2. In some embodiments, to detect a potential adversarial attack or other disturbance, computer system 1700 may use a diverse set of canary networks, as discussed in association with block 415 of Figure 4.

[0622] In block 810, in some embodiments, computer system 1700 may acquire additional data. In some embodiments, the additional data may comprise additional training data. In some embodiments, the additional data may comprise data obtained during operation of the current classifier system or from other deployed classifier systems. In some embodiments, the data may be generated or synthesized data. In some embodiments, computer system 1700 may generate extra data in regions selected by computer system 1700 by analyzing the results of the preliminary classifications.

[0623] In block 811, in some embodiments, computer system 1700 may apply the techniques of continual learning and growth such as those discussed in association with Figure 1.

[0624] In some embodiments, computer system 1700 may make additions and modifications to the network that are customized for the data item received in block 802.

[0625] In block 814, in some embodiments, computer system 1700 may optionally perform controlled semi-supervised learning using unlabeled data. In some situations, during deployment there may be no verification that a classification is correct. In some embodiments, computer system 1700 may acquire other data that is not labeled or classified. In some embodiments, during deployment a fraction of the classification results may be explicitly or implicitly confirmed by the end users or by another person while other classification results may be unconfirmed.

[0626] In some embodiments, computer system 1700 may perform additional training including unconfirmed data obtained during deployment by tentatively labeling each unconfirmed result with the best scoring label from the classifier. This process using 318976815.3 112Docket No.230458PCT unconfirmed labels from the classifier is known as semi-supervised learning, which is well known to those skilled in the art of machine learning. Semi-supervised learning often improves performance of machine learning systems when there is a limited amount of labeled training data. On the other hand, in some circumstances, semi-supervised learning may caus...

Claims

Docket No.230458PCT CLAIMS What is claimed is:

1. A method comprising: (a) growing, by a programmed computer system, a generative AI system by adding one or more explainable network elements to the generative AI system, wherein: the generative AI system comprises one or more machine learning networks trained such that the generative AI system generates textual passages in response to prompting and context; and each of the one or more explainable network elements is trained to discriminate two or more explainable sets of training data items for the generative AI system; (b) after adding the one or more explainable network elements, performing updated training, by the programmed computer system, of the generative AI system with the one or more explainable network elements added; (c) after (a) and (b), determining, by the programmed computer system, whether continued growth of the generative AI system is required; upon a determination at (c) that continued growth of the generative AI system is required, repeating steps (a) through (c); and upon a determination at step (c) that continued growth of the generative AI system is not required, deploying the generative AI system to generate textual passages.

2. The method of claim 1, wherein the generative AI system comprises an autoregressive next word predictor.

3. The method of claim 1, wherein the generative AI system comprises a large language model (LLM).

4. The method of claim 3, wherein the LLM comprises a transformer network.

5. The method of claim 4, wherein the transformer network comprises one or more attention blocks, and wherein each of the one or more attention blocks comprises one or more attention heads.

6. The method of claim 1, wherein: 318976815.3 285Docket No.230458PCT the generative AI system comprise a hybrid network; and one of the one or more explainable network elements comprises a cell of the hybrid network.

7. The method of claim 1, wherein: the generative AI system comprises a neural network; and one of the one or more explainable network elements comprises a node that is added to the neural network of the generative AI system.

8. The method of claim 1, wherein the one or more explainable network elements comprise a plurality of nodes organized into a new layer that is added to one of the machine learning networks of the generative AI system.

9. The method of claim 7, wherein the neural network comprises a transformer network, wherein the transformer network comprises one or more attention blocks, and wherein each of the one or more attention blocks comprises one or more attention heads.

10. The method of claim 7, wherein at least one of the one or more explainable network elements comprises a first node that is trained to classify each data item as belonging to a specific set out of a collection of 2 or more explainable sets.

11. The method of claim 10, wherein adding the one or more explainable network elements comprises: selecting, by the programmed computer system, a target node of the generative AI system; training, by the programmed computer system, based on computation of a regression, the first node to discriminate between a first set of one or more classification categories and a second set of one or more classification categories; and adding the first node to the generative AI system.

12. The method of claim 11, wherein training the first node to discriminate comprises: computing, by the programmed computer system, for two or more classification categories, the regression of a number of instances of each classification category as function of activation value of the target node; selecting, by the programmed computer system, a first set of one or more classification categories with positive regression coefficients; 318976815.3 286Docket No.230458PCT selecting, by the programmed computer system, a second set of one or more classification categories with negative regression coefficients; and training, by the programmed computer system, the first node to discriminate between the first and second sets.

13. The method of claim 11, wherein training the first node to discriminate comprises: computing, by the programmed computer system, for two or more classification categories, the regression of a number of instances of each category as function of activation value of the target node; selecting, by the programmed computer system, a first set of one or more classification categories with regression coefficients greater than a first threshold value; selecting, by the programmed computer system, a second set of one or more classification categories with regression coefficients less than a second threshold value, wherein the second threshold value is less than or equal to the first threshold value; and training, by the programmed computer system, the first node to discriminate between the first and second sets.

14. The method of one of claims 11 to 13, wherein the regression comprises a linear regression.

15. The method of one of claims 11 to 13, wherein the regression comprises a monotonic regression.

16. The method of one of claims 11 to 13, wherein each of the classification categories comprises a word.

17. The method of one of claims 11 to13, wherein: the neural network comprises a transformer network; and each of the classification categories comprises a named state of a hidden model of the transformer network.

18. The method of claim 10, wherein the updated training comprises back propagating to weights on direct incoming connections to the first node, without back propagating deeper than the weights on the direct incoming connections to the first node. 318976815.3 287Docket No.230458PCT 19. The method of claim 7, wherein: the one or more explainable network elements comprises a first node and a second node; and the updated training comprises soft tying the first and second nodes.

20. The method of claim 19, wherein the updated training further comprises counter tying a connection to the first node to a corresponding connection of the second node.

21. The method of claim 7, wherein: the one or more explainable network elements comprises a first node and a second node; and the updated training comprises imposing a node-to-node regularization link between the first and second nodes.

22. The method of claim 1, wherein deploying the generative AI system to generate textual passages comprises: generating, by the generative AI system, first and second textual passages; presenting, by a back-end computer system that comprises the generative AI system and the programmed computer system, the first and second textual passages to a user via a user interface; receiving, by the back-end computer system, a selection by the user of one of the first and second textual passages; and further training, by the programmed computer system, the generative AI system based on the selection.

23. The method of claim 1, wherein deploying the generative AI system to generate textual passages comprises: generating, by the generative AI system, a textual passage; generating, by an explanatory system of a back-end computer system that comprises the generative AI system and the programmed computer system, an explanation relevant to the textual passage generated by the generative AI system; presenting, by the back-end computer system, the textual passage and the explanation to a user via a user interface; receiving, by the back-end computer system, feedback from the user with respect to the explanation; and 318976815.3 288Docket No.230458PCT further training, by the programmed computer system, the generative AI system based on the feedback.

24. The method of claim 23 wherein: generating the explanation comprises generating first and second explanations, wherein each of the first and second explanations is relevant to the textual passage generated by the generative AI system; and the feedback comprises a selection by the user of either the first or second explanations.

25. The method of claim 23, wherein the feedback comprises a rating from the user of the explanation.

26. The method of claim 1, wherein growing the generative AI system comprises adding a probability model to the generative AI system, wherein the probability model is associated with one of the one or more explainable network elements.

27. The method of claim 26, wherein adding the probability model comprises adding the probability model from a repository of the programmed computer system.

28. The method of claim 26, wherein the probability model comprises a non-parametric probability model.

29. The method of claim 28, wherein the non-parametric probability model comprises a non-parametric conditional probability model.

30. The method of claim 26, wherein: the one or more explainable network elements comprise first and second explainable network elements; and the probability model comprises a non-parametric correlation correction model that is conditioned on activation values of the first and second explainable network elements.

31. The method of claim 26, wherein the probability model comprises a parametric probability model. 318976815.3 289Docket No.230458PCT 32. The method of claim 26, wherein the probability model comprises a template-type model.

33. The method of claim 32, wherein the template-type model represents a probability of activation values of a specified set of the one or more explainable network elements conditional on a specified event in a portion of a sequence that has not yet been observed by the generative AI system during training of the generative AI system.

34. The method of claim 26, wherein deploying the generative AI system comprises, for each position in a sequence for a textual passage to be generated: computing, by the generative AI system, a list of multiple candidate items for the position in the sequence based on a context for the textual passage; estimating, by the generative AI system, a probability of each candidate item in the list of multiple candidate items for the position in the sequence; and adding, by the generative AI system, one of the candidate items from the list to the position in the sequence based on the probabilities.

35. The method of claim 34, wherein: the generative AI system comprises multiple generative sub-systems that, collectively, generate the textual passage; and the method further comprises: broadcasting, by the generative AI system, activation values of a selected embedding node from each of the multiple generative sub-system to the other generative sub- systems; and revising, by the generative AI system, the selected embedding node in each of the generative sub-system using data-dependent regularization links across the selected embedding nodes.

36. The method of claim 35, wherein the data-dependent regularization links comprise “is equal to” relationship regularization links.

37. The method of claim 36, wherein deploying the generative AI system further comprises: selecting, by the generative AI system a set of key linguistic units relevant to the context, 318976815.3 290Docket No.230458PCT wherein each linguistic unit is a word or phrase; and loading, by the generative AI system, multiple example textual passages in which one or more of the key linguistic units in the set of key linguistic units appear in each of the multiple example textual passages, wherein computing the list of multiple candidate items comprises computing the list based on, at least in part, (i) word counts in the example textual passages and (ii) autoregressive prediction scores for words for the position.

38. The method of claim 28, wherein deploying the generative AI system comprises, for each position in a sequence for a textual passage to be generated: selecting, by the generative AI system a set of key linguistic units relevant to the context, wherein each linguistic unit is a word or phrase; loading, by the generative AI system, multiple example textual passages in which one or more of the key linguistic units in the set of key linguistic units appear in each of the multiple example textual passages; computing, by the generative AI system, a list of multiple candidate items for the position in the sequence based on a context for the textual passage, wherein computing the list of multiple candidate items comprises computing the list based on (i) word counts in the example textual passages and (ii) outputs from the non-parametric probability model; estimating, by the generative AI system, a probability of each candidate item in the list of multiple candidate items for the position in the sequence; and adding, by the generative AI system, one of the candidate items from the list to the position in the sequence based on the probabilities.

39. The method of claim 38, wherein: loading the multiple example textual passages comprises testing, by a semantic analysis system of a back-end computer system, candidate textual passages for semantic similarity with the context; and the back-end computer system further comprises the generative AI system and the programmed computer system.

40. The method of claim 39, wherein deploying the generative AI system to generate textual passages comprises: generating, by the generative AI system, first and second textual passages; 318976815.3 291Docket No.230458PCT presenting, by the back-end computer system, the first and second textual passages to a user via a user interface; receiving, by the back-end computer system, a selection by the user of one of the first and second textual passages; and further training, by the programmed computer system, the generative AI system based on the selection.

41. The method of claim 39, wherein: the back-end computer system further comprises an explanatory system; and deploying the generative AI system to generate textual passages comprises: generating, by the generative AI system, a textual passage; generating, by the explanatory system, an explanation relevant to the textual passage generated by the generative AI system; presenting, by the back-end computer system, the textual passage and the explanation to a user via a user interface; receiving, by the back-end computer system, feedback from the user with respect to the explanation; and further training, by the programmed computer system, the generative AI system based on the feedback.

42. A system comprising: a generative AI system that comprises one or more machine learning networks trained such that the generative AI system generates textual passages in response to prompting and context; and a programmed computer system in communication with the generative AI system, wherein the programmed computer system is configured to: (a) grow the generative AI system by adding one or more explainable network elements to the generative AI system, wherein each of the one or more explainable network elements is trained to discriminate two or more explainable sets of training data items for the generative AI system; (b) after adding the one or more explainable network elements, perform updated training of the generative AI system with the one or more explainable network elements added; (c) after (a) and (b), determine, whether continued growth of the generative AI system 318976815.3 292Docket No.230458PCT is required; and upon a determination at (c) that continued growth of the generative AI system is required, repeating steps (a) through (c); and wherein upon a determination at step (c) that continued growth of the generative AI system is not required, the generative AI system is configured to generate textual passages.

43. The system of claim 42, wherein the generative AI system comprises an autoregressive next word predictor.

44. The system of claim 42, wherein the generative AI system comprises a large language model (LLM).

45. The system of claim 44, wherein the LLM comprises a transformer network.

46. The system of claim 45, wherein the transformer network comprises one or more attention blocks, and wherein each of the one or more attention blocks comprises one or more attention heads.

47. The system of claim 42, wherein: the generative AI system comprise a hybrid network; and one of the one or more explainable network elements comprises a cell of the hybrid network.

48. The system of claim 42, wherein: the generative AI system comprises a neural network; and one of the one or more explainable network elements comprises a node that is added to the neural network of the generative AI system.

49. The system of claim 42, wherein the one or more explainable network elements comprise a plurality of nodes organized into a new layer that is added to one of the machine learning networks of the generative AI system.

50. The system of claim 48, wherein the neural network comprises a transformer network, wherein the transformer network comprises one or more attention blocks, and wherein each of the one or more attention blocks comprises one or more attention heads. 318976815.3 293Docket No.230458PCT 51. The system of claim 48, wherein at least one of the one or more explainable network elements comprises a first node that is trained to classify each data item as belonging to a specific set out of a collection of 2 or more explainable sets.

52. The system of claim 51, wherein the programmed computer system is configured to add the one or more explainable network elements by: selecting a target node of the generative AI system; training, based on computation of a regression, the first node to discriminate between a first set of one or more classification categories and a second set of one or more classification categories; and adding the first node to the generative AI system.

53. The system of claim 52, wherein the programmed computer system is configured to train the first node to discriminate by: computing for two or more classification categories, the regression of a number of instances of each classification category as function of activation value of the target node; selecting a first set of one or more classification categories with positive regression coefficients; selecting a second set of one or more classification categories with negative regression coefficients; and training the first node to discriminate between the first and second sets.

54. The system of claim 52, wherein the programmed computer system is configured to train the first node to discriminate by: computing, for two or more classification categories, the regression of a number of instances of each category as function of activation value of the target node; selecting, a first set of one or more classification categories with regression coefficients greater than a first threshold value; selecting, a second set of one or more classification categories with regression coefficients less than a second threshold value, wherein the second threshold value is less than or equal to the first threshold value; and training, the first node to discriminate between the first and second sets. 318976815.3 294Docket No.230458PCT 55. The system of one of claims 52 to 54, wherein the regression comprises a linear regression.

56. The system of one of claims 52 to 54, wherein the regression comprises a monotonic regression.

57. The system of one of claims 52 to 54, wherein each of the classification categories comprises a word.

58. The system of one of claims 52 to 54, wherein: the neural network comprises a transformer network; and each of the classification categories comprises a named state of a hidden model of the transformer network.

59. The system of claim 51, wherein the programmed computer system is configured perform the updated training by, in part, back propagating to weights on direct incoming connections to the first node, without back propagating deeper than the weights on the direct incoming connections to the first node.

60. The system of claim 48, wherein: the one or more explainable network elements comprises a first node and a second node; and the programmed computer system is configured to perform the updated training by, in part, soft tying the first and second nodes.

61. The system of claim 60, wherein the programmed computer system is configured to perform the updated training further by, in part, counter tying a connection to the first node to a corresponding connection of the second node.

62. The system of claim 48, wherein: the one or more explainable network elements comprises a first node and a second node; and the programmed computer system is configured to perform the updated training by, in part, imposing a node-to-node regularization link between the first and second nodes.

63. The system of claim 42, wherein: 318976815.3 295Docket No.230458PCT the system comprises a back-end computer system that comprises the generative AI system and the programmed computer system; the generative AI system is configured to generate first and second textual passages; the back-end computer system is configured to: present the first and second textual passages to a user via a user interface; and receive a selection by the user of one of the first and second textual passages; and the programmed computer system is further configured to train the generative AI system based on the selection.

64. The system of claim 42, wherein: the system comprises a back-end computer system that comprises the generative AI system, the programmed computer system, and an explanatory system; the generative AI system is configured to generate a textual passage; the explanatory system is configured to generate an explanation relevant to the textual passage generated by the generative AI system; the back-end computer system is configured to: present the textual passage and the explanation to a user via a user interface; and receive feedback from the user with respect to the explanation; and the programmed computer system is configured to further train the generative AI system based on the feedback.

65. The system of claim 64, wherein: the explanatory system is configured to generate the explanation by, in part, generating first and second explanations, wherein each of the first and second explanations is relevant to the textual passage generated by the generative AI system; and the feedback comprises a selection by the user of either the first or second explanations.

66. The system of claim 64, wherein the feedback comprises a rating from the user of the explanation.

67. The system of claim 42, wherein the programmed computer system is configured to grow the generative AI system by, in part, adding a probability model to the generative AI system, wherein the probability model is associated with one of the one or more explainable network elements. 318976815.3 296Docket No.230458PCT 68. The system of claim 67, wherein the programmed computer system is configured to add the probability model from a repository of the programmed computer system.

69. The system of claim 67, wherein the probability model comprises a non-parametric probability model.

70. The system of claim 69, wherein the non-parametric probability model comprises a non-parametric conditional probability model.

71. The system of claim 67, wherein: the one or more explainable network elements comprise first and second explainable network elements; and the probability model comprises a non-parametric correlation correction model that is conditioned on activation values of the first and second explainable network elements.

72. The system of claim 67, wherein the probability model comprises a parametric probability model.

73. The system of claim 67, wherein the probability model comprises a template-type model.

74. The system of claim 73, wherein the template-type model represents a probability of activation values of a specified set of the one or more explainable network elements conditional on a specified event in a portion of a sequence that has not yet been observed by the generative AI system during training of the generative AI system.

75. The system of claim 67, wherein to generate textual passages, the generative AI system is configured to, for each position in a sequence for a textual passage to be generated: compute a list of multiple candidate items for the position in the sequence based on a context for the textual passage; estimate a probability of each candidate item in the list of multiple candidate items for the position in the sequence; and add one of the candidate items from the list to the position in the sequence based on the 318976815.3 297Docket No.230458PCT probabilities.

76. The system of claim 75, wherein: the generative AI system comprises multiple generative sub-systems that, collectively, generate the textual passage; and the generative AI system is configured to: broadcast activation values of a selected embedding node from each of the multiple generative sub-system to the other generative sub-systems; and revise the selected embedding node in each of the generative sub-system using data- dependent regularization links across the selected embedding nodes.

77. The system of claim 76, wherein the data-dependent regularization links comprise “is equal to” relationship regularization links.

78. The system of claim 77, wherein, to generate textual passages, the generative AI system is configured to, in part:: select a set of key linguistic units relevant to the context, wherein each linguistic unit is a word or phrase; load multiple example textual passages in which one or more of the key linguistic units in the set of key linguistic units appear in each of the multiple example textual passages; and compute the list based on, at least in part, (i) word counts in the example textual passages and (ii) autoregressive prediction scores for words for the position.

79. The system of claim 69, wherein, to generate textual passages, the generative AI system is configured to, in part, for each position in a sequence for a textual passage to be generated: select a set of key linguistic units relevant to the context, wherein each linguistic unit is a word or phrase; load multiple example textual passages in which one or more of the key linguistic units in the set of key linguistic units appear in each of the multiple example textual passages; compute a list of multiple candidate items for the position in the sequence based on a context for the textual passage, wherein computing the list of multiple candidate items comprises computing the list based on (i) word counts in the example textual passages and (ii) outputs from the non-parametric probability model; 318976815.3 298Docket No.230458PCT estimate a probability of each candidate item in the list of multiple candidate items for the position in the sequence; and add one of the candidate items from the list to the position in the sequence based on the probabilities.

80. The system of claim 79, further comprising a back-end computer system that comprises the generative AI system, the programmed computer system, and a semantic analysis system that is configured to test candidate textual passages for semantic similarity with the context.

81. The system of claim 80, wherein the generative AI system is configured to generate textual passages by, in part: generating, by the generative AI system, first and second textual passages; presenting, by the back-end computer system, the first and second textual passages to a user via a user interface; receiving, by the back-end computer system, a selection by the user of one of the first and second textual passages; and further training, by the programmed computer system, the generative AI system based on the selection.

82. The system of claim 80, wherein: the back-end computer system further comprises an explanatory system; and the generative AI system is configured to generate textual passages by, in part: generating, by the generative AI system, a textual passage; generating, by the explanatory system, an explanation relevant to the textual passage generated by the generative AI system; presenting, by the back-end computer system, the textual passage and the explanation to a user via a user interface; receiving, by the back-end computer system, feedback from the user with respect to the explanation; and further training, by the programmed computer system, the generative AI system based on the feedback. 318976815.3 299

Images