Training multi-stage malleable hybrid networks

Abstract

Multi-stage hybrid network integrates relationship regularization links and explainable elements to improve alignment with human values, explainability, robustness, and efficiency. The network comprises neural components, event prediction elements, and probability models across multiple stages, with relationship constraints enforcing structured knowledge representation. Explainable elements provide interpretable rationales for decisions, enhancing transparency. Training incorporates supervised learning, human-guided refinement, semi-automated knowledge engineering, and adversarial robustness techniques. A Socratic reasoning module detects contradictions and refines outputs for logical consistency. Indexed model elements enable dynamic memory optimization for improved efficiency. Candidate outputs may be scored, verified, or selected using neural and symbolic criteria. The invention supports retry loops and configurable subsystem pipelines to improve output quality. Applications include text generation, speech recognition, translation, and decision support. By combining structured constraints, human oversight, and modular architectures, the system improves the trustworthiness, safety, and adaptability of AI systems across diverse modalities and tasks.

Description

IN THE UNITED STATES RECEIVING OFFICEPatent Cooperation Treaty (PCT) Application For:TRAINING MULTI-STAGE MALLEABLE HYBRID NETWORKSInventors: James K. Baker and Alexander LitzenbergerPRIORITY CLAIM

[0001] The present application claims priority to the following United States provisional patent applications: (1) Serial No. 63 / 668,504, filed July 8, 2024, titled “Training Multi-Stage Hybrid Networks”; (2) Serial No. 63 / 775,007, filed March 20, 2025, titled “Training Multi- Stage Hybrid Networks”; and (3) Serial No. 63 / 801,489, filed May 7, 2025, titled “Training Multi-Stage Malleable Hybrid Networks,” all of which are incorporated herein by reference.CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The present application is related to the following international applications:(i) PCT / US24 / 12671, with an international filing date of January 24, 2024, titled “Training Dynamic Hybrid Al Networks”; (ii) PCT / US24 / 30324, with an international filing date of May 21, 2024, titled “Training Human-Guided Al Networks”; and (iii) PCT / US24 / 39132, with an international filing date of July 23, 2024, titled “Explainable Adaptable Artificial Intelligence Networks.”BACKGROUND

[0003] Modem generative Al systems based on the transformer architecture have recently demonstrated remarkable improvements, generating text that is nearly indistinguishable from text that is written by a human author. However, despite this improvement in verisimilitude, many problems remain.

[0004] The pretraining of foundational transformer-based large language models (LLMs) requires a large amount of high-end, power-hungry computation. Text generation systems may “hallucinate” or fabricate false information and present it as factual. Both text and image generation systems may generate content that plagiarizes and that may violate copyrights. They may produce content that fails to align with human values and / or fail to meet the goals of their developers and end users. These systems may generate content that contains proprietary information such as confidential business and customer information. Generative Al systems are vulnerable to adversarial attacks. Current Al systems lack judgment. Theymake mistakes that no sensible person would make. Classification systems are often overconfident, giving a very high score to a wrong answer. They fail to follow the principle of Socrates to “know what you know and know what you do not yet know.”

[0005] A deeper problem is that, although state-of-the-art large language models trained on trillions of words of text have an enormous amount of stored knowledge, they lack understanding. Although a large language model can generate text that is hard to distinguish from text written by a human, not only may it produce text that is a fabrication as mentioned above, with additional context in a dialogue, they may produce two statements that contradict each other. If the human points out the contradiction, the Al system may still fail to understand. The Al system may agree with the human out of politeness but still maintain the truth of both contradictory statements.

[0006] For example, a large language model text generator requested to summarize a paragraph may produce text that is not a decent summary. A successful prompt engineering technique called “few shot learning” is to include a few examples of paragraphs and their summaries. However, the success of few-shot learning highlights the fact that the system does not understand the word “summarize.” Few-shot learning demonstrates the value of even a little human guidance, but it also shows that the remarkable quality of generated text is achieved despite, rather than because of, true understanding.

[0007] Some recent Al systems have eliminated the need for human knowledge engineering, which has been touted as a significant advance. For an even longer period, there has been an effort to avoid human guidance during the training of an Al system. However, the elimination of human knowledge engineering decreases the explainability and controllability of the Al system. It makes alignment with human values more difficult. It may decrease the ability of the Al system to generalize to new data, thus decreasing the efficacy. It may decrease the sensibility and thereby decrease the security. Rather than eliminating human knowledge engineering, a better goal is to semi-automate human knowledge engineering, making it more efficient and more effective as measured by the amount of human labor and the amount of computation.SUMMARY

[0008] The present invention relates, in one general aspect, to a multi-stage hybrid network architecture that significantly enhances the explainability, reliability, efficiency, and controllability of Al systems during both training and inference. Through a modular orchestration of generation, recognition, and verification components — combined withstructured constraints and symbolic reasoning — the invention addresses core limitations in modem Al, such as hallucinations, inconsistency, and low-confidence outputs.

[0009] A key technical improvement, in various embodiments, is the use of relationship regularization links — structural connections between elements across network stages that enforce semantic, logical, or numeric conditions (e.g., “is-equal-to,” “is-less-than”). These links act as persistent constraints during training and inference, ensuring model outputs remain bounded and consistent under perturbation. This contrasts with standard deep learning models trained solely via backpropagation, which often yield unstable or adversarially sensitive outputs.

[0010] The invention can also incorporate explainable elements — modules trained to output structured, interpretable rationales alongside task predictions. These may include symbolic parsers, attention-based metrics, or logic-consistency modules. Outputs are scored for explainability and can be filtered based on thresholds, enhancing trust and auditability in high-stakes domains like finance, legal, healthcare, and autonomous control.

[0011] To improve consistency and reduce hallucinations, the system can use a Socratic verification framework, applying sensibility checks both before and after synthesis stages. These checks may include logical rules, domain-specific templates, or consistency measures, and may be aggregated across multiple verifiers. This dual-stage validation improves robustness and safety of outputs used in downstream applications.

[0012] In some embodiments, a retry mechanism is invoked when outputs fail verification. The system dynamically adjusts generation parameters (e.g., sampling temperature, beam width) to regenerate improved candidates. The retry loop can use feedback from recognition and sensibility subsystems, optionally incorporating user feedback to refine future behavior. Candidate outputs can also be scored and compared in parallel, using both neural and symbolic evaluations to select a preferred output based on verifiable metrics. This multi-candidate selection process addresses the inefficiency and uncertainty of single-shot generation methods.

[0013] The invention can further include support for human-in-the-loop workflows, including options for human feedback, guided training, and interactive supervision. Human operators may select preferred outputs from among candidates or adjust parameters of generation and recognition models via a human-guided development, training, and control system (HGDTCS). Logging of candidate scores and outcomes supports continual learning and adaptive fine-tuning.

[0014] Training of the multi-stage hybrid network may leverage a combination oftechniques: event-prediction elements for probabilistic inference, speculative decoding pipelines, indexed modular components for memory efficiency, and mixture-of-experts subsystems for output diversity. Semi -automated knowledge engineering further reduces manual curation by integrating symbolic resources like ontologies and rule sets during training.

[0015] Security and integrity are enhanced, in various embodiments, through adversarial robustness modules, which use counter-tying constraints and symbolic checks to detect and neutralize perturbations or malformed inputs. The system can suppress synthesis or mark uncertain outputs based on confidence evaluations or rule violations, improving safety and performance in mission-critical environments.

[0016] Another innovative feature is the configurable architecture of subsystems — including generation, recognition, verification, and synthesis — which may be arranged in flexible sequences and applied across multiple data modalities such as text, speech, structured sequences, and graphs. In some embodiments, the invention further includes complementary subsystems trained with differing unit structures — for example, a speech recognition subsystem operating on phonemes, syllables, or words and a language model subsystem operating on tokens — wherein the computer system coordinates communication between the subsystems via intermediate word-level hypotheses. The computer system may compute directed acyclic graphs (DAGs) or tree structures to represent continuation hypotheses in each subsystem, and use alignment of those continuations to facilitate communication and improve joint performance. This configuration allows hybrid systems to combine the domainspecific strengths of each subsystem without requiring them to share identical representational units. Such flexibility enables parallel processing, cross-verification between models, and specialized inference pipelines tailored to domain requirements.

[0017] Overall, the disclosed invention addresses significant technical challenges in Al system design, including explainability, robustness, output control, modality generalization, and efficient retraining. Unlike conventional architectures, it enables structured, multi-phase output validation, dynamic reconfiguration during inference, and modular reuse across tasks. These advancements make Al systems more transparent, resilient, and adaptable to real- world conditions, satisfying the technical improvement criteria under patent law and overcoming limitations of purely statistical or black-box Al methods. These and other benefits that can be realized through embodiments of the present invention will be apparent from the description that follows.FIGURES

[0018] Various embodiments of the present invention are described herein by way of example in connection with the following figures.

[0019] Figure l is a flow chart of illustrative embodiment of an aspect of the invention in which computer system 800 may build and train a malleable multi-stage hybrid network.

[0020] Figure 1 A is a flowchart of an illustrative embodiment of an aspect of the invention in which computer system 800 may build and train a malleable hybrid network without multiple stages or without adding stages to an existing multi-stage network.

[0021] Figure 2 is a diagram of the major subsystems of an illustrative embodiment of the software of a Human-Guided Development, Training and Control System (HGDTCS), including a human team (260).

[0022] Figure 2A is a diagram of two lists of examples of use cases, a list of goals, and a list of types of techniques for improving performance on goals as a framework for discussing the relative effectiveness of the different types of techniques.

[0023] Figure 3 is a Venn diagram of example techniques that computer system 800 may use during training to improve the performance of the system on goals listed in block 210 of Figure 2.

[0024] Figure 3 A is a flow chart of an illustrative embodiment of the process of informal semi -automated knowledge engineering. In informal semi-automated knowledge engineering, computer system 800 may train an Al system to represent and utilize a body of knowledge with some human guidance but substantially less human labor than would be required for humans to manually produce a representation of the knowledge in a form that may be utilized by computers.

[0025] Figure 3B is a flow chart of an illustrative embodiment of a process by which computer system 800 may use position synchronous beam pruning to construct a directed acyclic graph (DAG) representing sequences of tokens, words and phrases with associated scores representing the output of a stage of a multi-stage network as a generator or recognizer producing a sequence of text as output.

[0026] Figure 3C is an illustrative embodiment of an aspect of the invention in which computer system 800 trains a hidden Markov process model based on sequences of training data and / or from the DAG in the interstage of a multi-stage network and / or from the event predictions and look-back probability models.

[0027] Figure 3D is a flow chart of an illustrative embodiment of a guide for selecting which of the techniques associated with the Venn diagram of Figure 3 to use for the goal ofimproving the alignment with human values.

[0028] Figure 3E is a flow chart of an illustrative embodiment of a guide for selecting which of the techniques associated with the Venn diagram of Figure 3 to use for the goal of improving the explainability of the system.

[0029] Figure 3F is a flow chart of an illustrative embodiment of a guide for selecting which of the techniques associated with the Venn diagram of Figure 3 to use for the goal of improving the efficacy of the system.

[0030] Figure 3G is a flow chart of an illustrative embodiment of a guide for selecting which of the techniques associated with the Venn diagram of Figure 3 to use for the goal of improving the efficiency of the system.

[0031] Figure 3H is a flow chart of an illustrative embodiment of a guide for selecting which of the techniques associated with the Venn diagram of Figure 3 to use for the goal of improving the security and sensibility of the system.

[0032] Figure 31 is a flow chart of an illustrative embodiment of a guide for selecting which of the techniques associated with the Venn diagram of Figure 3 to use for the goal of improving the generalizability of the system.

[0033] Figure 3 J is a high-level system diagram which computer system 800 may apply to various embodiments, each comprising two or more system designs with complementary strengths and weaknesses to achieve better overall performance in the combined result 384.

[0034] Figures 3K, 3L, 3M, 3N, and 30 are system diagrams illustrating alternative subsystem configurations for recognition, generation, sensibility verification, and synthesis across different data modalities. Each figure represents a variant pipeline architecture used in different Al tasks. Figure 3K illustrates a configuration for speech and text processing, showing a Text Generation System, Speech Recognition System, Sensibility Verification Control System, and Speech Synthesis System. Figure 3L shows a similar architecture specialized for sequence data, including Sequence Generation, Sequence Recognition, Sensibility Verification, and Sequence Synthesis subsystems. Figure 3M presents a variant focused on graph data, incorporating Graph Unit Generation, Graph Data Recognition, Sensibility Verification, and Graph Data Synthesis systems. Figure 3N illustrates a reordered architecture for sequence processing where Sequence Recognition precedes Sequence Generation, followed by Sensibility Verification and Synthesis. Figure 30 highlights a multistage verification-centric pipeline with Sequence Generation, Sequence Recognition, Sensibility Verification, and a Final Verification system. These figures collectively illustrate how the system can be adapted to various data types and processing goals while maintaininga modular sensibility verification architecture.

[0035] Figure 4 is a flow chart of an illustrative embodiment of an aspect of the invention in which computer system 800 performs generation using a multi-stage hybrid network.

[0036] Figure 5 is a flow chart of an illustrative embodiment of an aspect of the invention in which computer system 800 may train and add an explainable element as a companion element to a base element in a hybrid network.

[0037] Figure 6 is a diagram of a system comprising multiple subsystems in which computer system 800 may use a mixture-of-experts subsystem and / or a combining network with relationship links to train the subsystems to be diverse while combining and consolidating the output values of the subsystems.

[0038] Figure 6A is a block diagram of an illustrative embodiment of a generative model of a mixture of generators of explainable detector clusters, with each cluster represented by a generator-detector pair, in which the word “cluster” is given a broad definition as any set for which an explainable detector may be trained.

[0039] Figures 6B and 6C represent an illustrative embodiment of a system and method for pairing a generator with a detector that is useful in various embodiments of this invention. The illustrated method is used in the illustrative embodiment of Figure 6A to build a mixture of generators model (MGM). Figure 6B is a block diagram of an illustrative embodiment, and Figure 6C is a flow chart of an illustrative example of training the system. In the illustrative embodiment, detector 642 of Figure 6B and generator 652 of Figure 6B are trained by computer system 800 to detect and generate data related to a single explainable detector cluster.

[0040] Figure 7 is a diagram of a multi-stage system which computer system 800 may implement with N > 1 stages and N-l interstage subsystems. The diagram is of a software construct, not a system diagram of hardware.

[0041] Figure 8 is a system diagram of an illustrative example of a multiprocessor computer system 800 that may be used to implement aspects of the invention.

[0042] Figure 9 is a system diagram of an illustrative embodiment of a particular implementation of a multiprocessor computer system such as illustrated in Figure 8 that supports cascaded computations to implement aspects of the invention.

[0043] Figure 10 is a flow chart of an illustrative embodiment of an aspect of the invention in which computer system 800 trains a translation system using a multi-path chain of one-way translations in which each link in the chain translates from a source language to a target language.

[0044] Figure 11 is a flow chart of an illustrative embodiment of an aspect of the invention in which computer system 800 uses a multi-path chain of paired language translations to compute a robust composite translation.

[0045] Figure 12 is a flow chart of an illustrative embodiment for training alignment models related to images. In some embodiments, computer system 800 may train a model to align elements of the hybrid network with elements of a human interpretable representation of knowledge, such as a mereology, ontology, grammar, or semantic network.

[0046] Figure 13 is a flow chart and Figure 13 A is a corresponding block diagram of an illustrative embodiment of the training and use of a system for image generation with human guidance.

[0047] Figure 14 is a flow chart of an illustrative embodiment of the process of building and training an interactive, human-guided writer’s assistant.DESCRIPTION

[0048] The processes described in this disclosure may be implemented with a computer system executing software. An illustrative embodiment of a multiprocessor computer system 800 that can be employed with various embodiments of the present invention is illustrated in Figure 8. In some embodiments, the processor modules may be one or more clusters of hardware modules located in a data center. In some embodiments, the modules may be separate computers which are connected to each other over a wide area network (WAN) or over the internet. In some embodiments, one or more computer modules may be high-end workstations. In some embodiments, one or more computer modules may be laptop computers or edge devices.

[0049] Some embodiments of computer system 800 may comprise an Al-based Human- Guided Development, Training System and Control System (HGDTCS, 850 in Figure 8). The diagram of an illustrative embodiment of an HGDTCS is shown in Figure 2.

[0050] In some embodiments of the present invention, the computer system 800 may build, train, and / or use an Al system with a modular network architecture comprising one or more multi-stage hybrid networks. A hybrid network architecture may comprise one or more elements such as neural nodes, units (defined below), event prediction elements, and probability model elements. In some embodiments, event prediction elements and probability model elements may be indexed, enabling computer system 800 to store an inactive indexed element in slower memory or secondary storage and move a model to higher speed memory only when needed.

[0051] Definitions: As used herein, the following terms shall have the meanings indicated below. However, these definitions are provided for clarity and convenience and are not intended to limit the scope of the invention. Unless expressly stated otherwise, terms should be interpreted as having their ordinary and customary meanings as understood by those skilled in the art, and the definitions provided herein should be read in the context of the entire disclosure.

[0052] Neural node: A neural node is an element of a neural network that represents a simple artificial model of a neuron. An example of such a node N may comprise a set of incoming connections, a weight W;,N for each incoming connection i, a value Vi,N,(d) for each incoming connection for each data item d, a constant bias bx, and a single output value of a specified activation function, such as f\(d) = fN( iwi,Nvi,N d) + bw). In a neural network, the weights W;,N and the biases bnare called “learned parameters” whose values are adjusted during training to optimize a specified objective. An example of an objective for a classification problem is an error loss function, such as the Kullbach-Leibler divergenceP^log ( %V k'A J

[0053] Unit: A hybrid network may also have other types of elements, such as a “unit.” A unit may have multiple components and multiple output values. For example, a hybrid network unit may have a separate output for two alternatives being distinguished as well as a third output “BOTH” that represents that both of the two things are plausible for the current data item, and a fourth output “NEITHER” that represents that neither alternative is plausible. In some embodiments, a unit may have additional output values, such as a fifth output “I DON’T KNOW” that represents the unit asserting that there is insufficient information to decide. In some embodiments, a unit may comprise memory elements that may record attributes of the unit, and cells that may comprise computational capabilities as well as memory.

[0054] Element: An element is a node, a unit, a cell, or a probability model. In some contexts, a connection may be called an “element”.

[0055] Implicit local objective: In a neural network trained by back propagation, during training in some embodiments, computer system 800 may assign any node an implicit local objective negative of the sign of the back propagated derivative of the error loss function. That is, an implicit objective of 1.0 if the back propagated derivative is less than zero and an implicit objective of -1.0 if the back propagated derivative is greater than zero. The negative of the sign is used because the error loss function is to be minimized.

[0056] Error prediction node: In some embodiments, given a base node B, computer system 800 may create one or two new nodes as auxiliary nodes. In some embodiments, computer system 800 may create an auxiliary node J to predict for each data item whether base node B will make an error on its implicit local object. In some embodiments, computer system 800 may create a first new node N node to predict for each data item with a negative implicit local whether base node B will make an error on its implicit local objective and train a second auxiliary node P to predict for each data item with a positive implicit objective whether base node B will make an error on its implicit local objective. An illustrative embodiment of implicit local objectives and error prediction nodes is discussed in US patent 11, 847,566, which is incorporated herein by reference in its entirety.

[0057] Relationship regularization link: An element-to-element regularization link from element A to element B is a unidirectional link, with an associated specified relationship, in which element A is the source element and element B is the receiving element. In some embodiments, computer system 800 may impose a regularization penalty or reward on the receiving element B depending on whether the values of specified variables associated elements A and B satisfy the specified relationship for a given input data item. In some embodiments, computer system 800 may create a bidirectional link by using two unidirectional links. Unlike a connection in a non-recurrent neural network, a regularization link may go between a pair of elements in either direction, regardless of the relative positions of the pair of elements within the network. A regularization link may go from a source node in one network to a receiving node in a second network. For a relationship regularization link, computer system 800 may associate a specified quantitative relationship between a value associated with the first element and a value associated with the second element. For example, the value associated with a node may be the output value of the activation function for a specified data item input to the network. Exemplary details about knowledge sharing links are provided in U.S. Patent 11,741,340, issued August 29, 2023, titled “Data-dependent node-to-node knowledge sharing by regularization in deep learning,” assigned to D5AI LLC, by inventors James K. Baker et al., which is incorporated herein by reference in its entirety.

[0058] In some embodiments, computer system 800 may associate a hyperparameter such as a multiplicative factor on the regularization penalty or reward to control the strength of the regularizing effect. In some embodiments, computer system 800 may associate multiple hyperparameters comprising a multiplicative factor, which may have a negative value, and an additive term and / or a limiting value.

[0059] Some example relationships are “is-equal-to,” “is-not-equal-to,” “is-greater-than,”and “is-less-than.” In some embodiments, if the specified relationship is not satisfied, computer system 800 may impose a regularization penalty on the receiving element. In some embodiments, computer system 800 may use the “is-equal-to” relationship during training to push the value associated with the receiving element to be closer to the value associated with the source element. This relationship regularization is called “soft tying.”

[0060] In some embodiments, computer system 800 may use an “is-not-equal-to” relationship during training to push the value associated with the receiving element to be different from the value associated with the source element. For the “is-not-equal-to” relationship, computer system 800 may impose a regularization penalty that is maximized when the two values are equal or a regularization reward that is minimized when the two values are equal. Imposing an “is-not-equal-to” relationship is called “counter tying.”

[0061] In some embodiments, computer system 800 may use counter tying to create diversity between two elements. In some embodiments, computer system 800 may use counter tying between a plurality of pairs of elements in two subsystems to increase the diversity between the subsystems. More details about counter tying nodes of a neural network are provided in U.S. Patent 11,321,612, issued May 3, 2022, titled “Self-organizing partially ordered networks and soft tying learned parameters, such as connection weights,” assigned to D5AI LLC, by inventors James K. Baker et al., which is incorporated herein by reference in its entirety.

[0062] In some embodiments, computer system 800 may use relationships such as “is- greater-than” or “is-less-than” to represent the expected relationship for pairs of elements in a network representing structured knowledge such as a mereology, an ontology, or a collection of sets and subsets and operations such as union, intersection, and complement. In some embodiments, computer system 800 may use relationship links to represent logic primitives such as “AND,” “OR,” “IF-THEN,” “XOR,” or “NOT.”

[0063] Probability Model: A probability model is a model of the likelihood of one or more specified events occurring. In some embodiments of the invention, the probability may be conditional on one or more other specified events having occurred. In some embodiments, a parametric probability model is represented by a set of sufficient statistics, such as the mean and variance of a single Gaussian distribution or the means and covariance matrix of a multivariate Gaussian distribution.

[0064] In some embodiments, a non-parametric probability model is a discrete distribution over a specified finite set of possible values of specified observable events. The model is non-parametric in the sense of non-parametric statistics. In the sense of learnedparameters in a machine learning system, the number of learned parameters is the number of distinct points in the data space of possible values for a single scalar or a vector of predicted random variables.

[0065] Template Model: Is a model that does a computation similar to the estimate of the logarithm of the probability density function of a simple parametric probability model. In some embodiments, the logarithm of the probability density function may be computed by a portion of a neural network. In some embodiments, however, the learned parameters of the template model may be estimated by a statistical procedure, such as the Expectation- Maximization (EM) algorithm, rather than by gradient descent. In some embodiments, the learned parameters for a template model may initially be learned from a single example (one- shot learning). For example, computer system 800 may estimate the mean other measure of central tendency as the observed value in the one sample. Computer system 800 may estimate the measure of spread of the probability using a procedure such as empirical Bayes.

[0066] Hybrid network: A hybrid network may comprise any of several kinds of elements, such as neural nodes (usually just called “nodes”), units, explainable elements, event prediction elements, non-parametric probability models, parametric probability models, and indexed elements.

[0067] Explainable set: An explainable set is a set of data items that can be explained with a simple, relatively short explanation.

[0068] Explainable element: An explainable element is a node or other element that can be explained as a discriminator between two explainable sets or as a detector of an explainable set.

[0069] Partially explainable element: A partially explainable element is a node or other element that can be explained as a discriminator of two recognizable sets or as a detector of a recognizable set. A recognizable set is a set for which there is a detector in a pretrained system that can detect data items in the set and reject data items not in the set to a degree of accuracy specified by the system designer or a Human-guided development, training, and control system (HGDTCS).

[0070] Decisive element: A decisive element is a discriminator element that has been trained with an additional objective to increase the separation in the activation between the data items of two sets being discriminated from each other.

[0071] Safe or sensible element: Is an element, preferably as decisive element with a hyperparameter controlled by the HGDTCS to change the activation function to have a middle interval with a constant value or with a derivative with a magnitude less than aspecified value to make a change in the input value of less than a specified magnitude unable to change the discrimination result from one decisive interval to the other decisive interval.

[0072] Indexed element: An indexed element is a member of a family of elements such that each member of the family may be accessed by an associated numerical index, by a hash code, or by some specified retrieval procedure. In some embodiments, computer system 800 may locate inactive indexed elements in slower memory or in secondary storage.

[0073] Stage: A stage is a portion of a multi-stage hybrid network. Each stage may comprise one or more network layers. Each stage may also comprise one or more component subsystems such as attention blocks. Between successive stages, computer system 800 may enable specialized stage-specific input and output variables and inter-stage communication in addition to the values passed by connections from source nodes and units in a lower stage to receiving nodes and units in an upper stage. The concept of “stage” has properties and capabilities different from those of a layer or a set of layers in a neural network. For example, in some embodiments, computer system 800 may generate output values from a middle stage with target values such that computer system 800 may train the subnetwork up through the middle stage with target output values as well as training the full network, including the middle stage. In some embodiments, a middle stage may also receive direct input values, so the stage may also be trained separately from the lower stages as well as being trained in combination with the lower stages. In some embodiments, computer system 800 may use these additional training modes to improve the efficacy and efficiency of the training process.

[0074] Although most of the discussion in this disclosure is about multi-stage networks, some of the techniques for improvement in efficacy, efficiency and other goals may also apply to regular or “single stage” networks. Similarly, although some of the techniques only apply to networks that generate and / or recognize sequences, many of the techniques also apply to networks that do not involve sequences. Figure 3 is a Venn diagram showing how many techniques apply to various combinations of goals for each of the three overlapping sets of networks: (a) all networks (b) networks involving sequences, and (c) multi-stage networks.

[0075] Event prediction element: A hybrid network may comprise one or more event prediction elements. In some embodiments, an event prediction element may predict a change in the probability of a detectable “future” event conditioned on one or more observable conditions in the current context. A “future” event means an event that is not part of the current context. In a sequence that is being processed backwards by a module in a multi-stage hybrid network, a “future” event may refer to an event at an earlier position in the sequence and thus is not yet in the current context of the current module as the computation proceedsbackwards through the sequence.

[0076] In some embodiments, a “future” event for one module or stage may already be in the processed context of a second module or stage. In some embodiments, the detectable event may be that at least a specified number of a specified set of items will occur in a specified interval of a subsequence that is not currently part of the context for the current module. For example, the conditioning event may be that the activation of a specified node is greater than a specified value and the predicted event may be that at least one of a specified set of words occurs within the next N words for a specified value N. With some prediction models, computer system 800 may estimate an increase or a decrease in the likelihood of a specified future event condition on the observed random variable such as the activation value of a specified node or element.

[0077] In some embodiments, computer system 800 may represent the change in probability as the ratio of the probability given the observable condition to the unconditional probability. In some embodiments, computer system 800 may represent the change in the logarithm of the ratio of the conditional probability to the unconditional probability of the event. In some embodiments, computer system 800 may “prune” an event predictor for which the magnitude of the predicted log ratio is less than a specified minimum value.

[0078] Implicit objective and implicit error: In some embodiments, computer system 800 may define the implicit objective (or “target”) for the output of a node, or for a specified output of a unit, as the sign of the back propagated derivative of an error loss function for the network. That is, the target value may be +1.0 for a positive back propagated derivative and - 1.0 for a negative back propagated derivative. In some embodiments, computer system 800 may then designate the output value as an implicit error if the output is below a specified threshold T1 when the implicit target is positive or if the output is greater than a specified threshold T2 when the implicit target is negative. The implicit objective of a node may also be called the “local objective” or the “node-specific” objective.

[0079] Candidate list: a candidate list is a list of possible output values from a system or subsystem. In some embodiments, in a multi-stage recognition of generation system, computer system 800 may limit the output of a system, subsystem, or a stage of a multi-stage system to a subset of the total set of possible output values. For example, in some embodiments, computer system 800 may prune a candidate list, dropping candidates with values less than a specified value. In some embodiments, the specified value may be relative to the best score value.

[0080] Draft system, master system and speculative decoding: In some embodiments,computer system 800 may use a smaller, faster system as a preliminary system (called a “draft” system) to supply a list of candidates and a larger system (called as a verification system) to further evaluate candidates produced by the draft system. In recognition or generation of sequences, computer system 800 may use a draft system to produce candidate lists for future positions in the sequence beyond the current position being processed by the verification system. This process of future potential candidates is called “speculative decoding.”

[0081] Efficacy: In some embodiments, computer system 800 may attempt to maximize efficacy. Efficacy is a generalization of error rate as a measure of performance. That is, computer system 800 may attempt to measure and optimize a desired result for a specified use case including other objectives in addition to the error rate on a classification task. In some embodiments, computer system 800 may use dialogues with one or more end users to determine user preferences. In some embodiments, computer system 800 may use dialogues with one or more system developers to align the system being developed with human values. In some embodiments, computer system 800 may implement specialized subsystems to evaluate criteria other than error rate.

[0082] Efficiency: The relative efficiency of a system is a specified weighted sum of the computational resources used for a specified computational task, such as pre-training, finetuning, recognition, or generation. The components of the weighted sum may include the amount of CPU time, the amount of GPU time, the amount of CPU RAM, the amount of GPU RAM, and possibly additional terms. The relative weights for the respective components may depend on the type of CPU and GPU. For sustainability, efficiency may include the amount of electric power used for the computer equipment plus the amount of power needed for cooling and air conditioning.

[0083] Sensibility: Sensibility is a measure of the quality of the errors made by a system rather than the quantity. In qualitative terms, if a human observer would say “That error is stupid, no sensible person would make that mistake,” then the error is not sensible. In a classification task, the minimum Bayes error rate may be non-zero. However, in preferred embodiments of the invention, the goal is for the system to make no nonsensible mistakes.

[0084] Sustainability: Sustainability depends on several factors, including the impact on the environment in generating the electrical power required, the need over time to replace obsolete components, the possible degradation in the performance of the system from extended use or from failure to successfully adapt to changing needs and requirements.

[0085] Figure l is a flow chart of an illustrative embodiment of an aspect of the inventionin which computer system 800 may build and train a malleable multi-stage hybrid network that works with sequences, and Figure 1 A is a flowchart of a corresponding illustrative embodiment of an aspect of the invention in which computer system 800 may build and train a malleable hybrid network without adding any stages. In the illustrative embodiment, computer system 800 may arrange the set of stages to be trained in a linear order which herein is called a “chain” of stages. A subset of the chain of stages connected in a linear order consistent with the linear order of the full chain is herein called a “subchain.”

[0086] In block 101, in some embodiments, computer system 800 may obtain an initial version of the network to be trained and may also obtain the training and validation data. In block 101, computer system 800 may specify the chain of stages of the initial network. In some embodiments, the initial network may have only a single stage. In some embodiments, the initial network may be a network that has previously been trained on different data. In some embodiments, the initial network may be a network that has previously been trained and that is to be further trained on the same data or on a subset of the previous data.In block 102, in some embodiments, computer system 800 may select a subset of the training data for the current round of training.

[0087] In block 103, in some embodiments, computer system 800 may select the validation data for the current round of training. In preferred embodiments, the set of validation data is disjoint from the set of training data.

[0088] In block 104, in some embodiments, computer system 800 may select a subchain of the chain of stages.

[0089] In block 105, in some embodiments, computer system 800 may set stage-specific controls for each stage in the selected subchain.

[0090] In some embodiments, the network may comprise a mixture of different types of stages. For example, some stages may be generator networks and some stages may be classification or recognition networks. In some networks, in block 105, computer system 800 may specify for each stage in the selected subchain whether the stage is a generator or a classifier. In some embodiments, a stage may comprise a “mixture of generators” structure, which comprises multiple pairs of coordinated generators and detectors. In some embodiments, for some use cases, a stage may comprise other varieties of network types as subnetworks in a single stage.

[0091] In some embodiments, in block 105, for a system that works with sequences, computer system 800 may specify for each stage in the subchain being trained whether theprocessing is to proceed going forward through the sequence or backward through the sequence.

[0092] As an example, in speech recognition, a multi-stage hybrid network may comprise a set of one or more stages for acoustic recognition of words and subword components, a set of stages for syntactic analysis and parsing, a set of stages comprising a large language model trained from text, and a set of stages for speech synthesis to verify hypothesized word sequences. In some embodiments, in block 105, computer system 800 may assign these roles to specific subchains.

[0093] In block 106, in some embodiments, computer system 800 may train the selected subchain using any of the goal -oriented techniques discussed in association with Figure 3. Each technique is aimed at improving the performance of the system on one or more specific goals from the list of goals in block 210 for Figure 2:1) Alignment with human values2) Efficacy3) Efficiency and sustainability4) Explainability and controllability5) Security and sensibility6) Generalizability

[0094] Some techniques apply to multiple goals. In the discussion of the techniques associated with Figure 3, the techniques are grouped first by whether (a) the technique applies to all neural networks and hybrid networks, (b) the technique applies to all networks associated with sequences, (c) the technique only applies to multi-stage networks, or (d) the technique only applies to hybrid networks not to non-hybrid neural networks. Within the list for each type of network, the techniques are grouped by which one or more goals the technique is primarily aimed at improving.

[0095] In block 107, in some embodiments, computer system 800 may test on validation data. The illustrative embodiment illustrated in Figure 1 is aimed at pretraining networks for which there may be a trillion or more items of training data. In some embodiments, computer system 800 may set aside for validation data a million or more sets of validation data of 10,000 or more items each. A validation set of 10,000 items is sufficient for most purposes.Furthermore, computer system 800 may use multiple validation sets when necessary. In some embodiments, computer system 800 may use fewer than a million validation sets. However, in some embodiments computer system 800 may continue training and validation during deployment, so additional validation data is always useful. In some embodiments, computer system 800 may continue to collect additional training and validation data during deployment.

[0096] In some embodiments with much less total training data, computer system 800 may use fewer data items in a validation set.

[0097] In block 108, in some embodiments, computer system 800 determines whether to proceed to deployment or to continue the development and training process, based on criteria specified by the HGDTCS. If the decision is to proceed to deployment, computer system 800 goes to block 114. Otherwise, computer system 800 proceeds to block 109.

[0098] In block 109, computer system 800 may deploy the system for use by end users.

[0099] In block 110, computer system 800 may interact with an end user. In some embodiments, computer system 800 may interact with the end user via the Al dialog system in the HGDTCS (block 250 in Figure 2). In some embodiments, computer system 800 may ask the user’s preference among two or more alternative generation subsequences. In some embodiments, computer system 800 may ask the user’s opinion about the output being produced. In some embodiments, the computer system 800 may ask the user for suggestions for improvement.

[0100] In some use cases, computer system 800 may interact with the end user as part of an interactive joint task. In such tasks, in some embodiments, computer system 800 may gather information about relative performance and user satisfaction through the interaction with the user during the task. Examples of use cases with interactive cooperative joint tasks include: a) Group conversational chats b) Personal tutoring c) The Al studying as a member of a student study group on a particular subject d) Learning a foreign language. e) Working as a teaching assistant in a course. f) Working as a graduate research assistant.g) Co-editing and proofreading h) Co-authorship i) Active discussion during joint production of images or video j) Production of music

[0101] In some use cases not involving a joint task, computer system 800 may interact with the end user to better adapt the system to the user’s preferences. Example use cases that are not explicitly interactive joint tasks:1) Text generation2) Speech recognition: a) Transcription of lectures spoken with a foreign accent b) Speaker dependent speech recognition and synthesis c) Interactive personal dictation / transcription system d) Closed captioning of video3) Economic series prediction4) Weather prediction5) Stock market prediction6) Train image generation and recognition based on detailed descriptions.7) Simultaneous training of translation for many languages

[0102] In block 111, in some embodiments, computer system 800 may proactively obtain additional data by reading reference materials written for humans. In some embodiments, computer system 800 may proactively contact a human with knowledge on a specific topic and may acquire information or knowledge from a conversation with the human. For example, to acquire additional knowledge about a language, computer system 800 may have a dialogue with a native speaker of that language. For knowledge in a technical subject, computer system 800 may contact and have a dialogue with a human expert on the subject.

[0103] In block 112, in some embodiments, from data and knowledge acquired in block 110 and / or in block 111, computer system 800 may perform adaptive training. In some embodiments, in block 112, computer system 800 may also do validation testing.

[0104] In block 113, in some embodiments, computer system 800 may determine, based on criteria specified by the HGDTCS, whether to do more full training in addition to the adaptive training in block 112. If so, computer system 800 proceeds to block 114. Otherwise, computer system 800 returns to block 109.

[0105] In block 114, in some embodiments, computer system 800 may determine, based on criteria specified by the HGDTCS, whether to add an additional stage to the system. If so, computer system 800 proceeds to block 115. Otherwise, computer system 800 returns to block 102.

[0106] In block 115, in some embodiments, computer system 800 may add a stage to the network. In some embodiments, computer system 800 may add the new stage as a new final stage. In some embodiments, computer system 800 may insert a new stage between two existing stages. Computer system 800 then returns block 114.

[0107] Figure 1 A is a flowchart of an illustrative embodiment of an aspect of the invention in which computer system 800 may build and train a malleable hybrid network without multiple stages or without adding stages to a multi-stage initial network.

[0108] The flow chart illustrated in Figure 1 A, and the associated procedures are based on the procedures in Figure 1, with the omission of blocks 104, 105, 113 and 114.

[0109] In block 121, in some embodiments, computer system 800 may obtain an initial network to be trained and may also obtain the training and validation data.

[0110] In block 122, in some embodiments, computer system 800 may select a subset of the training data for the current round of training.[OHl] In block 123, in some embodiments, computer system 800 may select the validation data for the current round of training. In preferred embodiments, the set of validation is disjoint from the set of training data.

[0112] In block 126, in some embodiments, computer system 800 may train the network using any of the techniques discussed in association with blocks 300 and 301 of the Venn diagram of Figure 3. Each technique is aimed at improving the performance of the system on one or more specific goals from the list of goals in block 210 for Figure 2:1) Alignment with human values2) Efficacy3) Efficiency and sustainability4) Explainability and controllability5) Security and sensibility6) Generalizability

[0113] Some techniques apply to multiple goals. In the discussion of the techniques associated with Figure 3, the techniques are grouped first by whether (a) the technique applies to all neural networks and hybrid networks (block 300), (b) the technique applies to all networks associated with sequences (block 301), or (c) the technique only applies to multi-stage networks, which computer system 800 may use as described in association with Figure 1 but which are not used in association with the illustrative embodiment of Figure 1 A. Within the list for each type of network, the techniques are grouped by which one or more goals the technique is primarily aimed at improving.

[0114] In block 127, in some embodiments, computer system 800 may test on validation data.

[0115] In block 128, in some embodiments, computer system 800 determines whether to proceed to deployment or to continue the development and training process, based on criteria specified by the HGDTCS. If the decision is to proceed to deployment, computer system 800 proceeds to block 129. Otherwise, computer system 800 returns to block 122 for further training.

[0116] In block 129, computer system 800 may deploy the system for use by end users.

[0117] In block 130, computer system 800 may interact with the end user. In some embodiments, computer system 800 may interact with the end user via the Al dialog system in the HGDTCS (block 250 in Figure 2). In some embodiments, computer system 800 may ask the user’s preference among two or more alternative generation subsequences. In some embodiments, computer system 800 may ask the user’s opinion about output being produced. In some embodiments, the computer system 800 may ask the user for suggestions for improvement.

[0118] In some use cases, such as those listed in association with block 110 of Figure 1, computer system 800 may interact with the end user as part of an interactive joint task. In such tasks, in some embodiments, computer system 800 may gather information about relative performance and user satisfaction through the interaction with the user during thetask.

[0119] In block 131, in some embodiments, computer system 800 may proactively obtain additional data by reading reference materials written for humans. In some embodiments, computer system 800 may proactively contact a human with knowledge on a specific topic and may acquire information or knowledge from a conversation with the human. For example, to acquire additional knowledge about a language, computer system 800 may have a dialogue with a native speaker of that language. For knowledge in a technical subject, computer system 800 may contact and have a dialogue with a human expert on the subject.

[0120] In block 132, in some embodiments, from data and knowledge acquired in block 130 and / or in block 131, computer system 800 may perform adaptive training. In some embodiments, in block 132, computer system 800 may also do validation testing.

[0121] In block 133, in some embodiments, computer system 800 may determine, based on criteria specified by the HGDTCS, whether to do more full training in addition to the adaptive training in block 132. If so, computer system 800 returns to block 122. Otherwise, computer system 800 returns to block 129, continuing the use of the system in deployment.

[0122] Figure 2 is a diagram of the major subsystems of an illustrative embodiment of the software of a Human-Guided Development, Training and Control System (HGDTCS).

[0123] The Human-Guided Development, Training System and Control System (HGDTCS) may comprise a human team (260) in cooperation with an Al system. In some embodiments, the Al system may be implemented on computer system 800 and may comprise a dialogue system (250), Training Coach Al (261), and a Socratic supervisor (262), which is discussed in association with Figure 4.

[0124] Dialogue Al (261): In block 250, in some embodiments, to communicate with human developers, end users and experts, computer system 800 may implement a dialogue system (250) with a knowledge base about the system (252) and a knowledge base obtained from humans (253). In some embodiments, computer system 800 may obtain or train a large language model fine-tuned to the task of communicating with the user and other humans on topics related to the system and the user’s values and preferences. Computer system 800 may acquire information for knowledge base 252 from several sources: (a) information supplied by the system designer or the human team (260) in the HGDTCS, (b) from documentation about the system written for human developers and users, (b) from techniques used for achieving the goals in block 210, especially explainability and controllability (goal 4 in block 210), (c) from dialogue with the user and adaptive training (block 110 of Figure 1 and block 130 of Figure 1A), and (d) from proactive learning (block 111 of Figure 1 and block 131 ofFigure 1A).

[0125] In block 253, in some embodiments, computer system 800 may also obtain from humans knowledge other than knowledge about the system being developed. In some embodiments, computer system 800 may obtain knowledge from human sources such as text documents that are available on the web or by other means. In some embodiments, computer system 800 may acquire knowledge by fine-tuning and / or proactive learning on a topic relevant to a use case, such as the cooperative interactive joint task example use cases discussed in association with block 110 of Figure 1.

[0126] In some embodiments, the HGDTCS system is self-documenting and capable of explaining itself by means of its dialogue Al subsystem (250), its knowledge bases (252 and 253), and other components. In some embodiments, the dialogue system 250 may have conversations (a) about the HGDTCS(200), (b) about the dialogue system itself (250), (c) about multi-stage hybrid networks in general and about the system currently being built and trained (252), (d) about the list of goals (210 of Figure 2) and the techniques that computer system 800 may use to achieve those goals, and (e) and about knowledge acquired from or about one or more humans, including the current user (242). In some embodiments, computer system 800 may have a dialogue with a system specifier, with a system designer or developer, with a member of a human team that is part of the HGDTCS (block 260 of Figure 2), with an independent expert on a topic (proactive learning in block 111 of Figure 1 or block 131 of Figure 1 A), or with an end user (block 110 of Figure 1 or 130 of Figure 1 A).

[0127] Training Coach Al (block 261): The Training Coach Al (261) is an illustrative embodiment of an Al system trained to assist in the development and training of a malleable multi-stage hybrid network system.

[0128] In some embodiments, computer system 800 may use incremental growth techniques in the development of the malleable multi-stage hybrid network system. In some embodiments, to efficiently support incremental growth, computer system 800 may specify different learning rates for different parts of the system. For example, in some embodiments computer system 800 may add one stage at a time to a system with as many as 100 or more stages. At any given point in the development process, a different learning rate may be appropriate for the most recent stage than for earlier stages. Each part of the network that might be assigned a different learning rate needs a separate hyperparameter to specify the learning rate. In some embodiments, computer system 800 may obtain or train an Al system as a Training Coach Al, in particular to be trained to manage the specification of the customized learning rate hyperparameters. In some embodiments, the Train Coaching Al mayalso be trained in other tasks, such as those discussed in the other paragraphs of this section. In some embodiments, there may be multiple Training Coach Al systems.

[0129] In some embodiments, the HGDTCS (200) may comprise a set of goals (210), such as:1) Alignment with human values2) Efficacy3) Efficiency and sustainability4) Explainability and controllability5) Security and sensibility6) GeneralizabilityFigures 3, 3D, 3E, 3F, 3G, and 31 illustrate the techniques from (1) to (195) that apply to particular subsets of the goals 1) to 6) above. Descriptions of individual techniques are given in detail in the discussion of Figure 3.

[0130] In block 241, as part of the HGDTCS, computer system 800 may manage frequent, recurring validation. In some embodiments, computer system 800 may use recurring validation as one of the techniques to support the goals listed in block 210, as discussed in association with Figure 3. In benchmark testing, validation testing on 10,000 items is common. In some embodiments, computer system 800 may use 10,000 or more new items for each validation test. For training a large foundational generative Al system, computer system 800 may obtain trillions of items as training data. In some embodiments, computer system 800 may set aside from one to ten percent of the training data, which is enough for up to a million or more validation tests. In some embodiments, computer system 800 may use fewer validation sets and / or may use fewer data items per validation test.

[0131] In block 242, as part of HGDTCS, computer system 800 may implement lifelong learning by adaptive training from interaction with the user and other humans.

[0132] In some embodiments, computer system 800 may learn from and / or teach the user using dialog system 250 in the dialogue in block 110 of Figure 1 or block 130 of Figure 1 A and block 242 of Figure 2. In some embodiments, this dialogue with one or more users may continue through the lifetime of the use of the system.

[0133] In some embodiments, in block 110 of Figure 1, block 130 of Figure 1 A, and block 242 of Figure 2, computer system 800 may collect efficacy data and feedback from the user during interactive use of the system. In some embodiments, computer system 800 may use this data and feedback for adaptive training (block 112 of Figure 1) of the system or for reinforcement learning from human feedback. Adaptive training is well-known to those skilled in the art of machine learning. Reinforcement learning from human feedback is known to those skilled in the art of training generative Al systems.

[0134] In block 111 of Figure 1, block 131 of Figure 1 A, and block 242 of Figure 2, computer system 800 may proactively seek data from external sources. In some embodiments, computer system 800 may proactively seek data from the user’s private source or from public sources. In some embodiments, computer system 800 may contact outside experts or commercial data sources.

[0135] The goals listed in block 210 overlap and interact. Fortunately, the interaction is generally synergistic. For example, efficiency contributes strongly to sustainability and explainability is a prerequisite and contributes strongly to controllability, so these pairs of goals have been combined in the list. In some embodiments, especially generative Al, explainability may also contribute substantially to alignment with human values, efficiency, efficacy, and generalizability because elements designed to be easily explained require less computation to train, are understandable and controllable, and tend to generalize. In addition, in some embodiments, computer system 800 may train parameters associated with explainable elements and probability models with a factor of up to 1,000 less computation per parameter compared to training network connection weights.

[0136] In some embodiments, the training process may incorporate curriculum learning techniques, in which the training data is organized into progressively harder sequences of examples. The system may also apply adaptive scheduling, dynamically prioritizing training samples that yield high gradients, coverage improvements, or correction of prior errors. These techniques improve convergence stability and performance, particularly in multi-stage hybrid systems where different modules may mature at different rates.

[0137] In some training configurations, previously trained network components or subsystems may be reused via transfer learning. For instance, a pre-trained recognizer or verifier module may be integrated into a new hybrid network and optionally fine-tuned. The system may support layerwise transfer, module-level reuse, or warm-start initialization from other domains, improving sample efficiency and enabling rapid adaptation to new tasks or input modalities.

[0138] Figure 2A is a diagram of two lists of examples of use cases, a list of goals, and a list of types of techniques for improving performance on goals as a framework for discussing the relative effectiveness of the different types of techniques. Examples of interactive use cases include (block 221):1. Group conversation2. Personal tutor3. Study group4. Foreign language5. Teaching assistant6. Proofreading7. Research assistant8. Co-author9. Images, Video10. Musing production

[0139] Interactive use cases are important in various embodiments of the invention because interaction with the user(s) inherently provides human feedback and guidance for lifelong and proactive learning. In addition, most of the interactive use cases provide repeated, iterative improvement of generated material and thereby substantial knowledge of past and future context. Examples of generation and prediction use cases include (block 222):1. Text generation2. Speech recognition3. Economic time series4. Weather5. Stock market6. Image generation7. Language translation

[0140] Generally, for generation and prediction use cases, computer system 800 may make use of any of the techniques in Figure 3 for goal improvement. Generation and prediction inherently involve sequences (block 301 of Figure 3), they tend to require a depth of analysis that fits well with multi-stage hybrid networks (block 303 of Figure 3).

[0141] Block 223 contains a short list of high-level methods that may be used by computer system 800 in various embodiments of the invention:1. Human dialog2. Multi-stage3. T argeted incremental growth4. Explainability5. Lifelong learning6. Proactive learning7. Sequence modeling8. Full context9. Probability modeling10. Multimodal11. Internal autoencodersThese high-level methods may be implemented by computer system 800 implementing specialized systems in cooperation with a human team and / or an Al system in the HGDTCS. These techniques are especially important in fixing problems such as non-sensible errors. Non-sensible errors are errors made by an Al system that would not make sense to a human. By their basic nature, non-sensible errors do not necessarily behave as expected to changes in learned parameters during training by techniques such as gradient descent. In some embodiments of technique (176) and other techniques discussed in association with Figure 3 and Figures 3J to 30, computer system 800 may use systems with multiple complementary subsystems. In some embodiments, computer system 800 may detect that an error made by one subsystem is non-sensible in the perspective of a complementary subsystem. In some embodiments, having detected a non-sensible error, computer system 800 may use one or more of the methods 1 to 11 listed above to modify the system making the non-sensible error. In some embodiments, computer system 800 may repeatedly modify the subsystem making the non-sensible error untilthe error is corrected. Since traditional training by gradient descent may fail to correct a nonsensible error, in some embodiments, computer system 800 may select possible modifications to the subsystem based on human guidance such as by (1) human dialogue with HGDTCS team members, outside experts and human end users, and / or (5) interaction with an end user during lifelong learning. In some embodiments, computer system 800 may make modifications to the system architecture and / or computation procedure based on (2) adding, deleting, and / or temporarily by passing one or more stages in a multi-stage design, (3) targeted incremental growth in which computer systems 800 may add one or more nodes and or connections to fix an error or to escape from the vicinity of a stationary point (including a local minimum or even global minimum in an error loss function), (4) explainability, in which computer system 800 may add one or more easy-to-explain companion nodes to an existing polysemantic node, (6) proactive learning from reference books, from retrieval augmented generation (RAG), from active agents, and / or from chain of thought reasoning. (7) sequence modeling, in which computer system 800 may construct representations of multi-units subsequences and / or may use a multi-unit sequence as a unit in a structure such as a random variable in a probability model, (8) full context modeling, in which computer system 800 may store in a repository a context longer than the context of a transformer within the system and may directly retrieve such context by direct retrieval rather than by the attention mechanism using mechanisms such as key word search, concordances and / or graphical data bases (9) explicit probability modeling, including event prediction and / or look- back conditional parametric probability models, (10) multimodal modeling, in which computer system 800 may add, delete and / or modify individual models in the set of models in the multimodal system, and / or (11) internal autoencoders, in which computer system 800 may add one or more internal autoencoders in which an autoencoder may have an input node in an inner layer of the multi-stage model and / or output node that is in an inner layer of the multi-stage model . In general, these or other changes in the base system computed by methods other than gradient descent may enable computer system 800 as part of the HGDTCS to repeatedly retry the training of the modified subsystem and repeatedly to test the system on new validation data until a specified criterion is satisfied.

[0142] Block 210 repeats the list of goals from block 210 of Figure 2. The discussion of detailed techniques in association with Figure 3 is organized based on these goals.

[0143] Figure 3 is a Venn diagram of example techniques that computer system 800 may use during training to improve the performance of the system on goals listed in block 210 of Figures 2 and 2A.

[0144] The diagram in Figure 3 lists the number of techniques and associated goals for four subsets of the set of neural network and hybrid network systems Set 1 (Block 300): Any neural network or hybrid network, Set2 (Block 301): Any neural network or hybrid network involving sequences and Set 3 (Block 302): Any multi-stage hybrid network. Set 4 (Block 303): is a disjoint set of techniques that apply only to hybrid networks.

[0145] Note that both Set 2 and Set 3 are subsets of Set 1 in the Venn diagram. Thus, the techniques counted in block 300, may be applied to all networks, including the networks involving sequences (Set 2) and the multi-stage networks (Set 3). Note that some multi-stage networks may involve sequences and some multi-stage networks may not involve sequences. Some of the multi-stage techniques may apply both to networks involving sequences and to networks not involving sequences.

[0146] Some techniques apply to multiple goals. In the discussion of the techniques associated with Figure 3, the techniques are grouped first by whether (a) the technique applies to all neural networks and hybrid networks, (b) the technique applies to all networks associated with sequences, (c) the technique only applies to multi-stage networks, or (d) the technique only applies to hybrid networks. The techniques that only apply to hybrid networks generally use units with multiple output values. Within the list for each type of network, the techniques are grouped by which one or more goals the technique is primarily aimed at improving. The number in the title line of each block is the number of techniques discussed in association with that block. Each line item lists a set of goals. The number in each line item is the number of techniques discussed in association with the set of goals listed in the line item.

[0147] List of Techniques: The following list names the techniques that are organized and counted in the Venn diagram of Figure 3. Each technique is discussed further in the following discussion.

[0148] Block 300: Techniques that may be used by computer system 800 for any neural network or hybrid network: General prerequisite techniques:1) Malleability:2) Element-to-element regularization links:3) Incremental growth:Explainability, efficiency, efficacy, and alignment with Human Values:4) Internal autoencoders placed anywhere in network:5) Blocking back propagation:6) Error prediction and correction nodes:7) Creating a latent variable space:8) Training internal autoencoders stand alone:9) Explainable elements:10) Units with low bit qualitative outputAlignment with human values:11) Active defense against adversarial attacks and other perturbations to the input:Efficacy and Efficiency:12) Counter tying for diversity:Efficacy:13) Subsystem design variety:14) Recurring quick validation:15) Adding a connection:16) Adding a layer:17) Node splitting:18) Duplicating a model or subsystem:19) Monitoring added elements and explainable elements:20) Socratic supervisor verifying the Socratic principle:21) Socratic supervisor estimating precision and recall:Efficiency:22) Simplify the objectives of selected elements:23) Customized back propagation strength:Generalizability:24) Cooperatively generate realistic training data:25) Generate non-adversarial data near a decision boundary:26) Generate data to fool system:27) To detect problems, reduce the amount of training data:Security and sensibility:28) Running on private computers:29) Piecewise constant activations:30) Noi sy acti vati on :31) Training for robustness:32) Diversity for robustness and security:33) Diverse canary networks:

[0149] Block 301 : Techniques that may be used by computer system 800 for any neural or hybrid networks involving sequences:Explainability, efficiency, efficacy, alignment with human values:34) “Future event” prediction:35) Explainable event prediction:36) Parametric look-back probability models:37) Word-level and phrase-level semantic entities in higher stages:38) Autoencoder state space model:39) Mixture of sources in training:40) Paraphrasing and / or summarizing and switching between training text sources for greater variety:41) Mutual event prediction and look-back probability models:42) Train special models for words and selected phrases:Efficacy and Efficiency:43) Diverse subsystem ensemble combining network:44) Word count statistics by intervals:45) Increasing the size of the word count interval:46) Embed normalized word count vectors in a latent variable space:47) Sequence alignment:48) Paraphrase and / or summarize source training text for additional training data:49) For indexed models, only load models that are in the forecast candidate lists for future items in the look-ahead beam:50) Grow the network as more context is accumulated:Efficiency:51) Rank and prune candidate lists based on future event prediction and look-back probability models:52) During training, prune the candidate lists based on looking ahead from known future during training:53) Selective activation of look-back probability models:54) Simplify objective for longer range event prediction:55) Enduring mixture-of-experts routing:56) After pretraining, prune indexed models for sparsity:

[0150] Block 302: Techniques that may be used by computer system 800 for any multistage hybrid network:General techniques:57) Setting stage-specific conditions:58) Stage-specific selection of training data:59) Stage-specific hyperparameters:60) Stage-specific selection of generation or recognition:Alignment with human values:61) Human assistance in specifying and setting stage-specific conditions:62) Cooperation with humans in joint cooperative tasks:63) Al in a human-guided HGDTCS:64) Lifelong learning with human guidance:65) Interactive training on practice data:66) Learn by doing with multiple clients:Explainability, Efficacy, Efficiency and Alignment with Human Values:67) Creating multi-token entities during position synchronous beam pruning:68) Low perplexity sequence aligned directed acyclic graph (DAG) graph interstage: Efficacy and efficiency:69) Indexed event predictions:70) Repeated processing of full document:71) Repeated forward and backward processing:72) Take advantage of two-sided data:73) Stage-specific degree of use of indexed models based on multi-token entities:74) Creation of concordance:75) Using long-term properties of a document:76) Incremental addition of stages:77) Training of subchains:78) Direct connection of inner of stages to network input and output:79) Use of subchains to speed up inference or to improve performance:80) Joint development of mixture-of-experts and combining network:81) Efficiently growing width of a stage:82) Semi-automated prompt engineering:83) Lower stages advance beyond and guide higher stages:84) Efficient management of multi-stage training:85) Vary methodologies from stage to stage:86) Links from lower stages to higher stages:87) Fast bidirectional training in hidden Markov state space model:88) Train a small, fast generator as an error simulator:89) Efficiently growing a deep network with small stages:90) Training some stages on simplified objectives:91) Training a stage based on ranks of future items in candidate lists:92) Residuals of internal variables:Efficacy only:93) Training and validation stage-by-stage:94) Variety in stage design:95) User-specific customized pronunciation models:96) Mixture of models with fixed pruning:Efficiency only:97) Fewer active indexed models in higher stages:98) Skip indexed models that are not ready:99) Selective backpropagation to lower stages:100) Multi-channel inference on shared hardware:101) Simplify objective for elements in lower stages:102) Develop system trained for multiple distinct writing styles:103) Rerunning batches of training data with full look ahead: Generalizability:104) Validation and regularization to improve generalizability:105) Variety of stages design and methodology:Security and sensibility:106) Stages with noisy input:107) Estimate conversion or transformation from scores to estimated logarithms of probabilitiesAll together:108) Semi-automated knowledge engineering and human-guided training:

[0151] Block 303: Techniques that computer system 800 may use with hybrid networks or other advanced network architectures to support the techniques:109) Data-dependent substitution of back propagation function for security and sensibility:110) Substitution of backpropagation function to correct or avoid implicit errors:111) Substitution backpropagation to maintain explainability:) S ocrati c wi sdom output : ) Few-bit output representation: ) Bit vector latent spaces: ) Socratic active defense against adversarial attacks: ) Redundant subsystem for robustness and defense against adversarial attacks:) Testing reliability of subsystem with reverse adversarial attack: ) Delayed decision subchain selection: ) Computation and propagation of Socratic wisdom: ) Socratic wisdom defense against adversarial attacks and other perturbations:) Multi -token based language model: ) Word-based language model: ) Semi-hidden Markov process: ) Grammar-based transformer extension: ) Parser based conditional stochastic influence model: ) Pairwise word and token modeling: ) Hierarchical text organization for paired head words: a) Near neighbors in sequence: b) Within a phrase: c) Within a clause: d) Within a sentence: e) Within a paragraph: f) Within a chapter: g) Within a short document (e.g., an article or short story) h) Within a long document (e.g., a book): i) Within a set of related books (e.g. on-line or published encyclopedia or a series of novels): ) Pipeline parallelism for unlimited memory: ) Pipeline parallelism for enhanced throughput: ) Text-based indexing of word and token models: ) Text-based communication between stages and between models: ) Forward and backward conditional probability models: ) Models and templates based on head words in grammar: ) Models and templates based on named entities: ) Paraphrase or summarize passages from training samples:) Conditional probability models of word occurrence as a function of relative position in sequence: ) Model conditional probabilities conditioned on the same word or unit as conditionally independent: ) Models and templates based on head words in grammar: ) Models and templates based on named entities: ) Generate text by paraphrasing or summarizing a passage in from training samples: ) Use relationship regularization links: ) During training, incrementally increase the number of levels or stages: ) Incrementally grow an individual level or stage: ) Use ranked candidate lists from lower stages or levels to prune candidate lists or speculative decoding in higher stages or levels: ) Use candidate lists from lower stages or levels to help select the most effective experts in higher stages or levels based on mixture-of-experts (MOE) models:) Manipulate positional embeddings giving the positional embedding at a position in the sequence later than the position current being generated: ) Combining similar residuals: ) Attempt to prefill depth wise with more efficient approximations the residuals:) Use the logits of the LLM as implicit n-gram models of the local text: ) Train draft model to maximize the expected length of the completion: ) Hierarchy of draft models: ) Train n-gram models on specific tasks: ) Checking for an n-gram match to a text corpus: ) Measuring the statistical independence of different speculation methods: ) Determining sets of tokens that do not have significant causal impact: ) Tuning models to skip embedding / disembedding to speed up inference: ) Train models to have sets of discrete values at the residuals: ) Train up-projections from KV-caches of draft models: ) Quantify tokens and multi -token units based on information from speculative models which tokens are “Hard”: ) Approximate inference: ) Using backwards models to test the likelihood of proposed candidate sequences:) Aligning objectives of a verifier model with draft models:163) Combining systems with complementary capabilities, optionally with redundancy, regularization, and diversity:164) Determining strengths and weaknesses of specified subsystems:165) Train a base system primarily with gradient descent and a complementary system primarily without gradient descent:166) Train two complementary systems on different data domains:167) Use two subsystems that naturally use qualitatively different types of data:168) Combining training on a base domain with training on an independent verification domain:169) Combining two separately trained complementary subsystems with an independently trained combined system:170) Train a base system to suggest a candidate and train a complementary system to verify or reject the suggested candidate:171) Train a base system to generate a list of candidates and a complementary system to verify or reject each candidate on the list of candidates:172) Train a base system to recognize units associated with nodes on a graph and train a complementary system to align a sequence of units with a path of nodes in the graph:173) Train a base system to optimize the frequency with which the correct response is in a pruned list of candidates and train a complementary system to accept or reject each answer in the pruned list of candidates:174) A speech recognition system comprising a system to recognize sub-word units:175) Align a sequence of sub-word units to accept or reject word candidates produced by a language model:176) Train a verification system 383 to detect non-sensible errors:177) Coordinated node-specific implicit local objective and explicit node-specific artificial objective:178) Explicit local objectives for decisiveness:179) Testing for violations of sensibility:180) Localizing cause of violation of sensibility:181) Training a node to be sensible using a hyperparameter controlled local objective:182) Sensibility as a prerequisite to safety:183) Multi -phase training of decisiveness and sensibility:184) Socratic wisdom: Know what it is that you do not yet know:

[0152] In some embodiments, computer system 800 may set a flag indicating a lack of adecision on a data item d by a node S that has been trained to be sensible, if the activation of sensible S is in its flat activation interval.185) In a speech recognition system, train two complementary subsystems with and without audio input.186) Integrating n-gram modeling to make speculative decoding of text generation more accurate.187) Using look-back conditional probability models to represent n-grams efficiently.188) Prune candidate lists in look-back model n-gram computation.189) Use acoustic scores to modify candidate lists.190) Multiple cooperating systems designed and trained to have complementary capabilities.191) Cross-verification of sensibility.192) Cross-verification to lower average error rate.193) Dynamically configured distributed computation with cooperative complementary modules and systems.194) Complementary systems trained specifically to do final verification.195) Beam of sequences generated by multiple draft models.

[0153] Discussion of techniques: Block 300: Techniques that may be used by computer system 800 for any neural network or hybrid network:

[0154] General techniques:

[0155] 1) Malleability. In some embodiments, computer system 800 may make changes to the network to improve the performance or efficacy. In some embodiments, computer system 800 may add additional nodes and other elements to correct errors, improving efficacy, and / or to increase diversity, which may also contribute to improved efficacy and robustness. In some embodiments, computer system 800 may add elements or duplicate subsystems to improve explainability. In some embodiments, computer system 800 may add one or more entire layers. In some embodiments, computer system 800 may double the number of layers. In some embodiments, computer system 800 may delete elements from the network. In some embodiments, computer 800 may repeatedly make changes to the network.

[0156] 2) Element-to-element regularization links. In some embodiments, computer system 800 may create data-dependent relationship regularization links. In some embodiments, computer system 800 may link a pair of nodes, or more generally, a pair of elements. In this context, an “element” may be a node, a connection, a stand-alone elementwith an associated variable, a unit, or an internal element of a unit. In some embodiments, computer system 800 may link a value associated with one type of element to a value associated with a different type of element. For example, in some embodiments, computer system 800 may link the output value of a node with an attribute value stored in a memory cell within a unit. However, because connection weights are not data dependent, in some embodiments, computer system 800 may only link a connection weight only with another connection weight, using a link that is not data-dependent. A regularization link may have an associated hyperparameter called the “strength” of the link. Parameter tying, as in a convolutional neural network, is equivalent to the limit case of a set of “is-equal-to” relation links of infinite strength independent of the data item. The strength of a regularization link may be negative, such as in the case of counter tying.

[0157] Without loss of generality, each link is unidirectional, with a source element and a receiving element, because computer system 800 may create the equivalent of a bi-directional link by forming two unidirectional links. A link is not a connection. It does not transmit activations nor back propagate derivatives. Thus, any pair of elements may be linked in either direction regardless of their respective positions or layers in the network. In fact, the source element and the receiving element may be in separate networks.

[0158] During training, for each data item in a specified set, computer system 800 may add a regularization penalty or reward depending on whether the values associated with the pair of elements satisfy a specified relationship. In some embodiments, computer system 800 may control the amount and type of regularization based on hyperparameters specified by the HGDTCS. In some embodiments, there is no limitation on the form of the specified relationship between the values of the pair of elements associated with a regularization link. For example, the activation values of a pair of linked nodes may be mapped to Boolean values represented by 0 and 1 or to other symbolic values. Thus, relationship regularization links provide a bridge from a to a neural network or hybrid network to logical reasoning and other elements of symbolic Al. In some embodiments, computer system 800 may use symbolic-valued regularization links in association with representations of multi-node subsystems representing higher level structures such as ontologies and mereologies.

[0159] In some embodiments, computer system 800 may impose an “is-equal-to” relationship, “soft tying” two or more elements in a specified set. Such soft tying generalizes the concept of parameter sharing or “hard tying” of connection weights in convolutional networks.

[0160] Compared to parameter tying in convolutional neural networks, regularizationlinks: (1) may be applied to elements other than connection weights; (2) most applications are data dependent regularizations applied to nodes and other elements; (3) may be used for soft tying, with a hyperparameter controlling the strength; (4) may be associated with any relation, not just “is-equal-to”; (5) may be unidirectional (or effectively bi-directional), (6) may be applied selectively only for specific data items (7) may also be used to build on, generalize, and substantially improve transfer learning, and (8) may also be used for counter tying to create diversity for improved efficacy and robustness.

[0161] In transfer learning, the amount of computation for training a new system or subsystem may be reduced by initializing the parameters of the new system or subsystem from a previously trained system or subsystem, a process well-known to those skilled in the art of training neural networks. In some embodiments, computer system 800 may use soft tying to transfer knowledge from one system or subsystem to another not only at initialization but for every specified data item for the entire training process or as much of the training process as specified by the HGDTCS.

[0162] In some embodiments, computer system 800 may use soft tying in a much more targeted fashion. For example, in some embodiments, computer system 800 may tie an individual element in one stage of a multi-stage system to a specified element in a second stage of the multi-stage system. In some embodiments, computer system 800 may use such targeted soft tying to accelerate the training of higher stages for which the set of selected elements in the higher stage that correspond to elements in a lower stage may be a proper subset of the elements in the higher stage. This example illustrates one of many ways in which element-to-element regularization may contribute to improved efficiency. By applying targeted regularization, soft tying may also improve both efficacy and generalizability.

[0163] In some embodiments, computer system 800 may impose an “is-not-equal-to” relationship directional or bi-directional regularization between a specified pair of elements to create diversity, a process herein called “counter tying”. In some embodiments, computer system 800 may use this diversity to improve efficacy and / or robustness.

[0164] 3) Incremental growth. In some embodiments, computer system 800 may grow the multi-stage network by repeatedly adding more stages to a base network. Such incremental growth directly contributes to improved efficacy.

[0165] In some embodiments, computer system 800 may also improve efficacy by duplicating an element or duplicating an entire subsystem such that the expanded network corrects some of the errors of the previous network. In some embodiments, computer system 800 may repeatedly make such incremental expansions and improvements in efficacy.

[0166] In some embodiments, computer system 800 may improve explainability and efficacy by selecting one or more individual elements to which to add one or more companion nodes and / or duplicates of a selected element. For example, if in explaining a selected element, computer system 800 determines that the selected element is doing more than one task, then, in some embodiments, computer system 800 may create a separate element to be dedicated to each task. As a specific example, in an image segmentation and classification task, if computer system 800 determines that a selected node has a high activation value for a horse’s hoof in some images and for a horse’s tail in other images, computer system 800 may create two nodes, one trained to detect horses’ hooves, and the second trained to detect horses’ tails.

[0167] In some embodiments, if one or more elements have been duplicated, computer system 800 may create two or more copies of a subsystem, such as an attention head or a feed forward layer in a transformer network. In some embodiments, for each duplicated element, computer system 800 may assign each duplicated version of the element to a distinct subset of the set of duplicates of the subsystem. In some embodiments, computer system 800 may create diversity among the duplicate versions of a subsystem by counter tying with an “is-not- equal-to” data dependent relationship regularization link for selected pairs of corresponding elements among the duplicate versions of the subsystem.

[0168] In some embodiments, computer system 800 may increase the number of versions of a subsystem from one stage to the next and then, in some embodiments, reduce the number of versions of the duplicated subsystem by means of a combining network, such as shown in Figure 6. In some embodiments, computer system 800 may balance improvements in efficacy and efficiency by combining the duplication of subsystems with the reduction in subsystems from stage to stage.

[0169] In some embodiments, computer system 800 may create duplicates of an entire stage to distribute the computation and memory requirements, enabling computer system 800 to distribute the computation among multiple processors that cost less per computation and that use less electrical power and generate less heat and thus use less air conditioning. This use of this technique contributes to sustainability.

[0170] Explainability, efficiency, efficacy, and alignment with Human Values: 4) Internal autoencoders placed anywhere in network: In some embodiments, computer system 800 may create one or more internal autoencoders during the training of a system. An autoencoder is a network in which the network is trained to produce an output that matches the input as well as possible, subject to restrictions imposed by the architecture of the autoencoder, such ashaving a “bottleneck layer,” with a restricted number of elements or a sparsity constraint. Generally, both the input and the output are each a vector of values in a specified data space. An “internal autoencoder” is a subnetwork of a parent network in which either the input or the output of the internal autoencoder is an internal n-tuple of values rather than the input or output of the full network. Both the input and the output of the autoencoder may be internal to the network.

[0171] An autoencoder also comprises an encoder network from the input vector to the bottleneck layer and a decoder network from the bottleneck layer to the output vector.

[0172] Multiple autoencoders are used extensively in preferred embodiments of the many-language translation use case (Figure 10).

[0173] Since an autoencoder is trained to have its output match its input for each supplied input value, the target output is specified without the need for any human or machine labeling of the target values.

[0174] Because an autoencoder does not require label trained data, in some embodiments, computer system 800 may select any set of node or unit output values as the inputs to an internal autoencoder, which of course also specifies the target values for the outputs. In some embodiments, computer system 800 may select or create any set of nodes or cells as the latent variable space for the autoencoder. In some embodiments, computer system 800 may create a set of cells or nodes as the output values of the autoencoder. Computer system 800 may add outgoing connections from the output nodes or cells of the autoencoder. In some embodiments, computer system 800 may train the internal autoencoder with the target objective being that the output matches the input and may block back propagation from the nodes receiving connections from the autoencoder output nodes or cells. In some embodiments, an autoencoder may have multiple objectives and its output elements may receive back propagation from multiple connections.

[0175] 5) Blocking back propagation: In some embodiments, computer system 800 may block back propagation during gradient descent training from certain types of elements, such as the output values of an internal autoencoder or a judgment or error correction node. In some embodiments, computer system 800 may multiply the back propagated derivative value by a scaling factor 0 < a < 1.

[0176] 6) Error prediction and correction nodes: In some embodiments, computer system800 may select one or more base nodes for which to build error prediction and error correction companion nodes. In some embodiments, computer system 800 may train a companion node to predict for each input data item whether the base node will make an erroron the data item. In some embodiments, computer system 800 may create and train one companion node to predict whether the base node will make a false positive error and a second companion node to predict whether the base node will make a false negative error. In preferred embodiments, computer system 800 may block back propagation from the error correction node to the base node base because such back propagation may cause the training process to train the base node and the error prediction nodes to attempt to imitate each other or attempt to fool each other depending on whether the connection weights have the same sign of the opposite sign. In any case, back propagating to both may cause an unstable cycle in the training process.

[0177] Exemplary details about error correction nodes (also called “judgment nodes”) are provided in U.S. Patent 11,797,582, issued October 24, 2023, titled “Deep learning with judgment,” assigned to D5AI LLC, by inventor James K. Baker, which is also incorporated herein by reference in its entirety.

[0178] 7) Creating a latent variable space: In some embodiments, computer system 800 may create new nodes or cells as the bottleneck layer of an internal autoencoder. In some embodiments, computer system 800 may use existing nodes or cells in the bottleneck layer. In some embodiments, computer system 800 may use or copy existing nodes or cells in the bottleneck layer of an internal autoencoder. In preferred embodiments, the total number of nodes and cells in the bottleneck may be limited to a specified number and / or a sparsity condition may be enforced as a constraint or regularization. In some embodiments, back propagation to elements in the bottleneck from outgoing connections other than to the connections to the decoder portion of the autoencoder.

[0179] In some embodiments, computer system 800 may use the bottleneck layer of an internal autoencoder as a shared input space to two or more detectors or category classifiers. In some embodiments, the detectors or category classifiers may be outputs for the network. In some embodiments, they may be internal to the network.

[0180] In some embodiments, computer system 800 may create one or more explainable companion elements to an element in a latent variable space.

[0181] 8) Training internal autoencoders stand alone: In some embodiments, computer system 800 may train one or more internal autoencoders on a stand-alone basis on a specified set of training data. That is, computer system 800 may train the autoencoder by itself without training the rest of the network and, in some embodiments, computer system 800 may select different data for training the autoencoder than for training the rest of the network. In general, training the autoencoder by itself requires much less computation, so this techniquecontributes to efficiency. The lower dimensionality of the latent variable space may also improve the efficiency of elements receiving connections from the latent variable space and may also facilitate explainability.

[0182] In some embodiments, computer system 800 may make a copy of the elements in the bottleneck layer of an internal autoencoder but not connect the copies to the decoder of the autoencoder. In some embodiments, computer system 800 may add explainable nodes to the set of copies of the bottleneck elements.

[0183] 9) Explainable elements: In some embodiments, computer system 800 may select a base element and then create or train one or more companion elements to improve the efficacy and / or the explainability.

[0184] Adding companion nodes for explainability (or for another purpose) is an example of a process herein called “node splitting.” Generally, node splitting adds additional learned parameters, which improves efficacy as well as explainability. Many types of companion elements, including event prediction models, non-parametric probability models, and conditional probability models require much less training data than the network weights in a large system such as a large language model (on the order of 10,000 instances rather than billions or trillions) and much less computation. In some embodiments, based on hyperparameters specified by the HGDTCS, computer system 800 may reduce the total amount of computation per added parameter by a factor of 100 to 100,000 or more. Explainable elements contribute to efficiency as well as to explainability and efficacy.

[0185] In a large neural network, in a typical embodiment, during training each node is trained on all the training data, typically by stochastic gradient descent. Thus, the connection weights into the node are trained to an average of the gradient summed over all the data. The activation for each input data item is the result of this combined influence during training. It has been empirically observed that the pattern of high and low activation values is generally a response to a mixture of many categories. In language modeling the activation values represent many different words and different meanings. Such node responses are called “polysemantic.” A polysemantic node may be difficult for an Al system or human analyst to explain with a simple explanation. In this disclosure, the term “polysemantic” will be used to refer to any node or element for which the responses of high and / or low activation values correspond to a mixture of a plurality of different output categories or known labels such as words or tokens. In some embodiments, the computer system 800 as part of the HGDTCS may set one or more hyperparameters for testing whether a specified node or element is polysemantic.

[0186] In some embodiments, computer system 800 may create and train one or more explainable companion nodes to a polysemantic node. In some embodiments, to train an explainable companion element, computer system 800 may count, for each label or category, the number of instances in which the polysemantic element has an activation value more extreme than a specified threshold. In some embodiments, computer system 800 may collect such counts on a set of sample data that is a small fraction of the training data, such as 100,000 or fewer tokens out of a training set of a trillion or more tokens.

[0187] In some embodiments, computer system 800 may then select one or more of a specified set of a limited number of labels for which the number of extreme values satisfies a specified criterion. In some embodiments, computer system 800 may specify a criterion that the number of extreme values is greater than a specified fraction of all instances of the label. In some embodiments, computer system 800 may specify a criterion that the fraction of instances of the label in the sample data exceeds a specified multiple of the fraction of instances in a larger sample or in the whole of the training data.

[0188] An illustrative embodiment of a process for selecting explainable target sets and training an explainable element is discussed in association with Figure 5.

[0189] In some embodiments, an explainable companion element may be a simple, single node network receiving the same input connections as the base element. The explainable node may be trained as a discriminator of two disjoint explainable sets. The explainable companion node may be trained stand-alone just on the data of the two explainable sets. As a single node network with a limited amount of training data, computer system 800 may train the explainable companion node requiring much less computation per learned parameter than in the pre-training of a large neural network, such as a large language model. In some cases, the reduction in the amount of computation may be by a factor of 10,000 or more because computer system 800 may select two disjoint sets that are easy to discriminate.

[0190] In some embodiments, computer system 800 may create and train error prediction and error correction nodes for an explainable mode.

[0191] In some embodiments, computer system 800 may add explainable companion nodes to a node in the bottleneck layer of an internal autoencoder or other latent variable space.

[0192] 10) Units with low bit qualitative outputs: In some embodiments, computer system 800 may represent an output value of an element with a single bit, with the bit 0 representing an activation below a specified threshold T and the bit 1 representing an activation above T.

[0193] In some embodiments, computer system 800 may represent 4 possible qualitatively distinct outcomes: High, Low, Both, Neither.In an element trained as or interpreted as an implicit discrimination, computer system 800 may use one bit to represent the two alternate detections and have two additional bits, optionally stored in a cell, with one of the additional bits representing whether the computer system 800 has confidence that the implicit detector has satisfactory precision and the second additional bit representing whether computer system 800 has confidence that the detector has satisfactory recall. In some embodiments, computer system 800 may estimate the precision and recall from the most recent validation testing. In some embodiments, the precision and recall may have outgoing connections to other elements, but computer system 800 may block back propagation.

[0194] In some embodiments, computer system 800, acting as a Socratic Supervisor 262 of Figure 2, may compute the precision and recall along with estimates of the overlap of probability distributions and other useful diagnostic statistics as discussed in association with Figure 4.

[0195] Alignment with human values: 11) Active defense against adversarial attacks and other perturbations to the input: In some embodiments, computer system 800 or the HGDTCS may design a system to have an active defense. In some embodiments, in the active defense, a selected set of nodes may each have multiple activation functions. In some embodiments, each activation function may be characterized as having known intervals, such as constant intervals, that have a low value for the absolute value of the derivative. In some embodiments, computer system 800 may specify multiple activation functions, such that, for any given input, there is one or more activation functions such that computer system 800 may choose an activation function such that the value of the output of the activation function does not change significantly for a change less than a specified magnitude in the input. In preferred embodiments, computer system 800 may delay choosing which activation function to use for a given input until after the input has been received, so an attacker cannot adjust the input. This technique increases sensibility as well as security.

[0196] In some embodiments, computer system 800 may use an active defense with “last move” advantage. For example, in some embodiments of a text generator, for a suspicious prompt, computer system 800 may privately compute a tentative response and test the tentative response for violation of guard rails.

[0197] In some embodiments, computer system 800 may obtain or train a separate system to detect patterns characteristic of adversarial attacks. In some embodiments, computersystem 800 may obtain or train a diverse set of systems that are deliberately designed to have no defense against adversarial attacks. In some embodiments, computer system 800 may obtain or train a system to recognize the difference in the pattern of responses from one or more defended networks from the pattern of responses of the undefended networks. This technique increases sensibility as well as security.

[0198] Efficacy and Efficiency: 12) Counter tying for diversity: In some embodiments, computer system 800 may create diversity by counter tying one or more elements in a subsystem to corresponding elements in a second subsystem. In some embodiments, with an arbitrarily large number of such counter tying relationships, computer system 800 may create a larger number of diverse subsystems and train the diversity more efficiently than with conventional methods, such as boosting and bagging. Trained diversity may contribute both to improved efficacy and improved efficiency of the training process.

[0199] Efficacy: 13) Subsystem design variety: In a hybrid network, in some embodiments, computer system 800 may mix a variety of elements, such as nodes, cells, units, and probability models. In some embodiments, computer system 800 may mix an arbitrary variety of modules and subsystems. In some embodiments, in a multi-stage system, computer system 800 may mix stages that perform recognition with stages that perform generation. As illustrated in Figures 6 and 6a, a multi-stage system may comprise mixtures of expert classifiers and mixtures of generators.

[0200] 14) Recurring quick validation: In some embodiments, computer system 800 may frequently do repeated validation testing on new data. In some embodiments, computer system 800 may use 10,000 new data items for each validation test, a typical figure for the number of items in a benchmark test. In some embodiments, such as pretraining of a large language model, computer system 800 may obtain trillions of items of training data. Thus, if computer system 800 sets aside enough data for one million validation tests, the set aside data may still be less than one percent of the training data.

[0201] In some embodiments, computer system 800 may do frequent validation testing during training. For example, in training based on the flowchart of Figure 1, computer system 800 may do validation testing during blocks 107 and 111. In some embodiments, computer system 800 may do validation testing only after there is a major change in at least one stage. Recurring validation is primarily aimed at improving efficacy. Quick validation on a limited but adequate amount of set aside data is aimed at reducing the impact on efficiency.

[0202] 15) Adding a connection: In some embodiments, computer system 800 may add one or more connections to an existing feed forward non-recurrent neural network (the basenetwork) while guaranteeing no loss in performance (as measured on training data) by initializing the weight for the new connections to the value zero and setting the weight for all existing connections to be the same as their value in the base network. Computer system 800 may then resume training using stochastic gradient descent.

[0203] In some embodiments, after an amount of additional training specified by the HGDTCS, computer system 800 may verify the improved performance on a set of validation data set aside from the training data. If the improvement is not verified, computer system 800 may restore the base network. In some embodiments, computer system 800 may select a different set of new connections and try again. In some embodiments, computer system 800 may add regularization links and / or other forms of regularization if an addition to the network improves the performance on training data but not on validation data.

[0204] To predict which new connections may most quickly improve the performance on training data, in some embodiments, computer system 800 may make a slight addition to the back propagation in a stochastic gradient descent computation. Computer system 800 may temporarily add one or more candidate new connections with zero weight and, as part of the normal back propagation step, may compute the value of the update for each candidate new connections without performing the update except on selected connections. In some embodiments, computer system 800 may then select a subset of the new candidate connections with the highest magnitude updates from the initial zero-weight value to retain in the new network and drop the candidates with lower magnitude update values.

[0205] 16) Adding a layer: In some embodiments, computer system 800 may add one or more entire layers to an existing, trained base neural network. To assure no degradation in performance relative to the existing neural network, in some embodiments, computer system 800 may initialize all the weights on connection coming into or outgoing from the new layer to be zero.

[0206] In some embodiments, computer system 800 may add regularization links soft tying one or more nodes in the new layer, the preceding layer, and / or the following layer to nodes either of the two adjacent layers in the original network to speed up initial training of the new layer. In some embodiments, computer system 800 may gradually reduce the strength of the regularization links as training proceeds. In some embodiments, computer system 800 may replace some of the soft tying links with counter tying links to create diversity.

[0207] 17) Node splitting: In some embodiments, computer system 800 may replace a single base node with a set of two or more nodes. For example, in some embodiments,computer system 800 may add a node to predict when the base node makes an error on an implicit local objective. In some embodiments, computer system 800 may add a pair of new nodes, one new node to predict when the base node makes a false positive error on the implicit local objective and a second new node to predict when the base node makes a false negative error on the implicit local objective.

[0208] In some embodiments, computer system 800 may replace a single node with a plurality of new nodes and train each of the new nodes on a distinct, optionally disjoint, subset of the training data. In some embodiments, computer system 800 may initialize the weights on all the outgoing weights from one or more of the new nodes to be zero. In some embodiments, computer system 800 may initialize the incoming weights to one or more of the new nodes to be the same as the incoming weights of the base node. In some embodiments, computer system 800 may initialize the incoming weights for one or more of the new nodes to be zero.

[0209] In some embodiments, computer system 800 may train one or more of the new nodes on a selected proper subset of the training data items. In some embodiments, computer system 800 may select a subset of the training data with large positive back propagated derivatives and / or a subset with large magnitude negative derivatives. In some embodiments, computer system 800 may select two disjoint subsets of the training data by testing the direction of the gradient with respect to the weights on the incoming connection for each candidate data item in one set against the average gradient direction for the data items in the other set. In some embodiments, computer system 800 may use as a selection criterion that the cosine of the angle for the gradient of a candidate data item be negative or that it be below a threshold value set by a hyperparameter controlled by the HGDTCS.

[0210] 18) Duplicating a model or subsystem: In some embodiments, computer system800 may expand a network by duplication of an entire base subnetwork. In some embodiments, in duplicating a selected base subnetwork, computer system 800 may select a subset of the nodes in the base subnetwork and apply node splitting to each of the selected nodes. In some embodiments, computer system 800 may select two or more proper subsets for each node selected to be duplicated by node splitting. In some embodiments, computer system 800 may create up to up to 2Ndistinct new subsystems by specifying for each new subsystem a distinct subset of the N sets of duplicate nodes to be trained to have high activations for a first alternative training data subsets and low activation values for the complement of the first alternative training data set.

[0211] In some embodiments, computer system 800 may train the duplicatedsubnetworks to be diverse by counter tying selected pairs of corresponding nodes.

[0212] 19) Monitoring added elements and explainable elements: In some embodiments, after adding one or more nodes to a network, computer system 800 may test the performance of an added element with an element-specific performance test. In some embodiments, computer system 800 may test the performance on a previously added node based on the node’s implicit local objective. In some embodiments, computer system 800 may test the performance of an explainable node on a discrimination or detection task associated with the explanation even if the explainable node is a base node rather than an added node. Computer system 800 may perform the node-specific tests on validation data that has been set aside from the training data.

[0213] In some embodiments, if computer system 800 detects degradation in performance of a node on a node-specific objective, computer system 800 may take node-specific corrective action. In some embodiments, computer system 800 may do additional training of the node on its local implicit objective and / or on its explanation if it is an explainable node. In some embodiments, computer system 800 may select training data comprising nodespecific errors and comparable non-errorful data, so that node-specific corrective training requires much less computation than full network training.

[0214] In some embodiments, computer system 800 may impose node-specific regularization, either during ongoing training of the full network or on a node-specific training set.

[0215] In some embodiments, computer system 800 may add a node-specific error prediction node. In some embodiments, computer system 800 may add a node-specific error correction node. In some embodiments, computer system 800 may train an existing node or a new node on the implicit objective or explainable objective of a node with degraded performance. In some embodiments, for training the new node, computer system 800 may select a subset of the training data comprising a sample of data on which the node with degraded performance makes local errors and a comparable amount of non-errorful data.

[0216] 20) Socratic supervisor verifying the Socratic principle: In some embodiments, computer system 800, acting as the Socratic Supervisor (262 of Figure 2) may compute statistics testing whether an output score of one or more elements represents over confidence, over-fitting or other violations of the Socratic principle. In some embodiments, computer system 800 may estimate these statistics using measurements made on set aside validation data, as discussed in association with Figure 4.

[0217] 21) Socratic supervisor estimating precision and recall: In some embodiments,computer system 800, acting as the Socratic Supervisor (262 of Figure 2) may determine that an output value for an element represents an explicit or implicit detection. In some embodiments, computer system 800 may estimate the precision and / or the recall of the explicit or implicit detector. Computer system 800 may estimate these statistics using measurements made on set aside validation data, as discussed in association with Figure 4.

[0218] Efficiency: 22) Simplify the objectives of selected elements: In some embodiments, computer system 800 may change the implicit local objective of an element to an explicit objective that is easier to meet than the back propagated network objective. In some embodiments, in a classification task or in a language model or other generation of named entities, computer system 800, for a selected element, computer system 800 may selectively back propagate an error loss only under specified conditions. For example, computer system 800 may only back propagate the error loss if the training target fails to rank in the top N choices, where N is a hyperparameter specified by the HGDTCS.

[0219] 23) Customized back propagation strength: In some embodiments, computer system 800 may multiply selected back propagated derivatives of the network objective function by a specified scaling factor a. In some embodiments, 0 < a < 1. For example, in some embodiments, computer system 800 may pretrain an internal autoencoder stand alone. In some embodiments, computer system 800 may train the autoencoder with outgoing connections from the input elements, the bottleneck layer and / or the output layer. In some embodiments, computer system 800 may multiply the derivative back propagated from these outgoing connections by a scaling factor a < 1 specified by the HGDTCS.

[0220] Generalizability: 24) Cooperatively generate realistic training data: In some embodiments, computer system 800 may generate data to be used as additional training data. In some embodiments, computer system 800 may use a variational autoencoder, a stochastic categorical autoencoder network (SCAN), a cooperative generator trained to cooperate with a discriminator to detect real versus artificial data, a mixture of generators, or a generator that generates random perturbations on real data in non-adversarial directions.

[0221] In some embodiments, computer system 800 may implement a system for dividing the data space into disjoint or overlapping regions. For example, computer system 800 may divide the data space into regions with a decision tree. As another example, computer system 800 may divide the data space into regions with a classifier. As a third example, computer system 800 may divide the data space into regions using detectors trained in a mixture-of-generators system.

[0222] In some embodiments, computer system 800 may generate data and measure therelative frequency of data in each region for cooperatively generated data and for real data. In some embodiments, computer system 800 may select cooperatively generated data from each region based on a hyperparameter controlling the rate of sampling from the region. In some embodiments, computer system 800 may adjust a set of hyperparameters that control the regional sampling to match the relative frequencies of the regions in real data.

[0223] 25) Generate non-adversarial data near a decision boundary: In some embodiments, computer system 800 may generate data near a specified decision boundary defined by the activation of a specified discriminator element of the subsystem. In some embodiments, computer system 800 may randomly sample real data and / or realistic data generated cooperatively. Computer system 800 may then select from a set of the real or generated data one or more data items that are closest to the specified decision boundary as measured by the output of a discriminator.

[0224] In some embodiments, computer system 800 may then generate additional data by randomly perturbing the selected data items. In some embodiments, computer system 800 may compute the gradient of the difference in scores of the two competing choices at the decision boundary. In some embodiments, computer system 800 may preferentially select perturbations in directions closer to the perpendicular to the decision boundary. In some embodiments, computer system 800 may find and select at least one data item correctly recognized on each side of the data boundary and preferentially select directions close to a vector from a data correctly recognized data item on one side of the decision boundary to a data item correctly recognized on the other side of the decision boundary.

[0225] However, in contrast to adversarial attacks, computer system 800 would not make small changes in an adversarial direction that attempts to fool a classified by changing the classification decision without changing the correct answer. Instead, computer system 800 may make larger step changes that randomly sample the data space and that change the correct answer in proportion to the frequency of real data.

[0226] 26) Generate data to fool system: In some embodiments, computer system 800 may train a system to generate data that mimics observed errors made by a recognition system in data set aside from the training data. In some embodiments, computer system 800 may test the data generated to mimic the errorful training data and preferentially select from the mimic data the data items that cause errors. In some embodiments, computer system 800 may add the generated errorful data to a set of real training data and validation data.

[0227] 27) To detect problems, reduce the amount of training data: In some embodiments, computer system 800 may conduct performance evaluation data using asmaller set of training data than used in a selected benchmark test. In some embodiments, the HGDTCS, optionally with human guidance, may conduct experiments with changes in the system design including architectural changes in the network and changes in hyperparameters that control the training process. In some embodiments, computer system 800 may explore the space of alternate designs and controls using reinforcement learning.

[0228] Security and sensibility: 28) Running on private computers: In some embodiments, computer system 800 may implement pre-training for trillions of easy-to-train parameters on privately owned workstations with communication over a local area network or a wide area network or over the internet if secure communication is available. As easy-to- train parameters, computer system 800 may use parameters such as connections to explainable nodes and sufficient statistics for probability models. An illustrative embodiment is shown in Figure 8, with a LAN or local data bus. In some embodiments, computer system 800 may dedicate one or more private workstations if necessary to accommodate the memory and computation requirement. In some embodiments, computer system 800 and the HGDTCS may design the data communication between successive stages to fit within the bandwidth of the communication channel.

[0229] In some embodiments, computer system 800 may use smaller probability models with fewer parameters in lower stages and use candidate list pruning to reduce the number of active indexed models in higher stages to reduce the memory requirements and computation requirements to fit on privately owned computers rather than computers in a data center or in the cloud. This technique contributes to security by protecting private data.

[0230] 29) Piecewise constant activations: In some embodiments, computer system 800 may use piecewise constant activation functions to make it difficult for an adversary to compute a successful adversarial attack.

[0231] 30) Noisy activation: In some embodiments, computer system 800 may design a network to add small amplitude random noise to the output of each activation function. In some embodiments, computer system 800 may train the members of an ensemble of networks to each add independent random noise. In some embodiments, computer system 800 operating as the HGDTCS may run experiments to adjust hyperparameters controlling the amplitude of the noise, the number of members in the ensemble, and the ensemble combining rule, to minimize the effect of adversarial attacks or other perturbations to the input. This technique increases sensibility as well as security.

[0232] 31) Training for robustness: In some embodiments, computer system 800 may train a network for robustness against noise by training on noisy data. In some embodiments,computer system 800 may create noisy data by adding simulated noise to clean data. In some embodiments, computer system 800 may train a network for robustness against adversarial attacks by training on data that computer system 800 has created from clean data using simulated adversarial attacks.

[0233] 32) Diversity for robustness and security: In some embodiments, computer system800 may select an architecture with diversity and / or may create diversity by duplicating an element or a subsystem and then training the collection of duplicates to be diverse. In some embodiments, computer system 800 may use counter tying to create diversity among a set of duplicates. In some embodiments, computer system 800 may use a combining network and counter tying, discussed in association with Figure 6.

[0234] 33) Diverse canary networks: In some embodiments, computer system 800 may duplicate a network and create diversity by counter tying and back propagating for decorrelation of errors, as discussed in association with Figure 6. In some embodiments, computer system 800 may create and train a set of diverse robust networks and a second set of diverse networks that have not been trained to be robust or given any other defense against adversarial attacks (herein called “canary” networks). In some embodiments, computer system 800 may detect adversarial attacks by training a specialized classifier to detect the differences in the pattern of responses of the set of diverse robust networks and the pattern of responses in the set of diverse canary networks.

[0235] Block 301 : Techniques that may be used by computer system 800 for any neural or hybrid networks involving sequences:

[0236] Explainability, efficiency, efficacy, alignment with human values:

[0237] 34) “Future event” prediction: For a specified position t in a sequence, and specified token, word, or another observable item, define Event(J, L1, L2) = 3Z L < I < L2, and j G J such that Wt+i= j.In some embodiments, for a specified set J, and a specified interval [Li , L2] computer system 800 may train a discrete non-parametric probability model of the form:3.1a logprob model of the form:3.2where k denotes a specified element and dt is the input at position t. Such a probability model is called a “future event prediction model.” If the set J is explainable, the model is called an “explainable future event prediction model.”

[0238] In some embodiments, computer system 800 may add event prediction elements as companion elements to selected nodes or elements. In some embodiments, computer system 800 may represent the learned parameters of an event prediction element as a nonparametric discrete-valued probability distribution. Like explainable elements, computer system 800 may train an event prediction element with little computation per parameter. In some embodiments, computer system 800 may estimate each discrete probability from a simple frequency count.

[0239] 35) Explainable event prediction: In some embodiments, computer system 800 may train one or more event prediction elements that predict an increased likelihood for some events and decreased likelihood for other events. In some embodiments, computer system 800 may treat such an event predictor as a discriminator in the sense of the discussion about explainable elements in association with Figure 5.

[0240] In some embodiments, computer system 800 may add and train one or more explainable companion elements to an event prediction element. Computer system 800 may use the procedure discussed in association with Figure 5. However, for an event prediction node, in block 502, computer system 800 selects an item that is a named future event, such as a token or word in a text sequence, rather than an input data item or an input sequence that a language model generator is attempting to predict as the next token. In addition to improving explainability, computer system 800 may improve efficiency since, like the event prediction element itself, each explainable companion node can be trained with a relatively small amount of data and computation. Computer system 800 may train an explainable element by training it as a one-node neural network with connection weights initialized as the weights in the base node.

[0241] 36) Parametric look-back probability models: In some embodiments, computer system 800 may train one or more multivariate conditional probability models in which the probability is conditioned on a hypothesized potential event that has not yet been observed with a parametric model for one or more observable dependent variables. For example, in some embodiments, computer system 800 may train a mixture of Gaussian distributions conditioned on a specified future event, such as the occurrence of a specified word, with the observations being the activation values of specified nodes in the network at specified positions in the context. In some embodiments, computer system 800 may estimate the sufficient statistics for each Gaussian distribution as a stand-alone statistical estimation problem, that is without any back propagation for stochastic gradient descent.

[0242] In some embodiments, computer system 800 may train an indexed parametricprobability model trained in block 106 of Figure 1. For example, during training in block 106 of Figure 1, computer system 800 may train a parametric probability model of the form:3-3 Pt,X1,X2.Xn(k = Pr (X1(d) = x , X2(d) = x2,... Xn(d) = xn\Wt= k , orwhere X^d) is the activation of a specified node for the current input data d, Wtis a discretevalued random variable such as the identity of a word or token at a relative position t in a sequence,is a multivariate parametric probability model such as a multivariate Gaussian distribution or other multivariate exponential family distribution.

[0243] In some embodiments, computer system 800 may add conditional parametric models to improve efficacy of the system being trained. However, since the amount of computation per learned parameter for estimating the sufficient statistics of such a model is small, computer system 800 may also use such parametric probability models to improve efficiency. In some embodiments, the conditioning event may be a word or short phrases and each observed variable may be a node activation corresponding to a position in the context associated with a known token or word in a known context, so that each probability distribution can be explained in human understandable language.

[0244] 37) Word-level and phrase-level semantic entities in higher stages: In some embodiments, a high activation value for a specific element in a high stage of a multi-stage hybrid network language model may correspond to a particular word containing a specific token rather than to all instances of the token. Further, a high activation value of the specified element may correspond to a particular meaning for the word rather than to all instances of the word. Such a word plus associated meaning is herein called a “semantic entity.” In some embodiments, computer system 800 may associate a node activation even more narrowly to a word-in-context with several word-in-context semantic entities each associated with a further distinction within the dictionary definitions. In some embodiments, computer system 800 may build a dictionary of words and semantic entities.

[0245] In some embodiments, computer system 800 may associate each such word or semantic entity to an index or hash code. In some embodiments, computer system 800 may use the index or hash code of a word or semantic entity to gather information such as accumulated statistics for training future event prediction and / or look-back probability models.

[0246] The use of word-level and phrase-level semantic entities may improve efficacy, efficiency, and explainability.

[0247] 38) Autoencoder state space model: In some embodiments, computer system 800 may train a state space model of the context. In some embodiments, computer system 800 may train an autoencoder network with dual objectives: (1) to regenerate the context (the autoencoding objective), and (2) to generate the future of the training sequence. In some embodiments, for each objective, computer system 800 may simplify the objective. For example, in some embodiments, computer system 800 may simplify an objective to merely include tokens or words in a candidate list of a specified size. In some embodiments, computer system 800 may simplify an objective to generating a combined candidate list without predicting the position of each item in the sequence. In some embodiments, computer system 800 may simplify an objective to merely predict the items for a specified limited range of positions rather than a full context.

[0248] In some embodiments, computer system 800 may set increasingly more difficult objectives for higher stages of the network. In some embodiments, computer system 800 may create objectives that penalize the incorrect prediction of specified tokens or words that do not occur in the training text.

[0249] In some embodiments, computer system 800 may add explainable nodes to the state space model.

[0250] 39) Mixture of sources in training: In the training process for an Al system that generates or recognizes sequences, a natural approach is to process a training corpus proceeding sequentially through each batch of data, one batch at a time. However, in some embodiments, computer system 800 may instead simultaneously process training batches from multiple sources.

[0251] In an illustrative embodiment, as a preprocessing step, computer system 800 may use a search engine to find text on related topics. In some embodiments, computer system 800 may find related material by following HTML links on a website or citations in a technical article. In some embodiments, computer system 800 may process its training corpus to create links between similar passages or between instances of the same keyword. In some embodiments, computer system 800 may create links between instances of similar word embeddings or phrase embeddings.

[0252] In some embodiments, in the main training step, computer system 800 may proceed in sequential order through training text for a currently active sequential data source, sequentially moving a prediction window and training the model to correctly guess the next word or the word to fill in a masked position. However, in addition to sequentially processing the current source, in some embodiments, computer system 800 may use links to similardocuments, similar phrases, and similar keywords. In some embodiments, computer system 800 may use multiple data sources to generate candidates lists. In some embodiments, computer system 800 may switch back and forth among a plurality of sequential data sources.

[0253] In some embodiments, computer system 800 may use the linked sources to suggest additional candidate words for the current masked position and / or additional candidate words for a look-ahead beam. In some embodiments, computer system 800 may use the linked sources in computing the scores for ranking the candidate tokens or words for the next position or current masked position.

[0254] In some embodiments, computer system 800 may switch from the currently active source to a linked source as the new active source.

[0255] 40) Paraphrasing and / or summarizing and switching between training text sources for greater variety: In some embodiments, computer system 800 may paraphrase and / or summarize the currently active source and / or one or more of the alternate sources. In some embodiments, computer system 800 may use a large language model that has been previously fine-tuned to generate paraphrasing and / or summarizations. In some embodiments, computer system 800 may switch among multiple sources as well as paraphrasing and / or summarizing one or more of the sources.

[0256] 41) Mutual event prediction and look-back probability models: In some embodiments, computer system 800 may train event prediction elements to predict the conditioning variables for look-back probability models and may train parametric look-back probability models that model the activation levels of the event prediction elements in the context. In some embodiments, computer system 800 may restrict the event prediction elements to explainable event prediction elements.

[0257] 42) Train special models for words and selected phrases: In some embodiments, computer system 800 may create and train future event prediction models and / or look-back parametric probability models conditioned on words or selected phrases rather than being conditioned on the activation values of nodes in the network. In some embodiments, computer system 800 may directly build such models for every word in an extensive vocabulary. In some embodiments, computer system 800 may include rare words and selected phrases in the vocabulary.

[0258] In some embodiments, computer system 800 may use position synchronous beam pruning to build a directed acyclic graph (DAG) of the likely words and phrases following the current position, as described in association with Figure 3B.

[0259] In some embodiments, computer system 800 may train conditional probabilitymodels based on words and phrases that are specific to a selected use case or data domain. In some embodiments, computer system 800, training for a specific use case or domain, computer system 800 may construct a list of domain-specific named entities as part of the set of selected phrases.

[0260] Efficacy and Efficiency:

[0261] 43) Diverse subsystem ensemble combining network: In some embodiments, computer system 800 may create, grow, and train a set of duplicates of a subsystem as an internal diverse ensemble with a combining network. Internal diverse ensembles with combining networks are illustrated and explained in association with Figure 6. In some embodiments, computer system 800 may add a combing network to jointly optimize the performance of the networks in the ensemble, as described in US Patent 11,222,288, issued January 11, 2022, titled “Joint Optimization of Ensembles in Deep Learning,” assigned to D5AI LLC, by inventor James K. Baker, which is incorporated herein by reference in its entirety.

[0262] 44) Word count statistics by intervals (also called “frequency of occurrence vectors”): In some embodiments, computer system 800 may count the frequency of occurrence of tokens, words, or selected phrases for specified intervals in the context. In some embodiments, computer system 800 may normalize a vector of frequency of occurrence counts by dividing by the sum on the counts.

[0263] 45) Increasing the size of the word count interval: In some embodiments, as the length of the context grows as new items are generated or recognized, computer system 800 may combine the counts for two or more intervals to create intervals of increasing size for intervals at a greater distance from the current position. In some embodiments, computer system 800 may grow the size of the intervals at a geometric rate, making the total number of accumulation intervals grow only as the logarithm of the total length of the context. The geometric rate may be rapid, such as reducing every two intervals to one or the rate may be somewhat slower such as reducing every N intervals to N-l intervals, for a specified value N > 2.

[0264] 46) Embed normalized word count vectors in a latent variable space: In some embodiments, computer system 800 may map the normalized frequency of occurrence vectors into a latent space comparable to the embedding space for tokens. In some embodiments, computer system 800 may train the mapping from normalized frequency account vectors to the latent variable space by training an autoencoder with the latent space as bottleneck layer and the encoder of the autoencoder as the mapping from the normalizedfrequency of occurrence to the latent space. In some embodiments, computer system 800 may add explainable nodes to the latent variable space of word count vectors.

[0265] 47) Sequence alignment: In some use cases, the data may be in the form of a sequence, but the data sequence may not be aligned with the model sequence. For example, in audio such as speech or music the items in a sequence may vary in duration. In a sequence of local receptive fields in an image, a sequence of observations of local receptive fields may not necessarily be in a straight line and the receptive fields and / or the items in the model may vary in size and spacing.

[0266] In some embodiments, computer system 800 may train a model for a hidden Markov process and align each of two or more sequences of data to the sequence of Markov states. The use of a hidden Markov process improves efficiency. The ability to align sequences effectively is aimed at improving efficacy.

[0267] 48) Paraphrase and / or summarize source training text for additional training data:In some embodiments, computer system 800 may obtain or train a training support system to paraphrase and / or summarize source material. In some embodiments, for efficiency, computer system 800 may optimize the support system for efficiency rather than efficacy. In some embodiments, computer system 800 may use the support system to paraphrase and / or summarize the text in training data source material. In some embodiments, computer system 800 may paraphrase and / or summarize alternate sources as well as the currently active source. In some embodiments, computer system 800 may use the generation of paraphrases and / or summarizes to produce additional training data. In some embodiments, computer system 800 may produce additional training data for fine tuning to a specific domain with limited training data. In some embodiments, computer system 800 may use the generation of paraphrases and / or summaries to improve the generalizability of the system being trained. In some embodiments, computer system 800 may substitute paraphrased and / or summarized text for the original text to avoid plagiarism.

[0268] 49) For indexed models, only load models that are in the forecast candidate lists for future items in the look-ahead beam: In some embodiments, computer system 800 may compute not only a candidate list of choices for the current position but also a candidate list for future items. Pruning this future candidate list may reduce the amount of computation, improving efficiency.

[0269] In some embodiments, computer system 800 may make inactive indexed models in the candidate list active, preloading them from slower memory or secondary storage. Pruning the candidate list may also improve the efficiency of this preloading.

[0270] 50) Grow the network as more context is accumulated: In some embodiments, during pretraining and lifelong learning, computer system 800 may grow the network as more context is accumulated. In some embodiments, computer system 800 may start with a relatively small network and add additional stages as more context is accumulated. In some embodiments, computer system 800 may also increase the sizes of existing or new stages, adding additional attention heads, increasing the number of tokens, increasing the number of experts in an MOE architecture, adding additional explainable companion elements, adding additional indexed models.

[0271] In some embodiments, computer system 800 may maintain smaller networks as subsystems within the expanding network.

[0272] In some embodiments, computer system 800 may revert to a smaller subsystem when starting training on a new document.

[0273] In some embodiments, computer system 800 may revert to a smaller subsystem when starting training on a new data domain or use case.

[0274] Efficiency:

[0275] 51) Rank and prune candidate lists based on future event prediction and look-back probability models: In some embodiments, computer system 800 may prune the candidate list for choosing the next word based on scores from event prediction elements and probability models in addition to the scores computed by node activation in the network. This additional pruning may produce a narrower beam, resulting in greater efficiency.

[0276] 52) During training, prune the candidate lists based on looking ahead from known future during training: During training, the future sequence is known. In some embodiments, computer system 800 may preload into high-speed memory all the parametric look-back models for tokens within a specified future interval. In some embodiments, computer system 800 may preload word-based models and / or models for selected phrases. In some embodiments, computer system 800 may also preload precomputed lists of alternate tokens, words and phrases that may be substituted for the correct items in the future sequence in the training text. This technique improves the efficiency of training.

[0277] 53) Selective activation of look-back probability models: In some embodiments, based on estimates from lower stages and future event prediction from the current stage, computer system 800 may estimate the relative probability of future events on the candidate lists. In some embodiments, computer system 800 may use revised probability estimates from previous stages by including active future event predictions from the current stage and may also include the available look-back conditional probability models from previous stages.

[0278] In some embodiments, computer system 800 may then rescore and revise the rank of the candidates for the specified future prediction interval. In some embodiments, computer system 800 may then prune the future candidate lists to a size specified by hyperparameters controlled by the HGDTCS.

[0279] For example, in the lower stages, in some embodiments, computer system 800 may initialize the future candidate list to have a full vocabulary of N1 words. In middle stages, in some embodiments, computer system 800 may reduce the number of active indexed look-back models to N2, where N2 « Nl. In some embodiments, computer system 800 may add additional items from future prediction of rare words, named entities, and special phrases to expand the active candidate list to N3, where N2 < N3 < Nl. Computer system 800 may control the rate of reducing the size of the candidate list from stage to stage based on hyperparameter values specified by the HGDTCS, so the HGDTCS may control the number of active candidates to any specific target number.

[0280] In some embodiments, computer system 800 may continue to refine the probability estimates and gradually reduce the number of active event models from stage to stage to a number as low as N4 candidates in the highest stages, where N4 « N2. In some use cases, in which there is other information in addition to the prior items in the context, such as the audio at the current position speech recognition, the number of active candidates N4 in the highest stages may be as few as 10 or less. In other use cases, the number of candidates in the highest stage may be on the order of 100. In some embodiments, computer system 800, acting as part of the HGDTCS, may set hyperparameter values to control the size of the candidate list for any stage to any specific desired value.

[0281] 54) Simplify objective for longer range event prediction: In some embodiments, computer system 800 may simplify the objective for event prediction models that predict events to occur in an interval that is further from the current context. For example, computer system 800 may block back propagation of an error loss function to a long-range prediction element except when the rank of the supplied answer is greater than a specified rank N > 1.

[0282] 55) Enduring mixture-of-experts routing: In some embodiments, computer system800 may do a mixture-of-experts data routing in which the routing depends on an enduring property rather than on the current data item. For example, in a multiple purpose or multimodal generation system, computer system 800 may pretrain or fine tune the system on multiple data domains, multiple use cases, or multiple styles or multiple genera. In some embodiments, computer system 800 may train a mixture-of-experts data routing among the multiple subsystems. During deployment, computer system 800 may set an enduring MOEdata routing, changing the MOE data routing only if system 800 detects a change in the data or a change is specified by the HGDTCS.

[0283] 56) After pretraining, prune indexed models for sparsity: In some embodiments, after pretraining, computer system 800 may prune the parameters in event prediction and look-back probability models, selecting the parameters that have the greatest information content. In some embodiments, computer system 800 may estimate the information content by the ratio of the magnitude of the random variable being modeled divided by its standard deviation.

[0284] Block 302: Techniques that may be used by computer system 800 for any multistage hybrid network:

[0285] General techniques:

[0286] 57) Setting stage-specific conditions: In some embodiments, computer system800 may set stage-specific conditions for the current training pass for the current stage, as described in association with block 105 of Figure 1.

[0287] In some embodiments, when the input values to a stage are arranged as a directed graph rather than as a sequence, computer system 800 may select a specific linear order for the input data for the current stage. In some embodiments, computer system 800 may select a linear order consistent with the organization of the data. In some embodiments, computer system 800 may select a linear order inconsistent with the organization of the data. For unordered data, any linear order is consistent.

[0288] For sequential data, in some embodiments, computer system 800 may select either the forward sequence or the backward sequence. In some embodiments, computer system 800 may select a specific linear order for one or more stages and the opposite, or backward linear order for one or more other stages. In some embodiments, computer 800 may use this process of reversing the linear order to enable, in a higher stage, the processing of items in the middle of a sequence based on the complete context of both sides of the full sequence in the higher stage.

[0289] 58) Stage-specific selection of training data: In some embodiments, computer system 800 may select a stage-specific subset of the data. For example, in image recognition, in some embodiments, computer system 800 may select data from receptive fields on or close to a specified path through a selected set of receptive fields, rather than data from all the receptive fields. In some embodiments, in block 302, computer system 800 may select the current stage multiple times. In some embodiments, computer system 800 may select different paths through the receptive fields in successive passes through block 302 tocollectively select a multiplicity of paths through the data, such as multiple paths of receptive fields in an image recognizer.

[0290] 59) Stage-specific hyperparameters: In some embodiments, computer system 800 may select stage-specific values for specified hyperparameters and other training control variables. For example, in some embodiments, in lower stages, computer system 800 may set selection thresholds and pruning thresholds to have a larger number of active indexed models, such as item-indexed conditional parametric probability models, but computer system 800 may use models with fewer non-zero parameters. In higher stages, with more accurate ranking of the potential conditioning items, computer system 800 may set controls to have fewer active indexed models but may use models with a greater number of non-zero parameters per model. In some embodiments, rather than fixed hyperparameters, computer system 800, acting as a learning management system or in cooperation with a separate learning management system, may dynamically adjust the hyperparameters controlling the number of active models and the number of non-zero parameters per model, based on the computational resources and memory currently available.

[0291] 60) Stage-specific selection of generation or recognition: In some embodiments, computer system 800 may select to train the current stage as a generator. In some embodiments, computer system 800 may select to train the current stage as a recognizer. For example, in some embodiments, computer system 800 may implement one or more internal autoencoders. In some embodiments, computer system 800 may implement the encoder as a recognizer or part of a recognizer and may implement the decoder as a generator.

[0292] As another example, in some embodiments, computer system 800 may implement a hybrid recognition system, such as a simplified speech recognizer, as a base recognition subsystem with input data for each item but with little or no context. Computer system 800 may then implement a language model subsystem trained as a sequence generator of items, such as words or tokens.

[0293] Alignment with human values:

[0294] 61) Human assistance in specifying and setting stage-specific conditions: The human team (260 of Figure 2) is part of the HGDTCS. Along with the training coach Al (261 of Figure 2), the human team (260) guides and controls the process of computer system 800 developing and training the system being developed. In some embodiments, computer system 800 by way of the dialogue Al system 250 may get assistance from other humans via dialogues in blocks 104, 105, 109, 110, and 111 ofFigure 1.

[0295] The human team (260) may contribute to any or all the goals listed in block 210 ofFigure 2. For the goal of alignment with human values and the goal of sensibility, whether the goal has been achieved is a human judgment, so human assistance is essential to these goals.

[0296] In some embodiments, the goals of explainability and controllability may be prerequisites to human assistance, including from the human team (260) in the HGDTCS itself.

[0297] 62) Cooperation with humans in joint cooperative tasks: In some embodiments, computer system 800 may implement a use case that is an interactive, cooperative task with one or more humans and, optionally, with additional Al systems. Some examples of use cases with interactive cooperative tasks include the following examples from the discussion of block 110 of Figure 1 : a) Group conversational chats b) Personal tutoring c) Study as a student in a group on a particular subject, such as: a. Study ethics, philosophy, history b. Study machine learning c. Study deep learning d. Study large language models d) Learn a foreign language. e) Work as a teaching assistant in a course. f) Work as a graduate research assistant. g) Co-editing and proofreading h) Co-authorship i) Active discussion during joint production of images or video j) Production of music

[0298] In some embodiments, in some of these use cases, the task may involve going over the produced sequence multiple times. In some embodiments, computer system 800 may use a long look ahead beam. In some embodiments, computer system 800 may take advantage of two-sided data.

[0299] In some embodiments in which the use case is not explicitly an interactive task between a human and an Al system, computer system 800 may obtain human assistance through a dialogue with an end user in blocks 110 and 111 of Figure 1 and blocks 130 and 131 of Figure 1A. In addition, in some embodiments in which the use case is not explicitly an interactive task between a human and an Al system, computer system 800 may obtain human assistance through a dialogue with an outside expert in block 111 of Figure 1 and block 131of Figure 1A.

[0300] 63) Al in a human-guided HGDTCS: In the illustrative embodiment of Figure 2, the Human-Guided Development Training and Control System (HGDTCS) may comprise an Al training coach (261 in Figure 2) working cooperatively with a human team (260 in Figure 2). In addition, the HGDTCS may comprise a dialogue Al (250 in Figure 2). In some embodiments, computer system 800 may communicate with a member of the human team in the HGDTCS. In some embodiments, computer system 800 may communicate with a human system specifier. In some embodiments, computer system 800 may communicate with a human system designer or developer. In some embodiments, computer system 800 may communicate with a human system end user (block 110 of Figure 1). In some embodiments, computer system 800 may communicate with a human with specific knowledge or expertise as part of proactive learning (block 111 of Figure 1). In cooperative use cases, computer system 800 may have frequent interactions with a human co-author in production of their joint work.

[0301] 64) Lifelong learning with human guidance: “Lifelong learning” (also called“continual learning”) refers to additional training that a machine learning system does after the system is deployed and is being used by end users. In some embodiments, computer system 800 may continue training a system after it has been deployed (block 112 of Figure 1).

[0302] 65) Interactive training on practice data: In some embodiments, computer system800 may implement practice training in which a developer or end user may work interactively to duplicate or revise a sample of training data. In some embodiments, computer system 800 may attempt to generate a sequence matching the training text and enable the human participant to revise the proposed generated continuation from the current context. In some embodiments, computer system 800 may enable the human to choose from two or more alternative continuations. In some embodiments, computer system 800 may implement such interactive practice training as fine tuning to a specific data domain or use case. In some embodiments, computer system 800 may implement such interactive practice training as part of pretraining or a full training selected in block 113 of Figure 1.

[0303] 66) Learn by doing with multiple clients: In block 113 of Figure 1, computer system 800 may select to return to full training during deployment. In some embodiments, when approved by two or more end users, computer system 800 may apply node-to-node regularization links between two or more deployed systems in which computer system 800 has returned to full training with some amount of shared data from the original training set.

[0304] A widely deployed system may have millions of end users. In some embodiments, with many end users participating in coordination of knowledge sharing by regularization links, computer system 800 may obtain potential regularization links from every active user. In some embodiments, computer system 800 may validate each potential regularization link and then deploy the validated links more broadly.

[0305] In some embodiments, with permission, computer system 800 may accumulate and share user preferences obtained during interactive training on shared public practice data.

[0306] Explainability, Efficacy, Efficiency and Alignment with Human Values:

[0307] 67) Creating multi-token entities during position synchronous beam pruning: In some embodiments, computer system 800 may create and train indexed event prediction models and / or parametric look-back probability models indexed by words or phrases, as described in association with Figure 3B.

[0308] 68) Low perplexity sequence aligned directed acyclic graph (DAG) graph interstage: In some embodiments, computer system 800 may perform position synchronous beam pruning with the result represented as a low perplexity DAG by the procedure represented by the flowchart in Figure 3B.

[0309] Efficacy and efficiency:

[0310] 69) Indexed event predictions: In some embodiments, computer system 800 may index event prediction models by the identity of the token or multi-token associated with conditioning observation.

[0311] 70) Repeated processing of full document: In some embodiments, in either generation or recognition, computer system 800 may produce a preliminary version of a full document using a subchain up to and including a specific stage SI. In some embodiments, computer system 800 may then again produce a revised version of the full document in a later stage of the system. In some embodiments, in such a later stage S2, at positions p in the sequence, computer system 800 may use the ranked candidate lists from stage SI for positions beyond p in selecting the candidate lists for items in the candidate list for stage S2. In some embodiments, computer system 800 may use the relative rankings of the items in the candidate lists, including the identity of the top choice in selecting the candidate lists for stage S2. In some embodiments, in selecting the candidate lists for stage S2, computer system 800 may use the information from a plurality of lower stages.

[0312] In some embodiments, computer system 800 may use the information from lower stages to select in advance which indexed models to preload from slower memory or storage into higher speed memory.

[0313] 71) Repeated forward and backward processing: In some embodiments, computer system 800 may process a full document or a selected portion of a document in one direction (forward or backward) in the sequence in a stage S 1 and process the document or a selected portion of the document in the opposite direction in a later stage S2. In some embodiments, computer system 800 may do the process of alternating directions a plurality of times.

[0314] In some embodiments, in processing a document or portion of a document in a particular direction that has previously been processed in the opposite direction one or more times, at position p in the sequence being produced in the current stage, computer system 800 may combine the look-back probability estimate for the current stage and, optionally, from one or more earlier stages that processed in the same direction. In some embodiments, computer system 800 may also combine the look-back estimates from one or more earlier stages that are processed in the opposite direction as look-ahead probability estimates for the current stage.

[0315] 72) Take advantage of two-sided data: In some embodiments, for each candidate item X at position p in the current stage, computer system 800 may compute the conditional probability of the combined look-back and look-ahead probabilities for candidate value X as the product of the combined look-back probability estimate and the look ahead-probability estimate from the look-back models from stages processing in the opposite direction. Computer system 800 may model the look-back and look-ahead probability as conditionally independent, given the identity of the candidate item X. In some embodiments, computer system 800 may adjust the combined estimate by a specified factor. The specified factor may be specified by the HGDTCS or may be a learned parameter of the system being trained. In some embodiments, the specified factor may be represented as an addition to the combined log likelihoods of the forward and backward estimates.

[0316] In some embodiments, computer system 800 may obtain two-sided data without processing the full document. For example, in some embodiments, before presenting an output item at position pl to a user, to another system, or to a higher stage within the same system, computer system 800 may do forward processing in a stage up to position p2 > pl and then do backward processing from position p2 back to position pl.

[0317] 73) Stage-specific degree of use of indexed models based on multi -token entities:In some embodiments, for lower stages, computer system 800 may predominantly index models based on a specified list of tokens rather than multi-token words or phrases and, for higher stages, computer system 800 may increasingly index a larger proportion of models based on words and phrases.

[0318] In some embodiments, computer system 800 may gradually add words to the list of indexed entities, then phrases as computer system 800 progresses from lower stages to higher stages.

[0319] In some embodiments, in higher stages computer system 800 may index an unabridged dictionary. In some embodiments, computer system 800 may add named entities and other selected phrases to expand the unabridged dictionary.

[0320] In some embodiments, for the lowest stages, computer system 800 may index only the most frequently occurring tokens. In some embodiments, computer system 800 may use fewer indexed models in lower stages to improve efficiency.

[0321] 74) Creation of concordance: In some embodiments, computer system 800 may create a concordance for elements that represent selected multi-token words and phrases. For elements that have a low frequency of occurrence, computer system 800 may train conditional probability models that occur infrequently more efficiently with the aid of a concordance. In some embodiments, computer system 800 may build concordance-based models for words and phrases that have a special meaning, such as the jargon of a specific technical field.

[0322] 75) Using long-term properties of a document: In some embodiments, computer system 800 may create and train event predictors based on words and phrases that occur in the text of a prompt or that occur early in a document during training. In some embodiments, computer system 800 may identify the topic of a document, or a section of the document that serves as an introduction, abstract or summary. In some embodiments, computer system 800 may do a preliminary reading of a document to write a summary or to determine the gist of the document. In some embodiments, computer system 800 may read part of the document to determine its style.

[0323] In some embodiments, computer system 800 may use the tokens, words, and phrases in these short extracts from a document as the conditioning random variables in event prediction and / or as the conditional random variables in parametric look-back probability models. In some embodiments, computer system 800 may use these variables independent of the relative positions of the conditioning and conditioned variables.

[0324] 76) Incremental addition of stages: As explained in association with block 114 ofFigure 1, in some embodiments, computer system 800 may build a multi-stage system stage by stage or may add stages to a multi-stage system. In some embodiments, training a system with fewer stages and then adding stages may be more efficient than training the larger number of stages from scratch. Adding more stages to a system that has already been trainedto state of the art performance may further improve efficacy.

[0325] 77) Training of subchains: The use of subchains in training is explained in association with blocks 104, 105, and 106 of Figure 1.

[0326] 78) Direct connection of inner of stages to network input and output: In addition to forming and training subchains, in some embodiments, computer system 800 may connect the input for the system directly to the input of an inner stage as well as connecting the output from the immediately preceding lower layer to the input for the inner stage. Similarly, in some embodiments, computer system 800 may connect the output from the inner stage directly to the network output.

[0327] In some embodiments, computer system 800 may use these additional connections as a form of regularization to improve the efficacy of the training. In some embodiments, computer system 800 may use these additional connections to first train a stage as initial pretraining of the single stage, improving the efficiency of the training process.

[0328] 79) Use of subchains to speed up inference or to improve performance: In some embodiments, computer system 800, may select a subchain rather than a full chain to reduce the amount of computation during inference.

[0329] In some embodiments, computer system 800 may first compute a preliminary inference with a subchain and then use information from the computation with the subchain to improve the efficiency and / or the efficacy of the computation for a more complete chain. For example, in processing the more complete chain, computer system 800 may use the candidate lists computed with the subchain to better prune the candidate lists for the longer chain and / or to assure that the top candidates from the shorter chain are not pruned from a longer candidate list for the longer chain.

[0330] In some embodiments, computer system 800 may implement a mixture-of-experts comprising a plurality of subchains.

[0331] 80) Joint development of mixture-of-experts and combining network: The combination and joint development of a combining network and a mixture-of-experts is discussed in association with Figure 6. A mixture-of-experts (MOE) subsystem and a combining network have complementary properties so that computer system 800 may use them together to improve both efficacy and efficiency. In a multi-stage network, computer system 800 may also reduce the memory requirement of a mixture-of-experts. As explained in association with Figure 6, computer system 800 may implement the MOE routers in a lower stage such that computer system 800 may move the selected networks into high-speed memory by the time computer system 800 activates them in a higher stage.

[0332] 81) Efficiently growing width of a stage: In some embodiments, computer system800 may grow a stage in a multi-stage network by making duplicates of one or more elements and then make K copies of the entire stage. For each copy of the entire stage, for each element that is duplicated, computer system 800 may assign a randomly chosen one of the duplicates of the element. In some embodiments, computer system 800 may initialize each duplicate of the stage to have the same connection weights as the original version of the stage.

[0333] In some embodiments, computer system 800 may create counter-links among selected pairs of duplicate elements to train the duplicate versions of the original stage to be diverse. In some embodiments, computer system 800 may combine the outputs of the duplicates of the stage using a combining network such as shown in Figure 6.

[0334] 82) Semi -automated prompt engineering: There are known rules for improving the effectiveness of a prompt to a generative Al system which are discussed in books and on-line articles. One effective method for improving a prompt is to give one or more examples of the type of response desired. For example, if the desired result is for the Al system to generate a summary of a paragraph, instead of merely giving the instruction to summarize, a more effective prompt is to give one or more examples of instances of paragraphs and associated summaries. This method is commonly referred to as “one-shot learning,” “two-shot learning,” or “many-shot learning,” but a more descriptive name for the method would be “one-example prompting,” etc.

[0335] A simple, effective method to semi-automate prompt engineering is discussed in association with block 380 of Figure 3 A. As an example of semi -automated knowledge engineering, a human knowledge engineer may prepare a list of few-shot prompt examples. For each example, the human may designate one or more instruction key words or phrases, such as “summarize.” For each key word or phrase, the human may prepare a corresponding few-shot prompt. Computer system 800 may be trained to recognize any of the instruction key words or phrases and to add the corresponding few-shot examples to the prompt.

[0336] 83) Lower stages advance beyond and guide higher stages: In some embodiments, computer system 800 may design one or more lower stages to be simpler with fewer nodes and connections and, thus, to require less computation for inference and training. In some embodiments, computer system 800 may move the current position being generated by a lower stage to a position further along in the sequence being generated than the current position being generated in a higher stage. In some embodiments, computer system 800 may use the beam of alternatives being generated by the lower stage to help prune the beam ofcandidates being generated by one or more higher stages. Better candidate beam pruning in higher stages may contribute to improving both efficiency and efficacy.

[0337] 84) Efficient management of multi-stage training: The flow chart in Figure 1 is an illustrative embodiment in which computer system 800 may grow a multi-stage hybrid network by adding one stage at a time. In some embodiments, computer system 800 may advance lower stages to positions in a sequence beyond the position in a higher stage. In some embodiments, computer system 800 may use the information about future positions obtained from the lower stage to make beam pruning and / or candidate selection in higher stages more accurate and more efficient.

[0338] The flow chart in Figure 1 also illustrates frequent validation testing, adaptive training and proactive training. The flow chart of Figure 1 also illustrates using stage-specific controls, which increases the flexibility of the design. The flow chart also illustrates the training of multiple subchains, which computer system 800 may use in some embodiments to improve the efficiency and / or the efficacy of the training.

[0339] In some embodiments, using the capability of interstage modules to provide input directly to a stage or to route output from a stage directly to an objective and receive back propagation, computer system 800 may train a single stage by itself like a one-stage network. Training a one stage network requires less computation than training a multi-stage network and, in some embodiments, computer system 800 may use this step to initialize the stage for further training as a member of a multi-stage chain.

[0340] 85) Vary methodologies from stage to stage: In some embodiments, computer system 800 may build a multi-stage system in which some stages are generators, and some stages are recognizers or classifiers. In some embodiments, integrating and co-training generation and recognition may improve efficacy.

[0341] In some embodiments, for example in block 403 of Figure 4, computer system 800 may select a different order for processing the data for each member of a set of two or more stages of a multi-stage system. For example, computer system 800 may process sequential data, such as text or audio, going forward in one stage and backwards in another stage. In some embodiments, computer system 800 may process images or video by selecting a different sequence of receptive fields for each stage. For object segmentation and classification, in a stage, computer system 800 may follow a sequence of receptive fields that goes around the outside edge of an object. In some embodiments, computer system 800 may follow the outline of a different object in each stage. In some embodiments adding stages with multiple methodologies may improve efficacy. Having both forward and backwardanalysis may improve efficacy. Following outlines of objects may improve both efficiency and efficacy.

[0342] In some embodiments, computer system 800 may build a multi-stage system in which some stages process text and some stages process or generate audio, such as speech or music.

[0343] In some embodiments, computer system 800 may build a multi-stage system in which some stages use a different modeling methodology. For example, in a system designed for speech recognition or speech synthesis, computer system 800 may use a hidden Markov process model in one or more stages and hybrid networks in other stages. Computer system 800 may use varying modeling methodologies to improve efficiency.

[0344] 86) Links from lower stages to higher stages: In some embodiments, computer system 800 may add a regularization link from a node in a lower stage to a node in a higher stage. In some embodiments, the regularization may decrease the number of iterations required to train the node in the higher stage, improving efficiency. In general, regularization may improve generalization to new data, improving efficacy.

[0345] 87) Fast bidirectional training in hidden Markov state space model: At any interior position in a sequence of training data, the future context as well as the past context is known during training. In some use cases, for example, summarization of an article, chapter of a book, or a full book, the future context is also known during generation or recognition.

[0346] One technique that computer system 800 may use in such a situation is to first process in one direction using normal unidirectional procedures, and then process in the other direction. In some embodiments, computer system 800 may then pass through the data again, combining the results for the two unidirectional procedures.

[0347] In some embodiments, computer system 800 may instead model the data as a probabilistic function of a hidden Markov process. In some embodiments, computer system 800 may first complete a forward computation. Computer system 800 may then do a backwards computation. While doing the backwards computation, in some embodiments, at each position in the backward computation, computer system 800 may analyze the current position with complete context of both the past and future. In some embodiments, computer system 800 may compute the complete context by combining the previously computed forward pass up to the current position with the just computed backward pass with the Markov transition between the forward state and the next position in the backward state sequence.

[0348] Computer system 800 may improve efficacy by training in the full context.Computer system 800 may improve efficiency by using the Markov property of the hidden process, which requires much less computation than the general case.

[0349] 88) Train a small, fast generator as an error simulator: In some embodiments, when the system being developed or one of the subsystems is a sequence generator, computer system 800 may train a subsystem to generate sequences that simulate the sequences generated by the system or subsystem during training that fail to match the training sequence. For example, computer system 800 may train a low perplexity hidden Markov process to model instances of such errorful sequences. In some embodiments, computer system 800 may then use the Markov process model to generate simulated error sequences. In some embodiments, computer system 800 may then use simulated error sequences for additional training of the system or subsystem being developed.

[0350] 89) Efficiently growing a deep network with small stages: In some embodiments, computer system 800 may grow a multi-stage network stage-by-stage with design objectives to decrease the amount of computation per stage and increase the number of stages. In some embodiments, computer system 800 may subsequently add and train individual stages efficiently using techniques such as (a) doing initial training stage-by-stage, (b) first training each individual stage in isolation, (c) limited initial training (d) diagnosing and localizing the cause of an error with localized correction, (e) grow the stage by adding indexed event prediction elements, (f) grow the stage by adding indexed parametric probability models, (g) grow the stage by adding diverse mixtures of experts with combining networks, (h) in later training, stop backpropagation at a middle stage if the stage completes short list assignment correctly, and / or (i) limit the non-tokenized infrequent word vocabulary.

[0351] 90) Training some stages on pruning objective of higher stages: In some embodiments, computer system 800 may train some stages on a simplified objective. For example, for a network that is generating or recognizing a sequence, the network objective may be to match a sequence of training data. In some embodiments, computer system 800 may train some stages, especially lower stages on an objective of ranking a set of N output choices and to have the item matching the training item be within the N1 < N output choices. In some embodiments, computer system 800 may gradually decrease N, the number choices in higher stages. In some embodiments, computer system 800 may control the training on the simplified objective to reduce the amount of computation and improve the efficiency. In some embodiments, computer system 800 may tune the training on the simplified objective to have a regularizing effect to reduce the error and improve generalization.

[0352] 91) Training a stage based on ranks of future items in candidate lists: In someembodiments, for a specified stage in a multi-stage network, computer system 800 may compute the rank of the correct answer in the candidate list for a future position or interval in the training sequence. In some embodiments, computer system 800 may then compute an error loss function depending on the rank and the relative position of the stage within the multi-stage network. This technique improves efficacy by lowering the rate of errors caused by the correct answer not being on the candidate list. This technique improves efficiency by increasing the ability of computer system 800 to predict the indexed models to preload.

[0353] 92) Residuals of internal variables: In some embodiments, in a feed forward network or in a directed acyclic graph, for two elements whose output values represent estimates of the same underlying content, computer system 800 may create a connection from the lower elements to the higher element computing the sum of the two values. In the modified configuration, as computer system 800 trains the network, the training process will train the intervening network to estimate the residual, that is, the difference between the first value and the original second value. The sum will then represent the original second value. The intervening network will be trained to represent the difference, which computer system 800 can estimate more accurately if computer system 800 does not need to cancel out fluctuations that are common to the two signals but only learn their differences. In a known special architecture called “ResNef ’ (short for residual network) the residual connections are specified in the original design of the network. In a malleable hybrid network, the HGDTCS may change the network architecture during the training process and may insert a residual connection at any time between any ordered pair of elements.

[0354] In some embodiments, computer system 800 may determine whether to make such a residual connection by detecting correlations in the local implicit errors of a pair of nodes. In some embodiments, computer system 800 may test whether adding a residual connection has improved the efficacy of a network by testing on validation data.

[0355] Efficacy only:

[0356] 93) Training and validation stage-by-stage: In some embodiments, computer system 800 may do training and validation stage-by-stage, increasing efficacy and generalization ability. In some embodiments, computer system 800 may add one stage at a time in block 114 of Figure 1. In some embodiments, computer system 800 may add one stage at a time as a new highest stage. In some embodiments, computer system 800 may add one stage at a time and may add the new stage at a position in the network other than the highest stage.

[0357] In some embodiments, computer system 800 may train a multi-stage network witha plurality of chains and may add one stage at a time to a selected chain.

[0358] In some embodiments, computer system 800 may add a plurality of stages in block 114 but may limit the number of added stages to a specified limit before returning to 102 to resume training.

[0359] 94) Variety in stage design: In some embodiments, computer system 800 may implement a plurality of different designs for the size, architecture, and other characteristics of the stages in the multi-stage network. In some embodiments, computer system 800 may use as many different stage designs as there are stages.

[0360] In some embodiments, for each stage, the HGDTCS and computer system 800 may select the design from a wide variety of choices. For example, each stage may be either a generator or a recognizer. Each stage may have a different basic architecture. For example, a large language model stage may be based on a transformer architecture, an RWKV architecture, a state space architecture, or a mixture-of-experts. In some embodiments, each stage may be based on a different organization of the data. For example, in a speech recognition system, computer system 800 may have one stage recognize a sequence of spectral frames at a rate of 100 frames per second, have another stage recognize phonemes at an average rate of about 18 phonemes per second, have another stage recognize syllables at an average rate of about 6 syllables per second, and have another stage recognize words at an average rate of about 2 words per second. In some embodiments, computer system 800, in processing a stage, may estimate the part of speech of each word. In some embodiments, computer system 800, in processing a stage, may parse a sentence and determine the head word of each grammatical component.

[0361] In some embodiments, computer system 800 may use a transformer decoder architecture in some stages and a transformer encoder architecture or a transformer encoderdecoder architecture in other stages. In some embodiments, computer system 800 may use a low-perplexity directed acyclic graph in the interface between successive stages of the different types.

[0362] In some embodiments, computer system 800 may use a smaller set of tokens in some stages than in other stages. In some embodiments, computer system 800 may use multitoken elements in some stages and not in others.

[0363] In some embodiments, the HGDTCS and computer system 800 may avoid specifying the number of stages and number of designs in advance. In the illustrative embodiment of Figure 1, computer system 800 may start with an empty network with no stages or with a single stage. In blocks 114 and 115 id Figure 1, computer system 800 mayadd any number of stages before resuming training.

[0364] In the illustrative embodiment of Figure 1, by returning to block 102 from blocks 113 and 114, computer system 800 may change the selection of training data after any validation or any addition of a stage.

[0365] In some embodiments, computer system 800 may specify or change the design for a stage in block 311 based on previous validation testing. In some embodiments, computer system 800 may use this methodology to combine training and validation of each stage and subsystem design variety. Design variety is aimed primarily at improving efficacy.

[0366] 95) User-specific customized pronunciation models: In some embodiments, in a multi-stage system, computer system 800 may build and train user-specific pronunciation models. For example, in a multi-stage speech recognition system, in some embodiments, computer system 800, build and train a chain of stages as a language model with training data in the form of text. In some embodiments, computer system 800 may build and train a word recognizer with audio input, trained on labeled audio data.

[0367] In some embodiments, computer system 800 may build and train an allophone recognizer and / or a phoneme recognizer with audio input and trained on audio data with a transcript of the speech. In some embodiments, computer system 800 may align the audio of the speech with the word level transcript using a speech recognition system such as a hidden Markov process model without using the language model that computer system 800 trained from text. Alignment of a hidden Markov process model to a word-level transcription is well- known to those skilled in the art of speech recognition with a hidden Markov process model.

[0368] In some embodiments, computer system 800 may further align the sounds within each word with the sequence of phonemes in the standard pronunciation of the word. In some embodiments computer system 800 may use an allophone recognizer or low-level phonetic recognizer to relabel each phoneme. In some embodiments, computer system 800 may then collect statistics for a specific speaker to determine whether a standard phoneme is consistently relabeled for specific words by that speaker as being a different sound than in the standard pronunciation.

[0369] In some embodiments, computer system 800 may determine if a specific relabeled sound occurs more frequently in some contexts than others. If the relabeling frequency in one or more contexts is greater than a specified threshold, in some embodiments, computer system 800 may create a user-specific pronunciation dictionary containing the alternate pronunciation. If the standard label is retained for a fraction of its instances in a specified context is less than a specified threshold, in some embodiments, computer system 800 maydrop the standard pronunciation from the user-specific pronunciation dictionary.

[0370] 96) Mixture of models with fixed pruning: In some embodiments, computer system 800 may train multiple context models. Each model may have a different architecture, different learned parameters, a different objective, or be trained on different data. Some models may be trained on or fine-tuned to different use cases.

[0371] In some embodiments, computer system 800 may create a weighted mixture of a plurality of trained context models. In some embodiments, computer system 800 may train the mixture weights of the mixture on a specific task, or on a specific data domain. In some embodiments, computer system 800 may train the mixture weights to be sparse, using weight decay, sparsity regularization, and / or pruning of low magnitude weights.

[0372] In some embodiments, computer system 800 may implement this mixture of context models as a special case of a mixture of models with fixed pruning with on-going training of the non-zero weights.

[0373] In preferred embodiments, computer system 800 may freeze the zero weights and pruning decisions.

[0374] In some embodiments, computer system 800 may continue training the non-zero mixture weights during deployment on a specific task. In some embodiments, computer system 800 may continue fine-tuning the non-zero mixture weights during deployment. In some embodiments, computer system 800 may continue training non-zero weights through proactive learning, lifelong learning, adaptive training, and validation, such as in blocks 110, 111, 112 of Figure 1.

[0375] With permanent pruning the amount of computation and the amount of memory remained fixed after computer system 800 freezes the pruning and the zero weights. Thus, during deployment, in which computer system 800 may have less computer resources than during pretraining, computer system 800 may improve efficacy by having a mixture of models rather than a single model. In deployed systems, especially on edge devices, the amount of GPU memory may be much less than in the top of the one GPU boards used in Al data centers. Computer system 800 may even continue to train the mixture weights which may improve efficacy by fine-tuning to the specific deployment task and data. Compared to dynamically selected mixture-of-experts models, computer system 800 may reduce the memory by the same ratio as the reduction in the amount of computation.

[0376] Efficiency only:

[0377] 97) Fewer active indexed models in higher stages: In some embodiments, computer system 800 may measure the average rank of the correct answer counting downfrom the best scoring answer. In some embodiments, computer system 800 may make this average rank measurement at multiple stages, including some middle stages and some higher stages.

[0378] If the average rank of the correct answer is lower (i.e. closer to the top choice) for higher stages than for middle stages by more than a specified amount, then, in some embodiments, computer system 800 may adjust hyperparameters that affect the number of active index models such that the number of active index models in the highest stages is less than the number of active models in middle stages to optimize a specified trade-off between efficiency and efficacy.

[0379] If the average rank of the correct answer for higher stages is not lower than for middle stages by the specified amount, then computer system 800 may do additional training on all the components contributing to the rank of the correct answer in the higher stages, including the information passed to the higher stages and the amount of information gained from two-sided data, if any.

[0380] 98) Skip indexed models that are not ready: In some embodiments, if an indexed model is not available in local memory at the time necessary to do a computation using it, computer system 800 may skip the computation and substitute a default value. In discriminations with positive and negative values, computer system 800 may use zero as the default value. In some embodiments, computer system 800 may determine whether to skip such a computation based on an estimate of the relative effect on efficiency versus efficacy and on hyperparameters set by the HGDTCS specifying the relative value of marginal changes in the efficiency and efficacy. In some embodiments, computer system 800 may vary the specified relative values of efficiency and efficacy during the training.

[0381] 99) Selective backpropagation to lower stages: During back propagation, in some embodiments, for a specified fraction of the data, computer system 800 may terminate the back propagation at a middle stage and not propagate the derivatives of an objective to lower stages. In some embodiments, computer system 800, may base the fraction of data to be back propagated based on an estimate of the amount of additional training data required to adequately train the higher stages relative to the amount of additional training data required to adequately train the lower stages. In some embodiments, computer system 800 may base the estimate of the amount of training data required for a specified stage in part on the number of learned parameters in the stage. In a network built and trained stage-by-stage, computer system 800 may base the amount of additional training required for a specified stage in part on the amount of training the stage has already received.

[0382] In some embodiments, computer system 800 may implement stage skipping by controlling the selection of subchains. In some embodiments, computer system 800 may at first only train lower stages. In such embodiments, when training the higher stages, computer system 800 may base the estimate of the amount of training required for a specified stage in part on the amount of training computer system 800 has already done on the stage.

[0383] In some embodiments, computer system 800 may base the estimate of the amount of additional training required for the stage in part on measurement of the error rate of the stage on samples of training data and / or on samples of validation data.

[0384] 100) Multi-channel inference on shared hardware: In some embodiments, computer system 800 may cascade multiple channels of inference on a shared set of hardware processors, as discussed in association with Figure 9. With such a cascade architecture, computer system 800 may have N processors processing N channels of data for a model with N times as much high-speed memory as a single processor processing a single channel using the same model but needing to use slower memory or secondary storage.

[0385] 101) Simplify objective for elements in lower stages: In some embodiments, computer system 800 may simplify the objectives for elements in lower stages of a multistage network by using an error loss function based on the rank of the correct answer in candidate lists in higher layers. In some embodiments, computer system 800 may compute a stage-specific, data-item-specific hyperparameter that specifies a fractional multiplicative factor for the learning rate for the current stage for the current data item. In some embodiments, the multiplicative factor for a data item may be zero if the rank of the correct answer is less than a specified stage-specific target rank. In some embodiments, computer system 800 may specify a multiplicative learning rate factor that is a function of the rank. In some embodiments, computer system 800 may specify a different function for each stage.

[0386] 102) Develop system trained for multiple distinct writing styles and other differences: In some embodiments, computer system 800 may pretrain a system of diverse duplicates of one or more subsystems. In some embodiments, computer system 800 may pretrain a system with mixture-of-experts routers and / or with combining networks, as illustrated in Figure 6.

[0387] In some embodiments, computer system 800 may separately pretrain and / or fine tune the system on different selections of training data, on different modalities of input data, or on other known or recognizable differences in the data, such as (1) different parts of a document, (2) different writing styles, (3) different artistic styles, (4) different speakers in speech recognition, or (5) any other difference that computer system 800 can detect.

[0388] In some embodiments, computer system 800 may train an enduring mixture-of- experts data routing system to recognize the style or other type of difference and to recognize when the style or other difference changes during a generation or recognition task.

[0389] 103) Rerunning batches of training data with full look ahead: In some embodiments, even when training with a training set so large that computer system 800 may perform only one epoch of training, computer system 800 may rerun training on selected batches of training data. In rerunning the training on a selected batch of training data, computer system 800 may use the future candidate lists computed during previous training pass on the same selected data.

[0390] In this situation, computer system 800 may use the future candidate lists to enable tighter pruning in the second and later passes while also assuring that the correct answer and the best competing answers are not pruned. Computer system 800 may ensure that competing answers that may affect back propagated score differences are not pruned.

[0391] In some embodiments, computer system 800 may use knowledge from the previous passes to assure that necessary indexed models will be loaded into high-speed memory well in advance.

[0392] In some embodiments, in the first pass as well as the later passes, computer system 800, acting as part of the HGDTCS, may specify hyperparameter values that, for each pass, optimize efficacy versus efficiency trade-offs for this situation that may be different from the trade-offs in the standard single-pass situation.

[0393] Generalizability:

[0394] 104) Validation and regularization to improve generalizability:

[0395] In some embodiments, especially when implementing multiple stages and / or when implementing multiple duplicates of subsystems and / or when increasing the number of learned parameters by other means, computer system 800 may repeatedly use validation testing to measure the ability of the system to generalize to new data.

[0396] In some embodiments, if computer system 800 detects a degradation in the ability of the system to generalize to new data, computer system 800 may add data-dependent node- to-node “is-equal-to” regularization links between selected pairs of nodes. In some embodiments, computer system 800 may select pairs of nodes in which one node is in a first stage or a first of a set of duplicated subsystems and the second node is in a corresponding location in a second stage or a second of a set of duplicated subsystems.

[0397] In some embodiments, computer system 800 may impose other hyperparameter- controlled regularization.

[0398] 105) Variety of stages design and methodology: In some embodiments, computer system 800 may specify and / or may randomly create differences in the design and / or differences in the methodology among the stages in a system. For example, in some embodiments, computer system 800 may use only tokens in the lowest stages and may gradually add words and phrases to successively higher stages. In a task such as speech recognition, in some embodiments, computer system 800 may dedicate a subchain of stages to recognition from the audio signal with a limited language model such as a bi-gram language model. In some embodiments, computer system 800 may dedicate another subchain of stages to implement a large language model trained from text data rather than or in addition to training the large language model only as part of a network with audio input.

[0399] In some embodiments, computer system 800 may represent the output of audio recognition with a low perplexity directed acyclic graph (DAG). In some embodiments, computer system 800 may train a large language model trained on text data by fine tuning the large language model as a graph neural network with DAGs as input and a specified script as output.

[0400] In some embodiments, computer system 800 may have some stages process sequences in one direction and have other stages process sequences in the other direction, as discussed in association with the technique of two-sided data.

[0401] In some embodiments, computer system 800 may specify the size of the candidate list for choosing the item to be selected as the next element in the sequence being generated or recognized as a stage-specific hyperparameter. For example, computer system 800 may reduce the number of candidates in higher stages because more information has been accumulated so that the best choices may be concentrated on a shorter list. In some embodiments, computer system 800 may reduce other hyperparameters that control the number of active elements, such as the beam width of a look-ahead beam search.

[0402] In some embodiments, computer system 800 may specify significant differences between the design of lower stages and the design of higher stages. For example, computer system 800 may vary the number of indexed models and / or the number of parameters per model. In some embodiments, for higher stages, computer system 800 may train a larger total number of indexed models with more parameters per model but then control the number of active indexed models to be fewer than for lower stages.

[0403] Security and sensibility

[0404] 106) Stages with noisy connections: In some embodiments, computer system 800 may train one or more stages adding pseudo-random noise to the input values to selectedelements to improve robustness against noise, adversarial attacks, and other disturbances to the network input and or the internal variables of the system. In some embodiments, computer system 800 may use validation testing with noise added to the input and, optionally, added to internal variables to test the robustness of the system and to adjust the amount of noise to be used in the training.

[0405] In some embodiments, during training, computer system 800 may add noise to the output of selected elements but send the same noisy output value to all the elements receiving connections from a given element.

[0406] 107) Estimate conversion or transformation from scores to estimated logarithms of probabilities: In some embodiments, for an explainable discriminator element or partially explainable discriminator element, for each of the two explainable or recognizable sets being discriminated, computer system 800 may compute a histogram of the output value of the element for data items in the specified set. In some embodiments, computer system 800 may fit a monotonic regression function to the histogram data. In some embodiments, computer system 800 may then estimate a monotonic function to map the regression function to a logarithm function. In some embodiments, computer system 800 may then combine, compare, and perform other operations on the output values of the logarithm function as estimated logarithms of probabilities.

[0407] It is common practice to use what is called a “softmax,” which is defined by z7-exp (x(), which implicitly treats the variables xtas logarithms of probabilities even if nothing in the preceding network treats them as logarithms of probabilities or causes them to estimate logarithms of probabilities or even to be on the same scale. In contrast, in some embodiments, computer system 800 may transform the output of selected elements so that the output distributions approximate the logarithms of probability estimates.

[0408] All goals together:

[0409] 108) Semi -automated knowledge engineering and human-guided training: For this discussion, of semi-automated knowledge engineering, it is helpful to note three related but distinct concepts. First, formal knowledge engineering is a discipline in which computer scientists work with domain experts to develop a system that incorporates expert knowledge for an application in the domain of the human experts. As an example, knowledge engineering for medical diagnosis may represent knowledge obtained from top human medical diagnosticians. The system may use knowledge representations such as domain-specific ontologies and mereologies, constructed with guidance from the human domain experts. The team of computer scientists may use specialized computer languages such as frame languages. The development process tends to rely much more on human expertise than on a massive amount of training data. The discipline of knowledge engineering herein called “formal knowledge engineering” achieved prominence and development as a formal system in the 1980s. However, Al developers not part of this formal discipline of knowledge engineering continue to use the phrase “knowledge engineering” less formally.

[0410] Second, the “experts” in a large neural network with a “mixture-of-experts” are a plurality neural network subsystems that have been automatically trained to be different from each other and to be complementary. The expert subsystems may be trained on different selections of trained data, there may be architectural differences in the networks. In some embodiments of this invention, they may be trained to be diverse by counter tying nodes or connection weights or by decorrelation of errors back propagation from a combining network. Although a neural network may comprise a mixture-of-experts subsystem, that does not per se constitute formal knowledge engineering or informal supportive knowledge engineering.

[0411] Third, informal supportive knowledge engineering is the incorporation of human supplied knowledge in the design and training of an Al system or a component of an Al system without the formal framework of formal knowledge engineering. This usage of phrase “knowledge engineering” distinct from the formal discipline of knowledge engineering received prominence from 2010 to 2020 when developers of large neural networks began proclaiming that automatic training of their large systems had eliminated the need for the knowledge engineering. Most of the systems from which “knowledge engineering” had been “eliminated” had never had formal knowledge engineering. The performance of the new neural networks matched or exceeded the performance of the previous state-of-the-art systems.

[0412] Examples of informal knowledge engineering include the use of common engineering knowledge, such as techniques for signal processing that do not require expertise in a particular task, such as medical diagnosis.

[0413] Examples of informal knowledge engineering also include the use of common human knowledge such as the knowledge of grammar and of the meanings of most common words.

[0414] In preferred embodiments, computer system 800 may reduce the amount of human labor required for informal knowledge engineering while amplifying its beneficialeffects.

[0415] Example use cases: a) Communication between two users who are not fluent in the same language b) Many -language translation c) Speech recognition and synthesis d) Transcription of lectures and videos e) Meeting transcription f) Cleaning and correcting recordings for audio books g) Creation of series of illustrations h) Weather prediction i) Prediction of stock market and over economic time series

[0416] An illustrative embodiment of informal semi-automated knowledge engineering is shown in Figure 3A. Block 303 counts techniques that apply only to hybrid networks and not to ordinary neural networks.

[0417] Techniques for hybrid networks only: For this discussion, a network for which computer system 800, during back propagation, may substitute a different activation function for an element than the function used during activation is regarded as a hybrid network rather than a normal neural network. Some of the following techniques apply to such hybrid networks and are counted in block 303 of Figure 3. Some of the following techniques apply only to hybrid networks because they work with units that have or that are modified to have multiple output values.

[0418] 109) Data-dependent substitution of back propagation function for security and sensibility: In some embodiments, computer system 800, for some data items, computer system 800 may substitute a different value for the derivative of the activation function of an element rather than the actual derivative. For example, in some embodiments, for a classifier, computer system 800 may determine that for a specific data item the difference of the best output score and the second-best output score is less than a value specified by a hyperparameter controlled by the HGDTCS. In some embodiments, computer system 800, for one or more elements with unbounded activation functions, computer system 800 may substitute the derivative of a bounded activation function.

[0419] 110) Substitution of backpropagation function to correct or avoid implicit errors:In some embodiments, computer system 800 may substitute a different value for the derivative of the activation function to avoid or correct a local implicit error. For example, for an activation that is bounded or that is bounded on one side, such as a rectified linear unit,computer system 800 may substitute the derivative of an unbounded function, such as a linear function, during back propagation for a data item in which an element makes an error on the implicit local objective.

[0420] 111) Substitution of backpropagation to maintain explainability: In some embodiments, for an explainable element, computer system 800 may substitute a lower magnitude value or zero for the derivative of a data item that is in neither of the target sets being discriminated by an explainable element.

[0421] Alternatively, in some embodiments, for an explainable element, computer system 800 may substitute the sign of the input value to the activation function multiplied by a hyperparameter specified by the HGDTCS.

[0422] 112) Socratic wisdom output: In some embodiments, computer system 800 may add an extra output that estimates the reliability of the regular output of a specified element. If the specified element is a node, computer system 800 may convert the specified element to a unit. In some embodiments, computer system 800 may add a new node to the specified unit. Computer system 800 may then train the new node to predict whether the base output of the element is making an error on its local implicit objective. Computer system 800 may add the output of the hew error prediction node as an additional output for the specified unit. The estimation of the reliability or error rate of an element is herein called “Socratic wisdom,” referring to Socrates’ principle: “Know what you know and know what you do not yet know.”

[0423] 113) Few-bit output representation: In some embodiments, computer system 800 may represent the output of a unit with 4 bits or less, optionally including side information such as Socratic wisdom.

[0424] In the simplest case, computer system 800 may use a single bit to represent a node with a monotonic activation function whether the non-quantized value is above of below a specified threshold T. Note that, although the quantized output is a zero-one step function, the unit is not equivalent to a node with a zero-one step function as its activation function. During back propagation for training, computer system 800 may use the non-quantized activation function. In some embodiments, the threshold T may be a hyperparameter specified by the HGDTCS. In some embodiments, the threshold T may be a learned parameter trained as an output of a unit in the network itself or trained in the Learning Coach Al (261 in Figure 2).

[0425] In some embodiments, computer system 800 may add an additional bit to represent the output of a companion error to predict Socratic wisdom node.

[0426] In some embodiments, for an element representing a discrimination, such as an explainable unit discriminating between two explainable sets, computer system 800 may add an additional bit to indicate “BOTH” or “NEITHER”. In some embodiments, computer system 800 may add a Socratic wisdom bit to the both-or-neither bit, making a total of four bits for the discriminator unit.

[0427] 114) Bit vector latent spaces: In some embodiments, computer system 800 may represent the variables in a latent variable space with few-bit units, optionally with Socratic wisdom. In some embodiments, computer system 800 may use the few-bit encoding as the constraint that prevents a bottleneck layer of an autoencoder from making an exact copy of the input when the input values to the autoencoder have non-binary numerical values.

[0428] 115) Socratic active defense against adversarial attacks: In some embodiments, computer system 800 may train a Socratic wisdom node not only to predict false positive and false negative errors in a detection or discrimination task but also to detect adversarial attacks. For example, computer system 800 may use canary networks to detect adversarial attacks and combine that estimate with the node-specific error prediction. In some embodiments, to reduce the amount of computation and memory required, computer system 800 may train canary networks specific to lower stages of a multi-stage network.

[0429] 116) Redundant subsystem for robustness and defense against adversarial attacks:In some embodiments, computer system 800 may develop and train a system with redundant subsystems in which each subsystem estimates the reliability of its output using both error prediction and detection of adversarial attacks. In some embodiments, computer system 800 may select among the redundant subsystems using either mixture-of-experts data routers or delayed decision combining networks.

[0430] 117) Testing reliability of subsystem with reverse adversarial attack: In some embodiments, when an adversarial attack is detected or suspected, computer system 800 may test the reliability and robustness of an individual subsystem using a weak reverse adversarial attack. In some embodiments, computer system 800 may simulate a weak adversarial attack that is tuned only to succeed against non-robust systems.

[0431] 118) Delayed decision subchain selection: In some embodiments, computer system 800 may train different subchains for different conditions or different processing of the input data. In some embodiments, during inference (generation of recognition) computer system 800 may perform a plurality of computations with each computation using a different selection of subchains. In some embodiments, computer system 800 may train a specialized stage to combine the results of multiple subchains based on an estimate of the reliability ofeach subchain. The selection of the subchain to be used is made during the activation computation, so the selection is known during back propagation, reducing the amount of computation during training. Computer system 800 may also use delayed decision subchain selection during semi -automated knowledge engineering, as discussed in association with Figure 3 A. During generation or recognition, delayed decision subchain selection improves efficacy and may improve efficiency in higher stages.

[0432] 119) Computation and propagation of Socratic wisdom: In some embodiments, for each of a plurality of base elements, computer system 800 may train an associated element as a specialized reliability estimate herein called a “Socratic wisdom” element. In some embodiments computer system 800 may add the value of the Socratic wisdom element as an extra output value for the associated base element. In some embodiments, when the Socratic wisdom element determines that the base element output value is unreliable, computer system 800 may substitute a neutral output value for the activation value of the base element. In some embodiments, computer system 800 may train the Socratic element directly from statistics of the performance of the base element, without any back propagation from elements to which the Socratic element has outgoing connections. In some embodiments, a specific Socratic element may have incoming connections from Socratic elements associated with other base elements that are connected directly or indirectly into the base element of the specific Socratic element. In some embodiments, computer system 800 may train a Socratic element to rate its base element as unreliable based in part on the number of incoming connections from elements that are estimated to be unreliable.

[0433] 120) Socratic wisdom defense against adversarial attacks and other perturbations:In some embodiments, computer system 800 may train a system comprising Socratic elements on data with simulated adversarial attacks and / or noise and other perturbations to the input data. In some embodiments, computer system 800 may adjust the amount of noise and other perturbations on training data to make the error rate on training data comparable to the error rate on validation data during the training of the reliability estimates for the Socratic wisdom elements. During the training, computer system 800 may use the knowledge of which data items are from adversarial attacks to set the target objective of the Socratic wisdom elements to be the detection of unreliability. In some embodiments, computer system 800 may represent the Socratic wisdom value for an element with a single bit. Even in a “white box” attack, in which an attacker knows everything about the network being attacked including the connections and the values of learned connection weight parameters, computer system 800 may prevent changes caused in the activation of lower nodes from propagating tohigher levels by marking the changed values as unreliable.

[0434] 121) Multi -token-based language model: In some embodiments, computer system800 may train and / or use a large language model (LLM) to generate or classify a sequence of units in which a unit may comprise a subsequence comprising a plurality of tokens rather than a single token. In some embodiments, computer system 800 may use a unit comprising a plurality of tokens but less than a complete word. In some embodiments, computer system 800 may use a unit comprising a sequence of tokens representing a complete word. In some embodiments, computer system 800 may use a unit comprising a sequence of tokens representing more than a complete word. In some embodiments, computer system 800 may reduce the amount of computation in pre-training of the LLM or inference with LLM by using the larger units to reduce the number of computation steps. In some embodiments, computer system 800 may use special tokens as subunits to use words as units. For example, in some embodiments, computer system 800 may create a set of word initial tokens, a set of word middle tokens and a set of word final tokens. In some embodiments, a word middle token may be an empty subsequence containing no tokens. In some embodiments, computer system 800 may generate token sequences subject to the constraint that every generated initial, middle, final subsequence corresponds to a word in a specified vocabulary or in a specified training corpus, optionally with limited exceptions to allow out-of-vocabulary words to be added. In some embodiments, computer system 800 may reduce the amount of computation or the amount of memory required by limiting the set of tokens allowed after the word initial or word middle token to the subset of tokens that satisfy the constraint that the token subsequence can be extended to form a complete word.

[0435] 122) Overlapping tokens: In some embodiments, computer system 800 may generate tokens or multi-token units that represent overlapping text sequences. In some embodiments, computer system 800 may generate a token with a designated initial portion of text that overlaps with a final portion of text from the preceding token. In some embodiments, computer system 800 at each step may generate a multi -token subsequence subject to the constraint that the initial token of each multi-token matches the final token of the previous multi-token sequence. In some embodiments, computer system 800 may limit the set of multi-token units to two-token units that occur in a specified training corpus to reduce computation and memory requirements and to improve accuracy by reducing the perplexity of the allowed generated sequences. In some embodiments, computer system 800 may use a separate generation process to allow limited exceptions to the constraint subject to a limitation on the amount of computation and memory allocated to the separate generationprocess. In some embodiments, computer system 800 may train the system to optimize a specified trade-off of performance on a measurable objective versus the amount of computation resources used.

[0436] 123) Full words as units in a large language model: In some embodiments, computer system 800 may use full words as units in a large language model. In some embodiments, computer system 800 may be configured to train a language model to generate text on a word-by-word basis rather than token-by-token. In some embodiments, computer system 800 may compute embeddings and disembeddings for words in addition to or instead of for tokens. In some embodiments, computer system 800 may use word units in a draft model and / or in a verification model.

[0437] 124) Word-based language model: In some embodiments, computer system 800 may use word-based models in addition to or instead of models based on token embeddings. In some embodiments, computer system 800 may organize, store and access word models and other associated information based on a numerical index and / or word written as text. In some embodiments, computer system 800 may use the numerical index or text form of words for candidate lists for the current and future positions. In some embodiments, computer system 800 may access and prune candidate lists as described in association with Figure 3, blocks 406 and 408 of Figure 4 and in association with many of the techniques discussed in association with Figures 3 and 3 A to 31. In some embodiments, computer system 800 may use indexed word-based access to candidate list, thus avoiding the computation of disembedding in each access to an item on a candidate list. In some embodiments, computer system 800 may model a word as a mixture of one or more variants for a specific written word. In some embodiments, computer system 800 may compute conditional probability models, correlation, or attention weights for word-variant pairs rather than for token pairs. In some embodiments, computer system 800 may store a mixture model and other information in a data structure indexed by word identity. In some embodiments, computer system 800 may use a conditional random field (CRF) model in the training of and / or inference with word variants.

[0438] 125) Word sampling for efficient training: In some embodiments, computer system 800 may sample only as many instances of each word as necessary for estimating sufficient statistics or for training other learned parameters. In some embodiments, computer system 800 may use a concordance to select samples of a specific word from the training corpus. In some embodiments, for each selected sample of a particular word, computer system 800 may select a specified number of preceding and following words of text. In someembodiments, for estimating conditional parametric probability models, computer system 800 may select only a limited number of examples of a token, word or other condition event rather than all the instances in the training set. In some embodiments, computer system 800 may select only as many examples of a conditioning event to satisfy a specified criterion on the relative size of a confidence interval. In a language model training set with up to a trillion or more words, to estimate sufficient statistics of a parametric probability model for a word or token, computer system 800 may select 100 or fewer examples of the word or token, a very small fraction of the number of instances in the training set.

[0439] 126) Hidden Markov process: In some embodiments, computer system 800 may model a unit (i.e., a word, token or multi-token sequence) as a mixture of probability distributions. In some embodiments, computer system 800 may associate an instance of a unit with a state of the Markov process. In some embodiments, computer system 800 may associate the instance of a unit with a transition of the Markov process. In some embodiments, computer system 800 may use discrete probability distributions. In some embodiments, for a discrete-valued probability distribution, computer system 800 may associate a random variable that indicates whether a unit associated with the random variable occurs within a specified interval in the sequence being generated.

[0440] In some embodiments, computer system 800 may use parametric probability distributions. In some embodiments, for a parametric probability distribution, computer system 800 may associate a random variable with the activation of a specified node in the network or with the value of a cell within a hybrid unit in a hybrid network. In some embodiments, computer system 800 may train the node or cell to estimate the probability or expected value of a specific event. In some embodiments, the predicted event may be the number of occurrences of a specified unit with a specified interval of the sequence being generated. In preferred embodiments, the parametric probability distribution may be multivariate with a plurality of continuous valued random variables. In some embodiments, with discrete- valued conditioning variables, computer system 800 may use a conditional random field model (CRF) in addition to or instead of a hidden Markov process model.

[0441] In some embodiments, during training, computer system 800 may access the future portion of the training text and train a node or cell to estimate counts of occurrences of specified units for an interval that includes a portion of the training sequence that is in the future relative to position in the sequence relative to the position in the sequence associated with the node or cell at the time that the value of the node or cell is observed. In some embodiments, during inference, computer system 800 may use such future predictions inestimating the relative likelihood of items in the candidate list for the current and future positions in the sequence.

[0442] 127) Syntax and semantic supplement to word sequence model: In some embodiments, computer system 800 may organize the representation of the data corresponding to a body of text using data structures other than a sequence of words or units. In some embodiments, computer system 800 may organize the words in the sentence as nodes in a parse tree with respect to a specified grammar. The grammar may be a finite-state grammar or a context-free grammar. In some embodiments, computer system 800 may simplify the grammar to reduce the amount of computation and memory required.

[0443] In some embodiments, during development, computer system 800 may experiment with a plurality of grammars to determine the best trade-off of the amount of computation and memory versus the accuracy in matching the training data in a generation or recognition task. In some embodiments, computer system 800 may use a less restrictive grammar designed to over generate. Note that using no grammar is equivalent to a grammar that allows any sequence of words.

[0444] 128) Feature vector associated with each word variant: In some embodiments, computer system 800 may train a feature vector for each word variant. In some embodiments, computer system 800 may train one or more explainable or decisive nodes as the elements of a word-variant-specific feature vector.

[0445] 129) Use a parser-based conditional stochastic influence model: In some embodiments, computer system 800 may train a unidirectional or bidirectional conditional probability model conditioned on a word variant at a specific position in a parse tree of sentence in a specified grammar, with conditioned random variables associated with a word variant at a second position in the parse tree. In some embodiments, computer system 800 may use a CRF model to do part-of-speech tagging.

[0446] 130) Pairwise word and token modeling: In some embodiments, computer system800 may select to build one or more models associated with pairs of units in which each member of the pair is a specified token, multi-token or word. In some embodiments, for each pair, computer system 800 may select blocks of text from the training data to estimate statistics associated with occurrences of the members of the pair in the blocks of text. In some embodiments, computer system 800 may estimate the correlation or may estimate the conditional probability of one member of a pair occurring given the other’s occurrence. In some embodiments, computer system 800 may estimate higher order statistics of the cooccurrence of the members of the pair. In some embodiments, computer system 800 mayreduce the amount of computation for training by selecting blocks that comprise a small fraction of the training corpus for a large language model. In some embodiments, computer system 800 may use these statistics in place of or in addition to the correlation of token embedding vectors in an attention block. In some embodiments, computer system 800 may supplement a smaller transformer model with pairwise word and token modeling to match the performance or a larger transformer model.

[0447] 131) Use of human understandable knowledge representation structures: In some embodiments, computer system 800 may estimate co-occurrence statistics of words or units associated with specified elements in a general knowledge representation structure. In some embodiments, computer system 800 may use a mereology, an ontology, a semantic network, a dictionary, a thesaurus, a bi-lingual dictionary, or a pronunciation dictionary. In some embodiments, computer system 800 may build a data structure to represent a computable relationship among related units. For example, computer system 800 may build a data structure to represent steps of logic or reasoning. As another example, computer system 800 may build a data structure to represent the relationships among multiple documents by tracking the citations among the documents.

[0448] 132) Pipeline parallelism for increased throughput and reduced apparent latency:In some embodiments, computer system 800 may be a distributed system organized as illustrated in Figure 9. In some embodiments, in each non-final stage, computer system 800 may recognize or generate text beyond the text being processed by the next higher stage. As each unit of text is processed by one of the multi-stage stack segments in Figure 9, work on that unit of text is passed to the next higher multi-stage stack segment. In some embodiments, if there is an apparent latency in each stack segment, computer system 800 may overlap that latency with the amount of latency in the next higher stack segment on the preceding unit of text so that there is no accumulation of apparent latency with additional units of text. At the completion of a unit of text by the final multi-stage segment, there is only a non-apparent latency before the first unit of text is processed. Furthermore, the average throughput rate is multiplied by the number of multi-stage segment processing blocks.

[0449] 133) Pipeline parallelism for additional memory: In some embodiments, computer system 800 may also increase the total amount of GPU memory, CPU memory, and secondary storage available by the number of multi-stage segments. Because the hardware cost and the throughput are both scaled by the number N of multi-stage processing segments, the cost per token or word processed is constant with N times the throughput and N times the amount of memory. Computer system 800 may speed up both the pretraining and theinference by the factor of N.

[0450] 134) Text-based indexing of word and token models: In some embodiments, computer system 800 may represent each token, multi-token or word unit with the corresponding written text. This text-based representation enables, facilitates or accelerates several processes. In some embodiments, computer system 800 may eliminate errors that might be made in disembedding (i.e., attempting to map from an embedding to the corresponding text). In some embodiments, computer system 800 may collect statistics from a body of text, such as the training corpus or body of text or from text supplied by a human user without having to run inference on the body of text. For example, in some embodiments, computer system 800 may directly access text via a concordance as discussed in technique (74). In some embodiments, computer system 800 may acquire additional text from inaction with an end user as in blocks 110 of Figure 1 and 130 and 131 of Figure 2. In some embodiments, computer system 800 may proactively obtain additional text from written reference material in block 131 of Figure 1.

[0451] 135) Text-based indexing facilitating communication between stages and between models: In some embodiments, computer system 800 may use text-based indexing to coordinate and communicate between stages in a multi-stage hybrid network that generates, classifies or recognizes speech with text output. For example, in some embodiments, computer system 800 may use text-based indexing to communicate from a first stage to a second stage that is not the immediate successor stage to the first stage. In some embodiments, computer system 800 may use text-based indexing to transfer knowledge from one model or module to another. In some embodiments, computer system 800 may use textbased indexing to accumulate and coordinate knowledge acquired from multiple written reference works.

[0452] 136) Conditional probability models of word occurrence as a function of relative position in sequence: In some embodiments, computer system 800 may directly train conditional probability models directly from the text training corpus. The following procedure will be described in terms of words. However, in some embodiments, computer system 800 may perform a similar procedure for tokens and / or multi-token units that are not necessarily words. For each of the following probability estimates, computer system 800 may make statistical estimates a small fraction of the available training data subject to the requirement that the statistical estimates meet a specified requirement of the relative size of the confidence interval. The steps of the procedure can be as follows: a) Count the frequency of occurrence of each word.b) Select word pairs for which to estimate conditional probabilities. c) For each selected pair of words, count the frequency of the occurrence of both words occurring within an interval of a specified length. d) Select word pairs for which the probability of joint occurrence exceeds the product of the individual words in the pair based on specified criteria. e) For each selected pair of words, count the fraction of the times when the first word occurs in an interval of a specified size that the second word also occurs within the interval and the fraction of times when the second word occurs that the first word also occurs. f) Use these counts to estimate the conditional probability of the second word occurring given that the first word occurs and the conditional probability that the first word occurs given that the second word occurs. g) In some embodiments, computer system 800 may perform the computation in steps (e and f) for a plurality of interval lengths. h) In some embodiments, computer system 800 may perform computations in step (e - g) separately for the first word occurring first in the sequence and for the first word occurring second in the sequence.

[0453] 137) Model conditional probabilities conditioned on the same word or unit as conditionally independent: In some embodiments, computer system 800 may obtain a word candidate list. The word candidate list may come for candidate lists from lower stages. In some embodiments, computer system 800 may obtain the word candidate list from words that are estimated most likely to occur based on word candidate lists in lower stages. In some embodiments, computer system 800 may prune from the word candidate list words that are estimated as unlikely to occur based on estimates made in lower stages. For each word Wk in the word candidate list, computer system 800 may estimate the probability Pt(Wk) of occurrence of Wk at position t as the product for all i, for a selected set of words {Wi}, of the conditional probability of Wi occurring or not occurring in the context, as the case may be, subject to the condition that word Wk occurs in the position t. Computer system 800 may then estimate the probability of word Wk being the word in position t as Pt(Wk) divided by the sum of Pt(Wj) for all j that for which Wj is a candidate for position t.

[0454] 138) Models and templates based on head words in grammar: In some embodiments, computer system 800 may generate or evaluate a whole sentence at a time and may estimate probability models of word pairs based not only on their relative positions in a word sequence but also on their relative position in the parse tree of a candidate sentence. Insome embodiments, computer system 800 may estimate the joint probability of word pairs as the head words of two adjacent clauses or phrases. In some embodiments, computer system 800 may estimate the probability of a non-head word Wnas a subordinate to head word Wh as the conditional probability of Wngiven Wh.

[0455] 139) Models and templates based on named entities: In some embodiments, computer system 800 may detect named entities, for example computer system 800 may use conditional random field (CRF) models. Detection of named entities, including in multilingual text is known to those skilled in the art of generative Al and is available on various platforms. In some embodiments, computer system 800 may treat a named entity as a single unit. In some embodiments, computer system 800 may treat a named entity as a special word and add the named entity to the vocabulary. However, in some embodiments, computer system 800 may treat named entities and single word proper names as a special case different from other words. Computer system 800 may treat a named entity or proper name as more likely to recur within the same document than in general text. In some embodiments, rather than estimate the probability of a named entity in the same document not solely or primarily from counts of that named entity but rather apply empirical Bayes, using statistics of recurrence of named entities in general as a back-off probability estimate for the specified named entity.

[0456] 140) Generate text by paraphrasing or summarizing a passage from a training sample: In some embodiments, computer system 800 may generate text based on finding in the training corpus one or more text passages like the context of the text currently being generated. In some embodiments, computer system 800 may use a concordance to find passages with one or more words that are present in the current context. In some embodiments, computer system 800 may paraphrase or summarize the passages that follow the instances of the matching words.

[0457] 141) Use relationship regularization links: In some embodiments, computer system 800 may use soft tying regularization links among data items in a cluster to smooth the parameter estimation and or to keep the cluster well-defined. In some embodiments, computer system 800 may use counter tying links to increase diversity among a plurality of similar subsystems, such as the feed forward layers of two or more attention blocks.

[0458] 142) During training, incrementally increase the number of levels or stages: In some embodiments, computer system 800 may add an additional attention block level or an additional hybrid stage between two existing stages. In some embodiments, computer system 800 may regularize the initial training of the new stage by soft tying some nodes in the newstage with corresponding nodes in the preceding stage of the base network. Computer system 800 may also soft tie some of the nodes in the new stage with the corresponding nodes in the following stage of the base network. In some embodiments, computer system 800 may counter tie some nodes in the new stage to corresponding nodes in either the preceding or following stages in the newly expanded network to regularize the three stages in the new network to learn more quickly to be different from each other.

[0459] 143) Incrementally grow an individual level or stage: In some embodiments, computer system 800 may duplicate an attention head or other subsystem to increase the number of nodes and parameters to build a larger network with greater capacity to model the training data.

[0460] 144) Use ranked candidate lists from lower stages or levels to prune candidate lists or speculative decoding in higher stages or levels: In some embodiments, computer system 800, in a non-final stage, may prune the candidate list for a given position in the sequence to no more than a specified maximum number of entries. In some embodiments, computer system 800 may, in a non-final stage, prune the candidate list by pruning all entries with a worse probability estimate less than a specified value. In some embodiments, computer system 800 may restrict the candidate list at a higher level to items that have not been pruned at a lower stage plus a specified limited number of new candidates.

[0461] 145) Use candidate lists from lower stages or levels to help select the most effective experts in higher stages or levels based on mixture-of-experts (MOE) models: In some embodiments, computer system 800 may train a separate system M within the HGDTCS to estimate the effectiveness of a specified expert in a mixture-of-experts model based on measurement or observations that may be made at a lower stage of a multi-stage system. In some embodiments, computer system 800 may use the analysis of system M to select a subset of the experts for a higher stage in the multi-stage system.

[0462] 146) Manipulate positional embeddings giving the positional embedding at a position in the sequence later than the position current being generated: In some embodiments, during inference, computer system 800 may manipulate the positional embeddings. For example, in some embodiments, computer system 800 may manipulate the positional embeddings to generate a token, multi-token or word at a position further in the sequence than the current position. In some embodiments, computer system 800 may then estimate whether a specific token, multi-token or word is more likely to occur specifically at the current position or merely to occur in the near future of the sequence generically.

[0463] 147) Combining similar residuals: In some embodiments, if the residuals of asubset of candidates having the same attention mask have value more similar than a specific criterion, computer system 800 may combine the subset of residuals, saving computation in evaluating numerous possible continuations.

[0464] 148) Attempt to prefill depth wise with more efficient approximations of the residuals: In some embodiments, computer system 800 may compute the attention with no residual or with a first order approximation of the residual. In some cases, computer system 800 may then determine a shorter list of elements of the KV cache that have non-trivial contributions based on specified criteria. In some embodiments, computer system 800 may then proceed to the next level without needing to compute the results as a sophisticated prefetch.

[0465] 149) Use the logits of the LLM as implicit n-gram models of the local text: In some embodiments, computer system 800 may perform tree-like speculative decoding. In some embodiments, computer system 800 may then compute n-grams of the token, multitokens, or words for paths through the tree. In some embodiments, computer system 800 may add the probabilities estimated from the n-grams to the softmax values estimated from the attention block.

[0466] 150) Train draft model to maximize the expected length of the completion: In some embodiments, computer system 800 may perform speculative decoding. That is, for one or more levels, computer system 800 may use a draft model that requires less computation than a full model to generate one or more likely future sequences of varying length. In some embodiments, computer system 800 may then use the full model to verify such a future sequence, terminating the verification at the token preceding the first item in the sequence for which the full model disagrees with the draft model. Speculative decoding is known to those skilled in the art of generating text using a large language model (LLM). In some embodiments, computer system 800 may perform a computation to help maximize the expected length of the future sequence that is verified. For example, computer system 800 may estimate P(A), the probability of item A being the next item accepted by the full model, and E[A] = the expected number of additional tokens that the full model will accept given the A is the next item. In some embodiments, computer system 800 may explore a tree of possible continuations, limiting the set of possible branches at each branch point to the top K logits for a specified value of K. In some embodiments, computer system 800 may explore the tree of possible continuations for N steps by selecting at each active branch point a specified number C of continuations for which P(i4)(l + 7T

[0071] ) is maximized. In some embodiments, computer system 800 may limit the number C of continuations to C = 1. Insome embodiments, computer system 800 may terminate at less than N steps if the estimate of E[a] is less than a specified value.

[0467] 151) Hierarchy of draft models: In certain embodiments, draft models may themselves leverage secondary draft models to accelerate inference, forming a hierarchical structure. This hierarchy may consist of an arbitrary number of models, each progressively faster and typically requiring less computational power further down the hierarchy. Lower- level models generally permit greater parallel exploration of potential sequences, whereas ascending the hierarchy typically involves increased computation and latency but results in predictions that more closely match those of the verifier model.

[0468] In some embodiments, lower-level models within the hierarchy may employ bidirectional mechanisms to refine candidate sequence likelihoods. These models may also utilize distinct architectures, including but not limited to n-gram or state-space models. Additionally, certain embodiments allow computations performed by lower-level models — such as portions of the KV-cache or residual streams — to be reused by higher-level models, thus accelerating inference at higher levels of the hierarchy.

[0469] 152) Train n-gram models on specific tasks: In some embodiments, computer system 800 may train n-gram models on specific tasks such as A) Training using recent logits to approximate n-gram probabilities, B) Training parameters to maximize the expected length of completions, C) Training n-grams to match statistics of the current document or D) some combination of tasks (A), (B) and (C).

[0470] 153) Checking for an n-gram match to a text corpus: In some embodiments, computer system 800 may check the training data set or other text corpus for a match of a generated n-gram for a specified value of n, say n=8. In some embodiments, computer system 800 may attempt decoding an exact copy of the later context from text corpus for up to a specified number of items.

[0471] 154) Measuring the statistical independence of different speculation methods: In some embodiments, computer system 800 may measure the statistical independence of different speculation methods to optimize the computation / information gain + expected generation speed tradeoff.

[0472] 155) Determining sets of tokens that do not have significant causal impact: In some embodiments, computer system 800 may use information on speculative decoding to determine A) Sets of tokens that don’t have a significant causal impact on the generation of the next few tokens based on measured attention weights relative to each other - such that for speculating further we can group these together and do more efficient computation withoutfurther disambiguating which one is in fact chosen until verification.

[0473] 156) Tuning models to skip embedding / disembedding to speed up inference: In some embodiments, computer system 800 may tune models to skip embedding and disembedding.

[0474] 157) Train models to have sets of discrete values at the residuals: In some embodiments, computer system 800 may train models to have sets of discrete values at the residuals to facilitate further optimizations.

[0475] 158) Train up-projections from KV-caches of draft models: In some embodiments, computer system 800 may train up-projections from kv-caches of draft models to the KV-caches of the full model as well as down projections, save-memory, potentially give smaller models better embeddings, allow further optimizations.

[0476] 159) Quantify tokens and multi -token units based on information from speculative models which tokens are “Hard”: In some embodiments, based on information from speculative models, computer system 800 may quantify which tokens are “Hard” using the MLP and attention from the large model without running it on the whole KV cache or running the whole depth in sequence, do up-projections of the KV-cache of the small model and using the full KV-cache for the finalized portion of the sequence, use up-projections for higher layers of the residuals, calculate the results for all layers in parallel, recompute for pairs where the results from the previous levels do not match the results from the up- projection well enough.

[0477] 160) Approximate inference: In some embodiments, computer system 800 may use approximate inference. For example, in some embodiments, computer system 800 may train the verification model to accept a token that is not an exact match based on an admissibility criteria in which the verification model has been trained to accept substitutions that are judged to be harmless. In some embodiments, computer system 800 may train the draft model to present multiple options. In some embodiments, computer system 800 may train the verification model to adjust for any bias in the scores caused by some tokens being selected by multiple choices and other tokens being selected from a single choice.

[0478] 161) Bidirectional Modeling for Candidate Sequence Likelihood Estimation: In certain embodiments, computer system 800 utilizes bidirectional or backward models to assess the likelihood of proposed candidate sequences. By analyzing sequences in reverse order — from outcomes back toward their initial states — these backward models enable the early identification and rejection of candidate sequences that may ultimately prove invalid or lead to undesirable outcomes (“Whoops moments”). This capability enhances the efficiencyand effectiveness of reasoning processes, allowing computational resources to be more effectively allocated to exploring alternative and potentially more viable sequence completions. Additionally bi-directional models allow estimation of the likelihood of the sequence as a whole as opposed to the local criteria of the likelihood of the last token given the preceding sequence.

[0479] 162) Aligning objectives of a verifier model with draft models: In some embodiments, computer system 800 may tune a verifier model to retain performance on its objectives while also aligning aspects of its generation which do not affect those objectives to be more aligned with draft models. By these measures, computer system 800 may train elements that directly or indirectly receive incoming connections to be robust against changes in the incoming values from lower elements that have been rated as unreliable by their associated Socratic wisdom elements. Propagating a Socratic wisdom determination of elements rated as unreliable helps prevent the propagation of the changes in element activations caused by an adversarial attack. In some embodiments, computer system 800 may use the blocking of feed forward activations that are rated unreliable to emulate a training method called “dropout,” which is known to those skilled in the art of training neural networks as a technique for making the system being trained more robust. In dropout, during training, computer system 800 may randomly set a portion of the activations to zero. In some embodiments, computer system 800 may emulate dropout. In some embodiments, computer system 800, under control of the HGDTCS may exert finer control of the based on the additional information obtained from the Socratic wisdom elements.

[0480] 163) Combining systems with complementary capabilities, optionally with redundancy, regularization, and diversity: In some embodiments, for techniques (162-189), computer system 800 may use a combination of multiple systems with complementary capabilities, as illustrated in Figure 3 J. The system designer and / or computer system 800, acting as the HGDTCS may select a base system 382 and a complementary system 383 based on prior experience and / or current experimentation and validation testing. In some embodiments, computer system 800 may implement either system 382 or system 383 as a set of cooperative subsystems. In some embodiments, for either system 383 or system 383, computer system 800 may use soft tying and / or other regularization to make estimates of trained parameters more accurate and more robust. In some embodiments, for either system 383 or system 383, computer system 800 may use counter tying to make the set of cooperative subsystems more diverse. In preferred embodiments, computer system 800 may train a system 384 specifically for the task of combining results of a system of the type ofbase system 382 with a system of the type of complementary system 383.

[0481] 164) Determining strengths and weaknesses of specified subsystems: In some embodiments, computer system 800 may determine strengths and weaknesses of a system based on prior use and known behavior of the type of system. For example, a large language model generally requires a large quantity of training data, up to a trillion tokens or more. On the other hand, a parametric probability model may have adequate performance from a hundred examples or less.

[0482] In some embodiments, computer system 800 may determine strengths and weaknesses of a system by testing the system on new data that has been set aside from the training data.

[0483] 165) Train a base system primarily with gradient descent and a complementary system primarily without gradient descent: In some embodiments, computer system 800 may use stochastic gradient descent to train a neural network as a base system and may use a different machine learning method to train the complementary system. For example, in some embodiments, computer system 800 may use the expectation-maximization (EM) algorithm to train a system based on probability models.

[0484] 166) Train two complementary systems on different data domains: In some embodiments, computer system 800 may train the base system 382 on one data domain and train the complementary subsystem 382 on a separate data domain. In some embodiments, computer system 800 may use two data domains that are disjoint subsets of a larger data set. In some embodiments, computer system 800 may use training data for a specific task for training data for base system 382 and may use training data for a different task for training data for complementary system 383.

[0485] 167) Use two subsystems that naturally use qualitatively different types of data: In some embodiments, computer system 800 may use two subsystems trained on different tasks that naturally use different types of data. For example, in a speech recognition system, computer system 800 may use a neural network transformer-based large language model trained on text as one subsystem and an acoustic recognition system trained on audio as the other subsystem.

[0486] 168) Combining training on a base domain with training on an independent verification domain: In some embodiments, computer system 800 may train base system 382 as a recognition system on a base domain. In some embodiments, computer system 800 may vary the training algorithm and the training data selection to optimize the performance of the complementary system 383 on the task of verification of the output produced by the basesystem 382.

[0487] 169) Combining two separately trained complementary subsystems with an independently trained combined system: In some embodiments, computer system 800 may add a third system, a combining system, to two complementary systems trained by one of the techniques (164 - 167). In some embodiments, computer system 800 may train the third system using the union of the training data available to the two subsystems 382 and 383. In some embodiments, computer system 800 may use this third system as a diagnostic system to detect problems in the subsystems 382 and 383. In some embodiments, computer system 800 may make changes in the selection of training data for the two subsystems based on this diagnosis.

[0488] In some embodiments, computer system 800 may include the output from the two subsystems as input to the third system and train the combined system as a combining system with a single output replacing the outputs of the two subsystems 382 and 383.

[0489] 170) Train a base system to suggest a candidate and train a complementary system to verify or reject the suggested candidate: In some embodiments, computer system 800 may train base system 382 to suggest a candidate output and train complementary system 383 to verify or reject the candidate suggested by base system 382.

[0490] 171) Train a base system to generate a list of candidates and a complementary system to verify or reject each candidate on the list of candidates: In some embodiments, computer system 800 may train base system 382 to output a list of candidate answers. In some embodiments, computer system 800 may train complementary system 383 to verify or reject each candidate in the list. In some embodiments, computer system 800 may verify a plurality of candidates. In some embodiments, computer system 800 may consider only candidates on the candidate list supplied by base system 382. In some embodiments, computer system 800 may reduce the amount of computation or memory by not comparing the items on the candidate list with any other possible output values. In some cases, computer system 800 may reject all candidates provided by base system 382.

[0491] 172) Train a base system to recognize units associated with nodes on a graph and train a complementary system to align a sequence of units with a path of nodes in the graph: In some embodiments, rather than generating or recognizing a sequence, computer system 800 may train system 382 to recognize data units associated with nodes on a directed or undirected graph. In some embodiments, computer system 800 may train a separate system to align a sequence of units with a path through the graph. In some embodiments, computer system 800 may use the Dijkstra best path algorithm or the Bellman-Ford algorithm tocompute the best alignment. In some embodiments, computer system 800 may use a hidden Markov process model to compute the alignment.

[0492] 173) Train a base system to optimize the frequency with which the correct response is in a pruned list of candidates and train a complementary system to accept or reject each answer in the pruned list of candidates. In some embodiments, rather than training the base system 382 with the objective of having the correct answer as the output, computer system 800 may train the base system 382 on the task of having the correct answer appear in a list of candidates. In some embodiments, the list of candidates may be restricted to having no more than a specified number of candidates. In some embodiments, the list of candidates may be restricted to include no candidate with a score less than a specified threshold value relative to the best score. In some embodiments, the specified threshold value may be set as a specified amount worse than the value of the best scoring candidate. In some embodiments, computer system 800 may train complementary system 383 to accept or reject the candidates in the pruned candidate list as in technique 170 or 171.

[0493] In some embodiments, computer system 800 may train two subsystems 382 and 383 and may use either subsystem 382 or subsystem 383 to generate a candidate list and use the other subsystem to accept or reject candidates. For example, in technique 167, computer system 800 may use either the acoustic recognition subsystem or the language model subsystem to generate the candidate list and use the other subsystem as the verification subsystem.

[0494] 174) A speech recognition system comprising a system to recognize sub-word units: In some embodiments, computer system 800 may train a speech audio recognition system to recognize sequences of sub-word units. In various embodiments, computer system800 may recognize spectral frames, allophones, phonemes, diphones, demi-syllables, syllables, or other detectable acoustic segments as sub-word units.

[0495] In some embodiments, computer system 800 may use sub-word units to reduce the amount of computation and memory relative to a recognition system that directly recognizes words. In some embodiments, computer system 800 may use the recognition of a sequence of sub-word units as a lower stage in a multi-stage word recognition system. In some embodiments, computer system 800 may use the recognition of a sequence of subunits directly as a verification subsystem 383, as described in techniques 170 and 171.

[0496] 175) Align a sequence of sub-word units to accept or reject word candidates produced by a language model: In some embodiments, computer system 800 may use a language model as a base system 382. In some embodiments, the language model 382 may betrained on text and not on audio. In some embodiments, base system 382 may be a complete speech recognition system including both audio recognition and a language model. In some embodiments, computer system 800 may use a subword recognition system such as described in techniques 170, 171, and 173 as a verification system 383.

[0497] In some embodiments, computer system 800 may train the sub-word recognition system to detect errors that are not sensible when viewed purely as an audio recognition task. A language model trained from text or a speech recognition system including both audio recognition and a language model may choose a word that fits the language model very well but that does not sound like the actual word spoken. Such an error may be called an “error that is non-sensible from the perspective of the audio” or a similar phrase that emphasizes that the audio analysis system is independently verifying the output and or one or more inner nodes of a language model subsystem. In some embodiments, computer system 800 may reject a word candidate for which the match with the aligned sequence of sub-word units is worse than specified criteria.

[0498] 176) Train a verification system 383 to detect non-sensible errors: In some embodiments, computer system 800 may train a conventional recognition system as base recognition system 382. In some embodiments, computer system 800 may train a complementary system 383 to detect non-sensible errors that are non-sensible from the perspective of system 383. In some embodiments, during training, computer system 800 may modify the data and / or the operation of base system 382 on selected items of training data to test the sensitivity of the system to such disturbances. In some embodiments, computer system 800 may supply information about the effects of such disturbances to system 383. In some embodiments, computer system 800 may keep a record of data items on which the disturbance caused a non-sensible error.

[0499] In some embodiments, computer system 800 may train complementary system 383 to discriminate data items on which there is a non-sensible error from data items on which there is no error or an ordinary error. In some embodiments, computer system 800 may collect examples of non-sensible errors by tests on validation data set aside from the training data. In some embodiments, computer system 800 may collect examples of non-sensible errors from user feedback during deployment or simulated end use. In some embodiments, computer system 800 may train complementary system 383 based at least in part on detecting and correcting examples of such non-sensible errors. After such training, computer system 800 may designate such errors as “non-sensible from the perspective of system 383.”

[0500] 177) Coordinated node-specific implicit local objective and explicit node-specificartificial objective: In some embodiments, computer system 800 may define both an implicit local objective for a node and a node-specific explicit artificial objective for the node. For example, in some embodiments, computer system 800 may define an explicit artificial local objective as a defense against adversarial attacks.

[0501] 178) Explicit local objectives for decisiveness: In some embodiments, computer system 800 may, for a specified interval [nl, n2] of the domain of the activation function of a node N, multiply the back propagated derivative by a factor D > 1.0, where D may be the value of a hyperparameter specified by the HGDTCS. In some embodiments, computer system 800 may count the relative fraction of data items with activation input values in the interval [nl, n2] and use the count as an estimate of the degree of decisiveness of the node N. In some embodiments, computer system 800 may adjust the factor D based on the degree of decisiveness of the node N or based on the average degree of decisiveness of a set of nodes containing node N. In some embodiments, computer system 800 may increase the factor D for a node N until node N satisfies specified criteria for being decisive.

[0502] 179) Testing for violations of sensibility based on extreme activation values: In some embodiments, computer system 800 may test for violations of sensibility using either cross-verification or self-verification. In some embodiments, computer system 800 may test sensibility for inner nodes of a network as well as testing the output of the network.

[0503] In some embodiments, during training, computer system 800 may process a data item d multiple times to check consistency or to judge the progress of the training process. If the system makes an error on data item d in one instance and not in other instances, and if the activation of the node is at an extreme value compared to its activation on other data items, then, in some embodiments, computer system 800 may flag the node as making a non- sensible error on the data item d. In some embodiments, computer system 800 may make such a judgment for an inner node N when the system makes an error on the data and / or when node N makes an error on its implicit local objective.

[0504] In some embodiments of some tasks, computer system 800 may apply this technique without processing any data item d multiple times but may instead identify multiple data items to be similar situations based on specified criteria. For example in text generation or in a classification or recognition task using a language model, computer system 800 may specify all instances in which the word to be generated or recognized is the same word as similar for the purpose of checking consistency.

[0505] 180) Localizing cause of violation of sensibility: In some embodiments, if a network makes an error that is judged not to be sensible, then computer system 800 maycheck the output nodes to identify any output nodes that falsely have extreme high activation values or that falsely have extreme low activation values. In some embodiments, computer system 800 may flag such nodes as making non-sensible errors. In some embodiments, computer system 800 may apply the same test to inner nodes.

[0506] 181) Training a node to be sensible using a hyperparameter controlled local objective: In some embodiments, computer system 800 may, for a specified interval [nl, n2] of the domain of the activation function of a node N, multiply the back propagated derivative by a factor D. In some embodiments, once a specified training criteria has been met, computer system 800 may begin gradually decreasing the factor D for specified nodes. In some embodiments, computer system 800 may decrease the value of D to zero for selected nodes.

[0507] 182) Sensibility as a prerequisite to safety and claims of human level intelligence:In benchmarking artificial intelligence systems a system is claimed to have achieved human level intelligence on a given task if the Al makes fewer errors on average on the benchmark than the average human adult. Such a measure may be appropriate if the cost of all errors are comparable. However, in some circumstances, there may be no way to estimate the cost of a non-sensible error. Therefore, regardless of the average rate of normal errors, computer system 800 may flag an Al system as not achieving human level intelligence and as not being safe if the Al system makes more non-sensible errors than the average human adult. In situations in which safety has even greater importance, computer system 800 may impose an even tighter constraint on the number of non-sensible errors. In some embodiments, computer system 800 may detect errors that are non-sensible from the perspective of a complementary system, as described in association with techniques 175 and 176 and Figures 3J, 3K, 3L, 3M, 3N, and 30. In some embodiments, computer system 800 may detect non-sensible errors using techniques 179 and 180. In some embodiments, computer system 800 may learn of non-sensible errors from the HGDTCS or from end-users.

[0508] 183) Multi-phase training of decisiveness and sensibility: In some embodiments, computer system 800 may first train specified nodes to be decisive. In some embodiments, computer system 800 may judge the degree of decisiveness of a node N by the relative fraction F of the data for which the activation of node N is in the interval [nl,n2] mentioned in techniques 178 and 181. In some embodiments, computer system 800 may continue training nodes to be more decisive until specified criteria for the value of F are satisfied. When specified criteria are met, computer system 800 may change the training phase and switch selected nodes from training to be decisive to training to be sensible. In someembodiments, computer system 800 may make the switch by gradually lowering the value of multiplier D mentioned in technique 181. In some embodiments, computer system 800 may make the phase switch decision separately for each selected node. In some embodiments, computer system 800 may make the phase switch decision simultaneously for specified blocks of nodes or for the whole network.

[0509] 184) Socratic wisdom: Know what it is that you do not yet know: In some embodiments, computer system 800 may assess the knowledge of each node based on its average error rate and its non-sensible error rate. In some embodiments, computer system 800 may assess the performance of a node based on its implicit local objective. In some embodiments, computer system 800 may train the HGDTCS or a separate Al coach trained to evaluate a node. In some embodiments, computer system 800 may take corrective action based on a Socratic assessment. For example, in some embodiments, computer system 800 may add additional elements to the system and do incremental training.

[0510] 185) In a speech recognition system, train two complementary subsystems with and without audio input: In some embodiments, for speech recognition, computer system 800 may train and use two systems: 1) A language model system trained primarily on text (382 of Figure 3 J); and 2) A second system trained on training data including speech (383 of Figure 3 J). In some embodiments, computer system 800 may use the second system as a draft system with speculative decoding and use the first system as the main system to verify the speculative decoding. In some embodiments, whether or not using speculative decoding from a language model perspective (382), computer system 800 may have the second system (383) do preliminary speech recognition at least one word ahead of the main system. In some embodiments, computer system 800 may add or remove words or tokens from the candidate lists (of 382) based on the preliminary speech recognition done by the second system.

[0511] 186) Integrating n-gram modeling to make speculative decoding of text generation more accurate: In some embodiments, computer system 800 may integrate n-gram modeling to make text generation more accurate. In some embodiments, computer system 800 may approximate n-gram probabilities by an indirect computation, such as technique 152 and 153, rather than store probabilities for a full set of n-grams.

[0512] 187) Using look-back conditional probability models to represent n-grams efficiently: In some embodiments, for selected words on a candidate list and for selected words being tested to be put on the candidate list, computer system 800 may use look-back probability models that look back up to n-1 words to approximate n-gram probabilities.

[0513] 188) Prune candidate lists in look-back model n-gram computation: In someembodiments, computer system 800 may remove some words or tokens from a candidate list based on the look-back model probability estimates.

[0514] 189) Use acoustic scores to modify candidate lists: In some embodiments, in a speech recognition system, computer system 800 may modify the n-gram scores and the candidate list based on a speech acoustic model for the next interval of speech. In some embodiments, computer system 800 may implement word spotting over a longer interval of speech to detect a word that may be several words beyond the current word in the sequence. Computer system 800 may either add or delete items from a candidate list based on a combination of attention block prediction scores, n-gram scores, and acoustic scores.

[0515] 190) Multiple cooperating systems designed and trained to have complementary capabilities: As discussed in association with Figure 3J and elaborated further in association with Figures 3K, 3L, 3M, 3N and 30 computer system 800 may design and train systems with a plurality of subsystems with complementary properties and capabilities. In some embodiments, computer system 800 may design subsystems that may also be used standalone as a complete system. In some embodiments, computer system 800 may train these subsystems to have different strengths and weaknesses such that the combined system has greater capability and better performance than any of the subsystems alone. However, two systems that are designed and trained to have different strengths and weaknesses may lack a shared set of fundamental units. For example, in a speech recognition or synthesis system, the units in a sequence may be phonemes, syllables, or words. On the other hand, the units in a transformer-based large language model are generally tokens. A token is a sequence of letters that is generally less than a full word that does not generally correspond to a syllable or to a phoneme. In some embodiments, computer system 800 may compute a tree or other type of directed acyclic graph (DAG), to represent the set of the most likely continuations of current sequence in either recognition or generation. The units used by computer system 800 in the continuation tree or DAG of a speech recognizer or generator / synthesizer may be phonemes and / or syllables. In the tree or graph continuation of a transformer-based text generator, computer system 800 may use tokens. Computer system 800 may separately explore the best paths in continuation tree or DAG, exploring multiple continuation paths in parallel and pruning poorer scoring paths based on specified criteria for the audio speech recognition system and the text-based language model system . In some embodiments, computer system 800 may keep track of the back path of each active path and determine when the path completes a word. In some embodiments, computer system 800 may compute the continuations of two complementary systems in parallel and communicate the wordsequences of the non-pruned paths. Thus, in some embodiments, computer system 800 may use words and word sequences to achieve communication between complementary systems rather than tokens, phonemes, or syllables.

[0516] In some embodiments, computer system 800 may implement a one- or multi-way communication flow between two complementary subsystems — such as a speech recognition subsystem and a token-based language model subsystem — by coordinating their respective processing pipelines. For example, in a one-way communication scenario, computer system 800 may first cause the speech recognition subsystem to process audio input and generate a directed acyclic graph (DAG) or tree representing likely continuations of speech-based units such as phonemes, syllables, or words. Computer system 800 may then extract word-level hypotheses from non-pruned paths in the DAG, identifying sequences that represent coherent or high-probability interpretations of the audio input. These word sequences may then be transmitted by computer system 800 to the language model subsystem. Upon receipt, computer system 800 may further configure the language model subsystem to adjust its token-level generation behavior based on the received word sequences, such as by constraining token selection, modifying token probabilities, or pruning inconsistent branches in its decoding tree. By managing both subsystems and facilitating the flow of intermediate results, computer system 800 enables complementary architectures to work together without requiring the speech recognition system to operate on token-based units or the language model to process raw audio. This coordination improves the contextual accuracy of the generated output by leveraging the specialized strengths of each subsystem.

[0517] In a two-way communication mechanism between complementary subsystems, such as a speech recognition subsystem and a token-based language model subsystem, to enable dynamic exchange of intermediate results, computer system 800 may coordinate both subsystems to operate in parallel, each producing a continuation structure — such as a directed acyclic graph (DAG) or tree — based on their respective units (e.g., phonemes or words in the speech recognizer, tokens in the language model). Computer system 800 may monitor these structures in real time, identifying word-level alignments between candidate paths. Upon identifying a correspondence, computer system 800 may transmit partial hypotheses or high- confidence segments from the speech recognizer to the language model to constrain or guide token selection. Conversely, the language model may suggest semantically or contextually plausible continuations based on broader language understanding, which computer system 800 may feed back to the speech recognizer to refine or reprioritize its candidate paths. This iterative feedback loop allows the two subsystems to converge on mutually consistentoutputs, leveraging the domain-specific strengths of each subsystem while maintaining overall coherence in the decoded sequence. The two-way exchange of partial results increases robustness, especially in noisy or ambiguous inputs, by allowing each subsystem to influence and validate the other's hypotheses.

[0518] 191) Cross-verification of sensibility. All humans and all Al systems make errors.On a task to which it has been trained, a state-of-the-art Al system may make fewer errors than the average intelligent human. However, humans and Al systems may also make errors that are not sensible. Generally, when a human makes an error that is not sensible, the human will be able to understand that the error was not sensible upon further reflection or if the error is pointed out by another person. Some Al systems may have a greater tendency to make non- sensible errors than the average human. For example, an Al text generation may generate text the is not sensible in the context of a prompt or previously generated text. The Al text generation system may just make up text that is false or that doesn’t fit. The Al system may even make false text that may be very convincing. The phenomenon is well-known and is often referred to a “hallucination.” As the name implies, an Al text generation system that hallucinates is generally not aware that it has done so. However, a nonsensible error that is not detected may be more dangerous than a normal error. For example, a wrong but convincing medical diagnosis may be more dangerous than an error in which the Al is unsure. As discussed in association with Figures 3 J, 3K, 3L, 3M, 3N and 30, in some embodiments, computer system 800 may design and train complementary subsystems that may be able to detect and reject errors made by a base system that are non-sensible from the perspective of the complementary system.

[0519] 192) Cross-verification to lower average error rate: In some embodiments, by adjusting some error detection hyperparameters, computer system 800 may also be able to use complementary subsystems to reduce the number of ordinary errors.

[0520] 193) Dynamically configured distributed computation with cooperative complementary modules and systems: In some tasks, such as detecting errors that are nonsensible from the perspective of one or more complementary system in close to real time, it may be important to finish a computation as quickly as possible. As discussed in association with Figure 9, a distributed set of computers may be used in association with dynamic hybrid networks. Such a distributed set of computers may be used in pretraining including for targeted incremental growth. In some embodiments, with dynamic hybrid networks, a configuration of distributed computers may be able to modify the architecture and retrain in near real time in response to an end-user pointing out a serious error or a non-sensible error.In some embodiments, a set of distributed computers may detect a non-sensible error in near real time and take appropriate action.

[0521] 194) Complementary systems trained specifically to do final verification: In some embodiments, computer system 800 may train one or more subsystems specialized for the task of final verification of other subsystems, such as subsystems 394MV in Figure 30. In the illustrative embodiment of Figure 30, computer system 800 may design and train subsystems 391MV, 392MV and 393MV with tasks and training such as used for the comparable subsystems in Figure 3K, 3L, 3M and 3N. However, in some embodiments of the subsystems 394-MV illustrated in Figure 30, computer system 800 may train one or more subsystems specifically on the task of final verification.

[0522] For example, in some embodiments, computer system 800 may train one or more subsystems 30 by fine-tuning on errors detected in training data that computer system 800 has set aside from the training data used to train subsystems 392MV and 393MV. In some embodiments, computer system 800 may adjust hyperparameters in system 391MV to train a subsystem 394MV specifically to detect non-sensible errors and adjust the hyperparameters for another subsystem 394MV to detect all errors. In some embodiments, computer system 800 may train other subsystems in 394MV to detect specific types of errors. In some embodiments, computer system 800 may train multiple systems as an ensemble of cooperating systems on the same training objective. In some embodiments of subsystems 394MV, computer system 800 may train a set of complementary systems as discussed in association with Figure 3 J.

[0523] In some embodiments, in association with block 394-MV, computer system 800, may train a subsystem without some of the limitations applied in training the verification ability of subsystems associated with Figures 3K, 3L, 3M and 3N. As an example of a limitation that might be removed compared to the recognition systems associated with the other figures such as Figure 3K, in some embodiments, in block 392 of Figure 3K, computer system 800 may train a speech recognition system 392 with a weak language model or no language model so that speech recognition system 392 may be better able to detect a result from text generation system 393 specifically as not sensible from the perspective of the audio without audio recognition of system 392 being over ridden by a more powerful language model such as the language model in system 393. In some embodiments, computer system 800, in recognition system 393MV, may remove such constraints. In some embodiments, computer system 800, in system 393MV, may remove similar constraints compared to subsystems 392S and 392G.I l l

[0524] In some embodiments, computer system 800 may train a subsystem of 394MV to detect one or more distinct types of close calls: 1) a correct detection in which the correct answer has the highest output value but one or more other answers may have scores nearly as good, or 2) an error in which one or more wrong answers have scores slightly better than the correct answer. In some embodiments, computer system 800 may perform additional training or may apply targeted incremental growth to train the system to make more reliable correct choices on such data.

[0525] In some embodiments, in contrast to the system illustrated in Figure 3N, computer system 800 may not only enable sequence generation system 393MV to observe the current token or word being recognized by system 392MV, but may delay the sequence position of final verification system 394MV such that verification system 394MV may use output values from systems 392MV and 393MV that are in future positions in the sequence currently be verified by verification system 394MV. In some embodiments, computer system 800 may perform one or more rounds of partial or full forward-backward recognition and generation with two or more subsystems that are complementary in the sense of 382 and 383 of Figure 3J.

[0526] In some embodiments, computer system 800 may use multiple final verification systems 394MV, tuned to detect different types of errors as controlled by hyperparameters and / or training data selection by control system 391MV. For example, a system in verification system 394 MV may be tuned just to detect nonsensible errors or may be tuned to detect all errors. In some embodiments, computer system 800 may tune verification system 393MV to detect errors on specially selected data.

[0527] 195) Beam of sequences generated by multiple draft models: In some embodiments, computer system 800 may adjust the error detection threshold in systems such as illustrated in Figures 3K, 3L, 3M and 3N to detect ordinary errors and suspected errors rather than only non-sensible errors. In some embodiments, with a distributed set of computers such as shown in Figure 9, computer system 800 may implement a plurality of draft models. In some embodiments, computer system 800 may represent the generated sequences from the plurality of draft models as a beam of sequences. In some embodiments, in a sequence recognition application such as speech recognition, computer system 800 may align each unit in each sequence with an interval in the audio stream with an estimated beginning time and an estimated ending time. In some embodiments, computer system 800 may represent a sequence of units with associated time intervals as a graph with a unit in the sequence associated with an arc in the graph and an estimated time for each node. In someembodiments, computer system 800 may represent the combination of the sequences as a directed acyclic graph.

[0528] In some implementations, the verification subsystem may apply domain-specific constraints to generated outputs based on external knowledge resources, such as ontologies, taxonomies, or structured glossaries. These constraints may encode structural, logical, or semantic expectations for valid outputs and are used to filter or re-rank generated content based on its adherence to the encoded rules. Such constraints are particularly useful in regulated domains such as legal, medical, or financial language generation.

[0529] In some embodiments, computer system 800 may use one or more master or verification systems to choose a beam of paths through the directed acyclic graph. In some embodiments, computer system 800 may use one or more sequence recognition systems to detect units in the directed acyclic graph that a sequence recognition system detects as a possible non-sensible error or as an ordinary error with a match score in the sequence recognition system that is worse than a specified threshold. In some embodiments, computer system 800 may prune from the directed acyclic graph arcs that correspond to units that are worse from the perspective of the sequence recognition models than such a specified threshold. In some embodiments, computer system 800 may then perform a best-path search to find the best sequence of units based on the combined scores from the master models and the sequence recognition models. A best path search may be implemented, for example, by Richard Bellman’s minimum cost path algorithm using dynamic programming.

[0530] In some embodiments, computer system 800 may reduce the amount of computation required for one or more draft and master systems by pruning the candidate lists for the unit currently being recognized and for units within a specified look ahead interval.

[0531] Figure 3 A is a flow chart of an illustrative embodiment of the process of informal semi -automated knowledge engineering. In informal semi-automated knowledge engineering, computer system 800 may train an Al system to represent and utilize a body of knowledge with some human guidance but substantially less human labor than required for humans to produce a representation of the knowledge in a form that may be utilized by computers.

[0532] In block 371, a system designer, developer, end-user or the HGDTCS may select one or more writing styles. Examples of differences in style include non-fiction versus fiction, different genres of fiction, formal versus informal, technical versus non-technical, different domains of technical articles, short document versus long, and succinct versus wordy. Within technical writing a textbook may need a different style than a technical article. The style of an opinion piece such as an editorial may be different from the style of a factualreport. A review article may need a different style than a research article. The selected style may need to conform to the requirements of the intended publisher. The style selection may depend on the intended audience of readers. In some embodiments, computer system 800 may provide samples of variations of the selected writing styles for the same prompt and context. In some embodiments, a human specifier or the HGDTCS may request a plurality of samples of variations of each writing style. The human specifier may express a preference for specific variation of a specific writing style.

[0533] In some embodiments, computer system 800 may then set the style to the specific style and variation to be used in deployment.

[0534] In some embodiments, the HGDTCS may implement a multi-stage hybrid network with multiple chains collectively supporting a plurality of styles or variations within a style. In some embodiments, during deployment, computer system 800 may offer an end user a choice of two or more samples of generated text with different styles or variations of a style. In some embodiments, computer system 800 may adjust the selection or the style or variation of the style based on the preferences of the user.

[0535] In block 372, in some embodiments, computer system 800 may enable the user to provide an outline of the document to be produced. In some embodiments, computer system 800 may enable the user to provide some other type of condensed description of the intended contents, such as the list of chapters in a book, a synopsis of a plot, or a list of key points. In some embodiments, computer system 800 may use the outline or other condensed description as a set of anchor points in the generation process. In some embodiments, computer system 800 may generate text both forward and backward from each anchor point. In some embodiments, during the generation from an anchor point computer system 800 may include the other anchor points and any text that has been generated from them as additional context.

[0536] In some embodiments, when forward and backward text from two adjacent anchor points begin to overlap, computer system 800 may construct in an interstage a low perplexity directed acyclic graph (DAG) to represent the combined scores of the forward and backward candidate lists.

[0537] In block 373, optionally, a human developer, a human end user, or computer system 800 may specify one or more use cases and / or topic domains. In some embodiments, computer system 800 may obtain training data for each specified use case or topic domain. Computer system 800 may do pretraining and / or fine tuning for each use case or topic domain. In some embodiments, computer system 800 may train a separate multi-stage hybrid network for each of a plurality of use cases or topic domains. In some embodiments,computer system 800 may train a multi-modal multi-stage hybrid network with a distinct subchain trained on each of a plurality of use cases or topic domains.

[0538] In some embodiments, computer system 800 may test the performance of the trained system on data set aside as validation data. As part of the HGDTCS, one or more members of the human team (260 in Figure 2) may help judge the performance on the validation data. A member of the human team may drop a use case or a topic domain from a multi-modal network due to unsatisfactory performan...

Claims

CLAIMSWhat is claimed is:

1. A computer-implemented method for processing input data using a multi-stage hybrid network, the method comprising: receiving, by a computer system, a first input data associated with a task modality selected from the group consisting of speech, text, structured sequence, or graph data; processing, by the computer system, the first input data through a first stage comprising a generation subsystem configured to generate a first representation of the first input data in a target domain; processing, by the computer system, the first generated representation through a second stage comprising a recognition subsystem configured to evaluate semantic, structural, or contextual consistency of the first generated representation; applying, by the computer system, a sensibility verification subsystem configured to assess whether a first output of the recognition subsystem satisfies first predefined sensibility criteria based on first task-specific features prior to synthesis, wherein the first predefined sensibility criteria comprise one or more of task-specific logical, structural, or sensibility constraints for a first task; and based on a determination by the sensibility verification subsystem that the first output satisfies the predefined sensibility criteria, passing, by the computer system, the first output to a synthesis subsystem to render a first final output in a human usable format.

2. The method of claim 1, wherein, upon receiving, by the computer system, a second input data associated with the task modality, the method further comprises: processing, by the computer system, the second input data through the first stage comprising the generation subsystem configured to generate a second representation of the second input data in the target domain; processing, by the computer system, the second generated representation through the second stage comprising the recognition subsystem configured to evaluate semantic, structural, or contextual consistency of the second generated representation; applying, by the computer system, the sensibility verification subsystem configured to assess whether a second output of the recognition subsystem satisfies second predefined sensibility criteria based on second task-specific features prior to synthesis, wherein the second predefined sensibility criteria comprise one or more of task-specific logical,structural, or sensibility constraints; determining by the sensibility verification subsystem that the second output does not satisfy the predefined sensibility criteria; repeating, by the computer system, the generation, recognition, and verification stages to generate an updated second output from the second input data; and based on a determination by the sensibility verification subsystem that the updated second output satisfies the predefined sensibility criteria, passing, by the computer system, the updated second output to the synthesis subsystem to render a second final output in a human usable format.

3. The method of claim 1, wherein the generation subsystem comprises a neural network configured to produce a textual, symbolic, or graphical representation of the input data in a format adapted to a target task domain.

4. The method of claim 1, wherein the recognition subsystem comprises a consistency evaluator configured to detect at least one of a contradiction, redundancy, or domaininconsistent element in the generated representation.

5. The method of claim 1, wherein the predefined sensibility criteria comprise a taskspecific rule set selected based on a domain identifier included in the input data.

6. The method of claim 2, wherein repeating the generation, recognition, and verification stages comprises modifying a configuration parameter of the generation subsystem to alter its decoding behavior.

7. The method of claim 2, wherein the generation subsystem or the recognition subsystem comprises a plurality of parallel elements configured to operate concurrently on alternative configurations of the input data or intermediate representations.

8. The method of claim 2, further comprising, upon failure to satisfy the sensibility criteria after a predetermined number of repetitions, tagging the output as uncertain and suppressing synthesis.

9. The method of claim 2, wherein the updated second output is passed to a secondrecognition subsystem comprising a different model architecture than the recognition subsystem used in initial processing of the second input data.

10. The method of claim 2, further comprising receiving user feedback regarding the second output or the updated second output, and modifying at least one configuration parameter of the generation or recognition subsystem based on the user feedback prior to repeating the generation, recognition, and verification stages.

11. The method of claim 1, further comprising computing an explainability score for the first output based on a set of interpretable features, and applying a threshold to the explainability score as part of the predefined sensibility criteria.

12. The method of claim 11, wherein the explainability score or sensibility score is computed using internal activations of the recognition subsystem, including attention weights, hidden state magnitudes, or gradient-based saliency values.

13. The method of claim 1, wherein the generation subsystem is further configured to transform input data from one modality to another modality of the target domain prior to recognition.

14. The method of claim 11, wherein the generation subsystem is configured to produce multimodal outputs comprising a combination of two or more of text, image, audio, or symbolic formats.

15. The method of claim 1, wherein the recognition subsystem or sensibility verification subsystem includes a rule-based or symbolic reasoning component configured to detect violations of predefined logical or structural constraints within the generated representation.

16. The method of claim 5, wherein the task-specific rule set comprises logical constraints implemented using a symbolic rule engine that detects violations of expected entity relationships, value ranges, or dependency conditions defined in a task-specific ontology.

17. The method of claim 6, wherein the configuration parameter comprises a decodingtemperature, sampling strategy, or beam width associated with a neural sequence model in the generation subsystem, and modifying the parameter alters a distribution of generated candidate representations.

18. The method of claim 11, wherein computing the explainability score comprises calculating a weighted importance of internal features or attention values derived from intermediate layers of the recognition subsystem, and wherein the threshold is selected to reduce incorrect synthesis decisions due to low interpretability.

19. The method of claim 15, wherein the symbolic reasoning component comprises a grammar-based parser, a semantic type-checker, or a logic consistency module, each configured to identify non-conformities in the generated representation based on predefined structural templates or symbolic schemas.

20. A computer system comprising: one or more processors; and computer memory in communication with the one or more processors, wherein the computer memory stores instructions that when executed by the one or more processors cause the one or more processors to: receive input data associated with a task modality selected from the group consisting of speech, text, structured sequence, or graph data; process the input data through a first stage comprising a generation subsystem configured to generate a representation of the input data in a target domain; process the generated representation through a second stage comprising a recognition subsystem configured to evaluate semantic, structural, or contextual consistency of the generated representation; apply a sensibility verification subsystem configured to assess whether an output of the recognition subsystem satisfies predefined sensibility criteria based on taskspecific features prior to synthesis, wherein the predefined sensibility criteria comprise one or more of task-specific logical, structural, or sensibility constraints; upon a determination by the sensibility verification subsystem that the output satisfies the predefined sensibility criteria, passing, by the computer system, the output to a synthesis subsystem to render a final output in a human usable format; andupon a determination by the sensibility verification subsystem that the output does not satisfy the predefined sensibility criteria: repeat the generation, recognition, and verification stages to generate an updated output; and upon a determination by the sensibility verification subsystem that the updated output satisfies the predefined sensibility criteria, passing, by the computer system, the updated output to the synthesis subsystem to render a final output in a human usable format.

21. The computer system of claim 20, wherein the generation subsystem comprises a neural network configured to produce a textual, symbolic, or graphical representation of the input data in a format adapted to a target task domain.

22. The computer system of claim 20, wherein the recognition subsystem comprises a consistency evaluator configured to detect at least one of a contradiction, redundancy, or domain-inconsistent element in the generated representation.

23. The computer system of claim 20, wherein the predefined sensibility criteria comprise a task-specific rule set selected based on a domain identifier included in the input data.

24. The computer system of claim 20, wherein, upon a determination that the output does not satisfy the predefined sensibility criteria, the instructions further cause the computer system to modify a configuration parameter of the generation subsystem to alter its decoding behavior before repeating the generation, recognition, and verification stages.

25. The computer system of claim 24, wherein the configuration parameter comprises a decoding temperature, sampling strategy, or beam width associated with a neural sequence model in the generation subsystem, and modifying the parameter alters distribution of generated candidate representations.

26. The computer system of claim 20, wherein the instructions further cause the computer system, upon failure to satisfy the predefined sensibility criteria after a predetermined number of repetitions, to tag the output as uncertain and suppress synthesis.

27. The computer system of claim 20, wherein the updated output is passed to a second recognition subsystem comprising a different model architecture than the recognition subsystem used in initial processing of the input data.

28. The computer system of claim 20, wherein the instructions further cause the computer system to receive user feedback regarding the output or updated output, and to modify at least one configuration parameter of the generation or recognition subsystem based on the user feedback prior to repeating the generation, recognition, and verification stages.

29. The computer system of claim 20, wherein the instructions further cause the computer system to compute an explainability score for the output based on a set of interpretable features, and apply a threshold to the explainability score as part of the predefined sensibility criteria.

30. The computer system of claim 20, wherein the generation subsystem is further configured to transform input data from one modality to another modality of the target domain prior to processing by the recognition subsystem.

31. The computer system of claim 20, wherein the recognition subsystem or sensibility verification subsystem includes a rule-based or symbolic reasoning component configured to detect violations of predefined logical or structural constraints within the generated representation.

32. The computer system of claim 23, wherein the task-specific rule set comprises logical constraints implemented using a symbolic rule engine that detects violations of expected entity relationships, value ranges, or dependency conditions defined in a task-specific ontology.

33. The computer system of claim 29, wherein computing the explainability score comprises calculating a weighted importance of internal features or attention values derived from intermediate layers of the recognition subsystem, and wherein the threshold is selected to reduce incorrect synthesis decisions due to low interpretability.

34. The computer system of claim 31, wherein the symbolic reasoning component comprisesa grammar-based parser, a semantic type-checker, or a logic consistency module, each configured to identify non-conformities in the generated representation based on predefined structural templates or symbolic schemas.

35. The method of claim 1, further comprising applying a second sensibility verification subsystem after synthesis to perform a final pass / fail evaluation of the synthesized output prior to presentation or downstream use.

36. The computer system of claim 20, wherein the instructions further cause the computer system to apply a second sensibility verification subsystem after the synthesis subsystem to evaluate the final output and suppress output presentation if the verification fails.

37. The method of claim 1, wherein two or more sensibility verification subsystems are applied in sequence, and their respective outputs are aggregated to determine whether to pass the output to the synthesis subsystem.

38. The computer system of claim 20, wherein the computer memory stores instructions that, when executed, cause the computer system to apply a plurality of sensibility verification subsystems in sequence and aggregate their evaluation results to determine whether the output satisfies the predefined sensibility criteria.

39. The method of claim 1, wherein the recognition subsystem comprises both a learned model and a rule-based module configured to jointly assess consistency of the generated representation.

40. The computer system of claim 20, wherein the recognition subsystem comprises a learned neural model and a symbolic rule engine, each configured to evaluate different aspects of semantic, structural, or contextual consistency in the generated representation.

41. A computer-implemented method for training a multi-stage hybrid network, the method comprising: training, by a computer system, one or more stages of the multi-stage hybrid network using a plurality of techniques selected to optimize at least one of alignment with human values, explainability, efficiency, efficacy, generalizability, security, and sensibility, wherein theplurality of techniques comprises: employing event prediction elements to estimate changes in probability of specified future events based on observed conditions; integrating relationship regularization links between network elements to enforce specified relationships or diversity constraints; dynamically growing a network architecture by adding, modifying, or removing elements, layers, or stages based on performance criteria; using a mixture-of-experts model to diversify subsystem outputs and improve overall network performance; applying speculative decoding in a multi-stage recognition or generation system to constrain candidate output sequences; training explainable elements configured to improve interpretability of network decisions by representing detectable sets of input conditions; and / or implementing a human-guided development, training, and control system (HGDTCS) to enable iterative feedback-driven refinement of the network; validating, by the computer system, the trained multi-stage hybrid network using a set of validation data; and deploying, by the computer system, the trained multi-stage hybrid network for inference tasks including text generation, speech recognition, image recognition, multi-language translation, or other sequence processing tasks.

42. The method of claim 41, wherein the multi-stage hybrid network comprises at least one explainable element, the explainable element being configured to provide an interpretable rationale for its output based on a structured representation of knowledge.

43. The method of claim 41, wherein the relationship regularization links impose at least one of an “is-equal-to,” “is-not-equal-to,” “is-greater-than,” or “is-less-than” constraint between elements of the multi-stage hybrid network.

44. The method of claim 41, wherein the network architecture dynamically grows by incrementally adding stages, wherein each newly added stage is trained with stage-specific hyperparameters distinct from prior stages.

45. The method of claim 41, wherein the speculative decoding technique applies a draftmodel to generate candidate sequences and a verification model to refine candidate outputs using contextual constraints.

46. The method of claim 41, wherein the multi-stage hybrid network includes a mixture- of-experts subsystem, and the method further comprises training the mixture-of-experts subsystem using counter-tying constraints to ensure diversity among experts.

47. The method of claim 41, wherein training includes the use of informal semiautomated knowledge engineering, in which a subset of network elements are structured to represent human knowledge with reduced manual labor.

48. The method of claim 41, wherein at least one stage of the multi-stage hybrid network processes sequences of tokens using position synchronous beam pruning to construct a directed acyclic graph (DAG) representing possible token sequences.

49. The method of claim 41, wherein the training incorporates error prediction and correction nodes that predict and compensate for errors made by other nodes based on implicit local objectives.

50. The method of claim 41, wherein the multi-stage hybrid network includes a hierarchical text processing model that applies structured parsing based on grammar, ontology, or a semantic network to improve explainability.

51. The method of claim 41, wherein the multi-stage hybrid network is configured for a language translation task and is trained using a multi-path chain of one-way translations to improve robustness and generalizability.

52. The method of claim 41, wherein training includes lifelong learning mechanisms, enabling the network to adapt to new user interactions and refine its performance based on iterative human feedback.

53. The method of claim 41, wherein at least one stage of the network is designed to process bidirectional context by analyzing input sequences in both forward and backward directions.

54. The method of claim 41, wherein the multi-stage hybrid network is applied to speech recognition, and training includes aligning acoustic models with a language model to improve transcription accuracy.

55. The method of claim 41, wherein a human -guided development, training, and control system (HGDTCS) enables real-time interaction with human experts to iteratively refine training objectives and network configurations.

56. The method of claim 41, wherein the training includes generating synthetic training data through paraphrasing, summarization, or augmentation techniques to improve generalization to new data distributions.

57. The method of claim 41, wherein the multi-stage hybrid network comprises at least one subsystem dedicated to detecting adversarial attacks and applying countermeasures to maintain security and robustness.

58. The method of claim 41, wherein the network architecture includes an indexed element subsystem that dynamically loads and unloads indexed models based on runtime demands to optimize efficiency.

59. The method of claim 41, wherein training incorporates domain-specific adaptation, wherein the network is fine-tuned using specialized datasets for distinct application areas such as legal, medical, or financial text generation.

60. The method of claim 41, wherein a Socratic supervisor module is included to monitor contradictions in generated outputs and adjust network behavior to maintain logical consistency.

61. A computer system for training a multi-stage hybrid network, the computer system comprising: one or more processors; and computer memory in communication with the one or more processors, wherein the computer memory stores instructions that when executed by the one or more processors cause theone or more processors to: train one or more stages of the multi-stage hybrid network using a plurality of techniques selected to optimize at least one of alignment with human values, explainability, efficiency, efficacy, generalizability, security, and sensibility, wherein the plurality of techniques comprises: employing event prediction elements to estimate changes in probability of specified future events based on observed conditions; integrating relationship regularization links between network elements to enforce specified relationships or diversity constraints; dynamically growing a network architecture by adding, modifying, or removing elements, layers, or stages based on performance criteria; using a mixture-of-experts model to diversify subsystem outputs and improve overall network performance; applying speculative decoding in a multi-stage recognition or generation system to constrain candidate output sequences; training explainable elements configured to improve interpretability of network decisions by representing detectable sets of input conditions; and / or implementing a human-guided development, training, and control system (HGDTCS) to enable iterative feedback-driven refinement of the network; validate the trained multi-stage hybrid network using a set of validation data; and deploy the trained multi-stage hybrid network for inference tasks including text generation, speech recognition, image recognition, multi-language translation, or other sequence processing tasks.

62. A computer-implemented method for selecting a consistent representation from multiple generated candidates, the method comprising: receiving, by a computer system, input data associated with a task modality selected from the group consisting of speech, text, structured sequence, or graph data; generating, by a generation subsystem of a multi-stage hybrid network implemented by the computer system, a plurality of candidate representations of the input data in a target domain; processing, by a recognition subsystem of the multi-stage hybrid network, each candidate representation to evaluate semantic, structural, or contextual consistency;applying, by a sensibility verification subsystem of the multi-stage hybrid network, predefined sensibility criteria to each candidate representation to produce corresponding verification scores or status indicators; comparing, by the computer system, in a comparison step, the candidate representations based on a comparison of verification scores, consistency evaluations, or associated confidence values corresponding to each candidate representation; selecting, by the computer system, one of the candidate representations as a preferred output based on the comparison; and outputting, by the computer system, the preferred output as a final representation for downstream utilization.

63. The method of claim 62, wherein the comparison further comprises applying a voting scheme across the verification scores and consistency evaluations of the candidate representations to determine the preferred output.

64. The method of claim 62, wherein the verification scores are generated using a combination of a neural discriminator and a symbolic rule-based verifier.

65. The method of claim 62, further comprising discarding, by the computer system, one or more candidate representations having verification scores below a predefined threshold before the comparison step.

66. The method of claim 62, wherein the preferred output is selected based on both the verification scores and diversity of structure among the candidate representations.

67. The method of claim 62, further comprising logging, by the computer system, the verification scores and the preferred output for use in subsequent reinforcement or retraining operations.

68. The method of claim 62, further comprising presenting two or more of the candidate representations to a human user during training and receiving a user selection identifying a preferred representation, wherein the selection is used to reinforce or retrain one or more stages of the multi-stage hybrid network.

69. A computer system for selecting a consistent representation from multiple generated candidates, the computer system comprising: one or more processors; and computer memory in communication with the one or more processors, wherein the computer memory stores instructions that when executed by the one or more processors cause the one or more processors to: receive input data associated with a task modality selected from the group consisting of speech, text, structured sequence, or graph data; generate, via a generation subsystem of a multi-stage hybrid network implemented by the computer system, a plurality of candidate representations of the input data in a target domain; process, by a recognition subsystem of the multi-stage hybrid network, each candidate representation to evaluate semantic, structural, or contextual consistency; apply, by a sensibility verification subsystem of the multi-stage hybrid network, predefined sensibility criteria to each candidate representation to produce corresponding verification scores or status indicators; compare, in a comparison step, the candidate representations based on a comparison of verification scores, consistency evaluations, or associated confidence values corresponding to each candidate representation; select one of the candidate representations as a preferred output based on the comparison; and output the preferred output as a final representation for downstream utilization.

70. The computer system of claim 69, wherein the computer memory further stores instructions that, when executed, cause the computer system to apply a voting scheme across the verification scores and consistency evaluations of the candidate representations to determine the preferred output.

71. The computer system of claim 69, wherein the verification scores are produced by combining results from a neural discriminator and a symbolic rule-based verifier.

72. The computer system of claim 69, wherein the computer memory further stores instructions that, when executed, cause the computer system to discard one or more candidate representations having verification scores below a predefined threshold before performing thecomparison.

73. The computer system of claim 69, wherein the computer memory further stores instructions that, when executed, cause the computer system to log the verification scores and the preferred output for use in subsequent reinforcement or retraining operations.

74. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors of a computer system, cause the computer system to: receive a first input data associated with a task modality selected from the group consisting of speech, text, structured sequence, or graph data; process the first input data through a first stage comprising a generation subsystem configured to generate a first representation of the first input data in a target domain; process the first generated representation through a second stage comprising a recognition subsystem configured to evaluate semantic, structural, or contextual consistency of the first generated representation; apply a sensibility verification subsystem configured to assess whether a first output of the recognition subsystem satisfies first predefined sensibility criteria based on first taskspecific features prior to synthesis, wherein the first predefined sensibility criteria comprise one or more of task-specific logical, structural, or sensibility constraints for a first task; and based on a determination by the sensibility verification subsystem that the first output satisfies the predefined sensibility criteria, pass the first output to a synthesis subsystem to render a first final output in a human usable format.

75. The non-transitory computer-readable storage medium of claim 74, wherein the instructions further cause the computer system, upon receiving a second input data associated with the task modality, to: process the second input data through the first stage comprising the generation subsystem configured to generate a second representation of the second input data in the target domain; process the second generated representation through the second stage comprising the recognition subsystem configured to evaluate semantic, structural, or contextual consistency of the second generated representation; apply the sensibility verification subsystem to assess whether a second output of therecognition subsystem satisfies second predefined sensibility criteria based on second task-specific features prior to synthesis, wherein the second predefined sensibility criteria comprise one or more of task-specific logical, structural, or sensibility constraints; determine, by the sensibility verification subsystem, that the second output does not satisfy the second predefined sensibility criteria; repeat the generation, recognition, and verification stages to generate an updated second output from the second input data; and upon a determination by the sensibility verification subsystem that the updated second output satisfies the second predefined sensibility criteria, pass the updated second output to the synthesis subsystem to render a second final output in a human usable format.

76. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors of a computer system, cause the computer system to: train one or more stages of a multi-stage hybrid network using a plurality of techniques selected to optimize at least one of alignment with human values, explainability, efficiency, efficacy, generalizability, security, and sensibility, wherein the plurality of techniques comprises: employing event prediction elements to estimate changes in probability of specified future events based on observed conditions; integrating relationship regularization links between network elements to enforce specified relationships or diversity constraints; dynamically growing a network architecture by adding, modifying, or removing elements, layers, or stages based on performance criteria; using a mixture-of-experts model to diversify subsystem outputs and improve overall network performance; applying speculative decoding in a multi-stage recognition or generation system to constrain candidate output sequences; training explainable elements configured to improve interpretability of network decisions by representing detectable sets of input conditions; and / or implementing a human-guided development, training, and control system (HGDTCS) to enable iterative feedback-driven refinement of the network; validate the multi-stage hybrid network using a set of validation data; and deploy the multi-stage hybrid network for inference tasks including text generation, speech recognition, image recognition, multi-language translation, or other sequence processingtasks.

77. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors of a computer system, cause the computer system to: receive input data associated with a task modality selected from the group consisting of speech, text, structured sequence, or graph data; generate, via a generation subsystem of a multi-stage hybrid network, a plurality of candidate representations of the input data in a target domain; process, by a recognition subsystem of the multi-stage hybrid network, each candidate representation to evaluate semantic, structural, or contextual consistency; apply, by a sensibility verification subsystem of the multi-stage hybrid network, predefined sensibility criteria to each candidate representation to produce corresponding verification scores or status indicators; compare, in a comparison step, the candidate representations based on a comparison of verification scores, consistency evaluations, or associated confidence values corresponding to each candidate representation; select one of the candidate representations as a preferred output based on the comparison; and output the preferred output as a final representation for downstream utilization.

78. A computer program product comprising instructions which, when executed on a computer or a programmable device, cause the computer or programmable device to: receive a first input data associated with a task modality selected from the group consisting of speech, text, structured sequence, or graph data; process the first input data through a first stage comprising a generation subsystem configured to generate a first representation of the first input data in a target domain; process the first generated representation through a second stage comprising a recognition subsystem configured to evaluate semantic, structural, or contextual consistency of the first generated representation; apply a sensibility verification subsystem configured to assess whether a first output of the recognition subsystem satisfies first predefined sensibility criteria based on first taskspecific features prior to synthesis, wherein the first predefined sensibility criteria comprise one or more of task-specific logical, structural, or sensibility constraints for a first task; and based on a determination by the sensibility verification subsystem that the first outputsatisfies the predefined sensibility criteria, pass the first output to a synthesis subsystem to render a first final output in a human usable format.

79. The computer program product of claim 78, wherein the instructions further cause the computer to: receive a second input data associated with the task modality; process the second input data through the first stage comprising the generation subsystem to generate a second representation of the second input data in the target domain; process the second generated representation through the second stage comprising the recognition subsystem to evaluate semantic, structural, or contextual consistency of the second generated representation; apply the sensibility verification subsystem to assess whether a second output satisfies second predefined sensibility criteria based on second task-specific features, wherein the second predefined sensibility criteria comprise one or more of task-specific logical, structural, or sensibility constraints; determine that the second output does not satisfy the second predefined sensibility criteria; repeat the generation, recognition, and verification stages to generate an updated second output; and upon a determination that the updated second output satisfies the predefined sensibility criteria, pass the updated second output to the synthesis subsystem to render a second final output in a human usable format.

80. A computer program product comprising instructions which, when executed on a computer or a programmable device, cause the computer or programmable device to: train one or more stages of a multi-stage hybrid network using a plurality of techniques selected to optimize at least one of alignment with human values, explainability, efficiency, efficacy, generalizability, security, and sensibility, wherein the plurality of techniques comprises: employing event prediction elements to estimate changes in probability of specified future events based on observed conditions; integrating relationship regularization links between network elements to enforce specified relationships or diversity constraints; dynamically growing a network architecture by adding, modifying, or removing elements, layers, or stages based on performance criteria;using a mixture-of-experts model to diversify subsystem outputs and improve overall network performance; applying speculative decoding in a multi-stage recognition or generation system to constrain candidate output sequences; training explainable elements configured to improve interpretability of network decisions by representing detectable sets of input conditions; and / or implementing a human-guided development, training, and control system (HGDTCS) to enable iterative feedback-driven refinement of the network; validate the multi-stage hybrid network using a set of validation data; and deploy the multi-stage hybrid network for inference tasks including text generation, speech recognition, image recognition, multi-language translation, or other sequence processing tasks.

81. A computer program product comprising instructions which, when executed on a computer or a programmable device, cause the computer or programmable device to: receive input data associated with a task modality selected from the group consisting of speech, text, structured sequence, or graph data; generate, by a generation subsystem of a multi-stage hybrid network, a plurality of candidate representations of the input data in a target domain; process each candidate representation through a recognition subsystem to evaluate semantic, structural, or contextual consistency; apply predefined sensibility criteria to each candidate representation using a sensibility verification subsystem to produce corresponding verification scores or status indicators; compare the candidate representations based on a comparison of verification scores, consistency evaluations, or associated confidence values corresponding to each candidate representation; select one of the candidate representations as a preferred output based on the comparison; and output the preferred output as a final representation for downstream utilization.

82. The method of claim 1, further comprising: processing the first input data using a first subsystem and a second subsystem having complementary capabilities, wherein: the first subsystem is trained to process input data using audio-based units selected from phonemes, syllables, or words; andthe second subsystem is trained to process input data using token-based units, and computing, by the computer system, for each subsystem, a continuation structure comprising a directed acyclic graph structure representing likely sequence continuations based on subsystem-specific units; identifying, by the computer system, word-level alignments between the continuation structures of the first and second subsystems; and transmitting, by the computer system, intermediate results from the first subsystem to the second subsystem based on the word-level alignments.

83. The method of claim 82, further comprising: receiving, by the computer system, one or more continuation suggestions from the second subsystem based on the intermediate results; and modifying, by the computer system, one or more candidate paths of the continuation structure of the first subsystem based on the one or more continuation suggestions from the second subsystem.

84. The system of claim 20, wherein the computer system is configured to: process first input data using a first subsystem and a second subsystem having complementary capabilities, wherein: the first subsystem is trained to process input data using audio-based units selected from phonemes, syllables, or words; and the second subsystem is trained to process input data using token-based units; compute, for each of the first and second subsystems, a continuation structure comprising a directed acyclic graph representing likely sequence continuations based on subsystemspecific units; identify word-level alignments between candidate paths in the continuation structures of the first and second subsystems; and transmit intermediate results from the first subsystem to the second subsystem based on the word-level alignments.

85. The system of claim 84, wherein the computer system is further configured to: receive one or more continuation suggestions from the second subsystem based on the intermediate results; andmodify one or more candidate paths of the continuation structure of the first subsystem based on the received continuation suggestions.

86. A computer-implemented method for processing input data using cooperating subsystems with complementary capabilities, the method comprising: processing, by a first subsystem of a computer system, a first modality of input data based on audio-derived units selected from phonemes, syllables, or words; computing, by the computer system, a continuation structure comprising a directed acyclic graph (DAG) representing likely sequence continuations of the first modality based on the audio-derived units; extracting, by the computer system, word-level hypotheses from active paths in the continuation structure; and transmitting, by the computer system, the word-level hypotheses to a second subsystem trained to process input data using token-based units.

87. The method of claim 86, further comprising: receiving, by the computer system, contextually relevant feedback from the second subsystem based on the word-level hypotheses; and using the received feedback to influence continuation path selection or pruning in the continuation structure of the first subsystem.

88. A computer system comprising: a first subsystem configured to process a first modality of input data based on audio-derived units selected from phonemes, syllables, or words; a second subsystem trained to process input data using token-based units; and one or more processors and memory storing instructions that, when executed, cause the computer system to: compute a continuation structure comprising a directed acyclic graph (DAG) representing likely sequence continuations of the first modality based on the audio-derived units; extract word-level hypotheses from active paths in the continuation structure; and transmit the word-level hypotheses to the second subsystem.

89. The system of claim 88, wherein the instructions further cause the computer system to: receive contextually relevant feedback from the second subsystem based on the word-level hypotheses; and use the feedback to influence continuation path selection or pruning in the continuation structure of the first subsystem.

Images