Internodal behavior prediction in machine learning models

Abstract

A method includes accessing a machine learning model, including a plurality of nodes. Each node includes parameters configured to control generation of outputs of the node based on inputs of the node. A first input to the first node corresponding to a first data source is accessed. The parameters of a first node of the network are adjusted in accordance with an objective function configured to decrease an expected difference between a future first output and a future first input to the first node. The first output is generated based on the first input and the parameters of the first node. The first output is used as an input to the first data source. An inference is generated using the machine learning model, based on the first output.

Description

INTERNOD AL BEHAVIOR PREDICTION IN MACHINE LEARNINGMODELSTECHNICAL FIELD

[0001] The present disclosure relates generally to machine learning and, more particularly, to prediction of the behavior of nodes in a network of a machine learning models.BACKGROUND

[0002] Machine learning uses computer algorithms to automatically learn patterns and relationships in data, potentially without the need for explicit programming. A conventional machine learning process begins with a designer configuring or selecting an untrained machine learning model having a specific structure. The model is then trained to perform an inference task in a problem domain, using training data drawn from the problem domain. When training is complete, the trained model is deployed to perform the inference task at runtime.BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0003] In the drawings, which are not necessarily drawn to scale, like numerals may describe similar components in different views. To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the figure number in which that element is first introduced. Some non-limiting example embodiments of the subject matter are illustrated in the figures of the accompanying drawings in which:

[0004] FIG. 1 illustrates a machine-learning pipeline, providing context for examples described herein.

[0005] FIG. 2 illustrates training and use of an example machine-learning program, providing context for examples described herein.

[0006] FIG. 3 illustrates a first example network of a machine learning model, according to some examples.

[0007] FIG. 4 illustrates a further example network of a machine learning model, according to some examples.

[0008] FIG. 5 illustrates initial values of priors of a node of the network of FIG. 4, according to some examples.

[0009] FIG. 6 illustrates initial values of priors of the Bo submatrix of FIG. 5, according to some examples.

[0010] FIG. 7 illustrates differentiated values of priors of the Bo submatrix of FIG. 6, according to some examples.

[0011] FIG. 8 illustrates initial values of priors of the Bi submatrix of FIG. 5, according to some examples.

[0012] FIG. 9 illustrates differentiated values of priors of the Bi submatrix of FIG. 8, according to some examples.

[0013] FIG. 10 illustrates initial prior values and three different sets of differentiated values of priors of the B matrix of FIG. 5, according to some examples.

[0014] FIG. 11 illustrates a sequence showing generation and restructuring of a network of a machine learning model over time, according to some examples.

[0015] FIG. 12 illustrates a flowchart showing operations of a method for a node predicting the behavior of another node of a machine learning model, according to some examples.

[0016] FIG. 13 illustrates a flowchart showing operations of a method for adding a child node to a network of a machine learning model, according to some examples.

[0017] FIG. 14 is a diagrammatic representation of a machine in the form of a computer system within which a set of instructions may be executed to cause the machine to perform any one or more of the methodologies discussed herein, according to some examples.VO1DETAILED DESCRIPTION

[0018] Examples described in the present disclosure provide techniques for machine learning in which a machine learning model includes a network of nodes having inter-nodal communication of inputs and outputs. The nodes have parameters that are adjusted in response to inter-nodal communications over time to predict an input generated by, and received from, another node or from an entity outside of the network, such as another machine or an inference output process of the machine learning model. The node predicting the input is referred to herein as the predicting node, and the entity generating the input is referred to herein as the prediction target. In some examples, the predicting node learns to predict the input received from the prediction target conditioned in part on a previous output of the predicting node to the prediction target, thereby learning a relationship between its own output and a subsequently observed input from the prediction target. In some examples, the prediction node may also learn a generative model for generating the output of the the prediction node based on the predicted input from the prediction target.

[0019] Some existing machine learning model types, including artificial neural networks (also referred to herein simply as neural networks), Bayesian networks, and Markov models, are structured as a network of nodes. The structure of the network is predefined before training begins, and parameters of the model (which may include parameters representing the strength of connections between nodes of the network, also called weights) are adjusted during a training phase. Once the model is deemed to be trained, the training phase ends, and the trained model can be deployed to perform an inference task in an inference phase.

[0020] Examples of the Artificial Intelligence (Al) system described herein mitigate or eliminate the pitfalls described above (e.g., by including or otherwise implementing one or more examples of the machine learning model described herein). Some examples described herein allow a node to differentiate to predict a prediction target during runtime. In some examples, learning of a generative model by a predicting node provides a mechanism for supporting inference tasks, including tasks performed by entities outsideVO1 of the network of the machine learning model. The ability of a network to configure predicting nodes to support an external entity, and to configure other nodes to support the externally-predictive nodes, provides a recursive framework for expanding computational complexity to improve the performance of a task, differentiating the computational operations into functional clusters for performing or supporting sub-tasks, and improving efficiency of the computation, on a dynamic basis during runtime.

[0021] FIG. 1 is a flowchart depicting a machine-learning pipeline 200, and FIG. 2 is a schematic diagram illustrating the training and use of an example machine-learning program. FIG. 1 and FIG. 2 provide context for examples described herein by describing various techniques and approaches to structuring, training, validating, and deploying a machine learning model.

[0022] The machine-learning pipeline 200 may be used to generate a trained machine learning model, for example the trained machine-learning program 202 of FIG. 2, to perform operations associated with searches and query responses.

[0023] Broadly, machine learning may involve using computer algorithms to automatically learn patterns and relationships in data, potentially without the need for explicit programming. Machine learning algorithms can be divided into three main categories: supervised learning, unsupervised learning, and reinforcement learning.• Supervised learning involves training a model using labeled data to predict an output for new, unseen inputs. Examples of supervised learning algorithms include linear regression, decision trees, and neural networks.• Unsupervised learning involves training a model on unlabeled data to find hidden patterns and relationships in the data. Examples of unsupervised learning algorithms include clustering, principal component analysis, and generative models like autoencoders.• Reinforcement learning involves training a model to make decisions in a dynamic environment by receiving feedback in the form of rewards or penalties. Examples of reinforcement learning algorithms include Q- learning and policy gradient methods.VO1

[0024] Examples of specific machine learning algorithms that may be deployed, according to some examples, include logistic regression, which is a type of supervised learning algorithm used for binary classification tasks. Logistic regression models the probability of a binary response variable based on one or more predictor variables. Another example type of machine learning algorithm is Naive Bayes, which is another supervised learning algorithm used for classification tasks. Naive Bayes is based on Bayes' theorem and assumes that the predictor variables are independent of each other. Random Forest is another type of supervised learning algorithm used for classification, regression, and other tasks. Random Forest builds a collection of decision trees and combines their outputs to make predictions. Further examples include neural networks, which consist of interconnected layers of nodes (or neurons) that process information and make predictions based on the input data. Matrix factorization is another type of machine learning algorithm used for recommender systems and other tasks. Matrix factorization decomposes a matrix into two or more matrices to uncover hidden patterns or relationships in the data. Support Vector Machines (SVM) are a type of supervised learning algorithm used for classification, regression, and other tasks. SVM finds a hyperplane that separates the different classes in the data. Other types of machine learning algorithms include decision trees, k-nearest neighbors, clustering algorithms, and deep learning algorithms such as convolutional neural networks (CNN), recurrent neural networks (RNN), and transformer models. The choice of algorithm depends on the nature of the data, the complexity of the problem, and the performance requirements of the application.

[0025] The performance of machine learning models may be evaluated on a separate test set of data that was not used during training to ensure that the model can generalize to new, unseen data.

[0026] Although several specific examples of machine learning algorithms are discussed herein, the principles discussed herein can be applied to other machine learning algorithms as well. Deep learning algorithms such as convolutional neural networks, recurrent neural networks, and transformers, as well as more traditional machine learning algorithms like decision trees,VO1 random forests, and gradient boosting may be used in various machine learning applications.

[0027] Machine learning models are trained to perform specific types of inference tasks. Two example types of inference tasks in machine learning are classification tasks and regression tasks. Classification tasks, also referred to as categorization tasks, aim at classifying items into one of several category values (for example, is this object an apple or an orange?). Regression algorithms aim at quantifying some items (for example, by providing a value that is a real number) to perform a regression task.

[0028] Generating a trained machine-learning program 202 may include multiple phases that form part of the machine-learning pipeline 200, including for example the following phases illustrated in FIG. 1 :• Data collection and preprocessing 102: This phase may include acquiring and cleaning data to ensure that it is suitable for use in the machine learning model. This phase may also include removing duplicates, handling missing values, and converting data into a suitable format.• Feature engineering 104: This phase may include selecting and transforming the training data 206 to create features that are useful for predicting the target variable, or both. Feature engineering may include (1) receiving features 208 (e.g., as structured or labeled data in supervised learning), (2) identifying features 208 (e.g., unstructured or unlabeled data for unsupervised learning) in training data 206.• Model selection and training 106: This phase may include selecting an appropriate machine learning algorithm and training it on the preprocessed data. This phase may further involve splitting the data into training and testing sets, using cross-validation to evaluate the model, and tuning hyperparameters to improve performance.• Model evaluation 108: This phase may include evaluating the performance of a trained model (e.g., the trained machine-learning program 202) on a separate testing dataset. This phase can help determine if the model is overfitting or underfitting and determine whether the model is suitable for deployment.VO1• Prediction 110: This phase involves using a trained model (e.g., trained machine-learning program 202) to generate predictions on new, unseen data.• Validation, refinement or retraining 112: This phase may include updating a model based on feedback generated from the prediction phase, such as new data or user feedback.• Deployment 114: This phase may include integrating the trained model (e.g., the trained machine-learning program 202) into a more extensive system or application, such as a web service, mobile app, or loT device. This phase can involve setting up APIs, building a user interface, and ensuring that the model is scalable and can handle large volumes of data.

[0029] FIG. 2 illustrates further details of two example phases, namely a training phase 204 (e.g., part of the model selection and trainings 106) and a prediction phase 210 (part of prediction 110). Prior to the training phase 204, feature engineering 104 is used to identify features 208. This may include identifying informative, discriminating, and independent features for effectively operating the trained machine-learning program 202 in pattern recognition, classification, and regression. In some examples, the training data 206 includes labeled data, known for pre-identified features 208 and one or more outcomes. Each of the features 208 may be a variable or attribute, such as an individual measurable property of a process, article, system, or phenomenon represented by a data set (e.g., the training data 206). Features 208 may also be of different types, such as numeric features, strings, and graphs, and may include one or more of content 212, concepts 214, attributes 216, historical data 218, or user data 220, merely for example.

[0030] In training phase 204, also referred to herein as a training period, the machine-learning pipeline 200 uses the training data 206 to find correlations among the features 208 that affect a predicted outcome or prediction / inference data 222. The training data may also be referred to herein as reference data, because it is used as a reference for setting the trained parameter values of the model.VO1

[0031] With the training data 206 and the identified features 208, the trained machine-learning program 202 is trained during the training phase 204 during machine-learning program training 224. The machine-learning program training 224 appraises values of the features 208 as they correlate to the training data 206. The result of the training is the trained machinelearning program 202 (e.g., a trained or learned model).

[0032] Further, the training phase 204 may involve machine learning, in which the training data 206 is structured (e.g., labeled during preprocessing operations). The trained machine-learning program 202 implements a neural network 226 capable of performing, for example, classification and clustering operations. In other examples, the training phase 204 may involve deep learning, in which the training data 206 is unstructured, and the trained machine-learning program 202 implements a deep neural network 226 that can perform both feature extraction and classification / clustering operations.

[0033] In some examples, a neural network 226 may be generated during the training phase 204, and implemented within the trained machine-learning program 202. The neural network 226 includes a hierarchical (e.g., layered) organization of neurons, with each layer consisting of multiple neurons or nodes. Neurons in the input layer receive the input data, while neurons in the output layer produce the final output of the network. Between the input and output layers, there may be one or more hidden layers, each consisting of multiple neurons.

[0034] Each neuron in the neural network 226 operationally computes a function, such as an activation function, which takes as input the weighted sum of the outputs of the neurons in the previous layer, as well as a bias term. The output of this function is then passed as input to the neurons in the next layer. If the output of the activation function exceeds a certain threshold, an output is communicated from that neuron (e.g., transmitting neuron) to a connected neuron (e.g., receiving neuron) in successive layers. The connections between neurons have associated weights, which define the influence of the input from a transmitting neuron to a receiving neuron. During the training phase, these weights are adjusted by the learning algorithm to optimize the performance of the network. Different types ofVO1 neural networks may use different activation functions and learning algorithms, affecting their performance on different tasks. The layered organization of neurons and the use of activation functions and weights enable neural networks to model complex relationships between inputs and outputs, and to generalize to new inputs that were not seen during training.

[0035] In some examples, the neural network 226 may also be one of several different types of neural networks, such as a single-layer feedforward network, a Multilayer Perceptron (MLP), an Artificial Neural Network (ANN), a Recurrent Neural Network (RNN), a Long Short-Term Memory Network (LSTM), a Bidirectional Neural Network, a symmetrically connected neural network, a Deep Belief Network (DBN), a Convolutional Neural Network (CNN), a Generative Adversarial Network (GAN), an Autoencoder Neural Network (AE), a Restricted Boltzmann Machine (RBM), a Hopfield Network, a Self-Organizing Map (SOM), a Radial Basis Function Network (RBFN), a Spiking Neural Network (SNN), a Liquid State Machine (LSM), an Echo State Network (ESN), a Neural Turing Machine (NTM), or a Transformer Network, merely for example.

[0036] In addition to the training phase 204, a validation phase may be performed on a separate dataset known as the validation dataset. The validation dataset is used to tune the hyperparameters of a model, such as the learning rate and the regularization parameter. The hyperparameters are adjusted to improve the model's performance on the validation dataset.

[0037] Once a model is fully trained and validated, in a testing phase, the model may be tested on a new dataset. The testing dataset is used to evaluate the model's performance and ensure that the model has not overfitted the training data.

[0038] In prediction phase 210, also referred to herein as a runtime period, the trained machine-learning program 202 uses the features 208 for analyzing query data 228 to generate inferences, outcomes, or predictions, as examples of a prediction / inference data 222. For example, during prediction phase 210, the trained machine-learning program 202 generates an output. Query data 228 is provided as an input to the trained machine-learning program 202, and the trained machine-learning program 202 generates theVO1 prediction / inference data 222 as output, responsive to receipt of the query data 228. Query data processed by the model during the runtime phase may also be referred to herein as candidate data, because it is data that is a priori not associated with any particular inference output.

[0039] In some examples, the trained machine-learning program 202 may be a generative Al model. Generative Al is a term that may refer to any type of artificial intelligence that can create new content from training data 206. For example, generative Al can produce text, images, video, audio, code, or synthetic data similar to the original data but not identical.

[0040] Some of the techniques that may be used in generative Al are:• Convolutional Neural Networks (CNNs): CNNs may be used for image recognition and computer vision tasks. CNNs may, for example, be designed to extract features from images by using filters or kernels that scan the input image and highlight important patterns.• Recurrent Neural Networks (RNNs): RNNs may be used for processing sequential data, such as speech, text, and time series data, for example. RNNs employ feedback loops that allow them to capture temporal dependencies and remember past inputs. The feedback loops of an RNN can be considered a cycle within its network.• Generative adversarial networks (GANs): GNNs may include two neural networks: a generator and a discriminator. The generator network attempts to create realistic content that can “fool” the discriminator network, while the discriminator network attempts to distinguish between real and fake content. The generator and discriminator networks compete with each other and improve over time.• Variational autoencoders (VAEs): VAEs may encode input data into a latent space (e.g., a compressed representation) and then decode it back into output data. The latent space can be manipulated to generate new variations of the output data. VAEs may use self-attention mechanisms to process input data, allowing them to handle long text sequences and capture complex dependencies.• Transformer models: Transformer models may use attention mechanisms to learn the relationships between different parts of inputVO1 data (such as words or pixels) and generate output data based on these relationships. Transformer models can handle sequential data, such as text or speech, as well as non-sequential data, such as images or code.

[0041] In generative Al examples, the output prediction / inference data 222 include predictions, translations, summaries or media content.

[0042] FIG. 3 illustrates a first example network 300 of a machine learning model, such as trained machine-learning program 202. The network 300 includes a set of nodes, shown as first node 302, second node 304, and third node 306. The nodes are connected to each other by unidirectional connections, shown as a first connection 308 (from first node 302 to second node 304), a second connection 310 (from second node 304 to third node 306), and a third connection 312 (from third node 306 to first node 302). Each connection conveys or designates outputs of a transmitting node as inputs to a receiving node. The connections for which a given node is a transmitting node may be referred to as outgoing connections of that node; by the same token, those connections for which a node is the receiving node may be referred to as incoming connections of that node. In network 300, the transmitting node of first connection 308 is first node 302, and the receiving node of first connection 308 is second node 304; the transmitting node of second connection 310 is second node 304, and the receiving node of second connection 310 is third node 306; and the transmitting node of third connection 312 is third node 306, and the receiving node of third connection 312 is first node 302.

[0043] The nodes of the network 300 (and other networks of machine learning models described herein) may be described herein as receiving inputs, processing data, and generating outputs. Similarly, the connections of the described networks may be described as conveying, transmitting, or providing data from one node to another node. In some examples, these terms may refer to the operation of nodes and connections embodied as logical circuit components or other physical data processing or data communication components. For example, the nodes and connections of a network of a machine learning model as described herein may be implemented as circuit components of a field programmable gate arrayVO1(FPGA) or another dynamically reconfigurable logical circuit. In some examples, the use of binary inputs and outputs for the nodes of the network enable the implementation of the network as a circuit comprising a plurality of logic gates.

[0044] In some examples, the nodes and connections may be implemented as data, for example, as data structures or instructions stored in a memory and manipulated by a physical processor. The machine 1400 described with reference to FIG. 14 below provides an example of a computing system suitable for such an implementation. In some examples, the processor executes instructions to perform operations of a system, such as an artificial intelligence (Al) system, that manipulates the data corresponding to the machine learning model (including the nodes and connections of the network) to perform the operations described herein. For example, a node may be implemented as a data structure associated with one or more parameters (e.g., prior probability distributions, also called priors, as described below), and processing the inputs of a node to generate the node's outputs may be performed by the physical processor processing the parameter values of the node and the inputs of the node to generate the outputs of the node. A connection may be implemented as data designating or identifying a relationship between a transmitting node and a receiving node, such that the outputs of the transmitting node are treated by the physical processor as inputs to the receiving node. In some examples, the nodes and connections may include instructions executed by the processor to perform these operations, whereas in other examples, the instructions are separate from the nodes and connections, and cause the processor to process the data corresponding to the nodes and connections. Thus, in some examples, each node may be said to include or be associated with parameter values, and these parameter values may be used to control the processing of the inputs of the node in order to generate the outputs of the node. It will be appreciated that all references herein to a node processing data, to a connection conveying data, and so on, are all intended to encompass all suitable implementations of the nodes and connections as any combination of hardware and software.VO1

[0045] In some examples, as described below with reference to FIG. 12 and FIG. 12, the network may be implemented in software or in reconfigurable hardware during some phases requiring reconfiguration of the network (e.g., adjustment of parameter values of the nodes), and the behavior of the network at a point in time may be fixed by mapping the network's behavior to a set of interconnected logic gates to create a design for a digital circuit, which may then be constructed to implement the behavior of the network as a physical digital circuit.

[0046] In some examples, the nodes of the network 300 operate to process their inputs and generate their outputs at each of a sequence of discrete time steps. In some examples, all nodes of the network 300 operate at each time step; in other examples, some nodes may have different rates of operation such that some nodes of the network 300 may only process their inputs and generate (or update) their outputs at some time steps, such as every n time steps where n is a positive integer.

[0047] In some examples, a node generates its outputs at a current time step t by processing its inputs at a previous or prior time step t- . The inputs may be processed by applying various parameters (e.g., priors), such as matrices of priors, to the inputs, and computing one or more output values based on the processed parameters and inputs. In some examples, a node may generate only a single output value that is conveyed as an input to all connected nodes; in other examples, a node may generate multiple outputs to be conveyed to multiple connected nodes or as multiple output values conveyed to a single node.

[0048] In some examples, a single output generated by a node may be an activation value. An activation value may be regarded as a measure of how active the node is at the current time step based on its inputs from the previous time step. In some examples, the activation value of a given node is a continuous value, such as a floating-point value. In some examples, the activation value of a given node is a discrete value, such as a binary value (e.g., 0 or 1, corresponding to inactive or active, or “not firing” and “firing” in the context of an analogy of neuronal firing), or another member of a finite set of discrete values (e.g., a member of the set {0, 1, 2, 3} or aVO1 member of the set {00, 01, 10, 11 }). A discrete activation value may be determined by quantization, which may be either a stochastic quantization process or a deterministic quantization process. In a stochastic quantization process, for example, a distribution (e.g., a probability distribution) over all possible discrete output values may be generated by processing the inputs and priors, and a random or pseudo-random value may be used to select a discrete output value (or set of output values) from the probability distribution. In a deterministic quantization process, a single discrete output value (or set of output values) may instead be selected from the distribution of possible discrete output values, e.g., as the value or values having the highest probability or the highest prevalence.

[0049] In some examples, nodes may have connections to themselves, referred to as self-connections, in which they serve as both transmitting node and receiving node. In some such examples, a node may generate a single output for propagation to its connected nodes, and another output or set of outputs to itself, called self-outputs, via its self-connection(s). In some examples, a node may pass through its current inputs as self-outputs, thereby providing a one-time-step memory of its own inputs. This may enable the node to evaluate the accuracy of its own previous predictions of state transitions of the observed system, as described in greater detail below with reference to FIG. 5 through FIG. 10.

[0050] It will be appreciated that the three-node network 300 shown in FIG. 3 is intended as a simplified example, and that actual machine learning models may include networks having much larger number of nodes, such as thousands or millions of nodes. Thus, the network 300 may be regarded as a small section of a larger network, in which one or more of the illustrated nodes 302, 304, or 306 may have connections to or from one or more other unseen nodes of the larger network. In one such example, in the context of CNNs, first node 302 could be regarded as a neuron of a convolution layer, second node 304 could be regarded as a neuron of a rectified linear unit (ReLU) layer receiving an output activation map from the convolution layer, and third node 306 could be regarded as a neuron of a pooling layer receiving the ReLU layer output. The third connection 312 could be regarded as a back-connection to a previous layer as in a recurrent neural networkVO1(RNN), or the third connection 312 could be omitted (or set to a zero prior value by first node 302).

[0051] FIG. 4 illustrates a further example network 400, including first node 402, second node 404, and third node 406. In network 400, the connections between nodes are in bidirectional pairs: first connection 408 from first node 402 to second node 404 and first connection back 414 from second node 404 to first node 402; second connection 410 from second node 404 to third node 406 and second connection back 416 from third node 406 to second node 404; and third connection 412 from third node 406 to first node 402 and third connection back 418 from first node 402 to third node 406.

[0052] The network 400 also shows first node 402 having a bidirectional pair of connections to an environment 420: a connection from environment 422, and a connection to environment 424. In some examples, the environment 420 as shown in FIG. 4 can represent a data source or other entity residing in an environment outside of the network, such that a node connected to the environment 420 (such as first node 402), also referred to as an environment-facing node, acts as an input node and an output node for the network 400.

[0053] The connections to or from the environment 420 may act as connections to another node insofar as they convey inputs to, or receive outputs from, the environment-facing node. In some examples, the environment 420 can be regarded as an external data source (providing inputs to the network via the connection from environment 422) or a postprocessing stage for outputs of the network (e.g., generated data such as inference data). In some examples, multiple nodes of the network are environment-facing nodes, such that the inputs and outputs of the network at a given time step may comprise multiple values (e.g., multiple binary values for examples in which each node generates only binary outputs). Thus, for example, a network configured to perform an inference task on image data may have 1 million environment-facing nodes configured to each receive one bit of a 1000 x 1000 black and white pixel input image, or more environment-facing nodes for images having multi-bit encodings for eachVO1 pixel (e.g., greyscale or multi-channel images). A network configured to categorize images data into two categories may only need as few as one environment-facing node to provide an output to the environment 420: an output of “0” might indicate categorization into the first category, whereas an output of “1” might indicate categorization into the second category. However, in many cases, using an active inference model (described below) for the behavior of the network may provide benefits with respect to inference tasks involving bidirectional communication with an environment 420, such as control systems using feedback from the system being controlled: in some such cases, the environment 420 is controlled at least in part by the outputs received via the connection to environment 424, and the environment 420 provides feedback on its own state as inputs via connection from environment 422, such that the network can process the feedback to generate desired control outputs. Thus, in some examples, the behavior of the network may provide benefits in the context of executing the machine learning model on a first machine and using the machine learning model to control the operation of, or performance of a task by, a second machine.

[0054] Thus, in some examples, the environment 420 may represent an entity configured to perform a task based on inputs received from one or more environment-facing nodes (e.g., first node 402) via connections (e.g., connection to environment 424).

[0055] In the illustrated example, each node 402, 404, 406 has a selfconnection 426, which is a connection in which the node is both the transmitting node and the receiving node of the connection. A selfconnection 426 may act as a one-time-step delay loop feeding a node's outputs back to itself. A self-connection 426 can also serve as a one-timestep inverter (e.g., generating a 1 in response to a 0) or a number of other operations: for example, in some cases, a node can process its self-signal from the last timestep and its environment signal from the last timestep according to logic dictated by its parameters (e.g., priors). In some examples, self-connections 426 are used in learning to predict the output of a data source (e.g., another node or the environment 420), as described below.VO1

[0056] In some examples, a network with bidirectional pairs of connections, such as network 400, may be configured to perform inference according to the principles of active inference and the minimization of variational free energy (VFE) or expected free energy (EFE) against a generative model. Active inference is a framework for modeling Bayes- optimal behavior of an agent. In some examples herein using an active inference framework, an agent - such as a machine learning model, or even a node of a network driving the behavior of the machine learning model - acts to minimize an upper bound on the “surprise” resulting from the agent's observations. Surprise is a measure of the degree to which an agent’s current observations differ from its preferred observations and the informativeness of its observations. In some examples, VFE or EFE is a function of both reward and information gain, potentially in varying ratios, thereby biasing the node to both explore a parameter space and exploit its self-efficacy. In some examples herein using an active inference framework, agents seeking to minimize their surprise act on the observed system to solicit feedback that accords with the agent's desired observations: this activity of the agent is why the framework is referred to as “active” inference.

[0057] The active inference frameworks described herein propose that, even if an agent does not have the capability to directly minimize its surprise, it can at least seek to minimize a proxy for surprise, called variational free energy. The agent may minimize VFE OR EFE by performing computation in response to, and in anticipation of, its observations. Example implementations of active inference described herein may thus rely on the free energy principle, by which an agent reduces its surprise by generating inferences using an internal model and updating its internal model to predict the observed (and acted-upon) system.

[0058] Examples of computational operations implementing active inference are described below with reference to FIG. 5 through FIG. 10. The examples of FIG. 5 through FIG. 10 may be implemented by nodes of the network 400.

[0059] In some examples, a network such as network 400 may be used to implement a machine learning model operating according to an activeVO1 inference framework (also referred to as an active inference model). Each node of the network may be associated with parameters (e.g., priors) organized into one or more tensors (e.g., scalars, vectors, or matrices), also referred to as parameter tensors, each tensor modeling a different aspect of the node's behavior.

[0060] In some examples, the active inference behavior of the nodes of the network may be modeled as a partially observable Markov decision process (POMDP), characterized by the equation:POMDP equation wherein:• “T” indicates the time step, from a first time step T=1 to a final (or current) time step T=T,• “ot” indicates the observations o of the node (e.g., its inputs) at a given time step i=t,• “st” indicates the actual state 5 of the observed system (e.g., a hidden state that may differ from the observations of the node) at time step i=t,• "K" indicates a policy for the node, e.g., a possible sequence of actions (e.g., outputs to generate in response to inputs) that the node can take, and• “(|)” indicates an expected reward, e.g., a similarity between inputs received by the node and the node's preferred inputs.

[0061] The active inference behavior of a node may be implemented using three or more sets of priors organized into three or more tensors. Examples of tensors used to compute active inference behavior using a POMDP for a node of the network are illustrated and described with reference to FIG. 5.

[0062] FIG. 5 shows four tensors used by a POMDP of a node. The A tensor 502, B tensor 512 (consisting of the Bo subtensor 504 and Bi subtensor 506), C tensor 508, and D tensor 510 indicate four sets of priorsVO1 maintained by the node. Each of these sets of priors is used to contribute to the performance of an active inference task by the network (e.g., network 400). The values of the priors shown in the example of FIG. 5 are initial prior values that may be assigned to a newly initialized node of the network (e.g., network 400), also referred to as an undifferentiated node or a stem node. The node may be deployed within the network to contribute to an inference task during a runtime period, during which the node may differentiate itself by adjusting its prior values to diverge from their initial values in response to interactions with other nodes of the network. A node that has undergone at least partial differentiation may be referred to as a differentiated node.

[0063] In some examples, the inference task performed by the network is an active inference task. In active inference, as described herein, a machine learning model acts upon a system and observes the system.

[0064] In some examples, the A tensor 502 may be regarded as modeling a probabilistic mapping of the node's observations (e.g., its inputs at a given time step) to an inferred hidden state of the observed system (e.g., a node or set of nodes). This mapping may be referred to as an observational model, denoted as:Observational Model

[0065] In the example shown in FIG. 5, the A tensor 502 is initialized as an identity matrix mapping observed inputs to exactly equivalent hidden states. In some examples, the inputs received by a node are equivalent to the state of the observed system; thus, there are no hidden states, and the A tensor 502 may remain unchanged during differentiation.

[0066] In some examples, the B tensor 512 may be regarded as modeling a probabilistic mapping of the node's observations (e.g., its inputs) at a given time step T-1 to a predicted set of observations at a subsequent time step T.VO1Thus, the B tensor 512 may be referred to as a prediction parameter tensor.This mapping may be referred to as a transition model, denoted as:Transition Model

[0067] In some examples, the B tensor 512 includes a separate submatrix for each set of outputs of the node at the first time step: thus, the predicted inputs at the second time step are conditioned on the outputs of the node at the first time step. In the example shown in FIG. 5, the node has only a single output (e.g., a binary activation value) having only two possible states, 1 or 0. The B tensor 512 therefore consists of two submatrices: a Bo subtensor 504 corresponding to output 0, and a Bi subtensor 506 corresponding to output 1.

[0068] In the example shown in FIG. 5, Bo subtensor 504 and Bi subtensor 506 are both initialized as homogeneous matrices (e.g., all matrix elements set to a value of 0.5) mapping all observed inputs at the first time step with equal probabilities to predicted inputs at the second time step.

[0069] In some examples, the C tensor 508 may be regarded as modeling a prior probability distribution (also referred to simply as a prior) over the node's observations (e.g., its inputs). This prior over observations may be referred to as the node's preferences, prior preferences, or expectations, and is denoted as:Prior Over Observations

[0070] In the example shown in FIG. 5, the C tensor 508 is initialized as a zero vector mapping all observations to a zero reward. Thus, a node initialized according to the initial values shown in FIG. 5 has no preferred values for its inputs. In some examples, C is not 0 but is instead configuredVO1 to help the node predict the output of another node: the first node (having the C tensor 708) prefers or expects to see its self-signal match the output of a second node. The second node may be a node that the first node is configured to predict, and in some cases, the second node may be referred to as a parent node of the first node.

[0071] In some examples, the D tensor 510 may be regarded as modeling a prior over an initial hidden state at initial time step T=1. This prior over the initial hidden state is denoted as:Prior Over Initial Hidden State

[0072] In the example shown in FIG. 5, the D tensor 510 is initialized as a homogeneous vector (e.g., all vector components set to a value of 0.5) weighting all initial state values with equal prior probability. Thus, a node initialized according to the initial values shown in FIG. 5 has no a priori beliefs about an initial state of the observed system.

[0073] In some examples, additional tensors may be used, whereas in other examples, one or more of the four tensors shown in FIG. 7 may be omitted (such as the D tensor).

[0074] In some cases, the system applies an epistemic term that pertains to knowledge, understanding, and information. The system can prioritize epistemic terms such that the machine learning model focuses on gaining insight on the functions performed by the machine learning model and updating the model to learn from current inferences. For instance, the system increases the epistemic terms in the C tensor 508 such that the model focuses on certainty, uncertainty, and knowledge of the machine learning model.

[0075] In some examples, the B tensor 512 may be the primary or the only set of priors to be adjusted during differentiation of the node: in other words, each of the other sets of priors (the A tensor 502, C tensor 508, and D tensor 510) remains constant, whereas the transition model (implemented by the BVO1 tensor 512) is adjusted to attempt to predict state transitions by the observed system (e.g., other connected nodes), conditioned on the actions (e.g., outputs) of the node. However, in some examples, one or more of the other sets of priors may also be adjusted during differentiation.

[0076] FIG. 6 through FIG. 10 illustrate examples of how the priors of the B tensor 512 of a node may change during differentiation to adapt to different systems exhibiting different behaviors in response to the node's own actions.

[0077] FIG. 6 shows an initial Bo subtensor 504a, equivalent to the initial state of the Bo subtensor 504 shown in FIG. 5. The initial Bo subtensor 504a is the matrix of predictions for a future observation of the state of a system - in the illustrated example, the observed system is a single other node connected to the node associated with the initial Bo subtensor 504a, and the observed node generates a single output, having a one-bit binary value, at each time step. Similarly, the node associated with the initial Bo subtensor 504a also generates a single output, having a one-bit binary value, at each time step. Thus, the B tensor 512 of the node observing the system has two initial submatrices: a first submatrix, shown here in FIG. 6 as initial Bo subtensor 504a, conditioned on the output of the node having a “0” value, and a second submatrix, shown in FIG. 8 below as initial Bi submatrix 506a, conditioned on the output of the node having a “1” value.

[0078] The initial Bo subtensor 504a includes two columns of priors corresponding to two first time step inputs 602. The first time step inputs 602 correspond to the potential observation of two different states of the system (e.g., inputs to the node) at a previous time step T-1 (also referred to herein as the first time step). In this example, the node is assumed to see its inputs accurately; however, in other implementations, the agent may have some uncertainty as to whether its inputs are accurate or the result of noise (e.g., the A tensor 502 may not be an identity matrix). In this example, the node is assumed to see its inputs accurately; however, in other implementations, the agent may have some uncertainty as to whether its inputs are accurate or the result of noise (e.g., the A tensor 502 may not be an identity matrix). The states of the system may also be referred to as inputVO1 states of the node observing the system. The priors for 0 input at first time step 603 are in the left column, and the priors for 1 input at first time step 604 are in the right column. The two columns may thus be regarded as corresponding to two input states of the node; in some examples, each column (and its associated set of priors, 603 or 604) is only used in computation of the node's behavior when the node observes the corresponding input state.

[0079] Returning to FIG. 6, the columns of the initial Bo subtensor 504a are split into two rows, the second time step inputs 601, corresponding to observation of two different states of the system at a current time step T (also referred to herein as the second time step). The priors for 0 input at second time step 605 are in the top row, and the priors for 1 input at second time step 606 are in the bottom row.

[0080] It will be appreciated that, for nodes observing systems with more complex outputs (e.g., multiple connected nodes, or nodes generating outputs more complex than a single bit per time step), the initial Bo subtensor 504a would include more columns and rows. For example, a node connected to three other nodes, each outputting a single bit at each time step, would need an initial Bo subtensor 504a with 23= 8 columns and 8 rows to map all previously observed system states to all predicted current systems states. In some examples, a node may be able to determine that two or more of its inputs tend to be highly similar, and may have its B tensor 712 simplified by combining the two inputs into a single input value.

[0081] As described above, the initial Bo subtensor 504a is a probabilistic mapping of previous observations to predicted current observations, conditioned on the output of the node at time step T-1 having been the binary value 0. The Bi subtensor 506, by contrast, stores the prior values mapping previous observations to predicted current observations conditioned on the output of the node at time step T-1 having been the binary value 1. It will be appreciated that, for nodes having a more complex output than a single bit at each time step, a greater number of B submatrices would be required, equal to the number of possible output states (e.g., 8 submatrices for a 3 -bit binary output). However, in some examples, all inputs and all outputs may beVO1 treated as being independent of one another, and the B tensor 712 may include separate subtensors for the separate inputs, such that the 8 x 8 x 8 B tensor that would ordinarily be required (for 3 outputs and 3 dependent inputs) could be replaced by 3 separate subtensors of size 8 (2 x 2 x 2), each subtensor corresponding to a bidirectional connection with one other node.

[0082] Because the nodes of the network generate their outputs for a given time step T before receiving their neighboring nodes' outputs for time step T, the computations informed by the priors of the initial Bo subtensor 504a are performed after time step T-1 and before time step T. Thus, time step T may either be referred to as the “current” time step (meaning the time step for which the current outputs are being computed) or as the “next” or “future” time step (because the observations at time step T are not yet known).

[0083] During the runtime period, the node uses the prior values of initial Bo subtensor 504a to perform active inference, in which the initial Bo subtensor 504a predicts the state of the observed system, conditioned on the previous observed state of the system and conditioned on the node's own previous output. To the extent that observations diverge from the predicted observations of the second time step inputs 601, the node may adjust the priors of the B submatrices in an attempt to minimize future VFE or EFE resulting from the uncertainty of the predictions. Similarly, if the observations concur with the predicted observations of the second time step inputs 804, the node may adjust the priors of the B subtensors to increase the level of certainty of the expected observations. In this way, the initial Bo subtensor 504a may have its priors adjusted over time to predict the observed system, and thereby differentiate itself from its initial neutral, naive, undifferentiated state.

[0084] In some examples, the node selects its outputs to minimize VFE or EFE, as described above. VFE is a computation based on a generative model (e.g., a set of priors learned by a node or other agent) and observations (e.g., inputs to a node or other agent). VFE provides an upper bound on statistical surprise. There are various mechanisms to minimize VFE, including updating the generative model to more closely match observations, or performing a computation of expected free energy (EFE) across possibleVO1 actions and selecting one of the actions which leads to the lowest expected free energy. Operations involving the computation or minimization of EFE, as referred to herein, may be referred to as a computation or minimization of VFE.

[0085] To minimize EFE, a node may compute an expected distribution of free energy over multiple output states and select an output state that minimizes the expected free energy. The calculation of expected free energy may in some cases be affected by the node’s preferences (e.g., as represented by the C tensor 508), such that rows of the B tensor 512 corresponding to preferred observations of the node are expected to result in lower free energy than non-preferred rows of the B tensor 512. Thus, the node will calculate EFE distributed over all possible output states, then select an output that is at or near the minimum value of the EFE distribution. When operating in a deterministic mode, the node may select the output value that directly minimizes EFE. When operating in a stochastic mode, the node may weight various output values based on their corresponding EFE, or their proximity to the minimum value of the EFE distribution, and stochastically select an output value from the weighted distribution of output values.

[0086] Thus, the node’s preferences (e.g., the values of the parameters in the C tensor 508) may affect the degree to which the node operates to minimize VFE, EFE, or a combination of both VFE and EFE. A node with uniform zero parameter values in the C tensor 508 will have a close relationship between VFE and EFE, whereas a node with non-uniform parameter values in the C tensor 508 will tend to exhibit EFE behavior that diverges from that predicted solely from VFE.

[0087] In some cases, the node may switch between the stochastic mode and the deterministic mode based on the value of a stress parameter of the node that tracks the degree or frequency to which the node has predicted incorrectly over past time steps. The stress parameter may be adjusted downward on time steps when the node’s output predicts its target, and adjusted upward on time steps when the node’s output fails to predict its target. When the stress parameter value rises above a stress threshold, the node switches from the deterministic mode to the stochastic mode, and whenVO1 the stress parameter value falls below the stress threshold, the node switches back from the stochastic mode to the deterministic mode. In some cases, two different stress thresholds may be used, such that a falling stress threshold for switching back to the deterministic mode is lower than a rising stress threshold for switching to the stochastic mode.

[0088] In some examples, the stress threshold (and / or the falling and rising stress thresholds) may be a further parameter stored by the node. In some examples, each stress threshold may be a learned parameter that is adjusted during the learning process to assist in minimizing VFE or EFE, as described herein with reference to adjustment of the priors of the B tensor.

[0089] The prior values of the B tensor 512 may be adjusted over a sequence of time steps to predict outputs of another node or a data source, such as the environment 420 external to the network. After the node makes observations (e.g., inputs are accessed or received by the node) at a given time step, the priors of the node’s B tensor 512 may be adjusted based on the currently observed input state, based on the observed input state at the previous time step, and based on the output generated by the node at the previous time step. As described above, a columnar slice of the B tensor 512 is selected based on the node’s inputs observed and its output generated at the last time step, and a column of priors of one of the B subtensors (e.g., Bo subtensor 504 or Bi subtensor 506) intersected by the columnar slice is selected based on the comparison of the priors of the various individual columns in the columnar slice. The priors of this selected column may then be adjusted in accordance with the currently observed input state, such that a higher prior is assigned to the prior in the row corresponding to the currently observed input state, and all other priors in the row are decreased accordingly.

[0090] In some examples, the degree to which the priors of the column are adjusted is in proportion to the number of observations of each input state over time: thus, for example, if the input state of row 2 (e.g., the row of priors for 1 input at second time step 606 in the column of priors for 0 input at second time step 605) has been observed at 32% of time steps, it will be assigned a prior value of 0.32. In other examples, different degrees ofVO1 adjustment may be used, such as a degree of adjustment proportional to a learning rate that may vary under different conditions.

[0091] In some cases, the node’s preferences may affect the degree to which the priors of the column are adjusted. When a preferred row is observed, in addition to increasing the prior in the observed row and decreasing the other rows accordingly, all of the priors of the column may be adjusted to sharpen the distribution of prior values, such that high prior values are raised and low prior values are lowered, rendering the priors of the column less uniform. In contrast, when a non-preferred row is observed, in addition to increasing the prior in the observed row and decreasing the other rows accordingly, all of the priors of the column may be adjusted to partially flatten the distribution of prior values, such that high prior values are lowered and low prior values are raised, rendering the priors of the column more uniform. The partial flattening of the prior values may result in an increase in the effective learning rate for priors in the column in future time steps, because it is easier for a new observation to stand out relative to other observations when all older observations are relatively uniform. This increase in the effective learning rate after observing a non-preferred row may increase the likelihood or speed with which the node will reconfigure the priors of the column, thereby potentially finding a more optimal set of priors that may assist in avoiding the non-preferred row in the future. By the same token, a slowed effective learning rate after observing a preferred row may reduce the likelihood or speed with which the node will reconfigure the priors of the column, thereby potentially locking the node more tightly into its current configuration.

[0092] FIG. 7 shows the initial Bo subtensor 504a after having undergone differentiation to predict an observed system - in this example, another node - that is configured to employ a “tit for tat” strategy in an iterated prisoner's dilemma task. The tit for tat strategy is one in which an agent will defect (in this example, output a “1”) immediately after an opponent defects, and will cooperate (in this example, output a “0”) immediately after an opponent cooperates. Thus, the output of the observed node is a deterministic function of this node's own previous action: in this example, if the node output a “1” at time step T-1, then the observed node will output a “1” at time step T, andVO1 if the node output a “0” at time step T-1, then the observed node will output a “0” at time step T.

[0093] Thus, after the node has undergone differentiation to predict the observed node employing the tit for tat strategy, the node will adjust the priors of its initial Bo subtensor 504a to the values shown in the differentiated Bo subtensor 702: the priors for 0 input at second time step 605 are all 1, indicating a 100% probability that an output of “0” will result in a next observation of a “0”, and the priors for 1 input at second time step 606 are all 0, indicating a 0% probability that an output of “0” will result in a next observation of a “1”. These prior values are set across both columns, indicating that they apply regardless of the observed state of the system.

[0094] By differentiating nodes to learn to predict their observed systems, some examples described herein may train the nodes of the network to generate inferences about the behavior of the rest of the system including (for environment-facing nodes) the environment outside of the machine learning model.

[0095] FIG. 8 and FIG. 9 show the same process of differentiation to predict a “tit for tat” observed node as the differentiation adjusts the prior values of the Bi subtensor 506. Initial Bi submatrix 506a has the same values as initial Bo subtensor 504a. The first time step inputs 602 are mapped to the second time step inputs 601 using priors split into two columns (priors for 0 input at first time step 703 and priors for 1 input at first time step 704) and two rows (priors for 0 input at second time step 705 and priors for 1 input at second time step 706). The mapping of initial Bi submatrix 506a is conditioned on the output of the node at time step T-1 being “1”.

[0096] As the node differentiates to predict the observed tit for tat strategy of its neighboring node, the initial Bi submatrix 506a adjusts its prior values to those shown in the differentiated Bi submatrix 902: the priors for 0 input at second time step 705 are all 0, indicating a 0% probability that an output of “1” will result in a next observation of a “0”, and the priors for 1 input at second time step 706 are all 1, indicating a 100% probability that an output of “1” will result in a next observation of a “1”. These prior values are setVO1 across both columns, indicating that they apply regardless of the observed state of the system.

[0097] FIG. 10 shows an initial B tensor 512a (equivalent to B tensor 512 shown in FIG. 5), and its potential to differentiate into multiple possible differentiated values depending on the behavior of the observed system. In this example, the observed system is a single node playing an iterated prisoner's dilemma task, as in the examples of FIG. 6 through FIG. 9.

[0098] If the observed node employs a tit for tat strategy, the initial B tensor 512a will differentiate over time into the tit-for-tat differentiated B tensor 512b, whose Bo and Bi submatrices are equivalent to differentiated Bo subtensor 702 and differentiated Bi submatrix 902 shown in FIG. 7 and FIG. 9. The tit-for-tat differentiated B tensor 512b exhibits absolute certainty in its predictions: the action of the node is determinative of the next observed state of the system, due to the nature of the tit for tat strategy.

[0099] If the observed node employs an “always defect” strategy, the initial B tensor 512a will differentiate over time into the defector differentiated B tensor 512c. In an always defect strategy, the observed node always outputs “1”, irrespective of previous outputs and irrespective of inputs. As a result, the node observing the defector node will have complete certainty (priors 0 and 1) about the next observed state: it can predict with 100% confidence that the next observed state will always be “1”. Notably, this prediction is premised on the observed state being “1” as well: the 1 and 0 prior values are only in the right column of the Bo subtensor 504 and Bi subtensor 506. Because the node has never observed a system state of “0”, the priors in the left columns are completely undifferentiated from their initial values of 0.5.

[0100] In the final illustrated example, if the observed node employs a “random” strategy, the initial B tensor 512a will differentiate over time into a form resembling the random differentiated B tensor 512d. In a random strategy, the observed node outputs “0” or “1” randomly, irrespective of previous outputs and irrespective of inputs. As a result, the node observing the random node will remain largely undifferentiated (priors all at approximately 0.5) about the next observed state: it cannot predict the next observed state with any confidence.VO1

[0101] In some examples, a node may initially be generated as an undifferentiated node such as initial B tensor 512a, having a B tensor 512 with completely or largely uniform priors (e.g., 0.5 for all priors for two possible input states, 0.25 for all priors for four possible input states, and so on). Over a sequence of time steps, the parameters of the node (e.g., the priors of the B tensor 512) are adjusted such that the node learns a generative model configured to generate a value of a first output matching an expected value of a first input. In some examples, the node is configured to predict a data source (e.g., another node or the environment 420) that generates the first input, thereby learning to support a behavior (e.g., inference behavior) of the data source. Thus, for example, a data source may perform an inference task or other task based on its own output (designated as the first input to the first node), and first node has an objective function (e.g., the expected reward (|) of the POMDP equation above) configured to cause the first node to learn a generative model such that the first output of the first node contributes to and supports the task performed by the data source.

[0102] In some examples, the objective function may be in part a function of the C tensor 508, such that the first node minimizes its VFE or EFE in part by satisfying its preferences. For example, a first node may be configured to attempt to predict a first data source generating a first input. The first node may also have a self-input such that it can compare its own previous output with a current input: the C tensor 508 is configured to prefer input states in which the self-input (generated at time step t-1 but received at time step t) matches the currently observed first input at time step t. For example, the input state may have four possible values indicated as (selfinput received at time step t, first input received at time step t) = (0, 0)|(0, 1)|(1, 0)|(l, 1). Each input state may correspond to a row of the B tensor 512. The C tensor 508 will therefore be configured to prefer the first and fourth rows (0, 0) and (1, 1) and to not prefer the second and third rows (0, 1) and (1, 0). Thus, the C tensor 508 may be a vector with four values corresponding to these preference values for the four rows of the B tensor 512, such as (0.5, 0, 0, 0.5). In some examples, the C tensor 508 may be less completely configured to favor generation of outputs matching the predictedVO1 inputs from the prediction target, and the C tensor 508 may have more uniform priors in order to allow some exploration of non-preferred states: e.g., the C tensor 508 may have values (0.4, 0.1, 0.1, 0.4). The trade-off between exploration of non-preferred states and exploitation of self-efficacy to induce preferred states corresponds to the balance between a pragmatic term (encouraging exploitation) and an epistemic term (encouraging exploration) described below.

[0103] In some examples, the C tensor 508 may be uniform (e.g.: (0.25, 0.25, 0.25, 0.25)), indicating no preferred observations (e.g., no preferred rows of the B tensor 512), such that the objective function of the first node can be regarded as simply a minimization of surprise or expected surprise by minimizing VFE or EFE by improving the predictive certainty of the B tensor 512. Such a uniform configuration corresponds to a maximized epistemic term and a minimized pragmatic term.

[0104] As the first node learns to predict the data source, such that the first node can reliably generate the first output to match the first input at a given time step, the first node may be said to have learned a generative model configured to configured to generate a value of the first output matching an expected value of the first input. Examples are described herein in which nodes of the network are generated and configured to predict other nodes or the environment 420, such that inference tasks or other tasks performed by the other node or environment 420 are supported by the output of the first node. As used herein, the term "prediction target" refers to a node, environment, or other data source or entity whose outputs the first node is configured to predict and, in some cases, mirror (by generating an identical or similar output at the same time). Similarly, the node configured to predict the other node, environment, or other data source or entity may be referred to herein as a “predicting node”. In some examples, a node may be added to the network such that the newly added node inherits certain characteristics of another existing node: in such cases, the new node may be referred to as the child (or child node) of the existing node, and the existing node may be referred to as the parent (or parent node) of the child node. In some examples, a child node may be configured to support its parent. In some examples, an environment-facing node may be created when the network isVO1 first generated, and this environment-facing node may be referred to as a seed node; seed nodes may be configured in some examples to support the environment 420.

[0105] One potential benefit of some examples described herein is that the adjustment of the parameters (E.g., priors) of the nodes may continue during the runtime period, which may have the benefit of allowing the machine learning model to automatically adapt to changes in the environment 420, as well as changes internal to the network. After a first node has differentiated to learn to predict its prediction target, such that its priors are differentiated into a set of differentiated values (e.g., tit-for-tat differentiated B tensor 512b or defector differentiated B tensor 512c), changes to the observed system (e.g., changes to the prediction target, or changes to the network 400 or the environment 420 that cause changes in the outputs of the prediction target) may result in the first output of the first node no longer reliably matching the first input received from the prediction target. In such cases, it may be beneficial for the first node to reconfigure its priors to allow re-learn to predict the changed outputs of the prediction target.

[0106] In order for the first node to re-learn how to predict its prediction target, a de-differentiation operation may first be performed, in which the priors of the first node are adjusted, over a further sequence of time steps, to a set of de-differentiated values. The priors of the differentiated first node may be partially de-differentiated in some cases: for example, the priors of tit-for-tat differentiated B tensor 512b may be made more uniform by adjusting the 1 values to 0.75 and the 0 values to 0.25, without making them as fully uniform as initial B tensor 512a. In other cases, the priors of the differentiated first node may be substantially or fully de-differentiated: for example, the priors of tit-for-tat differentiated B tensor 512b can be adjusted over the further sequence of time steps to values that resemble initial B tensor 512a or random differentiated B tensor 512d.

[0107] After the first node has been partially or fully de-differentiated, the first node may then perform a re-differentiation operation to attempt to relearn how to predict its prediction target under the changed conditions. Redifferentiation may proceed in substantially the same way as the initialVO1 differentiation operation, such that the priors of the first node are adjusted, over an additional sequence of time steps, from the relatively uniform set of de-differentiated values to a less uniform set of re-differentiated values.

[0108] One mechanism for transitioning a node between differentiation, dedifferentiation, and re-differentiation modes of operation may be the balance between pragmatic and epistemic terms (e.g., using values of the C vector). Another mechanism may be a switch between a stochastic mode and a deterministic mode. As described above, the switch between a stochastic mode and a deterministic mode may be triggered by detecting failure to predict the prediction target.

[0109] Thus, a node may be regarded as having a set of parameter values (e.g., prior values) at a given time step that place it at a position along a gradient from undifferentiated to fully differentiated. The node's position along this gradient may change over time as it learns to predict its prediction target (differentiation), then de-differentiates in response to detecting failure to predict the prediction target, then re-differentiates to re-learn how to predict the prediction target under the changed conditions, and so on. In some examples, a more uniform set of priors or a more undifferentiated node corresponds to a higher effective learning rate. Thus, in some examples, the de-differentiation operation may place the node in a state that makes it more receptive to re-learning the generative model for the changed conditions.

[0110] Returning to FIG. 10, each of the differentiated states 512b, 512c, or 512d of the B tensor 512 of a node presents an example of a different degree of two capabilities of the node: predictive certainty, and self-efficacy.Predictive certainty refers to the degree of certainty in the node's predictions: the tit-for-tat differentiated B tensor 512b exhibits perfect predictive certainty, the defector differentiated B tensor 512c exhibits perfect predictive certainty only under some conditions (it is certain as long as the observed state is “1”, but would be completely uncertain at the next time step if it ever observed a “0”), and the random differentiated B tensor 512d has no predictive certainty.[OHl] The second capability of the node, self-efficacy, refers to a measure of the ability of the output of the node at the previous time step T-1 to affectVO1 the observations of the node at the current time step T. The self-efficacy of a node is high if the state of its previous outputs result in highly certain current observations. Thus, the tit-for-tat differentiated B tensor 512b has very high self-efficacy, because its own action is completely determinative of the next observation, which will always be exactly equal to the node's own previous output. The self-efficacy of the defector differentiated B tensor 512c, by contrast, is very low, even though its predictive certainty is perfect for half of the possible observed previous states; the previous output of the node has no effect on its next observation. Finally, the self-efficacy of the random differentiated B tensor 512d is also very low, because its outputs also have no statistically meaningful effect on later observations.

[0112] Thus, predictive certainty and self-efficacy are separate metrics which may or may not co-occur.

[0113] In nodes having no preferences, such as the initial POMDP tensors described with reference to FIG. 5 above having zero values for the C tensor 508, high self-efficacy maps directly to low VFE or low EFE. However, in examples having preferences, minimizing VFE or EFE may require preferences to be satisfied, which requires not only self-efficacy but specifically self-efficacy with respect to soliciting preferred observations. For example, a node with preference (e.g., with non-zero values for the priors of the C tensor 508) may learn how to reliably solicit a current “1” observation under certain previous output and observation states; however, if the node prefers to observe “0” states of the observed system, and has not learned a way to reliably solicit “0” observations, the self-efficacy of the node will be very low.

[0114] In some examples, the C tensor 508 has values that can be used to guide the determination of VFE or EFE. For example, VFE or EFE can be determined based on a pragmatic term (e.g., "does the node get what is expected?") and an epistemic term (e.g., "does the node learn?"). The C tensor 508 can be used to adjust the balance between the pragmatic term and epistemic term. For example, the C tensor 508 can be set to 0 in some examples described herein, whereby the node is maximizing epistemic gain. In some examples, a higher epistemic term corresponds to weighting theVO1 predictive certainty of a node higher than the self-efficacy of the node within the C tensor 508.

[0115] In some cases, a pragmatic term is used to compute VFE or EFE. The pragmatic term may focus on the practical use or application of a concept, emphasizing its utility in real-world scenarios. The pragmatic term may be emphasized to enable the machine learning model to focus on action, decision-making, and achieving specific outcomes. The pragmatic term may be increased by changing the C tensor 508 such that the machine learning model focuses more on improving accuracy, efficiency, performance, and the like. In some examples, a higher pragmatic term corresponds to weighting the self-efficacy of a node higher than the predictive certainty of the node within the C tensor 508.

[0116] Thus, in some examples, in the POMDP equation above, the reward function (|) may change between assigning a higher weight to predictive certainty (e.g., the degree of certainty indicated by the B tensor 512) and assigning a higher weight to self-efficacy (e.g., the degree to which the policy it is likely to achieve the desired next observation) depending on the relative values of the epistemic term and the pragmatic term.

[0117] In some examples, nodes have internal states, and predictive accuracy could also or alternatively reflect not the ability of a node to correctly predict the output of another node, but instead to predict an internal state of the other node (e.g. whether the other node is operating in a stochastic or a deterministic mode, how strongly the other node holds its priors, etc.).

[0118] In some examples, predictive certainty and self-efficacy may be measured with respect to individual connections of a node. The certainty with which a node can predict inputs received via a given connection may be regarded as the node's predictive certainty with respect to that connection. Similarly, the extent to which a node's outputs via a given connection affect the state of one or more inputs subsequently observed may be regarded as the self-efficacy of that connection. In some examples, self-efficacy is computed for pairs of connections, such that the self-efficacy of a bidirectional pair of connections between a first node and a second nodeVO1 measures the degree to which the first node's output(s) to the second node affect the first node's subsequent observation of the second node's output(s). In some examples, the self-efficacy of a connection from the first node to the second node may instead be a measure of the degree to which the first node's output(s) to the second node affect all observations of the first node with respect to all of its connected nodes.

[0119] In some examples, the self-efficacy of each of a node's connections can be combined to generate a nodal self-efficacy measure. In some examples, the nodal self-efficacy of a node is computed as the sum of the self-efficacy of each of its outbound connections. In other examples, the nodal self-efficacy of a node is computed as an average (such as an arithmetic or geometric mean, a median, or another averaging function), or as some other combination of the self-efficacy measures of each of its outbound connections.

[0120] In some examples, the self-efficacy of a connection can be computed as the mean of the absolute difference between the sub-matrices conditioned on each action, e.g., Bo subtensor 504 and Bi subtensor 506. (For nodes having multiple connections, self-efficacy for a given connection may be computed by isolating the observations and predictions of the B tensor 512 for the connection being measured.) Thus, the self-efficacy for tit-for-tat differentiated B tensor 512b is the maximum value of 1 :Self-efficacy (S) for tit-for-tat differentiated B tensor 512b

[0121] In contrast, the self-efficacy for defector differentiated B tensor 512c is the minimum value of 0:VO1Self-efficacy (S) for defector differentiated B tensor 512c

[0122] In some examples, the network (e.g., network 400) may be restructured over a sequence of time steps. Beginning from a small set of nodes or a single node, the network may grow by adding nodes, shrink by removing nodes, increase connectivity by adding connections, or simplify connectivity by removing connections.

[0123] FIG. 11 illustrates a sequence showing restructuring of a network of a machine learning model over time, according to techniques described herein. The network shown in FIG. 11 adds and removes nodes and connections over time. The connections shown in FIG. 11 may represent a pair of bidirectional connections, as in network 400.

[0124] The top row of FIG. 11, spanning from time to on the left to time ti on the right, begins on the left with a simple network at time to. A single first node 1102 has a connection to the environment, shown as first data source 1104.

[0125] Initially, the first data source 1104 provides input data to the first node 1102 of the machine learning architecture. The first data source 1104 can be any component, database, system, server, computing system, or the like, capable of providing or receiving data to the machine learning architecture. For example, the first data source 1104 may be a sensor that detects real-world information, a database that stores data for processing, a transmission module that receives data from a remote system, or any other component that can supply data to the architecture.

[0126] Moreover, in some cases, the machine learning model is part of or connected to another machine. As such, the output of the machine learning model changes the operation of the other machine. For example, the machine is connected to the first data source 1104, whereby characteristics of the machine are transmitted to the first data source 1104 and responses from theVO1 machine learning model are passed back to the first data source 1104. The responses can affect the operations of the connected machine.

[0127] The improved dynamic architecture of the machine learning model described herein dynamically restructures its internal architecture (e.g., the topology of the network) during runtime based on certain characteristics, such as certain characteristics of the network, environmental influences, system performance requirements and limitations, or changes thereof over time. In some cases, the machine learning model can test the restructured architecture using the same or different characteristics. As such, the machine learning model dynamically modifies its internal architecture to adapt to the needs and requirements of the machine learning model or the environment.

[0128] For example, the external first data source 1104 may be connected to a physical real-world machine (such as a robot) that sends sensory inputs into the machine learning model and adapts its movements and actions based on output of the machine learning model. Machine learning models can be connected to self-driving cars, enabling the machine learning model to interpret sensor data, make real-time decisions, and send responses to selfdriving cars on changing road conditions.

[0129] In some cases, machine learning models can be connected to a wide range of Internet of Things (loT) devices, including cameras, thermometers, and seismographs, enabling smart and adaptive automation in smart homes, cities, and industries.

[0130] In some cases, machine learning models can be connected to recommendation engines, providing personalized suggestions for products, content, or services. Machine learning models can be connected to computer vision applications, enabling object detection, facial recognition, and image understanding. Machine learning models can be connected to and providing instructions to gaming environments, providing adaptive gameplay, opponent Al, and personalized experiences.

[0131] The machine learning models described herein can interact with other systems and machines, and the model adapts to its new or changing environment dynamically by restructuring its internal architecture duringVO1 runtime. The model can adapt its structure to better suit new data or tasks by adding / pruning nodes and layers.

[0132] In some cases, the model reduces its size and computational requirements by pruning redundant or unnecessary nodes and connections. This allows the model to run faster and on lower power budgets. The model can grow certain areas such as adding nodes to a layer to specialize on a complex task while shrinking other areas. This lets the model focus computation where more attention needed.

[0133] In some cases, the model can route around failed nodes or layers by restructuring the internal architecture of the network. This makes the system more robust and stable in the face of hardware and network faults or noise. In some cases, the model can rapidly adapt to new tasks by transferring and reusing previously learned features and nodes.

[0134] In some cases, the model dynamically grows areas and prunes others, reflecting the demands of the current task. The model can compress unimportant nodes and connections to reduce storage footprint and memory requirements. The model can be personalized for different users by learning unique sub-structures that specialize to each user.

[0135] After initialization, the first node 1102 may be configured to attempt to predict the first data source 1104, such that, over time, the parameters (e.g., priors) or first node 1102 are adjusted, as described above, to differentiate first node 1102 to learn a generative model matching the output (e.g., first output) of first node 1102 to the input (e.g., first input) received from first data source 1104. Thus, for example, if the first node 1102 is configured to play an iterated prisoner's dilemma game with first data source 1104, and the first data source 1104 is configured to always defect, then the B tensor 512 of first node 1102 will eventually differentiate to the form of defector differentiated B tensor 512c. However, if the first data source 1104 later changes its behavior and begins to exhibit a tit for tat strategy, then the first node 1102 may begin to fail to predict with the first data source 1104 due to its priors in the form of defector differentiated B tensor 512c. In response to detecting such failure to predict, the Al system may initiate the de-differentiation operation for first node 1102, e.g., byVO1 adjusting its C tensor 508 to emphasize the epistemic term relative to the pragmatic term, or by switching the first node 1102 from a deterministic mode into a stochastic mode. After the first node 1102 partially or fully dedifferentiates, the first node 1102 may begin the re-differentiation operation, eventually re-learning to predict the first data source 1104 with priors approximating those of the tit-for-tat differentiated B tensor 512b.

[0136] During any of these operations of differentiation, de-differentiation, or re-differentiation, and in response to processing of inputs to generate outputs and inferences during the runtime period, the network may be restructured by adding two nodes, second node 1106 and third node 1108. The three nodes 1102, 1106, 1108 are all interconnected with each other, thereby form a first cluster 1110 of nodes in the network. Each added node may be configured to support an existing node of the first cluster 1110: for example, the first node 1102 may spawn the second node 1106, such that second node 1106 is the child of first node 1102 and is configured to support first node 1102 by mirroring the output of first node 1102. The third node 1108 may be spawned by either first node 1102 or second node 1106 and may be configured to mirror and support its parent node.

[0137] As time progresses further, just before time ti on the right side of the top row, the network may be further restructured to add another set of nodes (node 1114 and nodes 1116) and connections to form a second cluster 1112. The second node 1106 of the first cluster 1110 has a connection to a node 1114 of the second cluster 1112, potentially indicating that node 1114 is the child of second node 1106. Second node 1106 and node 1114 may be considered to be interface nodes providing inputs and outputs between the first cluster 1110 and second cluster 1112.

[0138] The bottom row of FIG. 11 spans from time ti on the left to time t2 on the right. On the left side of the bottom row, after time ti, the second cluster 1112 of the network is restructured to remove one of its nodes 1116.

[0139] On the right side of the bottom row, just before time t2, the second cluster 1112 is restructured again to add a further node 1116. It will be noted that, although the second cluster 1112 just before time ti and the second cluster 1112 just before time t2 each have five nodes, the pattern ofVO1 interconnection of the nodes of the second cluster 1112 is different at the two different times. Furthermore, it may be the case that the parameters (e.g., the priors of the various tensors) of each node may be different over time.

[0140] In the various network configurations shown in FIG. 11, the various nodes may support each other's operations, and ultimately support a task of the first data source 1104 (such as an inference task or a task performed by a second machine). Nodes within a cluster may support the interface nodes of the cluster to learn to predict the interface nodes with other clusters or other data sources. The learned prediction and mirroring of the interface nodes (or environment-facing nodes) may support the operations of the interconnected clusters or the environment 420 (e.g., first data source 1104). In some cases, the network may be rendered more accurate or more efficient by pruning nodes or connections between nodes; in other cases, accuracy may be improved by adding nodes or connecting previously-unconnected nodes. In some examples, added nodes may be connected to the environment 420 to improve support for the task performed by an external data source or other external entity.

[0141] By providing a network of a machine learning model that can be dynamically restructured during runtime, while performing an inference task for which it has been trained, an Al system may be provided that provides one or more benefits. The Al system may use a combined order of specific procedures that dynamically alters the structure of the machine learning model in real time and during inference. In response to certain input data (e.g., data from the environment 420), the Al system may both restructure the architecture of the machine learning model and perform an inference task on the restructured architecture. Not only do some examples and features of the Al system described herein eliminate the need for a separate training process from the inference process, but the dynamic nature of the Al system is different than the nature of traditional machine learning systems. The Al system does not simply automate traditional systems and concepts: instead, by leveraging dynamic architectural restructuring (e.g., by adding or removing connections, cycles, or nodes), the Al system naturally grows and prunes based on certain characteristics, such as certain characteristics of theVO1 network, environmental influences, system performance requirements and limitations, or changes thereof over time.

[0142] FIG. 12 illustrates an example method 1200 of adjusting a generative model of a node of a machine learning model to predict another entity. Although the example method depicts a particular sequence of operations, the sequence may be altered without departing from the scope of the present disclosure. For example, some of the operations depicted may be performed in parallel or in a different sequence that does not materially affect the function of the method. In other examples, different components of an example device or system that implements the method may perform functions at substantially the same time or in a specific sequence.

[0143] The method 1200 can be implemented by one or more processors executing instructions stored in a tangible computer readable storage medium. The tangible computer readable storage medium may also be used to store and retrieve the machine learning model.

[0144] The method 1200 will be described with reference to the example network 400 of FIG. 4. However, it will be appreciated that the method 1200 can be applied to various types of networks used in machine learning models, such as the examples described above with reference to FIG. 1 through FIG. 3.

[0145] Thus, in some examples, the machine learning model includes a network having a plurality of nodes and a plurality of connections between the nodes. Each node is configured to process one or more inputs to generate one or more outputs, and each connection is associated with a transmitting node and a receiving node and is configured to convey an output of the transmitting node as an input to the receiving node, at each time step of a sequence of discrete time steps. The network as a whole operates to generate inference data; for example, the environment 420 may receive outputs of the network as inference data, or the outputs of the network to the environment 420 may be further post-processed by other components of the machine learning model or an Al system using the machine learning model to generate the inference data. In some examples, the inference data may be generated on a slower time scale than the nodes of the network: for example,VO1 inference data may only be generated at every k time steps, or only when the network has generated certain patterns or quantities of output data. The inference data includes, or may be further processed to generate, an inference.

[0146] In some examples, the machine learning model may be at least partially trained to perform an inference task during a training period. During the training period, the machine learning model may be trained to perform the inference task by processing samples of reference data and applying a machine learning algorithm to train the machine learning model, as described above with reference to FIG. 1 and FIG. 2. The training of the network may not be complete: for example, only an initial training phase 204 may be performed, without refinement or retraining. In some examples, only a limited set of samples of reference data are used to partially train the machine learning model, after which the partially trained machine learning model is deployed to perform inference on candidate data during a runtime period. In some such examples, further training may be performed during or after the runtime period, using further reference data or candidate data.

[0147] The instructions, when executed by the one or more processors, may cause the one or more processors to perform the operations shown in FIG.12. The operations will be described as being performed by an Al system implemented by the instructions executed by the one or more processors.

[0148] At operation 1201, the Al system accesses the machine learning model. As described above, the machine learning model includes a plurality of nodes, and each node includes a generative model configured to control generation of outputs of the node based on inputs of the node. Each input of a node corresponds to a respective data source.

[0149] The Al system then begins operating the machine learning model and its network over a sequence of time steps. Each of operations 1202, 1203, 1204, and 1206 (as well as potentially operation 1207) may be performed during each time step of the sequence of time steps.

[0150] In some examples, the first node is added to the network as a child node of a parent node. In some such cases, the child node (e.g., the first node) is configured to learn to predict to its parent node. The parent nodeVO1 may be considered the prediction target, and the child node (e.g., the first node) may be considered the predicting node. An example method 1300 for a parent node to spawn a child node and add the child node to the network is described below with reference to FIG. 13.

[0151] At operation 1202, the Al system accesses inputs of the first node, including a first input to the first node corresponding to a first data source, as well as a second input corresponding to a second node of the plurality of nodes. As described above, the first data source may be another node or an entity external to the network, e.g., represented as environment 420. The second node has a second generative model that is different from the first generative model, thereby ensuring that the first node is provided at least one input that provides additional information about the operation of the network outside of the context of simply the first node and its prediction target. This additional information may ensure that the first node is able to keep its priors configured to predict other local behaviors in the network.

[0152] At operation 1203, the Al system adjusts a first generative model of the first node in accordance with an objective function. In some examples, the objective function is configured to decrease an expected difference between a first output to be generated by the first node (e.g., at operation 1204 below) and a future first input received by the first node from the first data source. The objective function may be a reward function, such as expected reward (]> set out in the POMDP equation above. In some examples, the objective function may be a consequence of the configuration of the C tensor 508, and may be regarded as representing the first node's preferences. Thus, in some examples, the objective function may itself be a function of adjustable parameters of the node. In some examples, the node's preferences may be limited to improvement of predictive certainty through the minimization of VFE: this may be the case, for example, when the C tensor 508 is populated with uniform priors, such that the epistemic term is maximized and the pragmatic term is minimized. In some examples, the node may also seek to minimize EFE to at least some degree as a result of non-uniform priors in the C tensor 508; nodes having preferred observations due to non-uniform priors of the C tensor 508 may seek to exploit their predictive certainty to solicit the preferred observations, thereby exhibitingVO1 and leveraging self-efficacy. A node seeking to predict another data source may in some cases exhibit a minimal pragmatic term, seeking only to resolve uncertainty in its predictions (e.g., by reducing the uncertainty in the priors of the B tensor 512) . For example, as described above, the first node may have priors of the C tensor 508 that cause it to prefer to observe input states (0, 0) and (1, 1) over input states (0, 1) or (1, 0) where the two inputs are (self-input, first input).

[0153] By generating a first output that mirrors the first input generated by the first data source, the first node may be said to support the behavior of the first data source. In some cases, the first data sources may generate the first input in further support of an inference task; in other cases, the first data source may itself perform an inference task.

[0154] In some examples, each of the first input and first output has a corresponding binary value (e.g., 0 or 1). At operation 1203, the first generative model may be adjusted to decrease an expected probability that the first input and first output will have different binary values. In other words, the likelihood that the first output will match the first input is increased through the learning effected by operation 1203.

[0155] In some examples, the first generative model includes a prediction parameter tensor such as B tensor 512. The prediction parameter tensor may include a probabilistic mapping of the inputs of the first node at a first time step (e.g., t-1) to a predicted set of inputs of the first node at a subsequent time step (e.g., t) subsequent to the first time step in a sequence of time steps. The adjusting of the first generative model at operation 1203 may include adjusting one or more parameters of the prediction parameter tensor, e.g., the priors of B tensor 512 may be updated during operation 1203, as described above.

[0156] At operation 1204, the Al system generates the first output based on the first input and the first generative model, as described above. The generation of the first output may in some cases also be based on one or more additional inputs of the first node, each input corresponding to a respective additional data source, such as another node or the environment 420.VO1

[0157] In some examples, the first node is itself supported by one or more other nodes of the network. In some cases, a first additional input to the first node is generated by a first additional node. The first additional node is configured to generate a value for the first additional input to match an expected value of the first output of the first node: in other words, the first additional node learns to predict some signal or property of the first node, which may enable the first additional node to mirror a signal of the first node. In some examples, the first node may be supported by multiple additional nodes of the network that have learned to predict (or in the process of learning to predict ) some signal or property of the first node.

[0158] At operation 1206, the Al system uses the first output as an input to the data source. The bi-directional pair of connections between nodes means that the first node may operate according to active inference, in which not only are the outputs of the first data source provided as inputs to the first node, but also vice-versa. By supporting the operation of the first data source, the first node may supply the first output to the first data source to improve the performance of the first data source in performing an inference task or supporting another entity, directly or indirectly, in performing an inference task. In neurological networks, groups of neurons may fire in synchrony to signal or create a shift in behavior of the system as a whole; in some examples described herein, groups or pairs of nodes may also learn to predict and mirror each other such that they generate identical or similar outputs in synchrony (e.g., at the same subset of time steps), and this synchronous activation of the nodes may result in an inference being generated by the system as a whole, as described at operation 1207.

[0159] At operation 1207, the Al system generates an inference using the machine learning model. The inference is based on the first output. In some examples, the synchronous output of identical outputs at the same time steps by the first node and first data source are effective to create or support the creation of inference data by the network. The inference data may be conveyed to the environment 420 by one or more environment-facing nodes. The environment 420 or another aspect or component of the machine learning model may include further processes to post-process the inference data to output an inference from the machine learning model, as a result ofVO1 the simultaneous generation, at a given time step or set of time steps, of identical outputs of the first node and first data source (as well as, optionally, one or more additional nodes or other data sources).

[0160] In some examples, the first node is an environment-facing node, and the first data source is an entity outside of the network. In some cases, the first data source may be configured to perform a task (e.g., generate the inference) based on (or in response to) the first output of the first node.

[0161] FIG. 13 illustrates an example method 1300 for adding a child node to a network of a machine learning model. Although the example method depicts a particular sequence of operations, the sequence may be altered without departing from the scope of the present disclosure. For example, some of the operations depicted may be performed in parallel or in a different sequence that does not materially affect the function of the method. In other examples, different components of an example device or system that implements the method may perform functions at substantially the same time or in a specific sequence.

[0162] The method 1300 can be implemented by one or more processors executing instructions stored in a tangible computer readable storage medium. The tangible computer readable storage medium may also be used to store and retrieve the machine learning model.

[0163] The method 1300 will be described with reference to the example network 400 of FIG. 4. However, it will be appreciated that the method 1300 can be applied to various types of networks used in machine learning models, such as the examples described above with reference to FIG. 1 through FIG. 3.

[0164] At operation 1301, an existing node of the network, referred to as a parent node, spawns a child node. The child node is added to the network. The child node has a connection to the parent node. In some examples, the child node has a set of preferences (e.g., an objective function) that cause it to attempt to predict and mirror the parent node.

[0165] At operation 1302, the child node may inherit one or more other connections from its parent node. In some examples, the child node inherits all of the parent node's connections. In some examples, the child nodeVO1 inherits only a subset of the connections of the parent node. In some examples, the child node is added to the network with only a single connection, to the parent node.

[0166] At operation 1303, the child node may inherit the generative model of its parent node. In some examples, the child node inherits all of the parent node's parameter values and the configurations of the parent node's parameter tensors. In some examples, the child node inherits only a subset of the parameter values of the parent node. For example, a child node may have different preferences from the parent node, insofar as the child node seeks to predict the parent node, whereas the parent node may seek to predict another data source. Furthermore, the parent node may have certain parameters (such as an age parameter indicating a number of time step for which the parent node has been operating) that are set to an initialized value in the child node that is different from the value of the parameter of the parent node (e.g., the age of the child node is set to an initialized value of 0). In some examples, the parameters of the parent node may be reconfigured when the child node is spawned: for example, the dimensionality of the parameter tensors (e.g., the A tensor 502, B tensor 512, C tensor 508, and D tensor 510) of the parent node may increase due to the added connection to the newly-added child node. In such cases, the child node may be spawned such that the parameters of the child node differ from those of the parent node at least with regard to the dimensionality of the B tensor 512 and C tensor 508 based on the different number of connections of the parent node and child node. However, the parameters that are copied form the parent node to the child node may include priors of the B tensor 512 that reflect a degree of predictive certainty that the parent node has developed with respect to its other connected nodes, such that the child node also inherits the parent node's learned predictive certainty.

[0167] Thus, when a parent node spawns a child node, the child node may inherit one or more characteristics of the parent node, including connections, parameters of the generative model of the parent node, or both.

[0168] At operation 1304, the child node seeks to match a first input received from the parent node with a first output of the child node. ThisVO1 mirroring may be achieved according to the techniques described above, e.g., with reference to method 1200.

[0169] FIG. 14 is a diagrammatic representation of the machine 1400 within which instructions 1402 (e.g., software, a program, an application, an applet, an app, or other executable code) for causing the machine 1400 to perform any one or more of the methodologies discussed herein may be executed. For example, the instructions 1402 may cause the machine 1400 to execute any one or more of the methods described herein. The instructions 1402 transform the general, non-programmed machine 1400 into a particular machine 1400 programmed to carry out the described and illustrated functions in the manner described. The machine 1400 may operate as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1400 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1400 may comprise, but not be limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular telephone, a smartphone, a mobile device, a wearable device (e.g., a smartwatch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions 1402, sequentially or otherwise, that specify actions to be taken by the machine 1400. Further, while a single machine 1400 is illustrated, the term “machine" shall also be taken to include a collection of machines that individually or jointly execute the instructions 1402 to perform any one or more of the methodologies discussed herein. In some examples, the machine 1400 may comprise both client and server systems, with certain operations of a particular method or algorithm being performed on the server-side and with certain operations of the particular method or algorithm being performed on the client-side.

[0170] The machine 1400 may include processors 1404, memory 1406, and input / output I / O components 1408, which may be configured to communicate with each other via a bus 1410. In an example, the processorsVO11404 (e.g., a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) Processor, a Complex Instruction Set Computing (CISC) Processor, a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Radio- Frequency Integrated Circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 1412 and a processor 1414 that execute the instructions 1402. The term "processor" is intended to include multi-core processors that may comprise two or more independent processors (sometimes referred to as “cores”) that may execute instructions contemporaneously. Although FIG. 14 shows multiple processors 1404, the machine 1400 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiples cores, or any combination thereof.

[0171] The memory 1406 includes a main memory 1416, a static memory 1418, and a storage unit 1420, both accessible to the processors 1404 via the bus 1410. The main memory 1406, the static memory 1418, and storage unit 1420 store the instructions 1402 embodying any one or more of the methodologies or functions described herein. The instructions 1402 may also reside, completely or partially, within the main memory 1416, within the static memory 1418, within machine-readable medium 1422 within the storage unit 1420, within at least one of the processors 1404 (e.g., within the processor’s cache memory), or any suitable combination thereof, during execution thereof by the machine 1400.

[0172] The I / O components 1408 may include a wide variety of components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I / O components 1408 that are included in a particular machine will depend on the type of machine. For example, portable machines such as mobile phones may include a touch input device or other such input mechanisms, while a headless server machine will likely not include such a touch input device. It will be appreciated that the I / O components 1408 may include many other components that are not shown in FIG. 14. In various examples, the I / O components 1408 may include user output componentsVO11424 and user input components 1426. The user output components 1424 may include visual components (e.g., a display such as a plasma display panel (PDP), a light-emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), haptic components (e.g., a vibratory motor, resistance mechanisms), other signal generators, and so forth. The user input components 1426 may include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo- optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or another pointing instrument), tactile input components (e.g., a physical button, a touch screen that provides location and force of touches or touch gestures, or other tactile input components), audio input components (e.g., a microphone), and the like.

[0173] Communication may be implemented using a wide variety of technologies. The I / O components 1408 further include communication components 1428 operable to couple the machine 1400 to a network 1430 or devices 1432 via respective coupling or connections. For example, the communication components 1428 may include a network interface component or another suitable device to interface with the network 1430. In further examples, the communication components 1428 may include wired communication components, wireless communication components, cellular communication components, Near Field Communication (NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and other communication components to provide communication via other modalities. The devices 1432 may be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a USB).

[0174] Moreover, the communication components 1428 may detect identifiers or include components operable to detect identifiers. For example, the communication components 1428 may include Radio Frequency Identification (RFID) tag reader components, NFC smart tag detection components, optical reader components (e.g., an optical sensor to detect onedimensional bar codes such as Universal Product Code (UPC) bar code,VO1 multi-dimensional bar codes such as Quick Response (QR) code, Aztec code, Data Matrix, Dataglyph™, MaxiCode, PDF417, Ultra Code, UCC RSS- 2D bar code, and other optical codes), or acoustic detection components (e.g., microphones to identify tagged audio signals). In addition, a variety of information may be derived via the communication components 1428, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi® signal triangulation, location via detecting an NFC beacon signal that may indicate a particular location, and so forth.

[0175] The various memories (e.g., main memory 1416, static memory 1418, and memory of the processors 1404) and storage unit 1420 may store one or more sets of instructions and data structures (e.g., software) embodying or used by any one or more of the methodologies or functions described herein. These instructions (e.g., the instructions 1402), when executed by processors 1404, cause various operations to implement the disclosed examples.

[0176] The instructions 1402 may be transmitted or received over the network 1430, using a transmission medium, via a network interface device (e.g., a network interface component included in the communication components 1428) and using any one of several well-known transfer protocols (e.g., hypertext transfer protocol (HTTP)). Similarly, the instructions 1402 may be transmitted or received using a transmission medium via a coupling (e.g., a peer-to-peer coupling) to the devices 1432.

[0177] Various examples described herein may provide systems, methods, and computer-readable storage media for adjusting a generative model of a node of a machine learning model to predict another entity. Various potential benefits may be realized by various examples described herein, as described throughout this specification.

[0178] Other technical features may be readily apparent to one skilled in the art from the figures, descriptions, and claims.

[0179] Example l is a method comprising: accessing a machine learning model, the machine learning model comprising a plurality of nodes; accessing a first input to the first node, the first input corresponding to a first data source; accessing a second input to the first node, the second inputVO1 corresponding to a second node of the plurality of nodes, the second node having a second generative model different from the first generative model; adjusting a first generative model of the first node in accordance with an objective function, the first generative model being configured to control generation of outputs of the node based on inputs of the node, the objective function being configured to decrease an expected difference between: a future first input of the first node; and a first output generated by the first node; generating the first output based on the first input, the second input, and the first generative model; using the first output as an input to the first data source; and generating an inference using the machine learning model, the inference being based on the first output.

[0180] In Example 2, the subject matter of Example 1 includes, wherein: the first input and first output each has a corresponding binary value; and the decreasing of the expected difference between the first output and the first input comprises: decreasing an expected probability that the first input and first output will have different binary values.

[0181] In Example 3, the subject matter of Examples 1-2 includes, wherein: the generating of the first output comprises generating the first output based on the first input, the second input, one or more additional inputs to the first node, and the first generative model; and each input of the one or more additional inputs to the first node corresponds to a corresponding additional node of the plurality of nodes.

[0182] In Example 4, the subject matter of Examples 1-3 includes, wherein: the second generative model is configured to generate a value for the second input to match an expected value of the first output of the first node.

[0183] In Example 5, the subject matter of Examples 1-4 includes, wherein: the first data source is an external data source outside of the plurality of nodes.

[0184] In Example 6, the subject matter of Examples 1-5 includes, wherein: the first data source is a third node of the plurality of nodes.

[0185] In Example 7, the subject matter of Example 6 includes, prior to the accessing of the first input to the first node: generating the first node, theVO1 first node having one or more characteristics identical to a corresponding one or more characteristics of the third node, the corresponding one or more characteristics including one or more of a connection of the third node to a second data source, the connection designating an input of the third node corresponding to the second data source; and a generative model of the third node; and adding the first node to the plurality of nodes.

[0186] In Example 8, the subject matter of Examples 1-7 includes, wherein: the first generative model includes: a prediction parameter tensor comprising a probabilistic mapping of the inputs of the first node at a first time step to a predicted set of inputs of the first node at a subsequent time step, the subsequent time step being subsequent to the first time step in a sequence of time steps; and the adjusting of the first generative model comprises adjusting one or more parameters of the prediction parameter tensor.

[0187] In Example 9, the subject matter of Examples 1-8 includes, wherein: the accessing of the first input, the accessing of the second input, the adjusting of the first generative model, and the generating of the first output are performed in association with each time step in a sequence of time steps; and the generating of the inference using the machine learning model is performed based on the first output corresponding to each time step in the sequence of time steps.

[0188] In Example 10, the subject matter of Examples 1-9 includes, wherein: the first data source is an entity outside of the plurality of nodes; and the entity is configured to perform a task based on the first output of the first node.

[0189] Example 11 is a system comprising: one or more processors; and a non-transitory computer readable storage medium storing: a machine learning model, comprising a plurality of nodes; and instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: accessing a first input to the first node, the first input corresponding to a first data source; accessing a second input to the first node, the second input corresponding to a second node of the plurality of nodes, the second node having a second generative modelVO1 different from the first generative model; adjusting a first generative model of the first node in accordance with an objective function, the first generative model being configured to control generation of outputs of the node based on inputs of the node, the objective function being configured to decrease an expected difference between: a future first input of the first node; and a first output generated by the first node; generating the first output based on the first input, the second input, and the first generative model; using the first output as an input to the first data source; and generating an inference using the machine learning model, the inference being based on the first output.

[0190] In Example 12, the subject matter of Example 11 includes, wherein: the first input and first output each has a corresponding binary value; and the decreasing of the expected difference between the first output and the first input comprises: decreasing an expected probability that the first input and first output will have different binary values.

[0191] In Example 13, the subject matter of Examples 11-12 includes, wherein: the generating of the first output comprises generating the first output based on the first input, the second input, one or more additional inputs to the first node, and the first generative model; and each input of the one or more additional inputs to the first node corresponds to a corresponding additional node of the plurality of nodes.

[0192] In Example 14, the subject matter of Examples 11-13 includes, wherein: the second generative model is configured to generate a value for the second input to match an expected value of the first output of the first node.

[0193] In Example 15, the subject matter of Examples 11-14 includes, wherein: the first data source is an external data source outside of the plurality of nodes.

[0194] In Example 16, the subject matter of Examples 11-15 includes, wherein: the first data source is a third node of the plurality of nodes.

[0195] In Example 17, the subject matter of Example 16 includes, wherein the operations further comprise: prior to the accessing of the first input to the first node: generating the first node, the first node having one or more characteristics identical to a corresponding one or more characteristics of theVO1 third node, the corresponding one or more characteristics including one or more of: a connection of the third node to a second data source, the connection designating an input of the third node corresponding to the second data source; and a generative model of the third node; and adding the first node to the plurality of nodes.

[0196] In Example 18, the subject matter of Examples 11-17 includes, wherein: the first generative model includes: a prediction parameter tensor comprising a probabilistic mapping of the inputs of the first node at a first time step to a predicted set of inputs of the first node at a subsequent time step, the subsequent time step being subsequent to the first time step in a sequence of time steps; and the adjusting of the first generative model comprises adjusting one or more parameters of the prediction parameter tensor.

[0197] In Example 19, the subject matter of Examples 11-18 includes, wherein: the accessing of the first input, the accessing of the second input, the adjusting of the first generative model, and the generating of the first output are performed in association with each time step in a sequence of time steps; and the generating of the inference using the machine learning model is performed based on the first output corresponding to each time step in the sequence of time steps.

[0198] Example 20 is a non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a system, cause the system to perform operations comprising: accessing a machine learning model, the machine learning model comprising a plurality of nodes; accessing a first input to the first node, the first input corresponding to a first data source; accessing a second input to the first node, the second input corresponding to a second node of the plurality of nodes, the second node having a second generative model different from the first generative model; adjusting a first generative model of the first node in accordance with an objective function, the first generative model being configured to control generation of outputs of the node based on inputs of the node, the objective function being configured to decrease an expected difference between: a future first input of the first node; and a first outputVO1 generated by the first node; generating the first output based on the first input, the second input, and the first generative model; using the first output as an input to the first data source; and generating an inference using the machine learning model, the inference being based on the first output.

[0199] Example 21 is at least one machine-readable medium including instructions that, when executed by processing circuitry, cause the processing circuitry to perform operations to implement of any of Examples 1-20.

[0200] Example 22 is an apparatus comprising means to implement of any of Examples 1-20.

[0201] Example 23 is a system to implement of any of Examples 1-20.

[0202] Example 24 is a method to implement of any of Examples 1-20.

[0203] Unless the context clearly requires otherwise, throughout the description and the claims, the words “comprise,” “comprising,” and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense, i.e., in the sense of “including, but not limited to.” As used herein, the terms “connected,” “coupled,” or any variant thereof means any connection or coupling, either direct or indirect, between two or more elements; the coupling or connection between the elements can be physical, logical, or a combination thereof. Additionally, the words “herein,” “above,” “below,” and words of similar import, when used in this application, refer to this application as a whole and not to any particular portions of this application. Where the context permits, words using the singular or plural number may also include the plural or singular number respectively. The word “or” in reference to a list of two or more items, covers all of the following interpretations of the word: any one of the items in the list, all of the items in the list, and any combination of the items in the list. Likewise, the term “and / or” in reference to a list of two or more items, covers all of the following interpretations of the word: any one of the items in the list, all of the items in the list, and any combination of the items in the list.

[0204] A “non-transitory” machine-readable medium, such as a “non- transitory” computer readable storage medium, as used herein, specifically excludes propagating signals per se. A non-transitory machine-readableVO1 medium may also be referred to herein as a tangible machine-readable medium. According to various example embodiments, instructions for execution by a machine, and other information such as machine learning models and the parameters thereof, can be communicated via a carrier medium (e.g., a machine-readable carrier medium). Examples of such a carrier medium include a non-transient carrier medium (e.g., a non-transitory machine-readable storage medium, such as a solid-state memory that is physically movable from one place to another place) and a transient carrier medium (e.g., a carrier wave or other propagating signal that communicates instructions or information).

[0205] Although some examples, e.g., those depicted in the drawings, include a particular sequence of operations, the sequence may be altered without departing from the scope of the present disclosure. For example, some of the operations depicted may be performed in parallel or in a different sequence that does not materially affect the functions as described in the examples. In other examples, different components of an example device or system that implements an example method may perform functions at substantially the same time or in a specific sequence.

[0206] The various features, steps, and processes described herein may be used independently of one another, or may be combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of this disclosure. In addition, certain method or process blocks may be omitted in some implementations.

Claims

CLAIMSWhat is claimed is:

1. A method comprising: accessing a machine learning model, the machine learning model comprising a plurality of nodes; accessing a first input to the first node, the first input corresponding to a first data source; accessing a second input to the first node, the second input corresponding to a second node of the plurality of nodes, the second node having a second generative model different from the first generative model; adjusting a first generative model of the first node in accordance with an objective function, the first generative model being configured to control generation of outputs of the node based on inputs of the node, the objective function being configured to decrease an expected difference between: a future first input of the first node; and a first output generated by the first node; generating the first output based on the first input, the second input, and the first generative model; using the first output as an input to the first data source; and generating an inference using the machine learning model, the inference being based on the first output.

2. The method of claim 1, wherein: the first input and first output each has a corresponding binary value; and the decreasing of the expected difference between the first output and the first input comprises: decreasing an expected probability that the first input and first output will have different binary values.

3. The method of claim 1, wherein: the generating of the first output comprises generating the first output based on the first input, the second input, one or more additional inputs to the first node, and the first generative model; andeach input of the one or more additional inputs to the first node corresponds to a corresponding additional node of the plurality of nodes.

4. The method of claim 1, wherein: the second generative model is configured to generate a value for the second input to match an expected value of the first output of the first node.

5. The method of claim 1, wherein: the first data source is an external data source outside of the plurality of nodes.

6. The method of claim 1, wherein: the first data source is a third node of the plurality of nodes.

7. The method of claim 6, further comprising: prior to the accessing of the first input to the first node: generating the first node, the first node having one or more characteristics identical to a corresponding one or more characteristics of the third node, the corresponding one or more characteristics including one or more of: a connection of the third node to a second data source, the connection designating an input of the third node corresponding to the second data source; and a generative model of the third node; and adding the first node to the plurality of nodes.

8. The method of claim 1, wherein: the first generative model includes: a prediction parameter tensor comprising a probabilistic mapping of the inputs of the first node at a first time step to a predicted set of inputs of the first node at a subsequent time step, the subsequent time step being subsequent to the first time step in a sequence of time steps; and the adjusting of the first generative model comprises adjusting one or more parameters of the prediction parameter tensor.VO19. The method of claim 1, wherein: the accessing of the first input, the accessing of the second input, the adjusting of the first generative model, and the generating of the first output are performed in association with each time step in a sequence of time steps; and the generating of the inference using the machine learning model is performed based on the first output corresponding to each time step in the sequence of time steps.

10. The method of claim 1, wherein: the first data source is an entity outside of the plurality of nodes; and the entity is configured to perform a task based on the first output of the first node.

11. A system comprising: one or more processors; and a non-transitory computer readable storage medium storing: a machine learning model, comprising a plurality of nodes; and instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: accessing a first input to the first node, the first input corresponding to a first data source; accessing a second input to the first node, the second input corresponding to a second node of the plurality of nodes, the second node having a second generative model different from the first generative model; adjusting a first generative model of the first node in accordance with an objective function, the first generative model being configured to control generation of outputs of the node based on inputs of the node, the objective function being configured to decrease an expected difference between: a future first input of the first node; and a first output generated by the first node;VO1 generating the first output based on the first input, the second input, and the first generative model; using the first output as an input to the first data source; and generating an inference using the machine learning model, the inference being based on the first output.

12. The system of claim 11, wherein: the first input and first output each has a corresponding binary value; and the decreasing of the expected difference between the first output and the first input comprises: decreasing an expected probability that the first input and first output will have different binary values.

13. The system of claim 11, wherein: the generating of the first output comprises generating the first output based on the first input, the second input, one or more additional inputs to the first node, and the first generative model; and each input of the one or more additional inputs to the first node corresponds to a corresponding additional node of the plurality of nodes.

14. The system of claim 11, wherein: the second generative model is configured to generate a value for the second input to match an expected value of the first output of the first node.

15. The system of claim 11, wherein: the first data source is an external data source outside of the plurality of nodes.

16. The system of claim 11, wherein: the first data source is a third node of the plurality of nodes.

17. The system of claim 16, wherein the operations further comprise: prior to the accessing of the first input to the first node:VO1 generating the first node, the first node having one or more characteristics identical to a corresponding one or more characteristics of the third node, the corresponding one or more characteristics including one or more of a connection of the third node to a second data source, the connection designating an input of the third node corresponding to the second data source; and a generative model of the third node; and adding the first node to the plurality of nodes.

18. The system of claim 11, wherein: the first generative model includes: a prediction parameter tensor comprising a probabilistic mapping of the inputs of the first node at a first time step to a predicted set of inputs of the first node at a subsequent time step, the subsequent time step being subsequent to the first time step in a sequence of time steps; and the adjusting of the first generative model comprises adjusting one or more parameters of the prediction parameter tensor.

19. The system of claim 11, wherein: the accessing of the first input, the accessing of the second input, the adjusting of the first generative model, and the generating of the first output are performed in association with each time step in a sequence of time steps; and the generating of the inference using the machine learning model is performed based on the first output corresponding to each time step in the sequence of time steps.

20. A non-transitory computer-readable storage medium, the computer- readable storage medium including instructions that when executed by a system, cause the system to perform operations comprising: accessing a machine learning model, the machine learning model comprising a plurality of nodes;VO1 accessing a first input to the first node, the first input corresponding to a first data source; accessing a second input to the first node, the second input corresponding to a second node of the plurality of nodes, the second node having a second generative model different from the first generative model; adjusting a first generative model of the first node in accordance with an objective function, the first generative model being configured to control generation of outputs of the node based on inputs of the node, the objective function being configured to decrease an expected difference between: a future first input of the first node; and a first output generated by the first node; generating the first output based on the first input, the second input, and the first generative model; using the first output as an input to the first data source; and generating an inference using the machine learning model, the inference being based on the first output.

Images