Dynamic internal cycles for machine learning models

Abstract

A machine learning model is accessed, including a network having a plurality of nodes and a plurality of connections. Each node includes parameter values configured to control processing of inputs of the node to generate outputs of the node. Each connection is associated with a transmitting node and a receiving node and is configured to designate an output of the transmitting node as an input to the receiving node. During a training period, the machine learning model is at least partially trained to perform an inference task. During a runtime period, the inputs of the plurality of nodes are processed and the outputs of the plurality of nodes are generated to perform the inference task with respect to candidate data, and the network of the machine learning model is restructured by changing presence of a first connection.

Description

DYNAMIC INTERNAL CYCLES FOR MACHINE LEARNINGMODELSTECHNICAL FIELD

[0001] The present disclosure relates generally to machine learning and, more particularly, to dynamic internal cyclic connectivity of nodes in machine learning models.BACKGROUND

[0002] Machine learning uses computer algorithms to automatically learn and extrapolate patterns and relationships in data, potentially without the need for explicit programming. A conventional machine learning process begins with a designer configuring or selecting an untrained machine learning model having a specific structure. The model is then trained to perform an inference task in a problem domain using training data drawn from the problem domain. When training is complete, the trained model is deployed to perform the inference task.BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0003] In the drawings, which are not necessarily drawn to scale, like numerals may describe similar components in different views. To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the figure number in which that element is first introduced. Some non-limiting examples are illustrated in the figures of the accompanying drawings in which:

[0004] FIG. 1 illustrates a machine-learning pipeline, providing context for examples described herein.

[0005] FIG. 2 illustrates training and use of an example machine-learning program, providing context for examples described herein.

[0006] FIG. 3 illustrates a first example network of a machine learning model, according to some examples.

[0007] FIG. 4 illustrates the first example network of FIG. 3 with a connection added to create a cycle, according to some examples.

[0008] FIG. 5 illustrates a further example network of a machine learning model, according to some examples.

[0009] FIG. 6 illustrates the second example network of FIG. 5 with a pair of connections added to create a cycle, according to some examples.

[0010] FIG. 7 illustrates initial values of priors of a node of the network of FIG. 5, according to some examples.

[0011] FIG. 8 illustrates initial values of priors of the Bo subtensor of FIG. 7, according to some examples.

[0012] FIG. 9 illustrates differentiated values of priors of the Bo subtensor of FIG. 8, according to some examples.

[0013] FIG. 10 illustrates initial values of priors of the Bi subtensor of FIG. 7, according to some examples.

[0014] FIG. 11 illustrates differentiated values of priors of the Bi subtensor of FIG. 10, according to some examples.

[0015] FIG. 12 illustrates initial prior values and three different sets of differentiated values of priors of the B tensor of FIG. 7, according to some examples.

[0016] FIG. 13 illustrates a flowchart showing operations of a method for adding a cycle to a network of a machine learning model, according to some examples.

[0017] FIG. 14 illustrates a flowchart showing operations of a method for removing a cycle from a network of a machine learning model, according to some examples.

[0018] FIG. 15 is a diagrammatic representation of a machine in the form of a computer system within which a set of instructions may be executed to cause the machine to perform any one or more of the methodologies discussed herein, according to some examples.

[0019] FIG. 16 illustrates a sequence showing restructuring of a network of a machine learning model over time, according to some examples.DETAILED DESCRIPTION

[0020] Examples of the present disclosure provide techniques for machine learning in which a machine learning model is generated, trained, and restructured on an ongoing basis while deployed to perform an inference task.

[0021] Some existing machine learning model types, including artificial neural networks (also referred to herein simply as neural networks), Bayesian networks, and Markov models, are structured as a network of nodes. The structure of the network is predefined before training begins, and parameters of the model (which may include parameters representing the strength of connections between nodes of the network, also called weights) are adjusted during a training phase. Once the model is deemed to be trained, the training phase ends, and the trained model can be deployed to perform an inference task in an inference phase.

[0022] Different types of machine learning models arrange their nodes according to different architectures. Deep learning techniques may use neural networks having large numbers of nodes (sometimes called deep neural networks or DNNs). However, the large number of nodes used in DNNs may require that connectivity among the nodes be limited in order to maintain a manageable number of parameters. Accordingly, the nodes of DNNs, such as convolutional neural networks (CNNs), are organized into a sequence of layers, with the nodes of one layer connected only to nodes of a subsequent layer, such that outputs of the nodes of a given layer propagate forward to nodes of the immediately subsequent layer. In some architectures, such as recurrent neural networks (RNNs), the network may include cycles whereby a node in a subsequent layer provides its outputs backward to one or more nodes in a previous layer. These cycles are pre-defined by the initial architecture of the machine learning model.

[0023] In some examples described herein, a network of nodes of a machine learning model can be restructured dynamically during a runtimeperiod while deployed to perform an inference task. Techniques are described whereby nodes and / or inter-nodal connections of a network may be created and / or removed to create or remove cycles in the network. By dynamically creating and / or removing connections between previously- unconnected nodes, some examples may enable the learning and / or inference behaviors of the machine learning model to be adjusted while the machine learning model is learning and / or when it is deployed to perform an inference task.

[0024] FIG. 1 is a flowchart depicting a machine-learning pipeline 200, and FIG. 2 is a schematic diagram illustrating the training and use of an example machine-learning program. FIG. 1 and FIG. 2 provide context for examples described herein by describing various techniques and approaches to structuring, training, validating, and deploying a machine learning model.

[0025] The machine-learning pipeline 200 may be used to generate a trained machine learning model, for example the trained machine-learning program 202 of FIG. 2, to perform operations associated with searches and query responses.

[0026] Broadly, machine learning may involve using computer algorithms to automatically learn and extrapolate patterns and relationships in data, potentially without the need for explicit programming. Machine learning algorithms can be divided into three main categories: supervised learning, unsupervised learning, and reinforcement learning.• Supervised learning involves training a model using labeled data to predict an output for new, unseen inputs. Examples of supervised learning algorithms include linear regression, decision trees, and neural networks.• Unsupervised learning involves training a model on unlabeled data to find hidden patterns and relationships in the data. Examples of unsupervised learning algorithms include clustering, principal component analysis, and generative models like autoencoders.• Reinforcement learning involves training a model to make decisions in a dynamic environment by receiving feedback in the form of rewardsor penalties. Examples of reinforcement learning algorithms include Q- learning and policy gradient methods.

[0027] Examples of specific machine learning algorithms that may be deployed, according to some examples, include logistic regression, which is a type of supervised learning algorithm used for binary classification tasks. Logistic regression models the probability of a binary response variable based on one or more predictor variables. Another example type of machine learning algorithm is Naive Bayes, which is another supervised learning algorithm used for classification tasks. Naive Bayes is based on Bayes' theorem and assumes that the predictor variables are independent of each other. Random Forest is another type of supervised learning algorithm used for classification, regression, and other tasks. Random Forest builds a collection of decision trees and combines their outputs to make predictions. Further examples include neural networks, which consist of interconnected layers of nodes (or neurons) that process information and make predictions based on the input data. Matrix factorization is another type of machine learning algorithm used for recommender systems and other tasks. Matrix factorization decomposes a matrix into two or more matrices to uncover hidden patterns or relationships in the data. Support Vector Machines (SVM) are a type of supervised learning algorithm used for classification, regression, and other tasks. SVM finds a hyperplane that separates the different classes in the data. Other types of machine learning algorithms include decision trees, k-nearest neighbors, clustering algorithms, and deep learning algorithms such as convolutional neural networks (CNN), recurrent neural networks (RNN), and transformer models. The choice of algorithm depends on the nature of the data, the complexity of the problem, and the performance requirements of the application.

[0028] The performance of machine learning models may be evaluated on a separate test set of data that was not used during training to ensure that the model can generalize to new, unseen data.

[0029] Although several specific examples of machine learning algorithms are discussed herein, the principles discussed herein can be applied to other machine learning algorithms as well. Deep learning algorithms such asconvolutional neural networks, recurrent neural networks, and transformers, as well as more traditional machine learning algorithms like decision trees, random forests, and gradient boosting may be used in various machine learning applications.

[0030] Machine learning models are trained to perform specific types of inference tasks. Two example types of inference tasks in machine learning are classification tasks and regression tasks. Classification tasks, also referred to as categorization tasks, aim at classifying items into one of several category values (for example, is this object an apple or an orange?). Regression algorithms aim at quantifying some items (for example, by providing a value that is a real number) to perform a regression task.

[0031] Generating a trained machine-learning program 202 may include multiple phases that form part of the machine-learning pipeline 200, including for example the following phases illustrated in FIG. 1 :• Data collection and preprocessing 102: This phase may include acquiring and cleaning data to ensure that it is suitable for use in the machine learning model. This phase may also include removing duplicates, handling missing values, and converting data into a suitable format.• Feature engineering 104: This phase may include selecting and transforming the training data 206 to create features that are useful for predicting the target variable. Feature engineering may include (1) receiving features 208 (e.g., as structured or labeled data in supervised learning) and / or (2) identifying features 208 (e.g., unstructured or unlabeled data for unsupervised learning) in training data 206.• Model selection and training 106: This phase may include selecting an appropriate machine learning algorithm and training it on the preprocessed data. This phase may further involve splitting the data into training and testing sets, using cross-validation to evaluate the model, and tuning hyperparameters to improve performance.• Model evaluation 108: This phase may include evaluating the performance of a trained model (e.g., the trained machine-learning program 202) on a separate testing dataset. This phase can helpdetermine if the model is overfitting or underfitting and determine whether the model is suitable for deployment.• Prediction 110: This phase involves using a trained model (e.g., trained machine-learning program 202) to generate predictions on new, unseen data.• Validation, refinement or retraining 112: This phase may include updating a model based on feedback generated from the prediction phase, such as new data or user feedback.• Deployment 114: This phase may include integrating the trained model (e.g., the trained machine-learning program 202) into a more extensive system or application, such as a web service, mobile app, or loT device. This phase can involve setting up APIs, building a user interface, and ensuring that the model is scalable and can handle large volumes of data.

[0032] FIG. 2 illustrates further details of two example phases, namely a training phase 204 (e.g., part of the model selection and trainings 106) and a prediction phase 210 (part of prediction 110). Prior to the training phase 204, feature engineering 104 is used to identify features 208. This may include identifying informative, discriminating, and independent features for effectively operating the trained machine-learning program 202 in pattern recognition, classification, and regression. In some examples, the training data 206 includes labeled data, known for pre-identified features 208 and one or more outcomes. Each of the features 208 may be a variable or attribute, such as an individual measurable property of a process, article, system, or phenomenon represented by a data set (e.g., the training data 206). Features 208 may also be of different types, such as numeric features, strings, and graphs, and may include one or more of content 212, concepts 214, attributes 216, historical data 218 , and / or user data 220, merely for example.

[0033] In training phase 204, also referred to herein as a training period, the machine-learning pipeline 200 uses the training data 206 to find correlations among the features 208 that affect a predicted outcome or prediction / inference data 222. The training data may also be referred toherein as reference data, because it is used as a reference for setting the trained parameter values of the model.

[0034] With the training data 206 and the identified features 208, the trained machine-learning program 202 is trained during the training phase 204 during machine-learning program training 224. The machine-learning program training 224 appraises values of the features 208 as they correlate to the training data 206. The result of the training is the trained machinelearning program 202 (e.g., a trained or learned model).

[0035] Further, the training phase 204 may involve machine learning, in which the training data 206 is structured (e.g., labeled during preprocessing operations). The trained machine-learning program 202 implements a neural network 226 capable of performing, for example, classification and clustering operations. In other examples, the training phase 204 may involve deep learning, in which the training data 206 is unstructured, and the trained machine-learning program 202 implements a deep neural network 226 that can perform both feature extraction and classification / clustering operations.

[0036] In some examples, a neural network 226 may be generated during the training phase 204, and implemented within the trained machine-learning program 202. The neural network 226 includes a hierarchical (e.g., layered) organization of neurons, with each layer consisting of multiple neurons or nodes. Neurons in the input layer receive the input data, while neurons in the output layer produce the final output of the network. Between the input and output layers, there may be one or more hidden layers, each consisting of multiple neurons.

[0037] Each neuron in the neural network 226 operationally computes a function, such as an activation function, which takes as input the weighted sum of the outputs of the neurons in the previous layer, as well as a bias term. The output of this function is then passed as input to the neurons in the next layer. If the output of the activation function exceeds a certain threshold, an output is communicated from that neuron (e.g., transmitting neuron) to a connected neuron (e.g., receiving neuron) in successive layers. The connections between neurons have associated weights, which define the influence of the input from a transmitting neuron to a receiving neuron.During the training phase, these weights are adjusted by the learning algorithm to optimize the performance of the network. Different types of neural networks may use different activation functions and learning algorithms, affecting their performance on different tasks. The layered organization of neurons and the use of activation functions and weights enable neural networks to model complex relationships between inputs and outputs, and to generalize to new inputs that were not seen during training.

[0038] In some examples, the neural network 226 may also be one of several different types of neural networks, such as a single-layer feedforward network, a Multilayer Perceptron (MLP), an Artificial Neural Network (ANN), a Recurrent Neural Network (RNN), a Long Short-Term Memory Network (LSTM), a Bidirectional Neural Network, a symmetrically connected neural network, a Deep Belief Network (DBN), a Convolutional Neural Network (CNN), a Generative Adversarial Network (GAN), an Autoencoder Neural Network (AE), a Restricted Boltzmann Machine (RBM), a Hopfield Network, a Self-Organizing Map (SOM), a Radial Basis Function Network (RBFN), a Spiking Neural Network (SNN), a Liquid State Machine (LSM), an Echo State Network (ESN), a Neural Turing Machine (NTM), or a Transformer Network, merely for example.

[0039] In addition to the training phase 204, a validation phase may be performed on a separate dataset known as the validation dataset. The validation dataset is used to tune the hyperparameters of a model, such as the learning rate and the regularization parameter. The hyperparameters are adjusted to improve the model's performance on the validation dataset.

[0040] Once a model is fully trained and validated, in a testing phase, the model may be tested on a new dataset. The testing dataset is used to evaluate the model's performance and ensure that the model has not overfitted the training data.

[0041] In prediction phase 210, also referred to herein as a runtime period, the trained machine-learning program 202 uses the features 208 for analyzing query data 228 to generate inferences, outcomes, or predictions, as examples of a prediction / inference data 222. For example, during prediction phase 210, the trained machine-learning program 202 generates an output.Query data 228 is provided as an input to the trained machine-learning program 202, and the trained machine-learning program 202 generates the prediction / inference data 222 as output, responsive to receipt of the query data 228. Query data processed by the model during the runtime phase may also be referred to herein as candidate data, because it is data that is a priori not associated with any particular inference output.

[0042] In some examples, the trained machine-learning program 202 may be a generative Al model. Generative Al is a term that may refer to any type of artificial intelligence that can create new content from training data 206. For example, generative Al can produce text, images, video, audio, code, or synthetic data similar to the original data but not identical.

[0043] Some of the techniques that may be used in generative Al are:• Convolutional Neural Networks (CNNs): CNNs may be used for image recognition and computer vision tasks. CNNs may, for example, be designed to extract features from images by using filters or kernels that scan the input image and highlight important patterns.• Recurrent Neural Networks (RNNs): RNNs may be used for processing sequential data, such as speech, text, and time series data, for example. RNNs employ feedback loops that allow them to capture temporal dependencies and remember past inputs. The feedback loops of an RNN can be considered a cycle within its network.• Generative adversarial networks (GANs): GNNs may include two neural networks: a generator and a discriminator. The generator network attempts to create realistic content that can “fool” the discriminator network, while the discriminator network attempts to distinguish between real and fake content. The generator and discriminator networks compete with each other and improve over time.• Variational autoencoders (VAEs): VAEs may encode input data into a latent space (e.g., a compressed representation) and then decode it back into output data. The latent space can be manipulated to generate new variations of the output data. VAEs may use self-attention mechanisms to process input data, allowing them to handle long text sequences and capture complex dependencies.• Transformer models: Transformer models may use attention mechanisms to learn the relationships between different parts of input data (such as words or pixels) and generate output data based on these relationships. Transformer models can handle sequential data, such as text or speech, as well as non-sequential data, such as images or code.

[0044] In generative Al examples, the output prediction / inference data 222 include predictions, translations, summaries or media content.

[0045] FIG. 3 illustrates a first example network 300 of a machine learning model, such as trained machine-learning program 202. The network 300 includes a set of nodes, shown as first node 302, second node 304, and third node 306. The nodes are connected to each other by unidirectional connections, shown as first connection 308 (from first node 302to second node 304) and second connection 310 (from second node 304 to third node 306). Each connection conveys or designates outputs of a transmitting node as inputs to a receiving node. The connections for which a given node is a transmitting node may be referred to as outgoing connections of that node; by the same token, those connections for which a node is the receiving node may be referred to as incoming connections of that node. In network 300, the transmitting node of first connection 308 is first node 302, and the receiving node of first connection 308 is second node 304; the transmitting node of second connection 310 is second node 304, and the receiving node of second connection 310 is third node 306.

[0046] The nodes of the network 300 (and other networks of machine learning models described herein) may be described herein as receiving inputs, processing data, and generating outputs. Similarly, the connections of the described networks may be described as conveying, transmitting, or providing data from one node to another node. In some examples, these terms may refer to the operation of nodes and / or connection embodied as logical circuit components or other physical data processing and / or data communication components. For example, the nodes and connections of a network of a machine learning model as described herein may be implemented as circuit components of a field programmable gate array (FPGA) or another dynamically reconfigurable logical circuit. However, insome examples, the nodes and / or connections may be implemented as data, for example, as data structures and / or instructions stored in a memory and manipulated by a physical processor. The machine 1500 described with reference to FIG. 15 below provides an example of a computing system suitable for such an implementation. For example, a node may be implemented as a data structure associated with one or more parameters (e.g., prior probability distributions, referred to herein as priors, as described below), and processing the inputs of a node to generate the node's outputs may be performed by the physical processor processing the parameter values of the node and the inputs of the node to generate the outputs of the node. A connection may be implemented as data designating or identifying a relationship between a transmitting node and a receiving node, such that the outputs of the transmitting node are treated by the physical processor as inputs to the receiving node. In some examples, the nodes and / or connections may include instructions executed by the processor to perform these operations, whereas in other examples, the instructions are separate from the nodes and connections, and cause the processor to process the data corresponding to the nodes and connections. Thus, in some examples, each node may be said to include or be associated with parameter values, and these parameter values may be used to control the processing of the inputs of the node in order to generate the outputs of the node. It will be appreciated that all references herein to a node processing data, to a connection conveying data, and so on, are all intended to encompass all suitable implementations of the nodes and / or connections as any combination of hardware and / or software.

[0047] In some examples, the nodes of the network 300 operate to process their inputs and generate their outputs at each of a sequence of time steps. In some examples, all nodes of the network 300 operate at each time step; in other examples, some nodes may have different rates of operation such that some nodes of the network 300 may only process their inputs and generate (or update) their outputs at some time steps, such as every n time steps where n is a positive integer.

[0048] In some examples, a node generates its outputs at a current time step t by processing its inputs at a previous or prior time step t- . The inputs may be processed by applying various parameters or priors, such as tensors of priors, to the inputs, and computing one or more output values based on the inputs and priors. In some examples, a node may generate only a single output value that is conveyed as an input to all connected nodes; in other examples, a node may generate multiple outputs to be conveyed to multiple connected nodes and / or as multiple output values conveyed to a single node.

[0049] In some examples, a single output generated by a node may be an activation value. An activation value may be regarded as a measure of how active the node is at the current time step based on its inputs from the previous time step. In some examples, the activation value may be continuous (such as a floating point value) - this approach may be taken by artificial neural networks such as CNNs, in which a given node (e.g., an artificial neuron in an artificial neural network) generates a floating-point output activation value as part of an activation map generated by the neural network layer inhabited by the neuron. In other examples, the activation value of a given node may be discrete, such as a binary value (e.g., 0 or 1, corresponding to inactive or active, or “not firing” and “firing” in the context of an analogy of neuronal firing). The discrete activation value may be determined by either a stochastic quantization process or a deterministic quantization process. In a stochastic quantization process, for example, a probability distribution over all possible discrete output values may be generated by processing the inputs and the priors, and a random or pseudorandom value may be used to select a discrete output value (or set of output values) from the probability distribution. In a deterministic quantization process, a single discrete output value (or set of output values) may instead be selected from the probability distribution as the value or values having the highest probability.

[0050] In some examples, nodes may have connections to themselves, referred to as self-connections, in which they serve as both transmitting node and receiving node. In some such examples, a node may generate a single output for propagation to its connected nodes, and another output or set ofoutputs to itself, called self-outputs (also referred to as self-signals or selfinputs), via its self-connection(s). In some examples, a node may pass through its current inputs as self-outputs, thereby providing a one-time-step memory of its own inputs. This may enable the node to evaluate the accuracy of its own previous predictions of state transitions of the observed system, as described in greater detail below with reference to FIG. 7 through FIG. 12.

[0051] It will be appreciated that the three-node network 300 shown in FIG. 3 is intended as a simplified example, and that actual machine learning models may include networks having much larger numbers of nodes, such as thousands or millions of nodes. Thus, the network 300 may be regarded as a small section of a larger network, in which one or more of the illustrated nodes 302, 304, and / or 306 may have connections to or from one or more other unseen nodes of the larger network. In one such example, in the context of CNNs, first node 302 could be regarded as a neuron of a convolution layer, second node 304 could be regarded as a neuron of a rectified linear unit (ReLU) layer receiving an activation outputs map from the convolution layer, and third node 306 could be regarded as a neuron of a pooling layer receiving the ReLU layer output. It will be appreciated that the network 300 shown in FIG. 3 does not contain any cycles, understood as a plurality of unique nodes in an ordered sequence, each node of the sequence sharing a connection with a next node in the sequence, and a final node of the sequence sharing a connection with an initial node of the sequence. In particular, network 300 can be regarded as a directed graph having no directed cycles, understood as a plurality of unique nodes in an ordered sequence, each node of the sequence having a connection (in this case, a directed connection) to a next node in the sequence, and a final node of the sequence having a connection to an initial node of the sequence. Instead, the output of each node shown in FIG. 3 is propagated in a forward direction from first node 302 to second node 304 to third node 306 without any feedback to previous nodes.

[0052] FIG. 4 illustrates a network 400, which is network 300 with a third connection 312 added from third node 306 to first node 302 to create a cycle. Examples of methods for adding a connection between nodes of an existingnetwork, and thereby forming a new cycle within the network, are described below with reference to FIG. 13. In some examples, the new connection (e.g., third connection 312) is added to the network after the model has been generated and at least partially trained, during a runtime period in which the machine learning model, including the network, has been deployed to repeatedly perform the inference task for which the machine learning model has been trained. By adding a new connection and thereby forming a new cycle during runtime, the operation of the network may be reconfigured dynamically while performing an inference task, thereby potentially enhancing the performance of the machine learning model to perform its inference task and / or dynamically adapting the machine learning model to new conditions, e.g., new types of input data that benefit from adding cycles to the network.

[0053] In some examples, instead of adding a new connection to the network, a connection may be removed from the network. Removal of a previous connection from the network can be illustrated by a transition from network 400 to network 300, with third connection 312 being removed. Examples of methods for removing a connection between nodes from an existing network, and thereby removing a previous cycle within the network, are described below with reference to FIG. 14. As with adding a connection, in some examples, the previous connection (e.g., third connection 312) is removed from the network after the model has been generated and at least partially trained, during the runtime period. By removing a connection and thereby removing a cycle during runtime, the operation of the network may be reconfigured dynamically while performing the inference task, thereby potentially enhancing the performance of the machine learning model and / or dynamically adapting the machine learning model to new conditions. In some examples, removing even a small subset of the connections within a highly-connected network may dramatically reduce resources used by the machine learning model, such as storage, memory, processor, and power usage.

[0054] Thus, various examples described herein may add and / or remove connections of a network. Such addition or removal of a connection may be referred to collectively as changing presence of the connection.

[0055] FIG. 5 illustrates a further example network 500, including first node 502, second node 504, and third node 506. In network 500, the connections between nodes are in bidirectional pairs: first connection 508 from first node 502 to second node 504 and first connection back 514 from second node 504 to first node 502; and second connection 510 from second node 504 to third node 506 and second connection back 516 from third node 506 to second node 504.

[0056] The network 500 also shows first node 502 having a bidirectional pair of connections to an environment 520: a connection from environment 522, and a connection to environment 524. In some examples, the environment 520 can represent an entity outside of the network, such that a node connected to the environment 520 (such as first node 502), also referred to as an environment-facing node, acts as an input node and / or an output node for the network 500. The connections to or from the environment 520 may act as connections to another node insofar as they convey inputs to, and / or receive outputs from, the environment-facing node.

[0057] In the illustrated example, third node 506 has a self-connection 532, which is a connection in which the third node 506 is both the transmitting node and the receiving node of the connection. A self-connection 532 may act as a one-time-step delay loop feeding a node's outputs back to itself. A self-connection 532 can also serve as a one-timestep inverter (e.g., generating a 1 in response to a 0) or a number of other operations: for example, in some cases, a node can process its self-signal from the last timestep and its environment signal from the last timestep according to logic dictated by its parameters (e.g., priors).

[0058] In some examples, a network with bidirectional pairs of connections, such as network 500, may be configured to perform inference according to the principles of active inference and the minimization of variational free energy (VFE) or expected free energy (EFE). Active inference is a framework for modeling Bayes-optimal behavior of an agent.In some examples herein using an active inference framework, an agent - such as a machine learning model, or even a node of a network driving the behavior of the machine learning model - acts to minimize an upper bound on the “surprise” resulting from the agent's observations. Surprise is a measure of the degree to which an agent’s current observations differ from its preferred observations and the informativeness of its observations. In some examples, VFE or EFE is a function of both reward and information gain, potentially in varying ratios, thereby biasing the node to both explore a parameter space and exploit known strategies. In some examples herein using an active inference framework, agents seeking to minimize their surprise may act on the observed system to solicit feedback that accords with the agent's desired observations: this activity of the agent is why the framework is referred to as “active” inference.

[0059] The active inference frameworks described herein propose that, even if an agent does not have the capability to directly minimize its surprise, it can at least seek to minimize a proxy for surprise, called variational free energy. The agent may minimize VFE or EFE by performing computation in response to, and in anticipation of, its observations. Example implementations of active inference described herein may thus rely on the free energy principle, by which an agent reduces its surprise by generating inferences using an internal model and updating its internal model to align with the observed (and acted-upon) system.

[0060] Examples of computational operations implementing active inference are described below with reference to FIG. 7 through FIG. 12. The examples of FIG. 7 through FIG. 12 may be implemented by nodes of the network 500.

[0061] FIG. 6 illustrates a network 600, which is network 500 with a pair of connections added between third node 506 and first node 502 to create a cycle. Third connection 512 is added from third node 506 to first node 502, and third connection back 518 is added from third node 506 to first node 502.

[0062] Because network 600 includes only pairs of bidirectional connections between each pair of connected nodes, any cycle of three ormore nodes in network 600 may be regarded as two distinct directed cycles, each directed cycle traversing the same trail in a direction opposite the other (e.g., one directed cycle from first node 502 to second node 504 to third node 506 to first node 502 and another directed cycle from first node 502 to third node 506 to second node 504 to first node 502).

[0063] Examples of methods for adding a connection between nodes of an existing network, and thereby forming a new cycle within the network, are described below with reference to FIG. 13. In some examples, the new connection or connections (e.g., third connection 512 and third connection back 518) are added to the network during the runtime period. The potential benefits of forming new cycles within the network 500 may include those described above with reference to adding a new cycle to network 300.

[0064] As in the case of network 400 described above, in some examples one or more connections may be removed from the network instead of added. Removal of one or more previous connections from the network can be illustrated by a transition from network 600 to network 500, with third connection 512 and third connection back 518 being removed. Examples of methods for removing a connection between nodes from an existing network, and thereby removing a previous cycle within the network, are described below with reference to FIG. 14. In some examples, the new connection or connections (e.g., third connection 512 and third connection back 518) are removed from the network during the runtime period. The potential benefits of removing previous cycles from the network 600 may include those described above with reference to removing a cycle from network 400.

[0065] In some examples, a network such as network 500 or network 600 may be used to implement a machine learning model operating according to an active inference framework (also referred to as an active inference model). Each node of the network may be associated with parameters or priors organized into one or more tensors (e.g., scalars, vectors, or matrices), each tensor modeling a different aspect of the node's behavior.

[0066] In some examples, the active inference behavior of the nodes of the network may be modeled as a partially observable Markov decision process (POMDP), characterized by the equation:POMDP equation wherein:• “T” indicates the time step, from a first time step T=1 to a final (or current) time step T=T,• “ot” indicates the observations o of the node (e.g., its inputs) at a given time step i=t,• “st” indicates the actual state 5 of the observed system (e.g., a hidden state that may differ from the observations of the node) at time step i=t,• "K" indicates a policy for the node, e.g., a possible sequence of actions (e.g., outputs to generate in response to inputs) that the node can take, and• “(|)” indicates an expected reward, e.g., a similarity between inputs received by the node and the node's preferred inputs.

[0067] The active inference behavior of a node may be implemented using three or more sets of priors organized into three or more tensors. Examples of tensors used to compute active inference behavior using a POMDP for a node of the network are illustrated and described with reference to FIG. 7.

[0068] FIG. 7 shows four tensors used by a POMDP of a node. The A tensor 702, B tensor 712 (consisting of the Bo subtensor 704 and Bi subtensor 706), C tensor 708, and D tensor 710 indicate four sets of priors maintained by the node. Each of these sets of priors is used to contribute to the performance of an active inference task by the network (e.g., network 500). The values of the priors shown in the example of FIG. 7 are initial prior values that may be assigned to a newly-initialized node of the network (e.g., network 500), also referred to as an undifferentiated node or a stem node. The node may be deployed within the network to contribute to an inference task during a runtime period, during which the node may differentiate itself by adjusting its prior values to diverge from their initialvalues in response to interactions with other nodes of the network. A node that has undergone at least partial differentiation may be referred to as a differentiated node.

[0069] In some examples, the inference task performed by the network is an active inference task. In active inference, as described herein, a machine learning model acts upon a system and observes the system.

[0070] In some examples, the A tensor 702 may be regarded as modeling a probabilistic mapping of the node's observations (e.g., its inputs at a given time step) to an inferred hidden state of the observed system (e.g., a node or set of nodes). This mapping may be referred to as an observational model, denoted as:Observational Model

[0071] In the example shown in FIG. 7, the A tensor 702 is initialized as an identity tensor mapping observed inputs to exactly equivalent hidden states. In some examples, the inputs received by a node are equivalent to the state of the observed system; thus, there are no hidden states, and the A tensor 702 may remain unchanged during differentiation.

[0072] In some examples, the B tensor 712 may be regarded as modeling a probabilistic mapping of the node's observations (e.g., its inputs) at a given time step T-1 to a predicted set of observations at a subsequent time step T. This mapping may be referred to as a transition model, denoted as:Transition Model

[0073] In some examples, the B tensor 712 includes a separate subtensor for each set of outputs of the node at the first time step: thus, the predictedinputs at the second time step are conditioned on the outputs of the node at the first time step. In the example shown in FIG. 7, the node has only a single output (e.g., a binary activation value) having only two possible states, 1 or 0. The B tensor 712 therefore consists of two submatrices: a Bo subtensor 704 corresponding to output 0, and a Bi subtensor 706 corresponding to output 1.

[0074] In the example shown in FIG. 7, Bo subtensor 704 and Bi subtensor 706 are both initialized as homogeneous tensors (e.g., all tensor elements set to a value of 0.5) mapping all observed inputs at the first time step with equal probabilities to predicted inputs at the second time step.

[0075] In some examples, the C tensor 708 may be regarded as modeling a prior probability distribution (also referred to simply as a prior) over the node's observations (e.g., its inputs). This prior over observations may be referred to as the node's preferences, prior preferences, or expectations, and is denoted as:Prior Over Observations

[0076] In the example shown in FIG. 7, the C tensor 708 is initialized as a zero vector mapping all observations to a zero reward. Thus, a node initialized according to the initial values shown in FIG. 7 has no preferred values for its inputs. In some examples, C is not 0 but is instead configured to drive the node to predict the output of another node: the first node (having the C tensor 708) prefers or expects to see its self-signal match the output of a second node (or the environment 520). The second node may be a node that the first node is configured to align with, and in some cases, the second node may be referred to as a parent node of the first node. The first mode may also be said to support the second node if the first node prefers to align with the second node.

[0077] In some examples, the D tensor 710 may be regarded as modeling a prior over an initial hidden state at initial time step T=1. This prior over the initial hidden state is denoted as:Prior Over Initial Hidden State

[0078] In the example shown in FIG. 7, the D tensor 710 is initialized as a homogeneous vector (e.g., all vector components set to a value of 0.5) prioring all initial state values equally. Thus, a node initialized according to the initial values shown in FIG. 7 has no a priori beliefs about an initial state of the observed system.

[0079] In some examples, additional tensors may be used, whereas in other examples, one or more of the four tensors shown in FIG. 7 may be omitted (such as the D tensor).

[0080] In some cases, the system applies an epistemic term that pertains to knowledge, understanding, and information, balanced against a pragmatic term that pertains to achieving a goal. The system can prioritize the epistemic term such that the machine learning model focuses on gaining insight on the functions performed by the machine learning model and updating the model to learn from current inferences. For instance, the system may increase the epistemic term relative to the pragmatic term by adjusting the parameters of the C tensor 708 such that the model focuses on certainty, uncertainty, and knowledge of the machine learning model.

[0081] In some examples, the B tensor 712 may be the primary or the only set of priors to be adjusted during differentiation of the node: in other words, each of the other sets of priors (the A tensor 702, C tensor 708, and D tensor 710) remains constant, whereas the transition model (implemented by the B tensor 712) is adjusted to attempt to predict state transitions by the observed system (e.g., other connected nodes), conditioned on the actions (e.g.,outputs) of the node. However, in some examples, one or more of the other sets of priors may also be adjusted during differentiation.

[0082] FIG. 8 through FIG. 12 illustrate examples of how the priors of the B tensor 712 of a node may change during differentiation to adapt to different systems exhibiting different behaviors in response to the node's own actions.

[0083] FIG. 8 shows an initial Bo subtensor 802, equivalent to the initial state of the Bo subtensor 704 shown in FIG. 7. The initial Bo subtensor 802 is the tensor of predictions for a future observation of the state of a system - in the illustrated example, the observed system is a single other node connected to the node associated with the initial Bo subtensor 802, and the observed node generates a single binary output of one bit value at each time step.

[0084] The initial Bo subtensor 802 includes two columns of priors corresponding to two first time step inputs 806. The first time step inputs 806 are observation of two different states of the system at a previous time step T-1 (also referred to herein as the first time step). The priors for 0 input at first time step 808 are in the left column, and the priors for 1 input at first time step 810 are in the right column.

[0085] These columns are split into two rows, the second time step inputs 804, corresponding to observation of two different states of the system at a current time step T (also referred to herein as the second time step). The priors for 0 input at second time step 812 are in the top row, and the priors for 1 input at second time step 814 are in the bottom row.

[0086] It will be appreciated that, for nodes observing systems with more complex outputs (e.g., multiple connected nodes and / or nodes generating outputs more complex than a single bit per time step), the initial Bo subtensor 802 would include more columns and rows. For example, a node connected to three other nodes, each outputting a single bit at each time step, would need an initial Bo subtensor 802 with 23= 8 columns and 8 rows to map all previously observed system states to all predicted current systems states. In some examples, a node may be able to determine that two or more of its inputs tend to be highly similar, and may have its B tensor 712 simplified by combining the two inputs into a single input value.

[0087] As described above, the initial Bo subtensor 802 is a probabilistic mapping of previous observations to predicted current observations, conditioned on the output of the node at time step T-1 having been the binary value 0. The Bi subtensor 706, by contrast, stores the prior values mapping previous observations to predicted current observations conditioned on the output of the node at time step T-1 having been the binary value 1. It will be appreciated that, for nodes having a more complex output than a single bit at each time step, a greater number of B subtensors would be required, equal to the number of possible output states (e.g., 8 subtensors for a 3 -bit binary output). However, in some examples, all inputs and all outputs may be treated as being independent of one another, and the B tensor 712 may include separate subtensors for the separate inputs, such that the 8 x 8 x 8 B tensor that would ordinarily be required (for 3 outputs and 3 dependent inputs) could be replaced by 3 separate subtensors of size 8 (2 x 2 x 2), each subtensor corresponding to a bidirectional connection with one other node.

[0088] Because the nodes of the network generate their outputs for a given time step T prior to receiving their neighboring nodes' outputs for time step T, the computations informed by the priors of the initial Bo subtensor 802 are performed after time step T-1 and before time step T. Thus, time step T may either be referred to as the “current” time step (meaning the time step for which the current outputs are being computed) or as the “next” or “future” time step (because the observations at time step T are not yet known).

[0089] During the runtime period, the node uses the prior values of initial Bo subtensor 802 to perform active inference, in which the initial Bo subtensor 802 predicts the state of the observed system, conditioned on the previous observed state of the system and conditioned on the node's own previous output. To the extent that observations diverge from the predicted observations of the second time step inputs 804, the node may adjust the priors of the B submatrices in an attempt to minimize future VFE resulting from the uncertainty of the predictions. Similarly, if the observations concur with the predicted observations of the second time step inputs 804, the node may adjust the priors of the B subtensors to increase the level of certainty of the expected observations. In this way, the initial Bo subtensor 802 may haveits priors adjusted over time to align with the observed system, and thereby differentiate itself from its initial neutral or naive state.

[0090] Thus, the prior values of the B tensor may be adjusted over a sequence of time steps to align the node with another node or a data source, such as the environment external to the network. After the node makes observations (e.g., inputs are accessed or received by the node) at a given time step, the priors of the node’s B tensor 712 may be adjusted based on the currently observed input state, based on the observed input state at the previous time step, and based on the output generated by the node at the previous time step. As described above, a columnar slice of the B tensor is selected based on the node’s inputs observed and its output generated at the last time step, and a column of priors of one of the B subtensors (e.g., Bo subtensor 704 or Bi subtensor 706) intersected by the columnar slice is selected based on the comparison of the priors of the various individual columns in the columnar slice. The priors of this selected column may then be adjusted in accordance with the currently observed input state, such that a higher prior is assigned to the prior in the row corresponding to the currently observed input state, and all other priors in the row are decreased accordingly.

[0091] In some examples, the degree to which the priors of the column are adjusted is in proportion to the number of observations of each input state over time: thus, for example, if the input state of row 2 (e.g., the row of priors for 1 input at second time step 814 in the column of priors for 0 input at second time step 812) has been observed at 32% of time steps, it will be assigned a prior value of 0.32. In other examples, different degrees of adjustment may be used, such as a degree of adjustment proportional to a learning rate that may vary under different conditions.

[0092] In some cases, the node’s preferences may affect the degree to which the priors of the column are adjusted. When a preferred row is observed, in addition to increasing the prior in the observed row and decreasing the other rows accordingly, all of the priors of the column may be adjusted to sharpen the distribution of prior values, such that high prior values are raised and low prior values are lowered, rendering the priors ofthe column less uniform. In contrast, when a non-preferred row is observed, in addition to increasing the prior in the observed row and decreasing the other rows accordingly, all of the priors of the column may be adjusted to partially flatten the distribution of prior values, such that high prior values are lowered and low prior values are raised, rendering the priors of the column more uniform. The partial flattening of the prior values may result in an increase in the effective learning rate for priors in the column in future time steps, because it is easier for a new observation to stand out relative to other observations when all older observations are relatively uniform. This increase in the effective learning rate after observing a non-preferred row may increase the likelihood or speed with which the node will reconfigure the priors of the column, thereby potentially finding a more optimal set of priors that may assist in avoiding the non-preferred row in the future. By the same token, a slowed effective learning rate after observing a preferred row may reduce the likelihood or speed with which the node will reconfigure the priors of the column, thereby potentially locking the node more tightly into its current configuration.

[0093] FIG. 9 shows the initial Bo subtensor 802 after having undergone differentiation to align itself with an observed system - in this example, another node - that is configured to employ a “tit for tat” strategy in an iterated prisoner's dilemma task. The tit for tat strategy is one in which an agent will defect (in this example, output a “1”) immediately after an opponent defects, and will cooperate (in this example, output a “0”) immediately after an opponent cooperates. Thus, the output of the observed node is a deterministic function of this node's own previous action: in this example, if the node output a “1” at time step T-1, then the observed node will output a “1” at time step T, and if the node output a “0” at time step T-1, then the observed node will output a “0” at time step T.

[0094] Thus, as the node undergoes differentiation to align with the observed node employing the tit for tat strategy, the node will adjust the priors of its initial Bo subtensor 802 to the values shown in the differentiated differentiated Bo subtensor 902: the priors for 0 input at second time step 812 are all 1, indicating a 100% probability that a previous output of “0” willresult in a current observation of a “0”, and the priors for 1 input at second time step 814 are all 0, indicating a 0% probability that a previous output of “0” will result in a current observation of a “1”. These prior values are set across both columns, indicating that they apply regardless of the previous observed state of the system.

[0095] By differentiating nodes to align them with their observed systems, some examples described herein may train the nodes of the network to generate inferences about the behavior of the rest of the system including (for environment-facing nodes) the environment outside of the machine learning model.

[0096] FIG. 10 and FIG. 11 show the same process of differentiation to align with a “tit for tat” observed node as the differentiation adjusts the prior values of the Bi subtensor 706. Initial Bi subtensor 1001 has the same values as initial Bo subtensor 802. The first time step inputs 1003 are mapped to the second time step inputs 1002 using priors split into two columns (priors for 0 input at first time step 1004 and priors for 1 input at first time step 1005) and two rows (priors for 0 input at second time step 1006 and priors for 1 input at second time step 1007). The mapping of initial Bi subtensor 1001 is conditioned on the output of the node at time step T-1 being “1”.

[0097] As the node differentiates to align with the observed tit for tat strategy of its neighboring node, the initial Bi subtensor 1001 adjusts its prior values to those shown in the differentiated Bi subtensor 1102: the priors for 0 input at second time step 1006 are all 0, indicating a 0% probability that a previous output of “1” will result in a current observation of a “0”, and the priors for 1 input at second time step 1007 are all 1, indicating a 100% probability that a previous output of “1” will result in a current observation of a “1”. These prior values are set across both columns, indicating that they apply regardless of the previous observed state of the system.

[0098] FIG. 12 shows an initial B tensor 1202 (equivalent to B tensor 712 shown in FIG. 7), and its potential to differentiate into multiple possible differentiated values depending on the behavior of the observed system. Inthis example, the observed system is a single node playing an iterated prisoner's dilemma task, as in the examples of FIG. 8 through FIG. 11.

[0099] If the observed node employs a tit for tat strategy, the initial B tensor 1202 will differentiate over time into the tit-for-tat differentiated B tensor 1204, whose Bo and Bi submatrices are equivalent to differentiated Bo subtensor 902 and differentiated Bi subtensor 1102 shown in FIG. 9 and FIG. 11. The tit-for-tat differentiated B tensor 1204 exhibits absolute certainty in its predictions: the previous action of the node is determinative of the next observed state of the system, due to the nature of the tit for tat strategy.

[0100] If the observed node employs an “always defect” strategy, the initial B tensor 1202 will differentiate over time into the defector differentiated B tensor 1206. In an always defect strategy, the observed node always outputs “1”, irrespective of previous outputs and irrespective of inputs. As a result, the node observing the defector node will have complete certainty (priors 0 and 1) about the next observed state: it can predict with 100% confidence that the next observed state will always be “1”. Notably, this prediction is premised on the previous observed state being “1” as well: the 1 and 0 prior values are only in the right column of the Bo subtensor 704 and Bi subtensor 706. Because the node has never observed a system state of “0”, the priors in the left columns are completely undifferentiated from their initial values of 0.5.

[0101] In the final illustrated example, if the observed node employs a “random” strategy, the initial B tensor 1202 will differentiate over time into a form resembling the random differentiated B tensor 1208. In a random strategy, the observed node outputs “0” or “1” randomly, irrespective of previous outputs and irrespective of inputs. As a result, the node observing the random node will remain largely undifferentiated (priors all at approximately 0.5) about the next observed state: it cannot predict the next observed state with any confidence.

[0102] Each of these differentiated states 1204, 1206, or 1208 of the B tensor 712 of a node presents an example of a different degree of two capabilities of the node: predictive certainty, and self-efficacy. Predictivecertainty refers to the degree of certainty in the node's predictions: the tit- for-tat differentiated B tensor 1204 exhibits perfect predictive certainty, the defector differentiated B tensor 1206 exhibits perfect predictive certainty only under some conditions (it is certain as long as the observed state is “1”, but would be completely uncertain at the next time step if it ever observed a “0”), and the random differentiated B tensor 1208 has no predictive certainty.

[0103] The second capability of the node, self-efficacy, refers to a measure of the ability of the output of the node at the previous time step T-1 to affect the observations of the node at the current time step T. The self-efficacy of a node is high if the state of its previous outputs result in highly certain current observations. Thus, the tit-for-tat differentiated B tensor 1204 has very high self-efficacy, because its own action is completely determinative of the next observation. The self-efficacy of the defector differentiated B tensor 1206, by contrast, is very low, even though its predictive certainty is perfect for half of the possible observed previous states; the previous output of the node has no effect on its next observation. Finally, the self-efficacy of the random differentiated B tensor 1208 is also very low, because its outputs also have no statistically meaningful effect on later observations.

[0104] Thus, predictive certainty and self-efficacy are separate metrics which may or may not co-occur.

[0105] In nodes having no preferences, such as the initial POMDP tensors described with reference to FIG. 7 above having zero values for the C tensor 708, high self-efficacy maps or corresponds directly to low VFE (and vice- versa: low self-efficacy maps or corresponds directly to high VFE). However, in examples having preferences, minimizing VFE may require preferences to be satisfied, which requires not only self-efficacy but specifically self-efficacy with respect to soliciting preferred observations. For example, a node with preference (e.g., with non-zero values for the priors of the C tensor 708) may learn how to reliably solicit a current “1” observation under certain previous output and observation states; however, if the node prefers to observe “0” states of the observed system, and has notlearned a way to reliably solicit “0” observations, the self-efficacy of the node will be very low.

[0106] In some examples, the C tensor 708 can be used to guide the determination of VFE. For example, VFE can be determined based on a pragmatic term (e.g., "does the node get what is expected?") and / or an epistemic term (e.g., "does the node learn?"). The C tensor 708 can be used to adjust the balance between the pragmatic term and epistemic term. For example, the C tensor 708 can be set to 0 in some examples described herein, whereby the node is maximizing epistemic gain.

[0107] In some cases, a pragmatic term is used to compute VFE. The pragmatic term may focus on the practical use or application of a concept, emphasizing its utility in real-world scenarios. The pragmatic term may be emphasized to enable the machine learning model to focus on action, decision-making, and achieving specific outcomes. The pragmatic term may be increased by changing the C tensor 708 such that the machine learning model focuses more on improving accuracy, efficiency, performance, and / or the like.

[0108] In some examples, predictive certainty and / or self-efficacy may be measured with respect to individual connections of a node. The certainty with which a node can predict inputs received via a given connection may be regarded as the node's predictive certainty with respect to that connection. Similarly, the extent to which a node's outputs via a given connection affect the state of one or more inputs subsequently observed may be regarded as the self-efficacy of that connection. In some examples, self-efficacy is computed for pairs of connections, such that the self-efficacy of a bidirectional pair of connections between a first node and a second node measures the degree to which the first node's output(s) to the second node affect the first node's subsequent observation of the second node's output(s). In some examples, the self-efficacy of a connection from the first node to the second node may instead be a measure of the degree to which the first node's output(s) to the second node affect all observations of the first node with respect to all of its connected nodes.

[0109] In some examples, the self-efficacy of each of a node's connections can be combined to generate a nodal self-efficacy measure. In some examples, the nodal self-efficacy of a node is computed as the sum of the self-efficacy of each of its outbound connections. In other examples, the nodal self-efficacy of a node is computed as an average (such as an arithmetic or geometric mean, a median, or another averaging function), or as some other combination of the self-efficacy measures of each of its outbound connections.

[0110] In some examples, the self-efficacy of a connection can be computed as the mean of the absolute difference between the sub-matrices conditioned on each action, e.g., Bo subtensor 704 and Bi subtensor 706. (For nodes having multiple connections, self-efficacy for a given connection may be computed by isolating the observations and predictions of the B tensor 712 for the connection being measured.) Thus, the self-efficacy for tit-for-tat differentiated B tensor 1204 is the maximum value of 1 :Self-efficacy (S) for tit-for-tat differentiated B tensor 1204 [OHl] In contrast, the self-efficacy for defector differentiated B tensor 1206 is the minimum value of 0:Self-efficacy (S) for defector differentiated B tensor 1206

[0112] FIG. 13 illustrates an example method 1300 for adding a connection between nodes of a network of a machine learning model during a runtime period. Although the example method depicts a particular sequence ofoperations, the sequence may be altered without departing from the scope of the present disclosure. For example, some of the operations depicted may be performed in parallel or in a different sequence that does not materially affect the function of the method. In other examples, different components of an example device or system that implements the method may perform functions at substantially the same time or in a specific sequence.

[0113] The method 1300 can be implemented by one or more processors executing instructions stored in a tangible computer readable storage medium. The tangible computer readable storage medium may also be used to store and / or retrieve the machine learning model.

[0114] The method 1300 will be described with reference to the example network 500 of FIG. 5 (and / or the network 600 of FIG. 6 after adding a connection). However, it will be appreciated that the method 1300 can be applied to various types of networks used in machine learning models, such as the examples described above with reference to FIG. 1 through FIG. 4.

[0115] Thus, in some examples, the machine learning model includes a network having a plurality of nodes and a plurality of connections between the nodes. Each node is configured to process one or more inputs to generate one or more outputs, and each connection is associated with a transmitting node and a receiving node and is configured to convey an output of the transmitting node as an input to the receiving node.

[0116] The instructions, when executed by the one or more processors, may cause the one or more processors to perform the operations shown in FIG.13.

[0117] At operation 1301, the machine learning model is generated and at least partially trained. The machine learning model includes the network of nodes and connections. In some examples, the machine learning model may be generated with a pre-determined allocation of nodes and connections. In some examples, the machine learning model may start as a single node, with new nodes added during a training or growth period. In some examples, the machine learning model may start as a large network, with nodes removed during a node pruning period. In some examples, the network may have one or more connections added and / or removed during the growth / learningand / or node pruning periods. In some examples, generating the machine learning model also includes initializing or otherwise generating values for data other than the network, such as various hyperparameter values or nodal parameter values that may control the operation of the machine learning model in addition to the operations of the nodes of the network. Once the machine learning model has been generated, it may be accessed (e.g., loaded into a memory) in order to perform the other operations of method 1300.

[0118] In some examples, the network includes one or more cycles. Each cycle includes a plurality of unique nodes in an ordered sequence (e.g., first node 502, second node 504, and third node 506, in that order, in network 600 in FIG. 6). In some examples, each node of the sequence shares a connection with a next node in the sequence, and a final node of the sequence shares a connection with an initial node of the sequence (e.g., first node 502 and second node 504 share both first connection 508 and first connection back 514, second node 504 and third node 506 share both second connection 510 and second connection back 516, and third node 506 and first node 502 share both third connection 512 and third connection back 518).

[0119] The machine learning model is at least partially trained to perform an inference task during a training period. During the training period, the machine learning model is trained to perform the inference task by processing samples of reference data and applying a machine learning algorithm to train the machine learning model, as described above with reference to FIG. 1 and FIG. 2. The training performed at operation 1301 may not be complete: for example, only an initial training phase 204 may be performed, without refinement and / or retraining. In some examples, only a limited set of samples of reference data are used to partially train the machine learning model at operation 1301, after which the partially trained machine learning model is deployed to perform inference on candidate data during a runtime period at operation 1302 (described below). In some such examples, further training may be performed during or after the runtime period, using further reference data and / or candidate data.

[0120] At operation 1302, the machine learning model is deployed for a runtime period to process samples of candidate data to iteratively perform aninference task. As described above, candidate data is data processed by the machine learning model that has no a priori association with a given inference, in contrast to reference data, which is used as a reference for training the machine learning model to perform the inference task.

[0121] At operation 1303, the machine learning model generates outputs of a first node (e.g., first node 502) and a second node (e.g., second node 504) of the network (e.g., network 500) at each of a plurality of time steps. As described above, the nodes of the network may all generate outputs at each time step, or some nodes may generate outputs only at a subset of time steps. Each node generates its outputs at a given time step based on inputs received at a previous time step.

[0122] In some examples, a node may generate only a single output, which is propagated via its outgoing connections to the receiving nodes of those connections. In some examples, some nodes may generate multiple different outputs, selectively propagated via its various outgoing connections perform one or more of the functions described above, such as receiving delayed selfoutputs, performing logical operations on self-outputs, and so on. In some examples, a node may have an outgoing connection to itself (such as selfconnection 532), such that it is both the transmitting node and the receiving node of the connection - such connections may act as one-time-step delay loop feeding a node's outputs back to itself.

[0123] At operation 1304, the outputs of the first node are compared to the outputs of the second node for each of the plurality of time steps. If the first node outputs are similar to the second node outputs for at least a step threshold number of time steps, the method 1300 proceeds to operation 1305; otherwise, the method 1300 returns to operation 1303, and the outputs of the first node and second node continue to be monitored for similarity.

[0124] In some examples, similarity between outputs of two nodes may be determined by a similarity metric, such as a difference between the outputs of the two nodes being below a threshold. A difference between outputs of a first node and outputs of a second node may be calculated as an arithmetic difference between a single output of the first node and a single output of the second node, a sum or average (e.g., mean, median, etc.) of the differencesbetween each of a plurality of corresponding outputs of the first node and second node, or any other metric for computing a difference between two sets of outputs. In some examples, the threshold to which the difference is compared may be a fixed threshold value, whereas in other examples it may be based on values of the outputs (e.g., a threshold equal to 10% of the value of an output of the first node). It will be appreciated that similarity between two sets of outputs may be determined by any suitable similarity metric.

[0125] In some examples, the plurality of time steps is a predetermined number of time steps, such as the last 20 time steps, or the last 20 time steps when both the first node and second node generated outputs. The step threshold number may be a number such as 15 or 19, such that the outputs of the first node and second node must be similar for at least 15 or 19 of the last 20 time steps in order for the method to proceed to operation 1305. It will be appreciated that other values for the number of time steps considered and / or the step threshold number can be used, and / or that other measures of similarity over time may be used, in various examples. For example, the degree of similarity at each given time step and the number of such similar outputs over time may be combined into a single value and compared to a metric to determine a degree of similarity over time. In some examples, the divergence or difference between the outputs of the first node and second node may be summed over the plurality of time steps (e.g., over the last 20 time steps), and this sum may be compared to a predetermined threshold to determine whether the divergence over time is small enough to proceed to operation 1305.

[0126] In some examples, the outputs of the first node are determined to be similar to the outputs of the second node if they satisfy an output similarity condition. In some examples, the output similarity condition is only satisfied if a first output value of the outputs of the first node is equal to a second output value of the outputs of the second node: for example, each node may output only a single one-bit binary value at each time step, and the output similarity condition is satisfied when the first node and second node both output a 1 or both output a 0 at a given time step, or at each of a stepthreshold number of time steps (e.g., 15) out of the plurality of time steps (e.g., the last 20 time steps).

[0127] In some examples, each node processes its inputs at a given time step to generate an activation value of the node. Some or all of the outputs of the node may be generated based on the node's activation value. In some such examples, the output similarity condition may be satisfied if the outputs of the first node are associated with a high activation value of the first node and the outputs of the second node are associated with a high activation value of the second node. For example, the first node may generate a single one-bit binary output of 1 or 0 based on whether an activation value of the node is high (e.g., equal to or above 0.5 on a normalized scale from 0 to 1) or low (e.g., below 0.5 on the normalized scale from 0 to 1). In various examples, the activation value may be mapped deterministically or stochastically to the output value: for example, in a stochastic system, an activation value of 0.7 for the first node may result in a randomly- determined 70% chance of the output being a 1 instead of a 0. Thus, in some examples, outputs (e.g., 1) that tend to be associated with high node activation values (e.g, above 0.5) will be considered to satisfy the output similarity condition. In the context of a biological analogy to neurons, this means that nodes (neurons, in the analogy) that have high activation values at the same time (neurons that fire in synchrony, in the analogy) will tend to form and / or strengthen dendritic connections (add inter-nodal connections, in the analogy) to each other. In some examples, the plurality of time steps may include only those time steps in which one or both of the nodes fires (e.g., generates an output associated with a high activation value): thus, for example, the first node and the second node may satisfy the output similarity condition if, out of the last 20 time steps in which the first node generated a 1 value for its output, the second node generated a 1 output in at least 19 of those time steps.

[0128] In some examples, the output similarity condition may be satisfied not only by both nodes having simultaneous outputs that are associated with a high activation value, but also by both nodes having simultaneous outputs that are associated with a low activation value. Thus, satisfying the outputsimilarity condition may further include determining that the one or more outputs of the first node are associated with a low activation value of the first node, and determining that the one or more outputs of the second node are associated with a low activation value of the second node. Thus, in some examples, the output similarity condition may be satisfied by nodes that fire together at the same time steps, but also by nodes that fail to fire at the same time steps.

[0129] At operation 1305, the network is restructured to add a connection from the first node (as the transmitting node) to the second node (as the receiving node). For example, starting from the example network 500, the third connection 512 of network 600 may be added from the third node 506 to the first node 502 based on a similarity of the outputs of the third node 506 and first node 502 over the plurality of time steps.

[0130] In some examples, a pair of bidirectional connections may be added between the first node and the second node at operation 1305 (e.g., third connection 512 and third connection back 518).

[0131] In some examples, one or more of the connections added to the network at operation 1305 may thereby create one or more new cycles. Adding the connection from the first node to the second node at operation 1305 may create a new cycle wherein the first node is the final node of the cycle and the second node is the initial node of the cycle. In some examples, the newly created cycle is a directed cycle wherein each node of the sequence is the transmitting node of the connection to the next node in the sequence. In some examples, more than one new cycle may be created as a result of adding one or more connections at operation 1305.

[0132] In some examples, the newly added cycle may be a 2-cycle in which the first node has a connection added to the second node, and the second node has a second connection added back to the first node. In some examples, the newly added cycle may be a cycle traversing more than two nodes.

[0133] In some architectures of the network, adding new cycles during the runtime period may result in the operation of the network, and thereby the operation of the machine learning model, to change after the machinelearning model has been deployed to perform the inference task on candidate data. In some examples, this may mean that a network initially configured and / or trained as a linear sequence of layers may be reconfigured dynamically, during runtime operations, to operate as a recurrent or other cyclical network type, in which feedback from downstream layers of the network is provided to previous layers of the network. This may enable the machine learning model to adapt to different patterns of candidate data (for example, an increase in the prevalence of data which follows a pattern that varies over multiple time steps) to improve its performance with respect to the inference task.

[0134] Thus, there may be circumstances in which adding new connections (and potentially new cycles) to the network, as described with respect to method 1300, may improve the performance of the machine learning model. However, in other circumstances, it may be beneficial to remove connections (and potentially, cycles) from the network. For example, performance of the machine learning model may be improved in some cases by reducing the degree of connectivity between nodes of the network. In some examples, removing connections and / or cycles from the network may assist in the segregation of nodes into layers, clusters, or other relatively self-contained structures with high internal connectivity and low external connectivity, thereby improving the efficiency and / or accuracy of the performance of the machine learning model in performing the inference task. In some examples, removing connections may improve efficiency and / or overfitting by reducing the number of computations that are performed at each time step. Thus, a method 1400 for removing connections from the network during the runtime period is described with reference to FIG. 14 below.

[0135] FIG. 14 illustrates an example method 1400 for removing a connection between nodes of a network of a machine learning model during a runtime period. Although the example method depicts a particular sequence of operations, the sequence may be altered without departing from the scope of the present disclosure. For example, some of the operations depicted may be performed in parallel or in a different sequence that does not materially affect the function of the method. In other examples, differentcomponents of an example device or system that implements the method may perform functions at substantially the same time or in a specific sequence.

[0136] As with method 1300, method 1400 can be implemented by one or more processors executing instructions stored in a tangible computer readable storage medium. The tangible computer readable storage medium may also be used to store and / or retrieve the machine learning model. The method 1400 will be described with reference to the example network 600 of FIG. 6 (and / or the network 500 of FIG. 5 after removing a connection). However, it will be appreciated that the method 1400 can be applied to various types of networks used in machine learning models, such as the examples described above with reference to FIG. 1 through FIG. 4.

[0137] Thus, in some examples, the machine learning model includes a network as described above with reference to FIG. 13. The instructions, when executed by the one or more processors, may cause the one or more processors to perform the operations shown in FIG. 14.

[0138] At operation 1401, the machine learning model is generated and trained. Operation 1401 may be regarded as equivalent to operation 1301 of method 1300.

[0139] At operation 1302, the machine learning model is deployed for a runtime period to process samples of candidate data to iteratively perform an inference task. Operation 1402 may be regarded as equivalent to operation 1302 of method 1300.

[0140] At operation 1303, the machine learning model generates outputs of a first node (e.g., first node 502) and a second node (e.g., second node 504) of the network (e.g., network 500) at each of a plurality of time steps. Operation 1403 may be regarded as equivalent to operation 1303 of method 1300.

[0141] At operation 1404, the machine learning model computes the self- efficacy of the first connection from the first node to the second node. In some examples, self-efficacy may be computed according to the techniques described with reference to FIG. 7 through FIG. 12 above, such as aPOMDP -based implementation of the computation of self-efficacy for individual connections of the network. For example, self-efficacy may be regarded in some examples as a measure of the ability of the one or more outputs of the first node to the second node via the first connection at a first time step to affect the one or more inputs of the first node at a second time step after the first time step. This measure may be computed, in some examples, based on a variational free energy (VFE) or expected free energy (EFE) of the first node with respect to the first connection. As described above, a low VFE or EFE of a connection may correspond to a high self- efficacy of the connection, and VFE or EFE may be a measure of difference between one or more preferred inputs of the first node and the one or more inputs received by the first node at the second time step. In other words, VFE or EFE may be inversely based on the similarity of the one or more preferred inputs of the first node and the one or more inputs received by the first node at the second time step.

[0142] In some examples, the generation of outputs of each node (e.g., at operation 1303 and operation 1403) may be performed by an optimization technique that seeks to reduce or minimize a difference between the preferred inputs (e.g., as indicated by the priors of the C tensor 708) and the predicted inputs (as indicated by the computations using the B tensor 712). Thus, for example, the outputs of the first node may be generated by computing one or more predicted inputs at the second time step (e.g., second time step inputs 804) as a function of the one or more inputs received at the first time step (e.g., the 0 input or 1 input at the first time step) and one or more possible outputs to be generated at the first time step (e.g., a 0 output at the first time step, corresponding to initial Bo subtensor 802, or a 1 output at the first time step, corresponding to differentiated Bo subtensor 902). Of these possible outputs, the machine learning model may select the output(s) that are likely to reduce or minimize the difference between the preferred inputs and the predicted inputs. An example of such an optimization computation is shown in the POMDP equation above.

[0143] In some examples, the preferred inputs of the first node at the second time step may include the one or more predicted inputs of the firstnode from the first time step, such that the VFE or EFE of the first node is lower the more similar the predicted inputs from the first time step are to the inputs received at the second time step. For example, the first node may include one or more self-connections that transmit its own predicted inputs computed at the first time step (e.g., based on the B tensor 712). This means that the first node will receive, as additional input(s) at the second time step, its own predict! on(s) from the first time step, which can be compared to the actual input(s) received from the second node at the second time step and factored into the VFE or EFE calculation, with the premise that the preferences of the first node include a preference that its predicted input(s) for a given time step align with its actual input(s) at that given time step. Thus, in some examples, a high predictive certainty of the first node with respect to the first connection may result in a lower VFE or EFE and / or a higher self-efficacy than would be the case if predictive certainty were lower.

[0144] VFE is a computation based on a generative model (e.g., a set of priors learned by a node or other agent) and observations (e.g., inputs to a node or other agent). VFE provides an upper bound on statistical surprise. There are various mechanisms to minimize VFE, including updating the generative model to more closely align with observations, or performing a computation of expected free energy (EFE) across possible actions and selecting one of the actions which leads to the lowest expected free energy. Operations involving the computation or minimization of EFE, as referred to herein, may be referred to as a computation or minimization of VFE.

[0145] To minimize EFE, a node may compute an expected distribution of free energy over multiple output states and select an output state that minimizes the expected free energy. The calculation of expected free energy may in some cases be affected by the node’s preferences (e.g., as represented by the C tensor 708), such that rows of the B tensor 712 corresponding to preferred observations of the node are expected to result in lower free energy than non-preferred rows of the B tensor 712. Thus, the node will calculate EFE distributed over all possible output states, then select an output that is at or near the minimum value of the EFE distribution. When operating in adeterministic mode, the node may select the output value that directly minimizes EFE. When operating in a stochastic mode, the node may weight various output values based on their corresponding EFE, or their proximity to the minimum value of the EFE distribution, and stochastically select an output value from the weighted distribution of output values.

[0146] Thus, the node’s preferences (e.g., the values of the parameters in the C tensor 708) may affect the degree to which the node operates to minimize VFE, EFE, or a combination of both VFE and EFE. A node with uniform zero parameter values in the C tensor 708 will have a close relationship between VFE and EFE, whereas a node with non-uniform parameter values in the C tensor 708 will tend to exhibit EFE behavior that diverges from that predicted solely from VFE.

[0147] In some cases, the priors of the B tensor 712 may be discretized before being used to compute VFE and / or EFE. A subtensor of the B tensor 712 is selected based on the node’s output at the previous time step. From this subtensor, a column is selected based on the node’s observation (e.g., input values) at the previous time step. This selected column from the selected subtensor is the only portion of the B tensor 712 used to compute VFE and / or EFE.

[0148] At operation 1405, the self-efficacy of the first connection is compared to a self-efficacy threshold. If the self-efficacy is below the self- efficacy threshold, the method 1400 proceeds to operation 1406; otherwise, the method 1400 returns to operation 1403 to continue monitoring the self- efficacy of the connections at subsequent time steps.

[0149] The self-efficacy threshold may be a fixed value or a dynamically computed value. In some examples, the self-efficacy threshold is computed based on a ranking of the self-efficacy of some or all of the connections of the network, such that those connections with the lowest self-efficacy (e.g., the five connections of the network with the lowest self-efficacy, the 5% of connections of the network with the lowest self-efficacy, or all connections that are not among the 1000 connections having the highest self-efficacy) are deemed to fall below the self-efficacy threshold. In some examples, the self- efficacy threshold is a fixed value, such as 0.2. In some examples, the self-efficacy threshold may change over time; for example, the self-efficacy threshold may be raised over time as the nodes of the network undergo more differentiation, or the self-efficacy threshold may be changed based on a hyperparameter value configurable by a designer. In some examples, the self-efficacy threshold may be different for different nodes or subsets of nodes (or connections, or subsets of connections) of the network; for example, the self-efficacy threshold used to evaluate outgoing connections of the first node may be computed based on a degree of differentiation of the first node and / or based on other characteristics of the first node. In some examples, one of the characteristics of the node in determining whether to remove a connection of the node is the age of the node, such as the number of time steps for which the node has been operating within the network.

[0150] At operation 1406, the network is restructured to remove a connection from the first node (as the transmitting node) to the second node (as the receiving node), in response to the determination at operation 1405. For example, starting from the example network 600, the third connection 512 of network 600 may be removed from the third node 506 to the first node 502 based on the determination at operation 1405 that the third connection 512 has a self-efficacy below the self-efficacy threshold.

[0151] In some examples, a pair of bidirectional connections may be removed between the first node and the second node at operation 1305 (e.g., third connection 512 and third connection back 518).

[0152] In some examples, one or more of the connections removed from the network at operation 1305 may thereby remove one or more previous cycles. Removing the connection from the first node to the second node at operation 1406 may remove a previous cycle wherein the first node is the final node of the cycle and the second node is the initial node of the cycle. In some examples, the removed previous cycle is a directed cycle wherein each node of the sequence is the transmitting node of the connection to the next node in the sequence. In some examples, more than one previous cycle may be removed as a result of removing one or more connections at operation 1305.

[0153] In some examples, the removed cycle may be a 2-cycle in which the first node has a connection added to the second node, and the second nodehas a second connection added back to the first node. In some examples, the removed cycle may be a cycle traversing more than two nodes.

[0154] In some examples, the self-efficacy computation of operation 1404, the self-efficacy comparison of operation 1405, and the connection removal of operation 1406 may be performed periodically for various connections of the network. For example, a poll of self-efficacy (e.g., determining self- efficacy as at operation 1404) may be performed for all connections of the network every set number of time steps, such as every 50 time steps. Based on the self-efficacy values determined for the entire set of connections of the network (e.g., at operation 1404), a self-efficacy threshold may be optionally computed (such as a self-efficacy threshold defined by the 5% lowest self- efficacy values in the network). All connections are compared to the self- efficacy threshold (e.g., at operation 1405), either explicitly (e.g., by mathematical comparison) or implicitly (e.g., because those connections are in the subset of the 5% lowest self-efficacy values), and those falling below the self-efficacy threshold may be removed (e.g., at operation 1406 described below).

[0155] In some examples, a predetermined self-efficacy threshold is used. In some example, the fixed self-efficacy threshold is very low, corresponding to a connection that has almost no effect on the outputs of its observations (e.g., the outputs of the receiving node of the connection, or all observations across all connections). For example, the self-efficacy threshold may be fixed at a value corresponding to a connection whose outputs have less than a 1 / 1000 effect on the outputs of the receiving node. In the example 5-node network described above, removing such a connection may lead to a <0.1% decrease in the nodal self-efficacy of the transmitting node, or a <0.06% decrease in average nodal self-efficacy across the whole network, in exchange for a >17% decrease in storage and energy usage. Thus, in some examples, removal of connections with low self-efficacy may result in very high gains in efficiency with very little cost to the nodal self-efficacy of the nodes of the network.

[0156] As described above with reference to FIG. 4, removing connections within a highly-connected network may dramatically reduce resources used by the machine learning model.

[0157] FIG. 15 is a diagrammatic representation of the machine 1500 within which instructions 1502 (e.g., software, a program, an application, an applet, an app, or other executable code) for causing the machine 1500 to perform any one or more of the methodologies discussed herein may be executed. For example, the instructions 1502 may cause the machine 1500 to execute any one or more of the methods described herein. The instructions 1502 transform the general, non-programmed machine 1500 into a particular machine 1500 programmed to carry out the described and illustrated functions in the manner described. The machine 1500 may operate as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1500 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1500 may comprise, but not be limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular telephone, a smartphone, a mobile device, a wearable device (e.g., a smartwatch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions 1502, sequentially or otherwise, that specify actions to be taken by the machine 1500. Further, while a single machine 1500 is illustrated, the term “machine" shall also be taken to include a collection of machines that individually or jointly execute the instructions 1502 to perform any one or more of the methodologies discussed herein. In some examples, the machine 1500 may comprise both client and server systems, with certain operations of a particular method or algorithm being performed on the server-side and with certain operations of the particular method or algorithm being performed on the client-side.

[0158] The machine 1500 may include processors 1504, memory 1506, and input / output I / O components 1508, which may be configured to communicate with each other via a bus 1510. In an example, the processors 1504 (e.g., a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) Processor, a Complex Instruction Set Computing (CISC) Processor, a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Radio- Frequency Integrated Circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 1512 and a processor 1514 that execute the instructions 1502. The term "processor" is intended to include multi-core processors that may comprise two or more independent processors (sometimes referred to as “cores”) that may execute instructions contemporaneously. Although FIG. 15 shows multiple processors 1504, the machine 1500 may include a single processor with a single-core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiples cores, or any combination thereof.

[0159] The memory 1506 includes a main memory 1516, a static memory 1518, and a storage unit 1520, both accessible to the processors 1504 via the bus 1510. The main memory 1506, the static memory 1518, and storage unit 1520 store the instructions 1502 embodying any one or more of the methodologies or functions described herein. The instructions 1502 may also reside, completely or partially, within the main memory 1516, within the static memory 1518, within machine-readable medium 1522 within the storage unit 1520, within at least one of the processors 1504 (e.g., within the processor’s cache memory), or any suitable combination thereof, during execution thereof by the machine 1500.

[0160] The I / O components 1508 may include a wide variety of components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I / O components 1508 that are included in a particular machine will depend on the type of machine. For example, portable machines such as mobile phones may include a touch input device or other such inputmechanisms, while a headless server machine will likely not include such a touch input device. It will be appreciated that the I / O components 1508 may include many other components that are not shown in FIG. 15. In various examples, the I / O components 1508 may include user output components 1524 and user input components 1526. The user output components 1524 may include visual components (e.g., a display such as a plasma display panel (PDP), a light-emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), haptic components (e.g., a vibratory motor, resistance mechanisms), other signal generators, and so forth. The user input components 1526 may include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo- optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or another pointing instrument), tactile input components (e.g., a physical button, a touch screen that provides location and force of touches or touch gestures, or other tactile input components), audio input components (e.g., a microphone), and the like.

[0161] Communication may be implemented using a wide variety of technologies. The I / O components 1508 further include communication components 1528 operable to couple the machine 1500 to a network 1530 or devices 1532 via respective coupling or connections. For example, the communication components 1528 may include a network interface component or another suitable device to interface with the network 1530. In further examples, the communication components 1528 may include wired communication components, wireless communication components, cellular communication components, Near Field Communication (NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and other communication components to provide communication via other modalities. The devices 1532 may be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a USB).

[0162] Moreover, the communication components 1528 may detect identifiers or include components operable to detect identifiers. For example, the communication components 1528 may include Radio Frequency Identification (RFID) tag reader components, NFC smart tag detection components, optical reader components (e.g., an optical sensor to detect onedimensional bar codes such as Universal Product Code (UPC) bar code, multi-dimensional bar codes such as Quick Response (QR) code, Aztec code, Data Matrix, Dataglyph™, MaxiCode, PDF417, Ultra Code, UCC RSS- 2D bar code, and other optical codes), or acoustic detection components (e.g., microphones to identify tagged audio signals). In addition, a variety of information may be derived via the communication components 1528, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi® signal triangulation, location via detecting an NFC beacon signal that may indicate a particular location, and so forth.

[0163] The various memories (e.g., main memory 1516, static memory 1518, and memory of the processors 1504) and storage unit 1520 may store one or more sets of instructions and data structures (e.g., software) embodying or used by any one or more of the methodologies or functions described herein. These instructions (e.g., the instructions 1502), when executed by processors 1504, cause various operations to implement the disclosed examples.

[0164] The instructions 1502 may be transmitted or received over the network 1530, using a transmission medium, via a network interface device (e.g., a network interface component included in the communication components 1528) and using any one of several well-known transfer protocols (e.g., hypertext transfer protocol (HTTP)). Similarly, the instructions 1502 may be transmitted or received using a transmission medium via a coupling (e.g., a peer-to-peer coupling) to the devices 1532.

[0165] In some examples, adding one or more connections to the network (e.g., at operation 1305) may be accompanied by adding one or more nodes to the network. A new node may be added having one or more connections to one or more existing nodes of the network. For example, a new node maybe added having a pair of bidirectional connections to and from the first node, and a pair of bidirectional connections to and from the second node.

[0166] Similarly, in some examples, removing one or more connections from the network (e.g., at operation 1406) may be accompanied by removing one or more nodes from the network. A node may be removed along with all of its connections to existing nodes of the network. For example, a node having a nodal self-efficacy value falling below a nodal self-efficacy threshold may be removed from the network.

[0167] FIG. 16 illustrates a sequence showing restructuring of a network of a machine learning model over time, according to techniques described herein. The network shown in FIG. 16 not only adds and removes connections, but also adds and removes nodes, over time. The connections shown in FIG. 16 may represent a pair of bidirectional connections, as in network 500.

[0168] The top row of FIG. 16, spanning from time to on the left to time ti on the right, begins on the left with a simple network at time to. A single node 1602 has a connection 1610 to the environment 520. As time progresses, and in response to processing of inputs to generate outputs and inferences during the runtime period, the network is restructured by adding two nodes 1606 and 1608, both having connections 1612, 1614, and 1616 to each other and to the initial node 1602. The three nodes 1602, 1606, and 1608 thereby form a first cluster 1604.

[0169] As time progresses further, just before time ti on the right side of the top row, the network is further restructured to add another set of nodes and connections to form a second cluster 1620. The node 1606 of the first cluster 1604 has a connection 1618 to a node 1622 of the second cluster 1620. In each case of a new connection, the connections may be added through the mechanisms described herein for adding connections (e.g., in response to determination of the similarity of the two connected nodes' outputs over time) and / or through addition of new nodes having connections to existing nodes.

[0170] The bottom row of FIG. 16 spans from time ti on the left to time t2 on the right. On the left side of the bottom row, after time ti, the secondcluster 1620 of the network is restructured to remove one of its nodes and the removed node's connections.

[0171] On the right side of the bottom row, just before time t2, the second cluster 1620 is restructured again to add a further node. It will be noted that, although the second cluster 1620 just before time ti and the 1620 just before time t2 each have five nodes, the pattern of interconnection of the nodes of the second cluster 1620 is different at the two different times. Furthermore, it may be the case that the parameters (e.g., the priors of the various tensors) of each node may be different over time.

[0172] By providing a network of a machine learning model that can be dynamically restructured during runtime, while performing an inference task for which it has been trained, an Al system may be provided that provides one or more benefits. The Al system may use a combined order of specific procedures that dynamically alters the structure of the machine learning model in real time and during inference. In response to certain input data (e.g., data from the environment 520), the Al system may both restructure the architecture of the machine learning model and perform an inference task on the restructured architecture. Not only do some examples and features of the Al system described herein eliminate the need for a separate training process from the inference process, but the dynamic nature of the Al system is different than the nature of traditional machine learning systems. The Al system does not simply automate traditional systems and concepts: instead, by leveraging dynamic architectural restructuring (e.g., by adding and / or removing connections, cycles, and / or nodes), the Al system naturally grows and prunes based on certain characteristics, such as certain characteristics of the network, environmental influences, system performance requirements and limitations, and / or changes thereof over time.

[0173] Various examples described herein may provide systems, method, and computer-readable storage media for adding and removing connections and / or cycles from a network of a machine learning model during a runtime period. In some examples, adding and / or removing connections and / or cycles from the network during the runtime period may allow the behavior of the machine learning model to be dynamically adjusted during runtime, therebychanging the operation of one or more machines being controlled by the machine learning model. Changes in the operation of a machine controlled by the machine learning model may include improving the performance of operation of the machine, such as improving accuracy and / or efficiency of operation, as described above. In some examples, dynamic restructuring of the network of the machine learning model through addition and / or removal of connections and / or cycles may allow a machine controlled by the machine learning model to adapt to new conditions (such as changes in the hardware configuration of the machine, changes in the external environment, changes in the goals of a user, and so on). In some examples, the machine learning model may be aligned with a user's goals or values dynamically during runtime in response to information received regarding a user's goals or values. Other technical features may be readily apparent to one skilled in the art from the figures, descriptions, and claims.

[0174] Example l is a method comprising: accessing a machine learning model, wherein: the machine learning model comprises a network having a plurality of nodes and a plurality of connections; each node comprises parameter values configured to control processing of inputs of the node to generate outputs of the node; and each connection is associated with a transmitting node and a receiving node and is configured to designate an output of the transmitting node as an input to the receiving node; during a training period, at least partially training the machine learning model to perform an inference task by processing a plurality of samples of reference data; during a runtime period, processing the inputs of the plurality of nodes and generating the outputs of the plurality of nodes to perform the inference task with respect to each of a plurality of samples of candidate data; and during the runtime period, restructuring the network of the machine learning model by changing presence of a first connection associated with a first node as the transmitting node and a second node as the receiving node.

[0175] In Example 2, the subject matter of Example 1 includes, wherein: the network of the machine learning model comprises one or more cycles, each cycle comprising a plurality of different nodes in an ordered sequence, each node of the sequence sharing a connection with a next node in thesequence, and a final node of the sequence sharing a connection with an initial node of the sequence.

[0176] In Example 3, the subject matter of Example 2 includes, wherein: the cycle is a directed cycle wherein each node of the sequence is the transmitting node of the connection to the next node in the sequence.

[0177] In Example 4, the subject matter of Example 3 includes, wherein: the changing of presence of the first connection comprises adding the first connection to the network, the adding of the first connection creating a new cycle wherein the first node is the final node of the cycle and the second node is the initial node of the cycle.

[0178] In Example 5, the subject matter of Example 4 includes, wherein: the adding of the first connection to the network is in response to determining that the outputs of the first node are similar to the outputs of the second node.

[0179] In Example 6, the subject matter of Example 5 includes, wherein: the outputs of the first node include, for each respective time step of a plurality of time steps, one or more outputs associated with the respective time step generated by processing one or more inputs of the first node associated with a time step prior to the respective time step; the outputs of the second node include, for each respective time step of a plurality of time steps, one or more outputs associated with the respective time step generated by processing one or more inputs of the second node associated with the time step prior to the respective time step; and the determining that the outputs of the first node are similar to the outputs of the second node comprises determining that, for at least a step threshold number of time steps of the plurality of time steps, the one or more outputs of the first node associated with the time step satisfy an output similarity condition relative to the one or more outputs of the second node associated with the time step.

[0180] In Example 7, the subject matter of Example 6 includes, wherein: the one or more outputs of the first node comprise a first output value; the one or more outputs of the second node comprise a second output value; and the determining that the one or more outputs of the first node satisfy the output similarity condition relative to the one or more outputs of the secondnode comprises: determining that the first output value is equal to the second output value.

[0181] In Example 8, the subject matter of Examples 6-7 includes, wherein: the processing of the one or more inputs of the first node or second node to generate the one or more outputs of the first node or second node comprises: processing the one or more inputs to generate an activation value of the node; and generating the one or more outputs based on the activation value; and the determining that the one or more outputs of the first node satisfy the output similarity condition relative to the one or more outputs of the second node comprises determining that the one or more outputs of the first node and the one or more outputs of the second node are both associated with high activation values of their respective nodes.

[0182] In Example 9, the subject matter of Examples 6-8 includes, wherein: the generating of the one or more inputs of the first node or second node to generate the one or more outputs of the first node or second node comprises: processing the one or more inputs to generate an activation value of the node; and generating the one or more outputs based on the activation value; and the determining that the one or more outputs of the first node satisfies the output similarity condition relative to the one or more outputs of the second node comprises determining that the one or more outputs of the first node and the one or more outputs of the second node are either both associated with high activation values, or both associated with low activation values, of their respective nodes.

[0183] In Example 10, the subject matter of Examples 3-9 includes, wherein: the changing of presence of the first connection comprises removing the first connection from the network, the removal of the first connection removing a previous cycle wherein the first node was the final node of the cycle and the second node was the initial node of the cycle.

[0184] In Example 11, the subject matter of Example 10 includes, wherein: the outputs of the first node include, for each respective time step of a plurality of time steps, one or more outputs associated with the respective time step generated by processing one or more inputs of the first node associated with a time step prior to the respective time step; and theremoving of the first connection is in response to determining that the first connection has a self-efficacy lower than a self-efficacy threshold, wherein self-efficacy is a measure of an ability of the one or more outputs of the first node to the second node via the first connection in association with a first time step to affect the one or more inputs of the first node associated with a second time step after the first time step.

[0185] In Example 12, the subject matter of Example 11 includes, wherein: the ability of the one or more outputs of the first node to the second node via the first connection associated with the first time step to affect the one or more inputs of the first node associated with the second time step is computed based on a variational free energy of the first node with respect to the first connection, wherein: a low variational free energy of a connection corresponds to a high self-efficacy of the connection, and a high variational free energy of the connection corresponds to a low self-efficacy of the connection; and variational free energy is a measure of difference between one or more preferred inputs of the first node and the one or more actual inputs received by the first node associated with the second time step.

[0186] In Example 13, the subject matter of Example 12 includes, generating the one or more outputs of the first node associated with the first time step by: computing one or more predicted inputs of the first node associated with the second time step as a function of the one or more inputs of the first node received in association with the first time step and one or more possible outputs of the first node to be generated in association with the first time step; and selecting the one or more outputs from the one or more possible outputs to reduce a difference between the preferred inputs and the predicted inputs.

[0187] In Example 14, the subject matter of Example 13 includes, wherein: the one or more preferred inputs of the first node associated with the second time step include the one or more predicted inputs of the first node associated with the first time step; and the variational free energy of the first node is inversely based on a similarity of the predicted inputs associated with the first time step to the inputs received in association with the second time step.

[0188] Example 15 is a system comprising: one or more processors; and a non-transitory computer readable storage medium storing: a machine learning model, wherein: the machine learning model comprises a network having a plurality of nodes and a plurality of connections; each node comprises parameter values configured to control processing of inputs of the node to generate outputs of the node; and each connection is associated with a transmitting node and a receiving node and is configured to designate an output of the transmitting node as an input to the receiving node; and instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: during a training period, at least partially training the machine learning model to perform an inference task by processing a plurality of samples of reference data; during a runtime period, processing the inputs of the plurality of nodes and generating the outputs of the plurality of nodes to perform the inference task with respect to each of a plurality of samples of candidate data; and during the runtime period, restructuring the network of the machine learning model by changing presence of a first connection associated with a first node as the transmitting node and a second node as the receiving node.

[0189] In Example 16, the subject matter of Example 15 includes, wherein: the network of the machine learning model comprises one or more directed cycles, each directed cycle comprising a plurality of different nodes in an ordered sequence, each node of the sequence being the transmitting node of a connection to a next node in the sequence, and a final node of the sequence being the transmitting node of a connection to an initial node of the sequence; the changing of presence of the first connection comprises adding the first connection to the network, the adding of the first connection creating a new cycle wherein the first node is the final node of the cycle and the second node is the initial node of the cycle; and the adding of the first connection to the network is in response to determining that the outputs of the first node are similar to the outputs of the second node.

[0190] In Example 17, the subject matter of Example 16 includes, wherein: the outputs of the first node include, for each respective time step of a plurality of time steps, one or more outputs associated with the respectivetime step generated by processing one or more inputs of the first node associated with a time step prior to the respective time step; the outputs of the second node include, for each respective time step of a plurality of time steps, one or more outputs associated with the respective time step generated by processing one or more inputs of the second node associated with the time step prior to the respective time step; and the determining that the outputs of the first node are similar to the outputs of the second node comprises determining that, for at least a step threshold number of time steps of the plurality of time steps, the one or more outputs of the first node associated with the time step satisfy an output similarity condition relative to the one or more outputs of the second node associated with the time step.

[0191] In Example 18, the subject matter of Examples 15-17 includes, wherein: the network of the machine learning model comprises one or more directed cycles, each directed cycle comprising a plurality of different nodes in an ordered sequence, each node of the sequence being the transmitting node of a connection to a next node in the sequence, and a final node of the sequence being the transmitting node of a connection to an initial node of the sequence; the changing of presence of the first connection comprises removing the first connection from the network, the removing of the first connection removing a previous cycle wherein the first node was the final node of the cycle and the second node was the initial node of the cycle; the outputs of the first node include, for each respective time step of a plurality of time steps, one or more outputs associated with the respective time step generated by processing one or more inputs of the first node associated with a time step prior to the respective time step; and the removing of the first connection is in response to determining that the first connection has a low self-efficacy, wherein self-efficacy is a measure of an ability of the one or more outputs of the first node to the second node via the first connection in association with a first time step to affect the one or more inputs of the first node associated with a second time step after the first time step.

[0192] In Example 19, the subject matter of Example 18 includes, wherein: the ability of the one or more outputs of the first node to the second node via the first connection associated with the first time step to affect the one ormore inputs of the first node associated with the second time step is computed based on a variational free energy of the first node with respect to the first connection, wherein: a low variational free energy of a connection corresponds to a high self-efficacy of the connection, and a high variational free energy of the connection corresponds to a low self-efficacy of the connection; and variational free energy is a measure of difference between one or more preferred inputs of the first node and the one or more actual inputs received by the first node associated with the second time step.

[0193] Example 20 is a non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a system, cause the system to perform operations comprising: accessing a machine learning model, wherein: the machine learning model comprises a network having a plurality of nodes and a plurality of connections; each node comprises parameter values configured to control processing of inputs of the node to generate outputs of the node; and each connection is associated with a transmitting node and a receiving node and is configured to designate an output of the transmitting node as an input to the receiving node; during a training period, at least partially training the machine learning model to perform an inference task by processing a plurality of samples of reference data; during a runtime period, processing the inputs of the plurality of nodes and generating the outputs of the plurality of nodes to perform the inference task with respect to each of a plurality of samples of candidate data; and during the runtime period, restructuring the network of the machine learning model by changing presence of a first connection associated with a first node as the transmitting node and a second node as the receiving node.

[0194] Example 21 is at least one machine-readable medium including instructions that, when executed by processing circuitry, cause the processing circuitry to perform operations to implement of any of Examples 1-20.

[0195] Example 22 is an apparatus comprising means to implement of any of Examples 1-20.

[0196] Example 23 is a system to implement of any of Examples 1-20.

[0197] Example 24 is a method to implement of any of Examples 1-20.

[0198] Unless the context clearly requires otherwise, throughout the description and the claims, the words “comprise,” “comprising,” and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense, i.e., in the sense of “including, but not limited to.” As used herein, the terms “connected,” “coupled,” or any variant thereof means any connection or coupling, either direct or indirect, between two or more elements; the coupling or connection between the elements can be physical, logical, or a combination thereof. Additionally, the words “herein,” “above,” “below,” and words of similar import, when used in this application, refer to this application as a whole and not to any particular portions of this application. Where the context permits, words using the singular or plural number may also include the plural or singular number respectively. The word “or” in reference to a list of two or more items, covers all of the following interpretations of the word: any one of the items in the list, all of the items in the list, and any combination of the items in the list. Likewise, the term “and / or” in reference to a list of two or more items, covers all of the following interpretations of the word: any one of the items in the list, all of the items in the list, and any combination of the items in the list.

[0199] A “non-transitory” machine-readable medium, such as a “non- transitory” computer readable storage medium, as used herein, specifically excludes propagating signals per se. A non-transitory machine-readable medium may also be referred to herein as a tangible machine-readable medium. According to various example embodiments, instructions for execution by a machine, and other information such as machine learning models and the parameters thereof, can be communicated via a carrier medium (e.g., a machine-readable carrier medium). Examples of such a carrier medium include a non-transient carrier medium (e.g., a non-transitory machine-readable storage medium, such as a solid-state memory that is physically movable from one place to another place) and a transient carrier medium (e.g., a carrier wave or other propagating signal that communicates instructions or information).

[0200] Although some examples, e.g., those depicted in the drawings, include a particular sequence of operations, the sequence may be altered without departing from the scope of the present disclosure. For example, some of the operations depicted may be performed in parallel or in a different sequence that does not materially affect the functions as described in the examples. In other examples, different components of an example device or system that implements an example method may perform functions at substantially the same time or in a specific sequence.

[0201] The various features, steps, and processes described herein may be used independently of one another, or may be combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of this disclosure. In addition, certain method or process blocks may be omitted in some implementations.

Claims

CLAIMSWhat is claimed is:

1. A method comprising: accessing a machine learning model, wherein: the machine learning model comprises a network having a plurality of nodes and a plurality of connections; each node comprises parameter values configured to control processing of inputs of the node to generate outputs of the node; and each connection is associated with a transmitting node and a receiving node and is configured to designate an output of the transmitting node as an input to the receiving node; during a training period, at least partially training the machine learning model to perform an inference task by processing a plurality of samples of reference data; during a runtime period, processing the inputs of the plurality of nodes and generating the outputs of the plurality of nodes to perform the inference task with respect to each of a plurality of samples of candidate data; and during the runtime period, restructuring the network of the machine learning model by changing presence of a first connection associated with a first node as the transmitting node and a second node as the receiving node.

2. The method of claim 1, wherein: the network of the machine learning model comprises one or more cycles, each cycle comprising a plurality of different nodes in an ordered sequence, each node of the sequence sharing a connection with a next node in the sequence, and a final node of the sequence sharing a connection with an initial node of the sequence.

3. The method of claim 2, wherein: the cycle is a directed cycle wherein each node of the sequence is the transmitting node of the connection to the next node in the sequence.

4. The method of claim 3, wherein: the changing of presence of the first connection comprises adding the first connection to the network, the adding of the first connection creating a new cycle wherein the first node is the final node of the cycle and the second node is the initial node of the cycle.

5. The method of claim 4, wherein: the adding of the first connection to the network is in response to determining that the outputs of the first node are similar to the outputs of the second node.

6. The method of claim 5, wherein: the outputs of the first node include, for each respective time step of a plurality of time steps, one or more outputs associated with the respective time step generated by processing one or more inputs of the first node associated with a time step prior to the respective time step; the outputs of the second node include, for each respective time step of a plurality of time steps, one or more outputs associated with the respective time step generated by processing one or more inputs of the second node associated with the time step prior to the respective time step; and the determining that the outputs of the first node are similar to the outputs of the second node comprises determining that, for at least a step threshold number of time steps of the plurality of time steps, the one or more outputs of the first node associated with the time step satisfy an output similarity condition relative to the one or more outputs of the second node associated with the time step.

7. The method of claim 6, wherein: the one or more outputs of the first node comprise a first output value; the one or more outputs of the second node comprise a second output value; andthe determining that the one or more outputs of the first node satisfy the output similarity condition relative to the one or more outputs of the second node comprises: determining that the first output value is equal to the second output value.

8. The method of claim 6, wherein: the processing of the one or more inputs of the first node or second node to generate the one or more outputs of the first node or second node comprises: processing the one or more inputs to generate an activation value of the node; and generating the one or more outputs based on the activation value; and the determining that the one or more outputs of the first node satisfy the output similarity condition relative to the one or more outputs of the second node comprises determining that the one or more outputs of the first node and the one or more outputs of the second node are both associated with high activation values of their respective nodes.

9. The method of claim 6, wherein: the generating of the one or more inputs of the first node or second node to generate the one or more outputs of the first node or second node comprises: processing the one or more inputs to generate an activation value of the node; and generating the one or more outputs based on the activation value; and the determining that the one or more outputs of the first node satisfies the output similarity condition relative to the one or more outputs of the second node comprises determining that the one or more outputs of the first node and the one or more outputs of the second node are either both associated with high activation values, or both associated with low activation values, of their respective nodes.

10. The method of claim 3, wherein: the changing of presence of the first connection comprises removing the first connection from the network, the removal of the first connection removing a previous cycle wherein the first node was the final node of the cycle and the second node was the initial node of the cycle.

11. The method of claim 10, wherein: the outputs of the first node include, for each respective time step of a plurality of time steps, one or more outputs associated with the respective time step generated by processing one or more inputs of the first node associated with a time step prior to the respective time step; and the removing of the first connection is in response to determining that the first connection has a self-efficacy lower than a self-efficacy threshold, wherein self-efficacy is a measure of an ability of the one or more outputs of the first node to the second node via the first connection in association with a first time step to affect the one or more inputs of the first node associated with a second time step after the first time step.

12. The method of claim 11, wherein: the ability of the one or more outputs of the first node to the second node via the first connection associated with the first time step to affect the one or more inputs of the first node associated with the second time step is computed based on a variational free energy of the first node with respect to the first connection, wherein: a low variational free energy of a connection corresponds to a high self-efficacy of the connection, and a high variational free energy of the connection corresponds to a low self-efficacy of the connection; and variational free energy is a measure of difference between one or more preferred inputs of the first node and the one or more actual inputs received by the first node associated with the second time step.

13. The method of claim 12, further comprising: generating the one or more outputs of the first node associated with the first time step by:computing one or more predicted inputs of the first node associated with the second time step as a function of the one or more inputs of the first node received in association with the first time step and one or more possible outputs of the first node to be generated in association with the first time step; and selecting the one or more outputs from the one or more possible outputs to reduce a difference between the preferred inputs and the predicted inputs.

14. The method of claim 13, wherein: the one or more preferred inputs of the first node associated with the second time step include the one or more predicted inputs of the first node associated with the first time step; and the variational free energy of the first node is inversely based on a similarity of the predicted inputs associated with the first time step to the inputs received in association with the second time step.

15. A system comprising: one or more processors; and a non-transitory computer readable storage medium storing: a machine learning model, wherein: the machine learning model comprises a network having a plurality of nodes and a plurality of connections; each node comprises parameter values configured to control processing of inputs of the node to generate outputs of the node; and each connection is associated with a transmitting node and a receiving node and is configured to designate an output of the transmitting node as an input to the receiving node; and instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:during a training period, at least partially training the machine learning model to perform an inference task by processing a plurality of samples of reference data; during a runtime period, processing the inputs of the plurality of nodes and generating the outputs of the plurality of nodes to perform the inference task with respect to each of a plurality of samples of candidate data; and during the runtime period, restructuring the network of the machine learning model by changing presence of a first connection associated with a first node as the transmitting node and a second node as the receiving node.

16. The system of claim 15, wherein: the network of the machine learning model comprises one or more directed cycles, each directed cycle comprising a plurality of different nodes in an ordered sequence, each node of the sequence being the transmitting node of a connection to a next node in the sequence, and a final node of the sequence being the transmitting node of a connection to an initial node of the sequence; the changing of presence of the first connection comprises adding the first connection to the network, the adding of the first connection creating a new cycle wherein the first node is the final node of the cycle and the second node is the initial node of the cycle; and the adding of the first connection to the network is in response to determining that the outputs of the first node are similar to the outputs of the second node.

17. The system of claim 16, wherein: the outputs of the first node include, for each respective time step of a plurality of time steps, one or more outputs associated with the respective time step generated by processing one or more inputs of the first node associated with a time step prior to the respective time step; the outputs of the second node include, for each respective time step of a plurality of time steps, one or more outputs associated with therespective time step generated by processing one or more inputs of the second node associated with the time step prior to the respective time step; and the determining that the outputs of the first node are similar to the outputs of the second node comprises determining that, for at least a step threshold number of time steps of the plurality of time steps, the one or more outputs of the first node associated with the time step satisfy an output similarity condition relative to the one or more outputs of the second node associated with the time step.

18. The system of claim 15, wherein: the network of the machine learning model comprises one or more directed cycles, each directed cycle comprising a plurality of different nodes in an ordered sequence, each node of the sequence being the transmitting node of a connection to a next node in the sequence, and a final node of the sequence being the transmitting node of a connection to an initial node of the sequence; the changing of presence of the first connection comprises removing the first connection from the network, the removing of the first connection removing a previous cycle wherein the first node was the final node of the cycle and the second node was the initial node of the cycle; the outputs of the first node include, for each respective time step of a plurality of time steps, one or more outputs associated with the respective time step generated by processing one or more inputs of the first node associated with a time step prior to the respective time step; and the removing of the first connection is in response to determining that the first connection has a low self-efficacy, wherein self-efficacy is a measure of an ability of the one or more outputs of the first node to the second node via the first connection in association with a first time step to affect the one or more inputs of the first node associated with a second time step after the first time step.

19. The system of claim 18, wherein:the ability of the one or more outputs of the first node to the second node via the first connection associated with the first time step to affect the one or more inputs of the first node associated with the second time step is computed based on a variational free energy of the first node with respect to the first connection, wherein: a low variational free energy of a connection corresponds to a high self-efficacy of the connection, and a high variational free energy of the connection corresponds to a low self-efficacy of the connection; and variational free energy is a measure of difference between one or more preferred inputs of the first node and the one or more actual inputs received by the first node associated with the second time step.

20. A non-transitory computer-readable storage medium, the computer- readable storage medium including instructions that when executed by a system, cause the system to perform operations comprising: accessing a machine learning model, wherein: the machine learning model comprises a network having a plurality of nodes and a plurality of connections; each node comprises parameter values configured to control processing of inputs of the node to generate outputs of the node; and each connection is associated with a transmitting node and a receiving node and is configured to designate an output of the transmitting node as an input to the receiving node; during a training period, at least partially training the machine learning model to perform an inference task by processing a plurality of samples of reference data; during a runtime period, processing the inputs of the plurality of nodes and generating the outputs of the plurality of nodes to perform the inference task with respect to each of a plurality of samples of candidate data; and during the runtime period, restructuring the network of the machine learning model by changing presence of a first connection associated with a first node as the transmitting node and a second node as the receiving node.

Images