Discrete internodal input / output for machine learning

Abstract

A method includes obtaining one or more inputs of a node of a network of a machine learning model, each input of the node having a respective input value, each input value being a member of a finite set of discrete values. The one or more inputs of the node are processed to generate one or more outputs of the node, each output having a respective output value, each output value being generated as a member of the finite set of discrete values by quantization. The machine learning model is used to generate inference data based at least in part on the one or more outputs of the node.

Description

DISCRETE INTERNODAL INPUT / OUTPUT FOR MACHINE LEARNINGTECHNICAL FIELD

[0001] The present disclosure relates generally to machine learning and, more particularly, to nodes in machine learning models having discrete input values and discrete output values.BACKGROUND

[0002] Machine learning uses computer algorithms to automatically learn patterns and relationships in data, potentially without the need for explicit programming. A conventional machine learning process begins with a designer configuring or selecting an untrained machine learning model having a specific structure. The model is then trained to perform an inference task in a problem domain using training data drawn from the problem domain. When training is complete, the trained model is deployed to perform the inference task at runtime.BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0003] In the drawings, which are not necessarily drawn to scale, like numerals may describe similar components in different views. To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the figure number in which that element is first introduced. Some non-limiting example embodiments of the subject matter are illustrated in the figures of the accompanying drawings in which:

[0004] FIG. 1 illustrates a machine-learning pipeline, providing context for examples described herein.

[0005] FIG. 2 illustrates training and use of an example machine-learning program, providing context for examples described herein.

[0006] FIG. 3 illustrates a first example network of a machine learning model, according to some examples.

[0007] FIG. 4 illustrates a further example network of a machine learning model, according to some examples.

[0008] FIG. 5 illustrates initial values of priors of a node of the network of FIG. 4, according to some examples.

[0009] FIG. 6 illustrates initial values of priors of the Bo submatrix of FIG. 5, according to some examples.

[0010] FIG. 7 illustrates differentiated values of priors of the Bo submatrix of FIG. 6, according to some examples.

[0011] FIG. 8 illustrates initial values of priors of the Bi submatrix of FIG. 5, according to some examples.

[0012] FIG. 9 illustrates differentiated values of priors of the Bi submatrix of FIG. 8, according to some examples.

[0013] FIG. 10 illustrates initial prior values and three different sets of differentiated values of priors of the B matrix of FIG. 5, according to some examples.

[0014] FIG. 11 illustrates prior values of an example Bo submatrix of a node configured to operate as a NAND logic gate, according to some examples.

[0015] FIG. 12 illustrates prior values of an example Bi submatrix of the node of FIG. 11, according to some examples.

[0016] FIG. 13 illustrates prior values of example B matrices of nodes configured to operate as NOR, XNOR, and NOT logic gates, according to some examples.

[0017] FIG. 14 illustrates prior values of example B matrices of nodes configured to operate as AND, OR, and XOR logic gates, according to some examples.

[0018] FIG. 15 illustrates a flowchart showing operations of a method for processing discrete inputs to generate discrete outputs of a node of a network of a machine learning model, according to some examples.

[0019] FIG. 16 illustrates a schematic diagram of a circuit design system using a network of a machine learning model, according to some examples.

[0020] FIG. 17 illustrates a flowchart showing operations of a method for revising the design of a digital circuit using a network of a machine learning model, according to some examples.

[0021] FIG. 18 is a diagrammatic representation of a machine in the form of a computer system within which a set of instructions may be executed to cause the machine to perform any one or more of the methodologies discussed herein, according to some examples.DETAILED DESCRIPTION

[0022] Examples described in the present disclosure provide techniques for machine learning in which a machine learning model includes a network of nodes having inter-nodal communication of inputs and outputs, and in which the inputs and outputs of the nodes are discrete values instead of continuous values. In some examples, the inputs and outputs are one-bit binary values. Using discrete values for inter-nodal inputs and outputs may allow the machine learning model to realize various benefits, as described in more detail below.

[0023] Some existing machine learning model types, including artificial neural networks (also referred to herein simply as neural networks), Bayesian networks, and Markov models, are structured as a network of nodes. The structure of the network is predefined before training begins, and parameters of the model (which may include parameters representing the strength of connections between nodes of the network, also called weights) are adjusted during a training phase. Once the model is deemed to be trained, the training phase ends, and the trained model can be deployed to perform an inference task in an inference phase.

[0024] Deep learning techniques may use neural networks having large numbers of nodes (sometimes called deep neural networks or DNNs). Insome cases, each node of a DNN receives one or more inputs encoded as continuous variables (e.g., floating-point variables) and uses these inputs and a set of nodal parameters (also sometimes encoded as floating-point numbers) to compute one or more outputs (which may also be encoded as floating-point numbers). Thus, propagating a set of inputs through a single node to generate that node's outputs may require a computationally intensive floating-point arithmetic operation that consumes a large quantity of processor cycles, memory, power, and time when iterated over the millions of parameters that characterize some DNNs.

[0025] Some neural networks are modeled on biological cellular networks. These neural networks generally use analog inputs and outputs, operate in continuous time instead of in discrete time steps, and / or use floating point values for inputs and outputs. In contrast, examples described herein may operate according to a sequence of discrete time steps and use only discrete values, such as single-bit binary values, for inputs and outputs.

[0026] One potential benefit of using binary inputs and outputs is that, in some cases, nodes of the network may have their parameters configured such that the nodes operate in accordance with Boolean logical operators such as AND, OR, or NOT operators. For example, a node may have parameter values that cause it to process two binary inputs ii and i2 to generate a single binary output 01 such that (01 = ii AND io). In some cases, the parameters of a node may be configured such that the node behaves as a logic gate, such as an AND, OR, XOR, NOT, NAND, NOR, or XNOR gate. This capability may be exploited in some examples by using the network of the machine learning model to model or simulate the behavior of a digital circuit by mapping the digital logic of the logic gates of the circuit to the parameter values of the nodes of the network. The network can then be adjusted through a training or optimization process, as described below. The adjusted network can then be translated back into circuit logic by mapping the parameter values of the nodes to logic gates, thereby generating a revised design for the digital circuit. This process can be performed in various ways in various examples: for example, the process may begin with a network, which is used to generate a design for a digital circuit; alternatively, theprocess may begin with an initial design for the digital circuit, translate the initial circuit design to a network, adjust the network, and re-translate the adjusted network to a revised design for the initial circuit. Examples of a circuit design system and a method for generating circuit designs are described with reference to FIG. 16 and FIG. 17 below.

[0027] Another potential benefit of using discrete inputs and outputs is that, in some examples, the set of inputs received by a given node may define a finite and relatively small space of possible input states. In some examples, the parameters of the nodes may be used to process these inputs by processing each possible input state according to a distinct subset of parameters. For example, the node may include or be associated with a set of parameters organized into a tensor. This parameter tensor may define a plurality of distinct portions, each portion corresponding to a distinct input state of the node. In some cases, only the portion of the parameter tensor corresponding to the current input state is used to generate the outputs of the node, thereby limiting the number of computations needed to generate the outputs for the node. Examples of the use of a portion of a parameter tensor to generate outputs of a node are described below with reference to FIG. 9 through FIG. 15.

[0028] FIG. 1 is a flowchart depicting a machine-learning pipeline 200, and FIG. 2 is a schematic diagram illustrating the training and use of an example machine-learning program. FIG. 1 and FIG. 2 provide context for examples described herein by describing various techniques and approaches to structuring, training, validating, and deploying a machine learning model.

[0029] The machine-learning pipeline 200 may be used to generate a trained machine learning model, for example the trained machine-learning program 202 of FIG. 2, to perform operations associated with searches and query responses.

[0030] Broadly, machine learning may involve using computer algorithms to automatically learn patterns and relationships in data, potentially without the need for explicit programming. Machine learning algorithms can be divided into three main categories: supervised learning, unsupervised learning, and reinforcement learning.• It will be appreciated that different values for the priors of the B tensors 512 shown in FIG. 11 through FIG. 14 may be used in some cases to achieve logic gate behavior. In some examples, a node may have a non-uniform C tensor 508, such that its behavior may be affected by its preferences. This may require that the priors of the B tensor 512 are different from those shown in the illustrated examples in order to achieve consistent logic gate behavior. In some examples, it may be possible to use less absolute values for the various priors shown in FIG. 11 through FIG. 14 and still achieve consistent logic gate behavior. For example, instead of a column having a one-hot array of priors (e.g., {0, 0, 0, 1 }), in some examples a column may include a less absolute distribution of priors (e.g., {0.1, 0.2, 0.1, 0.6}) and still achieve the same logic gate behavior as the illustrated examples. Supervised learning involves training a model using labeled data to predict an output for new, unseen inputs. Examples of supervised learning algorithms include linear regression, decision trees, and neural networks.• Unsupervised learning involves training a model on unlabeled data to find hidden patterns and relationships in the data. Examples of unsupervised learning algorithms include clustering, principal component analysis, and generative models like autoencoders.• Reinforcement learning involves training a model to make decisions in a dynamic environment by receiving feedback in the form of rewards or penalties. Examples of reinforcement learning algorithms include Q- learning and policy gradient methods.

[0031] Examples of specific machine learning algorithms that may be deployed, according to some examples, include logistic regression, which is a type of supervised learning algorithm used for binary classification tasks. Logistic regression models the probability of a binary response variable based on one or more predictor variables. Another example type of machine learning algorithm is Naive Bayes, which is another supervised learning algorithm used for classification tasks. Naive Bayes is based on Bayes' theorem and assumes that the predictor variables are independent of each other. Random Forest is another type of supervised learning algorithm usedfor classification, regression, and other tasks. Random Forest builds a collection of decision trees and combines their outputs to make predictions. Further examples include neural networks, which consist of interconnected layers of nodes (or neurons) that process information and make predictions based on the input data. Matrix factorization is another type of machine learning algorithm used for recommender systems and other tasks. Matrix factorization decomposes a matrix into two or more matrices to uncover hidden patterns or relationships in the data. Support Vector Machines (SVM) are a type of supervised learning algorithm used for classification, regression, and other tasks. SVM finds a hyperplane that separates the different classes in the data. Other types of machine learning algorithms include decision trees, k-nearest neighbors, clustering algorithms, and deep learning algorithms such as convolutional neural networks (CNN), recurrent neural networks (RNN), and transformer models. The choice of algorithm depends on the nature of the data, the complexity of the problem, and the performance requirements of the application.

[0032] The performance of machine learning models may be evaluated on a separate test set of data that was not used during training to ensure that the model can generalize to new, unseen data.

[0033] Although several specific examples of machine learning algorithms are discussed herein, the principles discussed herein can be applied to other machine learning algorithms as well. Deep learning algorithms such as convolutional neural networks, recurrent neural networks, and transformers, as well as more traditional machine learning algorithms like decision trees, random forests, and gradient boosting may be used in various machine learning applications.

[0034] Machine learning models are trained to perform specific types of inference tasks. Two example types of inference tasks in machine learning are classification tasks and regression tasks. Classification tasks, also referred to as categorization tasks, aim at classifying items into one of several category values (for example, is this object an apple or an orange?). Regression algorithms aim at quantifying some items (for example, by providing a value that is a real number) to perform a regression task.

[0035] Generating a trained machine-learning program 202 may include multiple phases that form part of the machine-learning pipeline 200, including for example the following phases illustrated in FIG. 1 :• Data collection and preprocessing 102: This phase may include acquiring and cleaning data to ensure that it is suitable for use in the machine learning model. This phase may also include removing duplicates, handling missing values, and converting data into a suitable format.• Feature engineering 104: This phase may include selecting and transforming the training data 206 to create features that are useful for predicting the target variable. Feature engineering may include (1) receiving features 208 (e.g., as structured or labeled data in supervised learning) and / or (2) identifying features 208 (e.g., unstructured or unlabeled data for unsupervised learning) in training data 206.• Model selection and training 106: This phase may include selecting an appropriate machine learning algorithm and training it on the preprocessed data. This phase may further involve splitting the data into training and testing sets, using cross-validation to evaluate the model, and tuning hyperparameters to improve performance.• Model evaluation 108: This phase may include evaluating the performance of a trained model (e.g., the trained machine-learning program 202) on a separate testing dataset. This phase can help determine if the model is overfitting or underfitting and determine whether the model is suitable for deployment.• Prediction 110: This phase involves using a trained model (e.g., trained machine-learning program 202) to generate predictions on new, unseen data.• Validation, refinement or retraining 112: This phase may include updating a model based on feedback generated from the prediction phase, such as new data or user feedback.• Deployment 114: This phase may include integrating the trained model (e.g., the trained machine-learning program 202) into a more extensive system or application, such as a web service, mobile app, orloT device. This phase can involve setting up APIs, building a user interface, and ensuring that the model is scalable and can handle large volumes of data.

[0036] FIG. 2 illustrates further details of two example phases, namely a training phase 204 (e.g., part of the model selection and trainings 106) and a prediction phase 210 (part of prediction 110). Prior to the training phase 204, feature engineering 104 is used to identify features 208. This may include identifying informative, discriminating, and independent features for effectively operating the trained machine-learning program 202 in pattern recognition, classification, and regression. In some examples, the training data 206 includes labeled data, known for pre-identified features 208 and one or more outcomes. Each of the features 208 may be a variable or attribute, such as an individual measurable property of a process, article, system, or phenomenon represented by a data set (e.g., the training data 206). Features 208 may also be of different types, such as numeric features, strings, and graphs, and may include one or more of content 212, concepts 214, attributes 216, historical data 218 , and / or user data 220, merely for example.

[0037] In training phase 204, also referred to herein as a training period, the machine-learning pipeline 200 uses the training data 206 to find correlations among the features 208 that affect a predicted outcome or prediction / inference data 222. The training data may also be referred to herein as reference data, because it is used as a reference for setting the trained parameter values of the model.

[0038] With the training data 206 and the identified features 208, the trained machine-learning program 202 is trained during the training phase 204 during machine-learning program training 224. The machine-learning program training 224 appraises values of the features 208 as they correlate to the training data 206. The result of the training is the trained machinelearning program 202 (e.g., a trained or learned model).

[0039] Further, the training phase 204 may involve machine learning, in which the training data 206 is structured (e.g., labeled during preprocessing operations). The trained machine-learning program 202 implements a neuralnetwork 226 capable of performing, for example, classification and clustering operations. In other examples, the training phase 204 may involve deep learning, in which the training data 206 is unstructured, and the trained machine-learning program 202 implements a deep neural network 226 that can perform both feature extraction and classification / clustering operations.

[0040] In some examples, a neural network 226 may be generated during the training phase 204, and implemented within the trained machine-learning program 202. The neural network 226 includes a hierarchical (e.g., layered) organization of neurons, with each layer consisting of multiple neurons or nodes. Neurons in the input layer receive the input data, while neurons in the output layer produce the final output of the network. Between the input and output layers, there may be one or more hidden layers, each consisting of multiple neurons.

[0041] Each neuron in the neural network 226 operationally computes a function, such as an activation function, which takes as input the weighted sum of the outputs of the neurons in the previous layer, as well as a bias term. The output of this function is then passed as input to the neurons in the next layer. If the output of the activation function exceeds a certain threshold, an output is communicated from that neuron (e.g., transmitting neuron) to a connected neuron (e.g., receiving neuron) in successive layers. The connections between neurons have associated weights, which define the influence of the input from a transmitting neuron to a receiving neuron. During the training phase, these weights are adjusted by the learning algorithm to optimize the performance of the network. Different types of neural networks may use different activation functions and learning algorithms, affecting their performance on different tasks. The layered organization of neurons and the use of activation functions and weights enable neural networks to model complex relationships between inputs and outputs, and to generalize to new inputs that were not seen during training.

[0042] In some examples, the neural network 226 may also be one of several different types of neural networks, such as a single-layer feedforward network, a Multilayer Perceptron (MLP), an Artificial Neural Network (ANN), a Recurrent Neural Network (RNN), a Long Short-TermMemory Network (LSTM), a Bidirectional Neural Network, a symmetrically connected neural network, a Deep Belief Network (DBN), a Convolutional Neural Network (CNN), a Generative Adversarial Network (GAN), an Autoencoder Neural Network (AE), a Restricted Boltzmann Machine (RBM), a Hopfield Network, a Self-Organizing Map (SOM), a Radial Basis Function Network (RBFN), a Spiking Neural Network (SNN), a Liquid State Machine (LSM), an Echo State Network (ESN), a Neural Turing Machine (NTM), or a Transformer Network, merely for example.

[0043] In addition to the training phase 204, a validation phase may be performed on a separate dataset known as the validation dataset. The validation dataset is used to tune the hyperparameters of a model, such as the learning rate and the regularization parameter. The hyperparameters are adjusted to improve the model's performance on the validation dataset.

[0044] Once a model is fully trained and validated, in a testing phase, the model may be tested on a new dataset. The testing dataset is used to evaluate the model's performance and ensure that the model has not overfitted the training data.

[0045] In prediction phase 210, also referred to herein as a runtime period, the trained machine-learning program 202 uses the features 208 for analyzing query data 228 to generate inferences, outcomes, or predictions, as examples of a prediction / inference data 222. For example, during prediction phase 210, the trained machine-learning program 202 generates an output. Query data 228 is provided as an input to the trained machine-learning program 202, and the trained machine-learning program 202 generates the prediction / inference data 222 as output, responsive to receipt of the query data 228. Query data processed by the model during the runtime phase may also be referred to herein as candidate data, because it is data that is a priori not associated with any particular inference output.

[0046] In some examples, the trained machine-learning program 202 may be a generative Al model. Generative Al is a term that may refer to any type of artificial intelligence that can create new content from training data 206. For example, generative Al can produce text, images, video, audio, code, or synthetic data similar to the original data but not identical.

[0047] Some of the techniques that may be used in generative Al are:• Convolutional Neural Networks (CNNs): CNNs may be used for image recognition and computer vision tasks. CNNs may, for example, be designed to extract features from images by using filters or kernels that scan the input image and highlight important patterns.• Recurrent Neural Networks (RNNs): RNNs may be used for processing sequential data, such as speech, text, and time series data, for example. RNNs employ feedback loops that allow them to capture temporal dependencies and remember past inputs. The feedback loops of an RNN can be considered a cycle within its network.• Generative adversarial networks (GANs): GNNs may include two neural networks: a generator and a discriminator. The generator network attempts to create realistic content that can “fool” the discriminator network, while the discriminator network attempts to distinguish between real and fake content. The generator and discriminator networks compete with each other and improve over time.• Variational autoencoders (VAEs): VAEs may encode input data into a latent space (e.g., a compressed representation) and then decode it back into output data. The latent space can be manipulated to generate new variations of the output data. VAEs may use self-attention mechanisms to process input data, allowing them to handle long text sequences and capture complex dependencies.• Transformer models: Transformer models may use attention mechanisms to learn the relationships between different parts of input data (such as words or pixels) and generate output data based on these relationships. Transformer models can handle sequential data, such as text or speech, as well as non-sequential data, such as images or code.

[0048] In generative Al examples, the output prediction / inference data 222 include predictions, translations, summaries or media content.

[0049] FIG. 3 illustrates a first example network 300 of a machine learning model, such as trained machine-learning program 202. The network 300 includes a set of nodes, shown as first node 302, second node 304, and third node 306. The nodes are connected to each other by unidirectionalconnections, shown as a first connection 308 (from first node 302 to second node 304), a second connection 310 (from second node 304 to third node 306), and a third connection 312 (from third node 306 to first node 302). Each connection conveys or designates outputs of a transmitting node as inputs to a receiving node. The connections for which a given node is a transmitting node may be referred to as outgoing connections of that node; by the same token, those connections for which a node is the receiving node may be referred to as incoming connections of that node. In network 300, the transmitting node of first connection 308 is first node 302, and the receiving node of first connection 308 is second node 304; the transmitting node of second connection 310 is second node 304, and the receiving node of second connection 310 is third node 306; and the transmitting node of third connection 312 is third node 306, and the receiving node of third connection 312 is first node 302.

[0050] The nodes of the network 300 (and other networks of machine learning models described herein) may be described herein as receiving inputs, processing data, and generating outputs. Similarly, the connections of the described networks may be described as conveying, transmitting, or providing data from one node to another node. In some examples, these terms may refer to the operation of nodes and / or connection embodied as logical circuit components or other physical data processing and / or data communication components. For example, the nodes and connections of a network of a machine learning model as described herein may be implemented as circuit components of a field programmable gate array (FPGA) or another dynamically reconfigurable logical circuit. In some examples, the use of binary inputs and outputs for the nodes of the network enable the implementation of the network as a circuit comprising a plurality of logic gates.

[0051] In some examples, the nodes and / or connections may be implemented as data, for example, as data structures and / or instructions stored in a memory and manipulated by a physical processor. The machine 1800 described with reference to FIG. 18 below provides an example of a computing system suitable for such an implementation. In some examples,the processor executes instructions to perform operations of a system, such as an artificial intelligence (Al) system, that manipulates the data corresponding to the machine learning model (including the nodes and connections of the network) to perform the operations described herein. For example, a node may be implemented as a data structure associated with one or more parameters (e.g., prior probability distributions, also called priors, as described below), and processing the inputs of a node to generate the node's outputs may be performed by the physical processor processing the parameter values of the node and the inputs of the node to generate the outputs of the node. A connection may be implemented as data designating or identifying a relationship between a transmitting node and a receiving node, such that the outputs of the transmitting node are treated by the physical processor as inputs to the receiving node. In some examples, the nodes and / or connections may include instructions executed by the processor to perform these operations, whereas in other examples, the instructions are separate from the nodes and connections, and cause the processor to process the data corresponding to the nodes and connections. Thus, in some examples, each node may be said to include or be associated with parameter values, and these parameter values may be used to control the processing of the inputs of the node in order to generate the outputs of the node. It will be appreciated that all references herein to a node processing data, to a connection conveying data, and so on, are all intended to encompass all suitable implementations of the nodes and / or connections as any combination of hardware and / or software.

[0052] In some examples, as described below with reference to FIG. 16 and FIG. 17, the network may be implemented in software or in reconfigurable hardware during some phases requiring reconfiguration of the network (e.g., adjustment of parameter values of the nodes), and the behavior of the network at a point in time may be fixed by mapping the network's behavior to a set of interconnected logic gates to create a design for a digital circuit, which may then be constructed to implement the behavior of the network as a physical digital circuit.

[0053] In some examples, the nodes of the network 300 operate to process their inputs and generate their outputs at each of a sequence of discrete time steps. In some examples, all nodes of the network 300 operate at each time step; in other examples, some nodes may have different rates of operation such that some nodes of the network 300 may only process their inputs and generate (or update) their outputs at some time steps, such as every n time steps where n is a positive integer.

[0054] In some examples, a node generates its outputs at a current time step t by processing its inputs at a previous or prior time step t- . The inputs may be processed by applying various parameters (e.g., priors), such as matrices of priors, to the inputs, and computing one or more output values based on the processed parameters and inputs. In some examples, a node may generate only a single output value that is conveyed as an input to all connected nodes; in other examples, a node may generate multiple outputs to be conveyed to multiple connected nodes and / or as multiple output values conveyed to a single node.

[0055] In some examples, a single output generated by a node may be an activation value. An activation value may be regarded as a measure of how active the node is at the current time step based on its inputs from the previous time step. The activation value of a given node is a discrete value, such as a binary value (e.g., 0 or 1, corresponding to inactive or active, or “not firing” and “firing” in the context of an analogy of neuronal firing), or another member of a finite set of discrete values (e.g., a member of the set {0, 1, 2, 3} or a member of the set {00, 01, 10, 11 }). The discrete activation value is determined by quantization, which may be either a stochastic quantization process or a deterministic quantization process. In a stochastic quantization process, for example, a distribution (e.g., a probability distribution) over all possible discrete output values may be generated by processing the inputs and priors, and a random or pseudo-random value may be used to select a discrete output value (or set of output values) from the probability distribution. In a deterministic quantization process, a single discrete output value (or set of output values) may instead be selected from the distribution of possible discrete output values, e.g., as the value or valueshaving the highest probability or the highest prevalence. Examples of distributions of output values are described below with reference to FIG. 6 through FIG. 12.

[0056] In some examples, nodes may have connections to themselves, referred to as self-connections, in which they serve as both transmitting node and receiving node. In some such examples, a node may generate a single output for propagation to its connected nodes, and another output or set of outputs to itself, called self-outputs, via its self-connection(s). In some examples, a node may pass through its current inputs as self-outputs, thereby providing a one-time-step memory of its own inputs. This may enable the node to evaluate the accuracy of its own previous predictions of state transitions of the observed system, as described in greater detail below with reference to FIG. 5 through FIG. 10.

[0057] It will be appreciated that the three-node network 300 shown in FIG. 3 is intended as a simplified example, and that actual machine learning models may include networks having much larger number of nodes, such as thousands or millions of nodes. Thus, the network 300 may be regarded as a small section of a larger network, in which one or more of the illustrated nodes 302, 304, and / or 306 may have connections to or from one or more other unseen nodes of the larger network. In one such example, in the context of CNNs, first node 302 could be regarded as a neuron of a convolution layer, second node 304 could be regarded as a neuron of a rectified linear unit (ReLU) layer receiving an output activation map from the convolution layer, and third node 306 could be regarded as a neuron of a pooling layer receiving the ReLU layer output. The third connection 312 could be regarded as a back-connection to a previous layer as in a recurrent neural network (RNN), or the third connection 312 could be omitted (or set to a zero prior value by first node 302).

[0058] FIG. 4 illustrates a further example network 400, including first node 402, second node 404, and third node 406. In network 400, the connections between nodes are in bidirectional pairs: first connection 408 from first node 402 to second node 404 and first connection back 414 from second node 404 to first node 402; second connection 410 from second node404 to third node 406 and second connection back 416 from third node 406 to second node 404; and third connection 412 from third node 406 to first node 402 and third connection back 418 from first node 402 to third node 406.

[0059] The network 400 also shows first node 402 having a bidirectional pair of connections to an environment 420: a connection from environment 422, and a connection to environment 424. In some examples, the environment 420 can represent an entity outside of the network, such that a node connected to the environment 420 (such as first node 402), also referred to as an environment-facing node, acts as an input node and / or an output node for the network 400. The connections to or from the environment 420 may act as connections to another node insofar as they convey inputs to, and / or receive outputs from, the environment-facing node. In some examples, the environment 420 can be regarded as an external data source (providing inputs to the network via the connection from environment 422) and / or a post-processing stage for outputs of the network (e.g., generated data such as inference data). In some examples, multiple nodes of the network are environment-facing nodes, such that the inputs and / or outputs of the network at a given time step may comprise multiple values (e.g., multiple binary values for examples in which each node generates only binary outputs). Thus, for example, a network configured to perform an inference task on image data may have 1 million environment-facing nodes configured to each receive one bit of a 1000 x 1000 black and white pixel input image, or more environment-facing nodes for images having multi -bit encodings for each pixel (e.g., greyscale or multi-channel images). A network configured to categorize images data into two categories may only need as few as one environment-facing node to provide an output to the environment 420: an output of “0” might indicate categorization into the first category, whereas an output of “1” might indicate categorization into the second category. However, in many cases, using an active inference model (described below) for the behavior of the network may provide benefits with respect to inference tasks involving bidirectional communication with an environment 420, such as control systems using feedback from the systembeing controlled: in some such cases, the environment 420 is controlled at least in part by the outputs received via the connection to environment 424, and the environment 420 provides feedback on its own state as inputs via connection from environment 422, such that the network can process the feedback to generate desired control outputs. Thus, in some examples, the behavior of the network may provide benefits in the context of executing the machine learning model on a first machine and using the machine learning model to control the operation of a second machine.

[0060] In the illustrated example, third node 406 has a self-connection 426, which is a connection in which the third node 406 is both the transmitting node and the receiving node of the connection. A self-connection 426 may act as a one-time-step delay loop feeding a node's outputs back to itself. A self-connection 426 can also serve as a one-timestep inverter (e.g., generating a 1 in response to a 0) or a number of other operations: for example, in some cases, a node can process its self-signal from the last timestep and its environment signal from the last timestep according to logic dictated by its parameters (e.g., priors).

[0061] In some examples, a network with bidirectional pairs of connections, such as network 400, may be configured to perform inference according to the principles of active inference and the minimization of variational free energy (VFE) or expected free energy (EFE) against a generative model. Active inference is a framework for modeling Bayes- optimal behavior of an agent. In some examples herein using an active inference framework, an agent - such as a machine learning model, or even a node of a network driving the behavior of the machine learning model - acts to minimize an upper bound on the “surprise” resulting from the agent's observations. Surprise is a measure of the degree to which an agent’s current observations differ from its preferred observations and the informativeness of its observations. In some examples, VFE or EFE is a function of both reward and information gain, potentially in varying ratios, thereby biasing the node to both explore a parameter space and exploit its self-efficacy. In some examples herein using an active inference framework, agents seeking to minimize their surprise act on the observed system to solicit feedback thataccords with the agent's desired observations: this activity of the agent is why the framework is referred to as “active” inference.

[0062] The active inference frameworks described herein propose that, even if an agent does not have the capability to directly minimize its surprise, it can at least seek to minimize a proxy for surprise, called variational free energy. The agent may minimize VFE OR EFE by performing computation in response to, and in anticipation of, its observations. Example implementations of active inference described herein may thus rely on the free energy principle, by which an agent reduces its surprise by generating inferences using an internal model and updating its internal model to predict the outputs of the observed (and acted-upon) system.

[0063] Examples of computational operations implementing active inference are described below with reference to FIG. 5 through FIG. 10. The examples of FIG. 5 through FIG. 10 may be implemented by nodes of the network 400.

[0064] In some examples, a network such as network 400 may be used to implement a machine learning model operating according to an active inference framework (also referred to as an active inference model). Each node of the network may be associated with parameters (e.g., priors) organized into one or more tensors (e.g., scalars, vectors, or matrices), also referred to as parameter tensors, each tensor modeling a different aspect of the node's behavior.

[0065] In some examples, the active inference behavior of the nodes of the network may be modeled as a partially observable Markov decision process (POMDP), characterized by the equation:• “T” indicates the time step, from a first time step T=1 to a final (or current) time step T=T,• “ot” indicates the observations o of the node (e.g., its inputs) at a given time step i=t,• “st” indicates the actual state 5 of the observed system (e.g., a hidden state that may differ from the observations of the node) at time step i=t,• "K" indicates a policy for the node, e.g., a possible sequence of actions (e.g., outputs to generate in response to inputs) that the node can take, and• “(|)” indicates an expected reward, e.g., a similarity between inputs received by the node and the node's preferred inputs.

[0066] The active inference behavior of a node may be implemented using three or more sets of priors organized into three or more tensors. Examples of tensors used to compute active inference behavior using a POMDP for a node of the network are illustrated and described with reference to FIG. 5.

[0067] FIG. 5 shows four tensors used by a POMDP of a node. The A tensor 502, B tensor 512 (consisting of the Bo subtensor 504 and Bi subtensor 506), C tensor 508, and D tensor 510 indicate four sets of priors maintained by the node. Each of these sets of priors is used to contribute to the performance of an active inference task by the network (e.g., network 400). The values of the priors shown in the example of FIG. 5 are initial prior values that may be assigned to a newly initialized node of the network (e.g., network 400), also referred to as an undifferentiated node or a stem node. The node may be deployed within the network to contribute to an inference task during a runtime period, during which the node may differentiate itself by adjusting its prior values to diverge from their initial values in response to interactions with other nodes of the network. A node that has undergone at least partial differentiation may be referred to as a differentiated node.

[0068] In some examples, the inference task performed by the network is an active inference task. In active inference, as described herein, a machine learning model acts upon a system and observes the system.

[0069] In some examples, the A tensor 502 may be regarded as modeling a probabilistic mapping of the node's observations (e.g., its inputs at a given time step) to an inferred hidden state of the observed system (e.g., a node or set of nodes). This mapping may be referred to as an observational model, denoted as:Observational Model

[0070] In the example shown in FIG. 5, the A tensor 502 is initialized as an identity matrix mapping observed inputs to exactly equivalent hidden states. In some examples, the inputs received by a node are equivalent to the state of the observed system; thus, there are no hidden states, and the A tensor 502 may remain unchanged during differentiation.

[0071] In some examples, the B tensor 512 may be regarded as modeling a probabilistic mapping of the node's observations (e.g., its inputs) at a given time step T-1 to a predicted set of observations at a subsequent time step T. This mapping may be referred to as a transition model, denoted as:Transition Model

[0072] In some examples, the B tensor 512 includes a separate submatrix for each set of outputs of the node at the first time step: thus, the predicted inputs at the second time step are conditioned on the outputs of the node at the first time step. In the example shown in FIG. 5, the node has only a single output (e.g., a binary activation value) having only two possible states, 1 or 0. The B tensor 512 therefore consists of two submatrices: a Bo subtensor 504 corresponding to output 0, and a Bi subtensor 506 corresponding to output 1.

[0073] In the example shown in FIG. 5, Bo subtensor 504 and Bi subtensor 506 are both initialized as homogeneous matrices (e.g., all matrix elements set to a value of 0.5) mapping all observed inputs at the first time step with equal probabilities to predicted inputs at the second time step.

[0074] In some examples, the C tensor 508 may be regarded as modeling a prior probability distribution (also referred to simply as a prior) over the node's observations (e.g., its inputs). This prior over observations may be referred to as the node's preferences, prior preferences, or expectations, and is denoted as:Prior Over Observations

[0075] In the example shown in FIG. 5, the C tensor 508 is initialized as a zero vector mapping all observations to a zero reward. Thus, a node initialized according to the initial values shown in FIG. 5 has no preferred values for its inputs. In some examples, C is not 0 but is instead configured to help the node predict the output of another node: the first node (having the C tensor 708) prefers or expects to see its self-signal match the output of a second node. The second node may be a node that the first node is configured to model or predict the outputs of, and in some cases, the second node may be referred to as a parent node of the first node.

[0076] In some examples, the D tensor 510 may be regarded as modeling a prior over an initial hidden state at initial time step T=1. This prior over the initial hidden state is denoted as:Prior Over Initial Hidden State

[0077] In the example shown in FIG. 5, the D tensor 510 is initialized as a homogeneous vector (e.g., all vector components set to a value of 0.5) weighting all initial state values with equal prior probability. Thus, a node initialized according to the initial values shown in FIG. 5 has no a priori beliefs about an initial state of the observed system.

[0078] In some examples, additional tensors may be used, whereas in other examples, one or more of the four tensors shown in FIG. 7 may be omitted (such as the D tensor).

[0079] In some cases, the system applies an epistemic term that pertains to knowledge, understanding, and information. The system can prioritize epistemic terms such that the machine learning model focuses on gaining insight on the functions performed by the machine learning model and updating the model to learn from current inferences. For instance, the system increases the epistemic terms in the C tensor 508 such that the model focuses on certainty, uncertainty, and knowledge of the machine learning model.

[0080] In some examples, the B tensor 512 may be the primary or the only set of priors to be adjusted during differentiation of the node: in other words, each of the other sets of priors (the A tensor 502, C tensor 508, and D tensor 510) remains constant, whereas the transition model (implemented by the B tensor 512) is adjusted to attempt to predict state transitions by the observed system (e.g., other connected nodes), conditioned on the actions (e.g., outputs) of the node. However, in some examples, one or more of the other sets of priors may also be adjusted during differentiation.

[0081] FIG. 6 through FIG. 10 illustrate examples of how the priors of the B tensor 512 of a node may change during differentiation to adapt to different systems exhibiting different behaviors in response to the node's own actions.

[0082] FIG. 6 shows an initial Bo subtensor 504a, equivalent to the initial state of the Bo subtensor 504 shown in FIG. 5. The initial Bo subtensor 504a is the matrix of predictions for a future observation of the state of a system - in the illustrated example, the observed system is a single other node connected to the node associated with the initial Bo subtensor 504a, and the observed node generates a single output, having a one-bit binary value, ateach time step. Similarly, the node associated with the initial Bo subtensor 504a also generates a single output, having a one-bit binary value, at each time step. Thus, the B tensor 512 of the node observing the system has two initial submatrices: a first submatrix, shown here in FIG. 6 as initial Bo subtensor 504a, conditioned on the output of the node having a “0” value, and a second submatrix, shown in FIG. 8 below as initial Bi submatrix 506a, conditioned on the output of the node having a “1” value.

[0083] The initial Bo subtensor 504a includes two columns of priors corresponding to two first time step inputs 602. The first time step inputs602 correspond to the potential observation of two different states of the system (e.g., inputs to the node) at a previous time step T-1 (also referred to herein as the first time step). In this example, the node is assumed to see its inputs accurately; however, in other implementations, the agent may have some uncertainty as to whether its inputs are accurate or the result of noise (e.g., the A tensor 502 may not be an identity matrix). The states of the system may also be referred to as input states of the node observing the system. The priors for 0 input at first time step 603 are in the left column, and the priors for 1 input at first time step 604 are in the right column. The two columns may thus be regarded as corresponding to two input states of the node; in some examples, each column (and its associated set of priors,603 or 604) is only used in computation of the node's behavior when the node observes the corresponding input state. Thus, when generating behavior for the node at a time step when the node observes an input value of “1”, only the priors for 1 input at first time step 604 are used to compute the node's behavior, which is to say, its output(s). The priors for 0 input at first time step 603 do not need to be consulted or processed during this computational step. Thus, by segmenting the space of possible inputs into a plurality of discrete input states (in this case, two input states), the number of parameters of the parameter tensor (in this example, a portion of the B tensor 512) that need to be processed may be significantly reduced relative to configurations in which an entire tensor of parameters is multiplied by an input tensor using matrix multiplication, such as in some conventional neural networks. Whereas FIG. 6 shows a node having only two possible inputstates, it will be appreciated that the potential benefit of this behavior may increase dramatically as the number of input states increases; a slightly more complex system having four possible observable states is observed by the node described with reference to FIG. 11 and FIG. 12 below, as two independent single-bit binary inputs giving rise to four possible input states.

[0084] Returning to FIG. 6, the columns of the initial Bo subtensor 504a are split into two rows, the second time step inputs 601, corresponding to observation of two different states of the system at a current time step T (also referred to herein as the second time step). The priors for 0 input at second time step 605 are in the top row, and the priors for 1 input at second time step 606 are in the bottom row.

[0085] It will be appreciated that, for nodes observing systems with more complex outputs (e.g., multiple connected nodes and / or nodes generating outputs more complex than a single bit per time step), the initial Bo subtensor 504a would include more columns and rows. For example, a node connected to three other nodes, each outputting a single bit at each time step, would need an initial Bo subtensor 504a with 23= 8 columns and 8 rows to map all previously observed system states to all predicted current systems states. In some examples, a node may be able to determine that two or more of its inputs tend to be highly similar, and may have its B tensor 712 simplified by combining the two inputs into a single input value.

[0086] As described above, the initial Bo subtensor 504a is a probabilistic mapping of previous observations to predicted current observations, conditioned on the output of the node at time step T-1 having been the binary value 0. The Bi subtensor 506, by contrast, stores the prior values mapping previous observations to predicted current observations conditioned on the output of the node at time step T-1 having been the binary value 1. It will be appreciated that, for nodes having a more complex output than a single bit at each time step, a greater number of B submatrices would be required, equal to the number of possible output states (e.g., 8 submatrices for a 3 -bit binary output). However, in some examples, all inputs and all outputs may be treated as being independent of one another, and the B tensor 712 may include separate subtensors for the separate inputs, such that the 8 x 8 x 8 Btensor that would ordinarily be required (for 3 outputs and 3 dependent inputs) could be replaced by 3 separate subtensors of size 8 (2 by 2 by 2), each subtensor corresponding to a bidirectional connection with one other node.

[0087] Because the nodes of the network generate their outputs for a given time step T before receiving their neighboring nodes' outputs for time step T, the computations informed by the priors of the initial Bo subtensor 504a are performed after time step T-1 and before time step T. Thus, time step T may either be referred to as the “current” time step (meaning the time step for which the current outputs are being computed) or as the “next” or “future” time step (because the observations at time step T are not yet known).

[0088] During the runtime period, the node uses the prior values of initial Bo subtensor 504a to perform active inference, in which the initial Bo subtensor 504a predicts the state of the observed system, conditioned on the previous observed state of the system and conditioned on the node's own previous output. To the extent that observations diverge from the predicted observations of the second time step inputs 601, the node may adjust the priors of the B submatrices in an attempt to minimize future VFE or EFE resulting from the uncertainty of the predictions. Similarly, if the observations concur with the predicted observations of the second time step inputs 804, the node may adjust the priors of the B subtensors to increase the level of certainty of the expected observations. In this way, the initial Bo subtensor 504a may have its priors adjusted over time to predict the outputs of the observed system, and thereby differentiate itself from its initial neutral, naive, undifferentiated state.

[0089] Thus, the prior values of the B tensor may be adjusted over a sequence of time steps to align the node with another node or a data source, such as the environment external to the network. After the node makes observations (e.g., inputs are accessed or received by the node) at a given time step, the priors of the node’s B tensor 512 may be adjusted based on the currently observed input state, based on the observed input state at the previous time step, and based on the output generated by the node at the previous time step. As described above, a columnar slice of the B tensor isselected based on the node’s inputs observed and its output generated at the last time step, and a column of priors of one of the B subtensors 504 or 506 intersected by the columnar slice is selected based on the comparison of the priors of the various individual columns in the columnar slice. The priors of this selected column may then be adjusted in accordance with the currently observed input state, such that a higher prior is assigned to the prior in the row corresponding to the currently observed input state, and all other priors in the row are decreased accordingly.

[0090] In some examples, the degree to which the priors of the column are adjusted is in proportion to the number of observations of each input state over time: thus, for example, if the input state of row 2 (e.g., the row of priors for 1 input at second time step 606 in the column of priors for 0 input at second time step 605) has been observed at 32% of time steps, it will be assigned a prior value of 0.32. In other examples, different degrees of adjustment may be used, such as a degree of adjustment proportional to a learning rate that may vary under different conditions.

[0091] In some cases, the node’s preferences may affect the degree to which the priors of the column are adjusted. When a preferred row is observed, in addition to increasing the prior in the observed row and decreasing the other rows accordingly, all of the priors of the column may be adjusted to sharpen the distribution of prior values, such that high prior values are raised and low prior values are lowered, rendering the priors of the column less uniform. In contrast, when a non-preferred row is observed, in addition to increasing the prior in the observed row and decreasing the other rows accordingly, all of the priors of the column may be adjusted to partially flatten the distribution of prior values, such that high prior values are lowered and low prior values are raised, rendering the priors of the column more uniform. The partial flattening of the prior values may result in an increase in the effective learning rate for priors in the column in future time steps, because it is easier for a new observation to stand out relative to other observations when all older observations are relatively uniform. This increase in the effective learning rate after observing a non-preferred row may increase the likelihood or speed with which the node will reconfigurethe priors of the column, thereby potentially finding a more optimal set of priors that may assist in avoiding the non-preferred row in the future. By the same token, a slowed effective learning rate after observing a preferred row may reduce the likelihood or speed with which the node will reconfigure the priors of the column, thereby potentially locking the node more tightly into its current configuration.

[0092] FIG. 7 shows the initial Bo subtensor 504a after having undergone differentiation to predict the outputs of an observed system - in this example, another node - that is configured to employ a “tit for tat” strategy in an iterated prisoner's dilemma task. The tit for tat strategy is one in which an agent will defect (in this example, output a “1”) immediately after an opponent defects, and will cooperate (in this example, output a “0”) immediately after an opponent cooperates. Thus, the output of the observed node is a deterministic function of this node's own previous action: in this example, if the node output a “1” at time step T-1, then the observed node will output a “1” at time step T, and if the node output a “0” at time step T-1, then the observed node will output a “0” at time step T.

[0093] Thus, after the node has undergone differentiation to predict the outputs of the observed node employing the tit for tat strategy, the node will adjust the priors of its initial Bo subtensor 504a to the values shown in the differentiated Bo subtensor 702: the priors for 0 input at second time step 605 are all 1, indicating a 100% probability that a previous output of “0” will result in a current observation of a “0”, and the priors for 1 input at second time step 606 are all 0, indicating a 0% probability that a previous output of “0” will result in a current observation of a “1”. These prior values are set across both columns, indicating that they apply regardless of the previous observed state of the system.

[0094] By differentiating nodes to predict the outputs of their observed systems, some examples described herein may train the nodes of the network to generate inferences about the behavior of the rest of the system including (for environment-facing nodes) the environment outside of the machine learning model.

[0095] FIG. 8 and FIG. 9 show the same process of differentiation to predict the outputs of a “tit for tat” observed node as the differentiation adjusts the prior values of the Bi subtensor 506. Initial Bi submatrix 506a has the same values as initial Bo subtensor 504a. The first time step inputs 602 are mapped to the second time step inputs 601 using priors split into two columns (priors for 0 input at first time step 703 and priors for 1 input at first time step 704) and two rows (priors for 0 input at second time step 705 and priors for 1 input at second time step 706). The mapping of initial Bi submatrix 506a is conditioned on the output of the node at time step T-1 being “1”.

[0096] As the node differentiates to predict the outputs of the observed tit for tat strategy of its neighboring node, the initial Bi submatrix 506a adjusts its prior values to those shown in the differentiated Bi submatrix 902: the priors for 0 input at second time step 705 are all 0, indicating a 0% probability that a previous output of “1” will result in a current observation of a “0”, and the priors for 1 input at second time step 706 are all 1, indicating a 100% probability that a previous output of “1” will result in a current observation of a “1”. These prior values are set across both columns, indicating that they apply regardless of the previous observed state of the system.

[0097] FIG. 10 shows an initial B tensor 512a (equivalent to B tensor 512 shown in FIG. 5), and its potential to differentiate into multiple possible differentiated values depending on the behavior of the observed system. In this example, the observed system is a single node playing an iterated prisoner's dilemma task, as in the examples of FIG. 6 through FIG. 9.

[0098] If the observed node employs a tit for tat strategy, the initial B tensor 512a will differentiate over time into the tit-for-tat differentiated B tensor 512b, whose Bo and Bi submatrices are equivalent to differentiated Bo subtensor 702 and differentiated Bi submatrix 902 shown in FIG. 7 and FIG. 9. The tit-for-tat differentiated B tensor 512b exhibits absolute certainty in its predictions: the previous action of the node is determinative of the next observed state of the system, due to the nature of the tit for tat strategy.

[0099] If the observed node employs an “always defect” strategy, the initial B tensor 512a will differentiate over time into the defector differentiated B tensor 512c. In an always defect strategy, the observed node always outputs “1”, irrespective of previous outputs and irrespective of inputs. As a result, the node observing the defector node will have complete certainty (priors 0 and 1) about the next observed state: it can predict with 100% confidence that the next observed state will always be “1”. Notably, this prediction is premised on the previous observed state being “1” as well: the 1 and 0 prior values are only in the right column of the Bo subtensor 504 and Bi subtensor 506. Because the node has never observed a system state of “0”, the priors in the left columns are completely undifferentiated from their initial values of 0.5.

[0100] In the final illustrated example, if the observed node employs a “random” strategy, the initial B tensor 512a will differentiate over time into a form resembling the random differentiated B tensor 512d. In a random strategy, the observed node outputs “0” or “1” randomly, irrespective of previous outputs and irrespective of inputs. As a result, the node observing the random node will remain largely undifferentiated (priors all at approximately 0.5) about the next observed state: it cannot predict the next observed state with any confidence.

[0101] Each of these differentiated states 512b, 512c, or 512d of the B tensor 512 of a node presents an example of a different degree of two capabilities of the node: predictive certainty, and self-efficacy. Predictive certainty refers to the degree of certainty in the node's predictions: the tit- for-tat differentiated B tensor 512b exhibits perfect predictive certainty, the defector differentiated B tensor 512c exhibits perfect predictive certainty only under some conditions (it is certain as long as the observed state is “1”, but would be completely uncertain at the next time step if it ever observed a “0”), and the random differentiated B tensor 512d has no predictive certainty.

[0102] The second capability of the node, self-efficacy, refers to a measure of the ability of the output of the node at the previous time step T-1 to affect the observations of the node at the current time step T. The self-efficacy of anode is high if the state of its previous outputs result in highly certain current observations. Thus, the tit-for-tat differentiated B tensor 512b has very high self-efficacy, because its own action is completely determinative of the next observation, which will always be exactly equal to the node's own previous output. The self-efficacy of the defector differentiated B tensor 512c, by contrast, is very low, even though its predictive certainty is perfect for half of the possible observed previous states; the previous output of the node has no effect on its next observation. Finally, the self-efficacy of the random differentiated B tensor 512d is also very low, because its outputs also have no statistically meaningful effect on later observations.

[0103] Thus, predictive certainty and self-efficacy are separate metrics which may or may not co-occur.

[0104] In nodes having no preferences, such as the initial POMDP tensors described with reference to FIG. 5 above having zero values for the C tensor 508, high self-efficacy maps directly to low VFE or low EFE. However, in examples having preferences, minimizing VFE or EFE may require preferences to be satisfied, which requires not only self-efficacy but specifically self-efficacy with respect to soliciting preferred observations. For example, a node with preference (e.g., with non-zero values for the priors of the C tensor 508) may learn how to reliably solicit a current “1” observation under certain previous output and observation states; however, if the node prefers to observe “0” states of the observed system, and has not learned a way to reliably solicit “0” observations, the self-efficacy of the node will be very low.

[0105] In some examples, the C tensor 508 has values that can be used to guide the determination of VFE or EFE. For example, VFE or EFE can be determined based on a pragmatic term (e.g., "does the node get what is expected?") and / or an epistemic term (e.g., "does the node learn?"). The C tensor 508 can be used to adjust the balance between the pragmatic term and epistemic term. For example, the C tensor 508 can be set to 0 in some examples described herein, whereby the node is maximizing epistemic gain. In some examples, a higher epistemic term corresponds to weighting thepredictive certainty of a node higher than the self-efficacy of the node within the C tensor 508.

[0106] In some cases, a pragmatic term is used to compute VFE or EFE. The pragmatic term may focus on the practical use or application of a concept, emphasizing its utility in real-world scenarios. The pragmatic term may be emphasized to enable the machine learning model to focus on action, decision-making, and achieving specific outcomes. The pragmatic term may be increased by changing the C tensor 508 such that the machine learning model focuses more on improving accuracy, efficiency, performance, and / or the like. In some examples, a higher pragmatic term corresponds to weighting the self-efficacy of a node higher than the predictive certainty of the node within the C tensor 508.

[0107] Thus, in some examples, in the POMDP equation above, the reward function (|) may change between assigning a higher weight to predictive certainty (e.g., the degree of certainty indicated by the B tensor 512) and assigning a higher weight to self-efficacy (e.g., the degree to which the policy it is likely to achieve the desired next observation) depending on the relative values of the epistemic term and the pragmatic term.

[0108] In some examples, nodes have internal states, and predictive accuracy could also or alternatively reflect not the ability of anode to correctly predict the output of another node, but instead to predict an internal state of the other node (e.g. whether the other node is operating in a stochastic or a deterministic mode, how strongly the other node holds its priors, etc.).

[0109] In some examples, predictive certainty and / or self-efficacy may be measured with respect to individual connections of a node. The certainty with which a node can predict inputs received via a given connection may be regarded as the node's predictive certainty with respect to that connection. Similarly, the extent to which a node's outputs via a given connection affect the state of one or more inputs subsequently observed may be regarded as the self-efficacy of that connection. In some examples, self-efficacy is computed for pairs of connections, such that the self-efficacy of a bidirectional pair of connections between a first node and a second nodemeasures the degree to which the first node's output(s) to the second node affect the first node's subsequent observation of the second node's output(s). In some examples, the self-efficacy of a connection from the first node to the second node may instead be a measure of the degree to which the first node's output(s) to the second node affect all observations of the first node with respect to all of its connected nodes.

[0110] In some examples, the self-efficacy of each of a node's connections can be combined to generate a nodal self-efficacy measure. In some examples, the nodal self-efficacy of a node is computed as the sum of the self-efficacy of each of its outbound connections. In other examples, the nodal self-efficacy of a node is computed as an average (such as an arithmetic or geometric mean, a median, or another averaging function), or as some other combination of the self-efficacy measures of each of its outbound connections.[OHl] In some examples, the self-efficacy of a connection can be computed as the mean of the absolute difference between the sub-matrices conditioned on each action, e g , Bo subtensor 504 and Bi subtensor 506 (For nodes having multiple connections, self-efficacy for a given connection may be computed by isolating the observations and predictions of the B tensor 512 for the connection being measured.) Thus, the self-efficacy for tit-for-tat differentiated B tensor 512b is the maximum value of 1 :Self-efficacy (S) for tit-for-tat differentiated B tensor 512b

[0112] In contrast, the self-efficacy for defector differentiated B tensor 512c is the minimum value of 0:Self-efficacy (S) for defector differentiated B tensor 512c

[0113] As described above, a node receiving binary inputs and generating binary outputs may in some cases operate according to the behavior of a Boolean operator or a logic gate. In some cases, a node may receive multiple binary inputs and / or outputs and may instead replicate the behavior of multiple Boolean operators or multiple logic gates. However, examples described herein that use a single node to reproduce the behavior of a logic gate may realize potential benefits, such as the ability to perform mappings from the logic of the network to the machine learning model (and its constituent nodes a connections) and the logic of a digital circuit, and vice- versa. For example, these mappings may enable simulation of circuit behavior, modification (including optimization) of circuit behavior using machine learning techniques, and manufacturing of revised circuits based on modified machine learning model networks.

[0114] Example submatrices of a B tensor 512 of a node configured to act as a logic gate are described with reference to FIG. 11 and FIG. 12 below.

[0115] FIG. 11 and FIG. 12 shows prior values of an example Bo submatrix 1102 and an example Bi submatrix 1202 of a node configured to operate as a NAND logic gate. In some examples, the parameter values of the Bo submatrix 1102 and Bi submatrix 1202 may be manually configured to an initial state as shown in FIG. 11 and FIG. 12 in order to intentionally replicate the behavior of a NAND gate. In some examples, the parameter values may converge on the illustrated configuration (or functionally similar sets of parameters) during a training, optimization, or inference period as a result of adjustment (e.g., to learn to predict the outputs of an observed system).

[0116] The parameter values of the Bo submatrix 1102 and Bi submatrix 1202 as shown in FIG. 11 and FIG. 12 may cause the node to replicate orsimulate the behavior of an AND gate in a system based on active inference and minimization of VFE or EFE, because by outputting outputs in response to observed inputs that correspond to the input-output mapping of an AND gate, the node minimizes its VFE or EFE. In some examples, VFE or EFE is an upper bound on statistical surprise - thus, the parameter values of the Bo submatrix 1102 and Bi submatrix 1202 may be regarded as a set of parameter values that minimize a more easily computable proxy for the surprise of the node, or maximize the certainty of the node, when the node generates outputs corresponding to outputs of an AND gate.

[0117] The B tensor 512 for a node simulating an AND gate should have rows and columns corresponding to each possible transition between hidden states, conditioned on the output of the node. This example assumes a perfect correspondence between the node's observations and any hidden states (represented by making the A tensor 502 an identity matrix, as described above). This means that each parameter value in the B tensor 512 should be equivalent to both the node's prediction about transitions between hidden states and transitions between observations.

[0118] The inputs from the previous time step T-1 (first time step inputs 602) fall into four possible input states corresponding to the four columns of Bo submatrix 1102 or Bi submatrix 1202: the priors for 00 input at first time step 1112 correspond to an input state of 00 (e.g., False and False), the priors for 01 input at first time step 1114 correspond to an input state of 01 (e.g., False and True), the priors for 10 input at first time step 1116 correspond to an input state of 10 (e.g., True and False), and priors for 11 input at first time step 1118 correspond to an input state of 11 (e.g., True and True).

[0119] The expected inputs from the current time step T (second time step inputs 601) also fall into the same four input states as the first time step inputs 602, and correspond to four rows: the priors for 00 input at second time step 1104 correspond to an expected next input state of 00 (e.g., False and False), the priors for 01 input at second time step 1106 correspond to an expected next input state of 01 (e.g., False and True), the priors for 10 input at second time step 1108 correspond to an expected next input state of 10(e.g., True and False), and priors for 11 input at second time step 1110 correspond to an expected next input state of 11 (e.g., True and True).

[0120] The B tensor 512 has a number of submatrices equal to the number of possible output states of the node. In this example, the node has a single one-bit binary output; thus, there are only two submatrices conditioned on the different output states, namely Bo submatrix 1102 and Bi submatrix 1202. If the node had two independent binary outputs, there would be four different submatrices corresponding to the (2 x 2) possible output states, and so on.

[0121] Thus, Bo submatrix 1102 represents the node's predictions about observed state transitions conditioned on the node outputting 0, whereas Bi submatrix 1202 represents the node's predictions about observed state transitions conditioned on the node outputting 1. It will be appreciated that the submatrices of the B tensor 512 may be regarded as slices through a larger B tensor 512 in some examples.

[0122] As described above, in some cases VFE or EFE may be decomposed into a pragmatic term and an epistemic term. In the example Bo submatrix 1102 and Bi submatrix 1202 shown in FIG. 11 and FIG. 12, the pragmatic term is minimized (e.g., by setting the entire C vector to 0), so that the node's behavior is dictated by a goal of maximizing epistemic gain. In this example, a node that deterministically acts to pursue maximum information gain will reproduce the behavior of a NAND gate if the B tensor 512 exhibits the following characteristics:• In response to observing an input state of 11 (True, True), then outputting 0 should lead to much greater opportunity to reduce uncertainty (e.g., to learn by updating priors) about the next timestep than outputting 1 (because True NAND True = False, so the node should be biased toward outputting a 0 in this case).• In response to observing any other input state (e.g., 00, 01, or 10), then outputting 1 should lead to much greater opportunity to reduce uncertainty about the next timestep than outputting 0 (because a NANDoperation applied to any other two inputs = True, so the node should be biased toward outputting a 1 in this case).

[0123] In order to achieve these characteristics, the B tensor 512 should:• Maximize the opportunity to reduce uncertainty (and, from the node's perspective, future learning) by presenting a uniform or near-uniform probability distribution (e.g., parameter values of 0.25 across all predicted next input states).• Minimize the opportunity to reduce uncertainty by presenting a highly non-uniform distribution, such as a one-hot probability distribution. (A one-hot is a group of bits among which the legal combinations of values are only those with a single high bit (e.g., 1) and all the other bits low (e.g., 0)).

[0124] The Bo submatrix 1102 shown in FIG. 11 therefore attempts to maximize the opportunity to reduce certainty by differentiating the priors corresponding to an output of 1 if the input state is 11, because the priors for 11 input at first time step 1118 are a uniform set of parameter priors {0.25, 0.25, 0.25, 0.25} representing complete uncertainty, meaning that there is a great deal of potential to improve certainty through further differentiation. Similarly, the Bi submatrix 1202 shown in FIG. 12 attempts to maximize the opportunity to reduce certainty for an output of 1 if the input state is 00, 01, or 10, because the priors for 00 input at first time step 1112, the priors for 01 input at first time step 1114, and the priors for 10 input at first time step 1116 are each a uniform set of parameter priors {0.25, 0.25, 0.25, 0.25}. All other columns of the Bo submatrix 1102 and Bi submatrix 1202 are one-hot sets of parameter values {0, 0, 0, 1 }, representing zero capacity to improve certainty for an output of 0 if the input state is 00, 01, or 10, and zero capacity to improve certainty for an output of 1 if the input state is 11.

[0125] When processing the input state according to an active inference framework using the POMDP equation set out above, these parameter values for the B tensor 512 result in behavior for the node that replicates the behavior of a NAND gate. Similar approaches are described below for replicating the behavior of other digital logic gates. Thus, in some examplesof a network of a machine learning model operating to minimize the VFE or EFE of each node, the nodes can be configured to replicate the behavior of digital logic gates through the minimization of VFE or EFE.

[0126] One further potential benefit of this capacity of the network and its nodes, when operating according to an active inference framework to minimize the VFE or EFE of each node, is that a node may be examined at a point in time to compare the current value of its parameters, or the input / output characteristics those parameters give rise to, to those of a digital logic gate, in order to understand the logical operation of the node and the extent to which the node is simulating or approximating the behavior of a given type of logic gate. This may assist in interpreting the state of the machine learning model at a given point in time and characterizing or approximating the behavior of the machine learning model using digital logic gates. For example, if a set of nodes are observed at time t to have parameters that are close to the values shown in FIG. 11 and FIG. 12, then those nodes may be considered to approximate the logical or functional behavior of NAND gates, and a digital circuit may be designed that approximates the function or behavior of these nodes using a corresponding set of NAND gates. Furthermore, even if the logic of the nodes is not used to construct a corresponding digital circuit, the function of the machine learning model as a whole may be better understood by a human observer through generating and presenting to the human observer a digital logic equivalent (or approximation) of the network of the machine learning model.

[0127] It will be appreciated that nodes simulating logic gates other than NAND gates can be configured using parameter values for the various columns of the B tensor 512 that either maximize or minimize uncertainty. For example, a B tensor 512 of a node simulating an AND gate can be constructed by swapping the parameter configurations of the Bo submatrix 1102 for those of the Bi submatrix 1202 and vice-versa. As a further example, a NOR gate can be constructed by replacing, in the Bo submatrix 1102, the priors for 01 input at first time step 1114 and priors for 10 input at first time step 1116 with uniform parameter priors {0.25, 0.25, 0.25, 0.25}, and replacing, in Bi submatrix 1202, the priors for 01 input at first time step1114 and priors for 10 input at first time step 1116 with one-hot parameter values {0, 0, 0, 1 }. An XNOR gate can be simulated by further modifying the B tensor 512 of the node simulating an NOR gate to replace, in the Bo submatrix 1102, the priors for 11 input at first time step 1118 with one-hot parameter priors {0, 0, 0, 1 }, and replacing, in Bi submatrix 1202, the priors for 11 input at first time step 1118 with uniform parameter priors {0.25, 0.25, 0.25, 0.25 } . Similarly, a NOT gate can be simulated by a node having a single input and having: 1) a Bo subtensor 504 as the initial Bo subtensor 504a but with priors for 0 input at first time step 603 equal to {0, 1 } and priors for 1 input at first time step 604 equal to {0.5, 0.5}, and 2) a Bi subtensor 506 as the initial Bi submatrix 506a but with priors for 0 input at first time step 603 equal to {0.5, 0.5} and priors for 1 input at first time step 604 equal to {0, 1 }. These and further examples are described below with reference to FIG. 13 and FIG. 14.

[0128] FIG. 13 illustrates parameter values of example B matrices of nodes configured to operate as NOR, XNOR, and NOT logic gates. The node simulating an NOR gate has a NOR gate Bo subtensor 1302 and a NOR gate Bi subtensor 1304 as depicted. The node simulating an XNOR gate has an XNOR gate Bo subtensor 1306 and an XNOR gate Bi subtensor 1308 as depicted. The node simulating a NOT gate has a NOT gate Bo subtensor 1310 and a NOT gate Bi subtensor 1312 as depicted, and receives only a single input, unlike the two-bit inputs of the other logic gates depicted.

[0129] FIG. 14 illustrates parameter values of further example B matrices of nodes configured to operate as AND, OR, and XOR logic gates. The node simulating an AND gate has an AND gate Bo subtensor 1402 and an AND gate Bi subtensor 1404 as depicted. The node simulating an OR gate has an OR gate Bo subtensor 1406 and 1 OR gate Bi subtensor 1408 as depicted. The node simulating an XOR gate has an XOR gate Bo subtensor 1410 and an XOR gate Bi subtensor 1412 as depicted.

[0130] It will be appreciated that different values for the priors of the B tensors 512 shown in FIG. 11 through FIG. 14 may be used in some cases to achieve logic gate behavior. In some examples, a node may have a non- uniform C tensor 508, such that its behavior may be affected by itspreferences. This may require that the priors of the B tensor 512 are different from those shown in the illustrated examples in order to achieve consistent logic gate behavior. In some examples, it may be possible to use less absolute values for the various priors shown in FIG. 11 through FIG. 14 and still achieve consistent logic gate behavior. For example, instead of a column having a one-hot array of priors (e.g., {0, 0, 0, 1 }), in some examples a column may include a less absolute distribution of priors (e.g., {0.1, 0.2, 0.1, 0.6}) and still achieve the same logic gate behavior as the illustrated examples.

[0131] The examples of B tensors 512 shown in FIG. 11 to FIG. 14 demonstrate a potential benefit of machine learning models in which the nodes of the network generate one-bit binary outputs: namely, these nodes can be used as primitives to construct arbitrary digital logic. This may enable one or more of the techniques described herein, such as the use of a circuit design system for digital circuit design and / or redesign.

[0132] In some cases, a node may be configured to behave as two or more different logic gates with respect to different sets of outputs and inputs. For example, a node having three inputs (from nodes X, Y, and Z) may also have 3 outputs (to the same cells X, Y, and Z), not including the node’s selfsignal. The node may behave as a NAND with respect to inputs X and Y and output Z, but it may behave as an OR with respect to inputs Y and Z and output X. It will be appreciated that other configurations are possible.This may be regarded as a form of “polycomputing” in which the same units perform different computations.

[0133] FIG. 15 illustrates an example method 1500 of processing discrete inputs to generate discrete outputs of a node of a network of a machine learning model. Although the example method depicts a particular sequence of operations, the sequence may be altered without departing from the scope of the present disclosure. For example, some of the operations depicted may be performed in parallel or in a different sequence that does not materially affect the function of the method. In other examples, different components of an example device or system that implements the method may perform functions at substantially the same time or in a specific sequence.

[0134] The method 1500 can be implemented by one or more processors executing instructions stored in a tangible computer readable storage medium. The tangible computer readable storage medium may also be used to store and / or retrieve the machine learning model.

[0135] The method 1500 will be described with reference to the example network 400 of FIG. 4. However, it will be appreciated that the method 1500 can be applied to various types of networks used in machine learning models, such as the examples described above with reference to FIG. 1 through FIG. 3.

[0136] Thus, in some examples, the machine learning model includes a network having a plurality of nodes and a plurality of connections between the nodes. Each node is configured to process one or more inputs to generate one or more outputs, and each connection is associated with a transmitting node and a receiving node and is configured to convey an output of the transmitting node as an input to the receiving node, at each time step of a sequence of discrete time steps. The network as a whole operates to generate inference data; for example, the environment 420 may receive outputs of the network as inference data, or the outputs of the network to the environment 420 may be further post-processed by other components of the machine learning model or an Al system using the machine learning model to generate the inference data. In some examples, the inference data may be generated on a slower time scale than the nodes of the network: for example, inference data may only be generated at every k time steps, or only when the network has generated certain patterns or quantities of output data. The inference data includes, or may be further processed to generate, an inference.

[0137] In some examples, the machine learning model is at least partially trained to perform an inference task during a training period. During the training period, the machine learning model is trained to perform the inference task by processing samples of reference data and applying a machine learning algorithm to train the machine learning model, as described above with reference to FIG. 1 and FIG. 2. The training of the network may not be complete: for example, only an initial training phase 204may be performed, without refinement and / or retraining. In some examples, only a limited set of samples of reference data are used to partially train the machine learning model, after which the partially trained machine learning model is deployed to perform inference on candidate data during a runtime period. In some such examples, further training may be performed during or after the runtime period, using further reference data and / or candidate data.

[0138] The instructions, when executed by the one or more processors, may cause the one or more processors to perform the operations shown in FIG.15. The operations will be described as being performed by an Al system implemented by the instructions executed by the one or more processors.

[0139] At operation 1501, the Al system obtains discrete input(s) of a node of a network of a machine learning model. As described above, each input has a respective corresponding input value, which is a member of a finite set of discrete values (as opposed to a continuous value, such as a floating-point value). In some examples, each input has a binary value, such as 0 or 1. Different nodes of the network may receive different inputs and / or different numbers of inputs, as determined by the connections of the network. Thus, different nodes may be configured to observe different input states, and the number of possible input states observable by each node may differ.

[0140] At operation 1502, the Al system processes the input(s) of the node associated with a time step, based on parameters of the node, to generate discrete output(s) of the node associated with the time step, by quantization.

[0141] In some examples, each node processes its inputs at a given time step to generate an activation value of the node. Some or all of the outputs of the node may be generated based on the node's activation value. In various examples, the activation value may be quantized, either deterministically or stochastically, to the output value: for example, in a stochastic system, an activation value of 0.7 for a node may result in a randomly-determined 70% chance of the node's output being a 1 instead of a 0, whereas in a deterministic system, an activation value of 0.7 for a node may always result in the node's output being a 1 instead of a 0.

[0142] In some examples, a node, or all nodes, of the network may operate in a consistently stochastic quantization mode or in a consistentlydeterministic quantization mode. However, in some examples, a given node may switch between operating in a stochastic mode and operating in a deterministic mode at different time steps. In some such examples, operation 1502 may optionally include sub -op erations 1506, 1508, 1510, and 1512.

[0143] In some examples, the node selects its outputs to minimize VFE and / or EFE, as described above. VFE is a computation based on a generative model (e.g., a set of priors learned by a node or other agent) and observations (e.g., inputs to a node or other agent). VFE provides an upper bound on statistical surprise. There are various mechanisms to minimize VFE, including updating the generative model to more closely align with observations, or performing a computation of expected free energy (EFE) across possible actions and selecting one of the actions which leads to the lowest expected free energy. Operations involving the computation or minimization of EFE, as referred to herein, may be referred to as a computation or minimization of VFE.

[0144] To minimize EFE, a node may compute an expected distribution of free energy over multiple output states and select an output state that minimizes the expected free energy. The calculation of expected free energy may in some cases be affected by the node’s preferences (e.g., as represented by the C tensor 508), such that rows of the B tensor 512 corresponding to preferred observations of the node are expected to result in lower free energy than non-preferred rows of the B tensor. Thus, the node will calculate EFE distributed over all possible output states, then select an output that is at or near the minimum value of the EFE distribution. When operating in a deterministic mode, the node may select the output value that directly minimizes EFE. When operating in a stochastic mode, the node may weight various output values based on their corresponding EFE, or their proximity to the minimum value of the EFE distribution, and stochastically select an output value from the weighted distribution of output values.

[0145] Thus, the node’s preferences (e.g., the values of the parameters in the C tensor 508) may affect the degree to which the node operates to minimize VFE, EFE, or a combination of both VFE and EFE. A node with uniform zero parameter values in the C tensor 508 will have a closerelationship between VFE and EFE, whereas a node with non-uniform parameter values in the C tensor 508 will tend to exhibit EFE behavior that diverges from that predicted solely from VFE.

[0146] In some cases, the priors of the B tensor may be discretized before being used to compute VFE, EFE, or both. A columnal slice through the B tensor 512 (e.g., the columns of priors for 0 input at first time step 808 of both the Bo subtensor 504 and the Bi subtensor 506) is selected based on the node’s observation (e.g., input values) at the previous time step, including its self-input representing its own generated output from the previous time step. This selected columnar slice from the selected subtensor is the only portion of the B tensor 512 used to compute VFE or EFE. Based on comparing the prior values within the individual columns of the B subtensors, an output value for the current time step is selected, the output value corresponding to one of the B subtensors (e.g., a 0 output value for the Bo subtensor 504, or a 1 output value for the Bi subtensor 506). How these priors are compared, and how the subtensor column (and therefore the output value) are selected, depend upon how the node is configured to minimize VFE, EFE, or both.

[0147] In some cases, the selected column includes an array of continuous values of the priors, such as the floating-point numbers shown in the example B subtensors 504, 506. However, the computations required to compute VFE or EFE may be significantly simplified if the continuous values of the priors are first converted into discrete values, such as binary values, in a discretization operation before computing VFE and EFE. In some cases, discretizing the priors of the B subtensor column may also provide other benefits, such as improved stability of the output of the node.

[0148] Different examples may use different approaches to discretization of the priors of the selected column of the selected B subtensor. In a first example, a prior of the column may be treated as a prior value of 1 only if the continuous value of the prior is above a discretization threshold, such as 0.8 or 0.9. If one of the prior values in the column is above the discretization threshold, all other prior values in the column are treated as 0. The underlying continuous values of the priors continue to be tracked and adjusted as described herein, such that the high prior value may, upon furtherupdates, fall below the discretization threshold and cease to be treated as a 1, or the high prior value may continue to increase above the discretization threshold, due to further adjustment.

[0149] In a second example, the prior values of the column may be treated as a one-hot array (a single 1 value and all other values 0) based on a stochastic selection of one of the priors of the column to be treated as having a value of 1. For example, each prior in the column could have a chance of being selected as the high value proportional to its continuous prior value; alternatively, each prior could have a chance of being selected as the high value proportional to some function of its continuous prior value, such as the square of its continuous prior value. As in the first example above, the underlying continuous values of the priors continue to be tracked and adjusted.

[0150] Some examples may combine the first and second examples of prior discretization above. For example, the first example approach may be used to discretize any column in which one of the prior values exceeds the discretization threshold; if no such prior value exists in the column, the stochastic approach of the second example may be used instead.

[0151] In some examples, the discretization threshold may be a further parameter stored by the node. In some examples, the discretization threshold may be a learned parameter that is adjusted during the learning or alignment process to assist in minimizing VFE or EFE, as described herein with reference to adjustment of the priors of the B tensor 512.

[0152] At operation 1506, the Al system processes a portion of the node's parameter tensor corresponding to the node's discrete input state to generate a distribution of possible output values.

[0153] The input state may be, as described above, a state corresponding to the values of all inputs received in association with a time step (e.g., previous time step T-1). The portion of the parameter tensor corresponding to the inputs state may be, for example, a slice, column, row, or other subset of a tensor of parameters associated with the node and used to generate outputs of the node. For example, a single column of the Bo subtensor 504 (e.g., priors for 0 input at first time step 603 of differentiated Bo subtensor 702)and the corresponding column of the Bi subtensor 506 (e.g., priors for 0 input at first time step 603 of differentiated Bi submatrix 902) may constitute a portion of a parameter tensor of a node corresponding to an input state (in this example, input state “0”) at the time step (in this example, previous time step T-1). Similarly, a single column of the Bo submatrix 1102 (e.g., priors for 00 input at first time step 1112 of Bo submatrix 1102) and the corresponding column of the Bi submatrix 1202 (e.g., priors for 00 input at first time step 1112 of Bi submatrix 1202) may constitute a portion of a parameter tensor of a node corresponding to an input state (in this example, input state “00”) at the time step (in this example, previous time step T-1).

[0154] As described above, by limiting the processing of nodal inputs to an operation using only those parameters corresponding to the currently observed input state, the computational intensity and complexity of generating the node's outputs may be reduced relative to techniques in which an entire parameter tensor of the node is used to process the node's inputs.

[0155] It will be appreciated that, in some examples, processing of the inputs may involve more than the portion of the parameter tensor corresponding to the input state, and operation 1502 may be modified accordingly. For example, a system using the POMDP framework described above may set the A tensor 502 to a set of values other than the identity matrix shown in FIG. 5, if the system accounts for a potential mismatch between observed states and hidden states of the observed system. In such systems, the POMDP equation set out above may require multiple input states to be considered at a single time step, because the hidden state of the system may be observed as multiple possible observed states, and by the same token, a single observed input state may correspond to multiple possible hidden states. Thus, multiple columns of the B tensor 512 may be considered in selecting output values for the node, instead of the single column corresponding to the observed input state.

[0156] The distribution of possible output values for an output may take different forms in different examples. In some cases, the distribution is a normalized or non-normalized probability distribution, in which each possible output value is associated with a normalized probability (e.g., allprobabilities in the distribution sum to 100%) or a non-normalized weight (e.g., all of the weights in the distribution need not sum to a specific value, but function to define relative probabilities in relation to each other, not absolute probabilities). In some examples, the POMDP equation set out above (or a computational implementation or approximation thereof) provides a means to compute a probability distribution over multiple output states of a node at a given time step t.

[0157] At operation 1508, the Al system determines a current mode of the node. In some examples, a given node may operate in either a stochastic mode or a deterministic mode, and some nodes may switch or transition between the two modes at different times (e.g., at different time steps). For example, nodes may be set by the Al system to operate in a stochastic mode when their predictive performance and / or self-efficacy is low, and set to operate in a deterministic mode when their predictive performance and / or self-efficacy is high. In some examples, predictive performance at a given time step may be determined based on whether the node successfully predicts a property of another entity (e.g., another node, itself, or the environment 420). A node may be said to be a good predictor of another entity if the node's preferences are satisfied by the inputs received from, or observations made of, the entity. In some examples, the predictive ability of a node with an entity is measured over multiple time steps, and a metric may be used to determine whether a prediction accuracy threshold or a prediction inaccuracy threshold has been met.

[0158] An example of a prediction threshold being met is that, for the last N time steps (where N is a positive integer), the node has accurately predicted an output of the entity. If the prediction threshold is met by a node in a stochastic mode, the node may be switched to a deterministic mode. This switch may effectively lock the node into some of its current priors and / or behaviors by eliminating randomness in the generation of its outputs, which could result in a prediction inaccuracy.

[0159] An example of a prediction inaccuracy threshold being met is that, for the last K time steps (where K is a positive integer), the node has not accurately predicted an output of the entity. If the prediction inaccuracythreshold is met by a node in a deterministic mode, the node may be switched to a stochastic mode. This switch may effectively introduce randomness into the node's behavior, allowing it stochastically explore a parameter space to find an optimal (or improved) parameter configuration that is more likely to result in prediction inaccuracy of the entity.

[0160] It will be appreciated that other prediction accuracy thresholds and / or prediction inaccuracy thresholds could be used in some examples. For example, a threshold could measure a degree of predictive accuracy at each time step (instead of a binary yes / no measurement), and / or a threshold number of time steps over a longer time course (e.g., the threshold could be satisfied if 15 out of the last 20 time steps resulted in predictive accuracy).

[0161] In some examples, nodes of the network of the machine learning model may be switched into a stochastic mode during a training period or when in a learning mode, and switched into a deterministic mode during a runtime period or when in an inference mode. For example, the entire machine learning model (and all nodes thereof) could be operated in a training mode during a training period, and then, after training is complete and the parameters of the nodes have converged on a trained configuration, the entire machine learning model (and all nodes thereof) could be operated in an inference mode during a runtime period. In another example, some subset of the nodes of the network may be determined to have converged on a trained parameter configuration, and those nodes could be switched into a deterministic mode, even while other nodes of the network (whose parameters are determined to not be in a fully trained parameter configuration) may remain in a stochastic mode.

[0162] In some cases, the node may switch between the stochastic mode and the deterministic mode based on the value of a stress parameter of the node that tracks the degree or frequency to which the node has predicted incorrectly over past time steps. The stress parameter may be adjusted downward on time steps when the node’s output predicts its target, and adjusted upward on time steps when the node’s output fails to predict its target. When the stress parameter value rises above a stress threshold, the node switches from the deterministic mode to the stochastic mode, and whenthe stress parameter value falls below the stress threshold, the node switches back from the stochastic mode to the deterministic mode. In some cases, two different stress thresholds may be used, such that a falling stress threshold for switching back to the deterministic mode is lower than a rising stress threshold for switching to the stochastic mode.

[0163] In some examples, the stress threshold (and / or the falling and rising stress thresholds) may be a further parameter stored by the node. In some examples, each stress threshold may be a learned parameter that is adjusted during the learning or alignment process to assist in minimizing VFE or EFE, as described herein with reference to adjustment of the priors of the B tensor.

[0164] At operation 1510, in response to determining that the node is in a stochastic mode at operation 1508, the Al system stochastically selects values for the node's outputs from the distribution of possible output values. For example, as described above, a probability distribution indicating, for a specific output of the node, an output value of 0 associated with a distribution weight of 0.7 and an output value of 1 associated with a distribution weight of 0.3 would mean that “0” will be selected 70% of the time and “1” will be selected 30% of the time, based on generation of a random or pseudo-random number to select among the values indicated by the distribution of values for the output at the current time step.

[0165] At operation 1510, in response to determining that the node is in a deterministic mode at operation 1508, the Al system deterministically selects values for the node's outputs from the distribution of possible output values at operation 1512. In the example above, in which the distribution associates a weight or probability of 0.7 with output value “0” and associates 0.3 with an output value of “1”, and the node is in a deterministic mode, the output value “0” will always be selected as the value associated with the highest weight or probability.

[0166] In some examples, operation 1502 may omit or vary one or more of sub -op erations 1506, 1508, 1510, and / or 1512.

[0167] At operation 1503, the Al system uses the machine learning model to generate inference data based at least in part on the discrete outputs of thenode generated at operation 1502. The inference data is an output of the machine learning model that represents the output of the inference task for which the model has been trained, and may include classification data, generative data, control data for controlling an external system, and / or other types of inference data suitable for generation by a machine learning model performing its inference task. In some examples, the inference data is generated as (or based on) outputs to the environment 420 by various environment-facing nodes of the network. As described above, in some cases the inference data may be generated at a different pace than the outputs of the individual nodes of the network, such that inference data is only generated at a subset of time steps.

[0168] In some cases, the inference data may be regarded as inter-nodal outputs internal to the network. For example, after the node generates its outputs at operation 1502, those outputs are conveyed to one or more other nodes of the network via the connections of the initial node. These connected nodes may, in turn, process their inputs to generate a further set of outputs at the subsequent time step. The outputs of the connected nodes in response to the outputs of the first node may be regarded as inference data in some examples.

[0169] At operation 1504, the Al system processes the inference data to adjust the parameters of the node. In some cases, the adjustment is performed in response to inference data in the form of outputs at a subsequent time step from the other nodes connected to the initial node; the POMDP framework described above may provide a framework for adjusting the various parameter tensors of the initial node based on subsequent inputs received. In some examples, the inference data is received by the environment 420, which in turn generates inputs back to the network, and these inputs eventually propagate back to the initial node, causing that node to adjust its parameter values. In some examples, the Al system makes adjustments to the network based on the inference data output by the network as a whole, such that the network may be modified by adding or removing nodes or connections and / or adjusting parameter values of nodes.

[0170] Thus, in some examples, a method 1500 is provided by which one or more inputs of a node of a network of a machine learning model are obtained. Each input of the node has a respective input value, and each input value is a member of a finite set of discrete values. The one or more inputs of the node are processed to generate one or more outputs of the node. Each output has a respective output value, and each output value is generated as a member of the finite set of discrete values by quantization. The machine learning model generates inference data based at least in part on the one or more outputs of the node.

[0171] In some examples, the finite set of discrete values is a set of two binary values.

[0172] In some examples, the node includes a plurality of parameters structured as a parameter tensor having a dimension (e.g., number of columns of B tensor 512) proportional in size to a number of possible input states of the node. The number of possible input states of the node is a product of a respective number of possible input values of each of the one or more inputs (e.g., three inputs having independent binary values = 2 x 2 x 2 = 8 possible input states). In some examples, the processing of the one or more inputs of the node to generate the one or more outputs includes processing a portion of the parameter tensor to generate the one or more outputs, where the portion corresponds to a current input state of the node, and the current input state includes the input values of the one or more inputs (e.g., “11” indicates one input state of a node having two binary inputs).

[0173] In some examples, the processing of the portion of the parameter tensor to generate the one or more outputs includes processing the portion of the parameter tensor to generate a respective distribution of possible output values for each of the one or more outputs (e.g., a probability distribution of output values, as described above). A respective output value is selected for each output based on the respective distribution of possible output values of the output. This selection may be a stochastic selection or a deterministic selection in different examples. In some examples, the selection is performed stochastically in response to determining that the node is in a stochasticmode. In some examples, the selection is performed deterministically in response to determining that the node is in a deterministic mode.

[0174] In some examples, some or all of the operations of method 1500 may be performed with respect to each of a subset, or each of the full set, of nodes of the network, sequentially and / or in parallel, with respect to one or multiple time steps. Thus, in some examples, operations 1501 and 1502 (optionally including 1506, 1508, 1510, and / or 1512) are performed for each node of the network. The inference data may be generated at operation 1503 based on the outputs of some or all of the nodes of the network; for example, the inference data may be generated as a result of the propagation, through the nodes of the network (via the connections) of the outputs of various nodes (and / or the environment 420) to other nodes (and / or the environment 420) over multiple time steps.

[0175] Thus, in some examples, the method 1500 includes, for each node of a plurality of nodes of a network of a machine learning model, and in association with each time step of an ordered sequence of discrete time steps, performing one or more of the operations described above. A respective one or more inputs of the node associated with the time step are obtained. The respective one or more inputs of the node includes at least one output of another node of the network associated with a previous time step of the ordered sequence of discrete time steps. The respective one or more inputs of the node are processed to generate a respective one or more outputs of the node associated with the time step. The machine learning model is used to generate inference data based at least in part on the respective one or more outputs of each node associated with each time step of the ordered sequence of discrete time steps.

[0176] FIG. 16 illustrates a schematic diagram of a circuit design system 1600 using a network of a machine learning model (e.g., network 400). In some examples, a circuit design system 1600 may be used to design and / or redesign digital circuits using nodes of a network of a machine learning model having discrete (e.g., binary) inter-nodal communications, as described herein.

[0177] The circuit design system 1600 takes as an (optional) input an initial digital circuit design 1602, and generates as an output a further digital circuit design (shown as revised digital circuit design 1620). The circuit design system 1600 includes several functional components, shown as circuit simulation 1604, optimization 1608 (shown as including inference 1610 and parameter adjustment 1614 components), and network to circuit mapping 1618. The circuit design system 1600 may generate one or more intermediate data products, including a network 1606, an adjusted network 1616, and inference data 1612.

[0178] The circuit design system 1600 may be implemented, in various examples, as any suitable combination of computing hardware and / or software.

[0179] Operation of the circuit design system 1600 will be described with reference to method 1700 shown in FIG. 17.

[0180] FIG. 17 illustrates a flowchart showing operations of a method 1700 for revising the design of a digital circuit using a network of a machine learning model. Although the example method 1700 depicts a particular sequence of operations, the sequence may be altered without departing from the scope of the present disclosure. For example, some of the operations depicted may be performed in parallel or in a different sequence that does not materially affect the function of the method 1700. In other examples, different components of an example device or system that implements the method 1700 may perform functions at substantially the same time or in a specific sequence.

[0181] The method 1700 can be implemented by one or more processors executing instructions stored in a tangible computer readable storage medium. The tangible computer readable storage medium may also be used to store and / or retrieve the machine learning model.

[0182] The method 1700 will be described with reference to the example circuit design system 1600 of FIG. 16 and the example network 400 of FIG. 4. However, it will be appreciated that the method 1700 can be applied to various types of circuit design systems and / or various types of networks used in machine learning models.

[0183] Some of the operations shown in FIG. 17 share reference numerals with operations of method 1500 shown in FIG. 15. Unless otherwise, the description of these operations in reference to method 1500 above applies equally to these operations in the context of method 1700.

[0184] At operation 1702, the circuit design system 1600 configures the parameters of a first node, and additional parameters of additional nodes, of a network 1606 of a machine learning model to correspond to the logic of an initial digital circuit design 1602. The initial digital circuit design 1602 may be the design of an existing circuit or simply a design for a hypothetical circuit. The initial digital circuit design 1602 may be encoded or otherwise represented as data, such as circuit design data encoded in a known data format for representing digital circuit designs (e.g., logical diagrams, circuit diagrams, schematic drawings, fabrication data such as Gerber files or ODB++ files, printed circuit board layout files, and so on). The circuit simulation 1604 components of the circuit design system 1600 parses or otherwise processes the design data for the initial digital circuit design 1602 and generates a structure and set of parameter values for a network 1606 of a machine learning model that simulates the logic of the initial digital circuit described by the initial digital circuit design 1602.

[0185] For example, the circuit simulation 1604 component may determine that the initial digital circuit design 1602 consists of a specific set of digital logic gates arranged to have a specific set of interconnections between their respective inputs and outputs. The circuit simulation 1604 component then generates a network 1606 having nodes that simulate the behavior of the logic gates (as shown in FIG. 11 to FIG. 14), and having connections between nodes that simulate the behavior of the connections between the logic gates. In some examples, this may mean that some of the nodes of the network 1606 are only interconnected using unidirectional connections (as in network 300), or that the parameters of the nodes are configured to minimize or eliminate the effect of one of the pair of bidirectional connections as used in network 400.

[0186] In some examples, nodes and connections of the network may be generated and configured to simulate the behavior of circuit componentsother than logic gates. For example, there may be configurations of nodal parameters, inter-nodal connections, and / or self-connections that may configure a node to simulate the behavior of a latch or a clock generator. The circuit simulation 1604 component may generate the network 1606 to include such non-logic gate circuit components in some cases.

[0187] In some examples, operation 1702 is omitted. Instead, the network 1606 may be generated or initialized according to various other techniques, such as initializing and training the network 1606, according to one or more of the machine learning techniques described above, to perform a given inference task. This initialized and / or trained network 1606 may then be used as the basis for generating a new circuit design, as described below, instead of modifying an existing initial digital circuit design 1602.

[0188] The next four operations 1501, 1502, 1503, and 1504 of method 1700 may be performed in accordance with their counterparts described above with reference to method 1700, using the network 1606 as the network that includes the node or nodes subject to those operations. In some examples, these operation are performed by the optimization 1608 components: operation 1503 is performed by an inference 1610 component to generate the inference data 1612, and the inference data 1612 is used by a parameter adjustment 1614 component to adjust the parameters of the node(s) of the network 1606 to generate an adjusted network 1616 having adjusted parameter values for at least one of the nodes. These operations may be repeated one or more times to continue to adjust the parameters of the network 1606, potentially improving or optimizing a desired behavior of the network 1606 and its machine learning model, e.g., by improving performance of the inference task. In some examples, the inference task may be defined by a user of the circuit design system 1600, or it may be inferred from the initial digital circuit design 1602 by the circuit simulation 1604 component or the optimization 1608 component. For example, a user of the circuit design system 1600 may implicitly define the inference task through the objective function and training data used in training the machine learning model (e.g., as part of the operations of the optimization 1608 component).

[0189] At some point during the iterative operation of the optimization 1608 component, an end condition may be satisfied (e.g., based on convergence of parameter values, reaching a performance threshold for performance of the inference task, or some other suitable end condition for optimization or training). After the end condition is satisfied, the method 1700 proceeds to operation 1704.

[0190] At operation 1704, the circuit design system 1600 generates a revised design for the initial digital circuit based on the adjusted network 1616. A network to circuit mapping 1618 component may be used to perform a similar mapping between digital circuit logic and network characteristics (e.g., node parameter values and network topology) as that performed by the circuit simulation 1604 component, but in reverse. In some examples, the behavior of the nodes of the network may be mapped to digital circuit logic (e.g., logic gates) using approximation: for example, a given node may be mapped to a logic gate based on a closest fit of the node's parameter values to the behavior of the logic gate.

[0191] In some examples, the revised digital circuit design 1620 may be used to configure a field programmable gate array (FPGA). The FGPA may be configured to implement the logic of the logic gates of the revised digital circuit design 1620. In some examples, the optimization 1608 component may continue or resume its operations after the FPGA has been configured, and the FPGA may subsequently be modified again based on a further revised digital circuit design 1620 generated based on a further adjusted network 1616.

[0192] Through the operations of method 1700, a new circuit design may be generated, and / or an existing circuit design may be modified, by optimizing or otherwise adjusting a network of a machine learning model, using machine learning techniques described herein, followed by translation of the optimized or adjusted machine learning model into logic gates of the revised digital circuit design 1620. It will be appreciated that, in examples in which a new circuit design is generated using method 1700, the revised digital circuit design 1620 may instead be a regarded as a new digital circuit design.

[0193] Thus, in some examples, the operations of method 1500 may be modified, expanded, or specified such that the node includes one or more parameters configured to map the one or more inputs to the one or more outputs according to logic corresponding to one or more Boolean operators. In some examples, the one or more Boolean operators operate as a binary logic gate. In some examples, the inference data is processed to adjust the one or more parameters of the node, thereby generating an adjusted network of the machine learning model, and a design for a digital circuit is generated based on the adjusted network of the machine learning model.

[0194] In some examples, prior to obtaining the one or more inputs of the node, the method 1700 operates to configure the one or more parameters of the node, and configure additional parameters of additional nodes of the network of the machine learning model, to correspond to the logic of an initial digital circuit, where the design for the digital circuit includes a revised design for the initial digital circuit.

[0195] In some examples, a configuration for a field programmable gate array is generated based on the network of the machine learning model after adjusting the one or more parameters.

[0196] FIG. 18 is a diagrammatic representation of the machine 1800 within which instructions 1802 (e.g., software, a program, an application, an applet, an app, or other executable code) for causing the machine 1800 to perform any one or more of the methodologies discussed herein may be executed. For example, the instructions 1802 may cause the machine 1800 to execute any one or more of the methods described herein. The instructions 1802 transform the general, non-programmed machine 1800 into a particular machine 1800 programmed to carry out the described and illustrated functions in the manner described. The machine 1800 may operate as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1800 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1800 may comprise, but not be limited to, a server computer, a client computer, a personal computer (PC), a tabletcomputer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular telephone, a smartphone, a mobile device, a wearable device (e.g., a smartwatch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions 1802, sequentially or otherwise, that specify actions to be taken by the machine 1800. Further, while a single machine 1800 is illustrated, the term “machine" shall also be taken to include a collection of machines that individually or jointly execute the instructions 1802 to perform any one or more of the methodologies discussed herein. In some examples, the machine 1800 may comprise both client and server systems, with certain operations of a particular method or algorithm being performed on the server-side and with certain operations of the particular method or algorithm being performed on the client-side.

[0197] The machine 1800 may include processors 1804, memory 1806, and input / output I / O components 1808, which may be configured to communicate with each other via a bus 1810. In an example, the processors 1804 (e.g., a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) Processor, a Complex Instruction Set Computing (CISC) Processor, a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Radio- Frequency Integrated Circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 1812 and a processor 1814 that execute the instructions 1802. The term "processor" is intended to include multi-core processors that may comprise two or more independent processors (sometimes referred to as “cores”) that may execute instructions contemporaneously. Although FIG. 18 shows multiple processors 1804, the machine 1800 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiples cores, or any combination thereof.

[0198] The memory 1806 includes a main memory 1816, a static memory 1818, and a storage unit 1820, both accessible to the processors 1804 via thebus 1810. The main memory 1806, the static memory 1818, and storage unit 1820 store the instructions 1802 embodying any one or more of the methodologies or functions described herein. The instructions 1802 may also reside, completely or partially, within the main memory 1816, within the static memory 1818, within machine-readable medium 1822 within the storage unit 1820, within at least one of the processors 1804 (e.g., within the processor’s cache memory), or any suitable combination thereof, during execution thereof by the machine 1800.

[0199] The I / O components 1808 may include a wide variety of components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I / O components 1808 that are included in a particular machine will depend on the type of machine. For example, portable machines such as mobile phones may include a touch input device or other such input mechanisms, while a headless server machine will likely not include such a touch input device. It will be appreciated that the I / O components 1808 may include many other components that are not shown in FIG. 18. In various examples, the I / O components 1808 may include user output components 1824 and user input components 1826. The user output components 1824 may include visual components (e.g., a display such as a plasma display panel (PDP), a light-emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), haptic components (e.g., a vibratory motor, resistance mechanisms), other signal generators, and so forth. The user input components 1826 may include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo- optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or another pointing instrument), tactile input components (e.g., a physical button, a touch screen that provides location and force of touches or touch gestures, or other tactile input components), audio input components (e.g., a microphone), and the like.

[0200] Communication may be implemented using a wide variety of technologies. The I / O components 1808 further include communication components 1828 operable to couple the machine 1800 to a network 1830 or devices 1832 via respective coupling or connections. For example, the communication components 1828 may include a network interface component or another suitable device to interface with the network 1830. In further examples, the communication components 1828 may include wired communication components, wireless communication components, cellular communication components, Near Field Communication (NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and other communication components to provide communication via other modalities. The devices 1832 may be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a USB).

[0201] Moreover, the communication components 1828 may detect identifiers or include components operable to detect identifiers. For example, the communication components 1828 may include Radio Frequency Identification (RFID) tag reader components, NFC smart tag detection components, optical reader components (e.g., an optical sensor to detect onedimensional bar codes such as Universal Product Code (UPC) bar code, multi-dimensional bar codes such as Quick Response (QR) code, Aztec code, Data Matrix, Dataglyph™, MaxiCode, PDF417, Ultra Code, UCC RSS- 2D bar code, and other optical codes), or acoustic detection components (e.g., microphones to identify tagged audio signals). In addition, a variety of information may be derived via the communication components 1828, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi® signal triangulation, location via detecting an NFC beacon signal that may indicate a particular location, and so forth.

[0202] The various memories (e.g., main memory 1816, static memory 1818, and memory of the processors 1804) and storage unit 1820 may store one or more sets of instructions and data structures (e.g., software) embodying or used by any one or more of the methodologies or functions described herein. These instructions (e.g., the instructions 1802), whenexecuted by processors 1804, cause various operations to implement the disclosed examples.

[0203] The instructions 1802 may be transmitted or received over the network 1830, using a transmission medium, via a network interface device (e.g., a network interface component included in the communication components 1828) and using any one of several well-known transfer protocols (e.g., hypertext transfer protocol (HTTP)). Similarly, the instructions 1802 may be transmitted or received using a transmission medium via a coupling (e.g., a peer-to-peer coupling) to the devices 1832.

[0204] Various examples described herein may provide systems, methods, and computer-readable storage media for processing discrete inputs to generate discrete outputs of a node of a network of a machine learning model. Various potential benefits may be realized by various examples described herein, as described throughout this specification. Some examples may attempt to address one or more technical problems in the field of machine learning and / or circuit design, such as problems relating to computational efficiency, circuit simulation, and / or circuit optimization. These examples may provide systems and methods that reduce the consumption of computational resources of machine learning techniques and / or improve the performance of circuits by reducing processing cycles, power consumption, and / or time required for various operations thereof.

[0205] Other technical features may be readily apparent to one skilled in the art from the figures, descriptions, and claims.

[0206] Example l is a method comprising: accessing one or more inputs of a node of a network of a machine learning model, each input of the node having a respective input value, each input value being a member of a finite set of discrete values; processing the one or more inputs of the node to generate one or more outputs of the node, each output having a respective output value, each output value being a member of the finite set of discrete values; and using the machine learning model to generate inference data comprising an inference based on the one or more outputs of the node.

[0207] In Example 2, the subject matter of Example 1 includes, wherein: the finite set of discrete values is a set of two binary values.

[0208] In Example 3, the subject matter of Example 2 includes, wherein: the node includes one or more parameters that are processed to map the one or more inputs to the one or more outputs according to logic corresponding to one or more Boolean operators.

[0209] In Example 4, the subject matter of Example 3 includes, wherein: the one or more Boolean operators operate as a binary logic gate.

[0210] In Example 5, the subject matter of Example 4 includes, generating an adjusted network of the machine learning model by adjusting the one or more parameters of the node based on the inference data; and generating a design for a digital circuit based on the adjusted network of the machine learning model.

[0211] In Example 6, the subject matter of Example 5 includes, prior to obtaining the one or more inputs of the node: configuring the one or more parameters of the node, and configuring additional parameters of additional nodes of the network of the machine learning model, to correspond to the logic of an initial digital circuit; wherein the design for the digital circuit comprises a revised design for the initial digital circuit.

[0212] In Example 7, the subject matter of Examples 4-6 includes, adjusting the one or more parameters of the node based on the inference data; and generating a configuration for a field programmable gate array based on the network of the machine learning model after adjusting the one or more parameters.

[0213] In Example 8, the subject matter of Examples 1-7 includes, wherein: the node includes a plurality of parameters structured as a parameter tensor having a dimension proportional in size to a number of possible input states of the node, the number of possible input states of the node being a product of a respective number of possible input values of each of the one or more inputs; and the generating of the one or more outputs comprises: generating the one or more outputs by processing a portion of the parameter tensor, the portion corresponding to a current input state of the node, the current input state comprising the input values of the one or more inputs.

[0214] In Example 9, the subject matter of Example 8 includes, wherein: the processing of the portion of the parameter tensor to generate the one or more outputs comprises: processing the portion of the parameter tensor to generate a respective distribution of possible output values for each of the one or more outputs; and selecting a respective output value for each output based on the respective distribution of possible output values of the output.

[0215] In Example 10, the subject matter of Example 9 includes, wherein: the selecting of the respective output value for each output based on the respective distribution of possible output values of the output comprises: selecting an output value from the respective distribution of possible output values via a stochastic process.

[0216] In Example 11, the subject matter of Examples 9-10 includes, wherein: the selecting of the respective output value for each output based on the respective distribution of possible output values of the output comprises: selecting an output value from the respective distribution of possible output values via a deterministic process.

[0217] In Example 12, the subject matter of Examples 9-11 includes, wherein: the selecting of the respective output value for each output based on the respective distribution of possible output values of the output is performed by a stochastic process in response to determining that the node is in a stochastic mode.

[0218] In Example 13, the subject matter of Examples 9-12 includes, wherein: the selecting of the respective output value for each output based on the respective distribution of possible output values of the output is performed by a deterministic process in response to determining that the node is in a deterministic mode.

[0219] In Example 14, the subject matter of Examples 9-13 includes, wherein: the portion of the parameter tensor comprises an array of continuous parameter values; and the processing of the portion of the parameter tensor to generate the respective distribution of possible output values for a first output of the one or more outputs comprises: reducing a computational complexity of generating the distribution of possible output values for the first output by discretizing the continuous parameter values togenerate an array of discrete parameter values; and processing the array of discrete parameter values to generate the distribution of possible output values for the first output.

[0220] In Example 15, the subject matter of Example 14 includes, wherein: the discretizing of the continuous parameter values to generate an array of discrete parameter values comprises: in response to determining that a first parameter value of the continuous parameter values is above a discretization threshold: discretizing the first parameter value to a value of 1; and discretizing each other continuous parameter value to a value of 0.

[0221] In Example 16, the subject matter of Example 15 includes: the discretization threshold is a learned parameter; and the method further comprises adjusting the discretization threshold based on the inference data.

[0222] Example 17 is a non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a system, cause the system to perform operations comprising: accessing one or more inputs of a node of a network of a machine learning model, each input of the node having a respective input value, each input value being a member of a finite set of discrete values; processing the one or more inputs of the node to generate one or more outputs of the node, each output having a respective output value, each output value being a member of the finite set of discrete values; and using the machine learning model to generate inference data comprising an inference based on the one or more outputs of the node.

[0223] Example 18 is a system comprising: one or more processors; and a non-transitory computer readable storage medium storing: a machine learning model, comprising a network having a plurality of nodes; and instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: accessing one or more inputs of a node of the network of the machine learning model, each input of the node having a respective input value, each input value being a member of a finite set of discrete values; processing the one or more inputs of the node to generate one or more outputs of the node, each output having a respective output value, each output value being a member of the finite setof discrete values; and using the machine learning model to generate inference data comprising an inference based on the one or more outputs of the node.

[0224] In Example 19, the subject matter of Example 18 includes, wherein: the finite set of discrete values is a set of two binary values.

[0225] In Example 20, the subject matter of Example 19 includes, wherein: the node includes one or more parameters that are processed to map the one or more inputs to the one or more outputs according to logic corresponding to one or more Boolean operators.

[0226] Example 21 is at least one machine-readable medium including instructions that, when executed by processing circuitry, cause the processing circuitry to perform operations to implement of any of Examples 1-20.

[0227] Example 22 is an apparatus comprising means to implement of any of Examples 1-20.

[0228] Example 23 is a system to implement of any of Examples 1-20.

[0229] Example 24 is a method to implement of any of Examples 1-20.

[0230] Unless the context clearly requires otherwise, throughout the description and the claims, the words “comprise,” “comprising,” and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense, i.e., in the sense of “including, but not limited to.” As used herein, the terms “connected,” “coupled,” or any variant thereof means any connection or coupling, either direct or indirect, between two or more elements; the coupling or connection between the elements can be physical, logical, or a combination thereof. Additionally, the words “herein,” “above,” “below,” and words of similar import, when used in this application, refer to this application as a whole and not to any particular portions of this application. Where the context permits, words using the singular or plural number may also include the plural or singular number respectively. The word “or” in reference to a list of two or more items, covers all of the following interpretations of the word: any one of the items in the list, all of the items in the list, and any combination of the items in the list. Likewise,the term “and / or” in reference to a list of two or more items, covers all of the following interpretations of the word: any one of the items in the list, all of the items in the list, and any combination of the items in the list.

[0231] A “non-transitory” machine-readable medium, such as a “non- transitory” computer readable storage medium, as used herein, specifically excludes propagating signals per se. A non-transitory machine-readable medium may also be referred to herein as a tangible machine-readable medium. According to various example embodiments, instructions for execution by a machine, and other information such as machine learning models and the parameters thereof, can be communicated via a carrier medium (e.g., a machine-readable carrier medium). Examples of such a carrier medium include a non-transient carrier medium (e.g., a non-transitory machine-readable storage medium, such as a solid-state memory that is physically movable from one place to another place) and a transient carrier medium (e.g., a carrier wave or other propagating signal that communicates instructions or information).

[0232] Although some examples, e.g., those depicted in the drawings, include a particular sequence of operations, the sequence may be altered without departing from the scope of the present disclosure. For example, some of the operations depicted may be performed in parallel or in a different sequence that does not materially affect the functions as described in the examples. In other examples, different components of an example device or system that implements an example method may perform functions at substantially the same time or in a specific sequence.

[0233] The various features, steps, and processes described herein may be used independently of one another, or may be combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of this disclosure. In addition, certain method or process blocks may be omitted in some implementations.

Claims

CLAIMSWhat is claimed is:

1. A method comprising: accessing one or more inputs of a node of a network of a machine learning model, each input of the node having a respective input value, each input value being a member of a finite set of discrete values; processing the one or more inputs of the node to generate one or more outputs of the node, each output having a respective output value, each output value being a member of the finite set of discrete values; and using the machine learning model to generate inference data comprising an inference based on the one or more outputs of the node.

2. The method of claim 1, wherein: the finite set of discrete values is a set of two binary values.

3. The method of claim 2, wherein: the node includes one or more parameters that are processed to map the one or more inputs to the one or more outputs according to logic corresponding to one or more Boolean operators.

4. The method of claim 3, wherein: the one or more Boolean operators operate as a binary logic gate.

5. The method of claim 4, further comprising: generating an adjusted network of the machine learning model by adjusting the one or more parameters of the node based on the inference data; and generating a design for a digital circuit based on the adjusted network of the machine learning model.

6. The method of claim 5, further comprising: prior to obtaining the one or more inputs of the node: configuring the one or more parameters of the node, and configuring additional parameters of additional nodes of the networkof the machine learning model, to correspond to the logic of an initial digital circuit; wherein the design for the digital circuit comprises a revised design for the initial digital circuit.

7. The method of claim 4, further comprising: adjusting the one or more parameters of the node based on the inference data; and generating a configuration for a field programmable gate array based on the network of the machine learning model after adjusting the one or more parameters.

8. The method of claim 1, wherein: the node includes a plurality of parameters structured as a parameter tensor having a dimension proportional in size to a number of possible input states of the node, the number of possible input states of the node being a product of a respective number of possible input values of each of the one or more inputs; and the generating of the one or more outputs comprises: generating the one or more outputs by processing a portion of the parameter tensor, the portion corresponding to a current input state of the node, the current input state comprising the input values of the one or more inputs.

9. The method of claim 8, wherein: the processing of the portion of the parameter tensor to generate the one or more outputs comprises: processing the portion of the parameter tensor to generate a respective distribution of possible output values for each of the one or more outputs; and selecting a respective output value for each output based on the respective distribution of possible output values of the output.

10. The method of claim 9, wherein:the selecting of the respective output value for each output based on the respective distribution of possible output values of the output comprises: selecting an output value from the respective distribution of possible output values via a stochastic process.

11. The method of claim 9, wherein: the selecting of the respective output value for each output based on the respective distribution of possible output values of the output comprises: selecting an output value from the respective distribution of possible output values via a deterministic process.

12. The method of claim 9, wherein: the selecting of the respective output value for each output based on the respective distribution of possible output values of the output is performed by a stochastic process in response to determining that the node is in a stochastic mode.

13. The method of claim 9, wherein: the selecting of the respective output value for each output based on the respective distribution of possible output values of the output is performed by a deterministic process in response to determining that the node is in a deterministic mode.

14. The method of claim 9, wherein: the portion of the parameter tensor comprises an array of continuous parameter values; and the processing of the portion of the parameter tensor to generate the respective distribution of possible output values for a first output of the one or more outputs comprises: reducing a computational complexity of generating the distribution of possible output values for the first output by discretizing the continuous parameter values to generate an array of discrete parameter values; and processing the array of discrete parameter values to generate the distribution of possible output values for the first output.

15. The method of claim 14, wherein: the discretizing of the continuous parameter values to generate an array of discrete parameter values comprises: in response to determining that a first parameter value of the continuous parameter values is above a discretization threshold: discretizing the first parameter value to a value of 1; and discretizing each other continuous parameter value to a value of 0.

16. The method of claim 15: the discretization threshold is a learned parameter; and the method further comprises adjusting the discretization threshold based on the inference data.

17. A non-transitory computer-readable storage medium, the computer- readable storage medium including instructions that when executed by a system, cause the system to perform operations comprising: accessing one or more inputs of a node of a network of a machine learning model, each input of the node having a respective input value, each input value being a member of a finite set of discrete values; processing the one or more inputs of the node to generate one or more outputs of the node, each output having a respective output value, each output value being a member of the finite set of discrete values; and using the machine learning model to generate inference data comprising an inference based on the one or more outputs of the node.

18. A system comprising: one or more processors; and a non-transitory computer readable storage medium storing: a machine learning model, comprising a network having a plurality of nodes; and instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:accessing one or more inputs of a node of the network of the machine learning model, each input of the node having a respective input value, each input value being a member of a finite set of discrete values; processing the one or more inputs of the node to generate one or more outputs of the node, each output having a respective output value, each output value being a member of the finite set of discrete values; and using the machine learning model to generate inference data comprising an inference based on the one or more outputs of the node.

19. The system of claim 18, wherein: the finite set of discrete values is a set of two binary values.

20. The system of claim 19, wherein: the node includes one or more parameters that are processed to map the one or more inputs to the one or more outputs according to logic corresponding to one or more Boolean operators.

Images