Nodal self-determination based on free energy

Abstract

A method includes calculating, by a node within a machine learning model, free energy measures corresponding to all nodal actions in a predetermined plurality of nodal actions that are performable by the node within the machine learning model during an upcoming timestep. A nodal action is selected to be performed by the node during the upcoming timestep based on comparisons among the free energy measures. The node performs the selected nodal action during the upcoming timestep.

Description

NODAL SELF-DETERMINATION BASED ON FREE ENERGYTECHNICAL FIELD

[0001] The present disclosure relates generally to the technical field of special-purpose machines that facilitate artificial intelligence (Al), including software-configured computerized variants of such special-purpose machines and improvements to such variants, and to the technologies by which such special-purpose machines become improved compared to other specialpurpose machines that facilitate Al. Specifically, the present disclosure addresses systems and methods to facilitate self-determination of nodes of a machine learning model based on free energy calculations.BACKGROUND

[0002] Within the field of artificial intelligence (Al), machine learning models implement one or more algorithms or other techniques for performance on computer hardware and are configured to learn from experiences processing data and make decisions (e.g., predictions, inferences, or categorizations) without explicit programming. Machine learning models may utilize one or more statistical methods, mathematical optimization, pattern recognition techniques, or any suitable combination thereof, to identify patterns or other relationships within data. Machine learning models are now widely used across various domains, such as image recognition, speech recognition, natural language processing, classifiers, recommendation generators, and anomaly detectors. In some traditional machine learning models, the model is trained using historical information that is labeled with one or more features, and after training, the trained machine learning model is provided with new unlabeled input data to generate inferences about the new data.BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0003] In the drawings, which are not necessarily drawn to scale, like numerals may describe similar components in different views. To easilyidentify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the figure number in which that element is first introduced. Some non-limiting example embodiments of the subject matter are illustrated in the figures of the accompanying drawings in which:

[0004] FIG. l is a network diagram illustrating a network environment suitable for self-determination of machine learning model nodes based on free energy, according to some examples.

[0005] FIG. 2 is a block diagram illustrating components of an Al machine suitable for self-determination of machine learning model nodes based on free energy, according to some examples.

[0006] FIG. 3 illustrates a first example group of nodes of a machine learning model, according to some examples.

[0007] FIG. 4 illustrates a further example group of nodes of a machine learning model, according to some examples.

[0008] FIG. 5 illustrates initial values of priors of a node of the group of nodes of FIG. 4, according to some examples.

[0009] FIG. 6 illustrates initial values of priors of the Bo submatrix of FIG. 5, according to some examples.

[0010] FIG. 7 illustrates differentiated values of priors of the Bo submatrix of FIG. 6, according to some examples.

[0011] FIG. 8 illustrates initial values of priors of the Bi submatrix of FIG. 5, according to some examples.

[0012] FIG. 9 illustrates differentiated values of priors of the Bi submatrix of FIG. 8, according to some examples.

[0013] FIG. 10 illustrates initial prior values and three different sets of differentiated values of priors of the B matrix of FIG. 5, according to some examples.

[0014] FIG. 11 illustrates a sequence showing generation and restructuring of a group of nodes of a machine learning model over time, according to some examples.

[0015] FIG. 12 illustrates a set of nodal actions, according to some examples.

[0016] FIG. 13 illustrates a flowchart showing operations of a method for nodal self-determination based on free energy, according to some examples.

[0017] FIG. 14 is a diagrammatic representation of a machine in the form of a computer system within which a set of instructions may be executed to cause the machine to perform any one or more of the methodologies discussed herein, according to some examples.DETAILED DESCRIPTION

[0018] Examples described herein provide methods (e.g., algorithms) for enabling self-determination, based on free energy, of nodes of a machine learning model, and example systems (e.g., special-purpose machines configured by special-purpose software) configured to facilitate self- determination, based on free energy, of nodes of a machine learning model. Examples merely typify possible variations. Unless explicitly stated otherwise, structures (e.g., structural components, such as modules) are optional and may be combined or subdivided, and operations (e.g., in a procedure, algorithm, or other function) may vary in sequence or be combined or subdivided. In the following description, for purposes of explanation, numerous specific details are set forth to provide a thorough understanding of various example embodiments. It will be evident to one skilled in the art, however, that the present subject matter may be practiced without these specific details.

[0019] Certain types of Al models (e.g., a machine learning model, which may form all or part of an Al engine) include multiple nodes (e.g., a plurality of nodes, which may be arranged, grouped, or otherwise structured into one or more groups, such as layers). In such Al models, one or more of such nodes may be configured to each produce outputs (e.g., future outputs) based on (e.g., from) inputs (e.g., future inputs) provided to that node. The production of outputs from inputs by a node may be performed using or otherwise based on a generative model of that node. The generative model of a node may be, include, or otherwise be defined by a set of numerical weights, biases, or both (e.g., stored as a tensor), and the generative model ofthe node may accordingly determine how the node processes inputs received by the node into outputs produced by that same node.

[0020] Regarding types of inputs to a node, for example, a node may be configured to receive directed input (e.g., specifically targeted for processing by the node as part of performing a function of the node, such as an outputting function, which may be shared in common by a group of nodes that include the node), and the directed input may be received by the node from one or more sources (e.g., one or more nodes inside or outside the group of nodes). Additionally, that same node may be configured to receive incidental or observational input (e.g., contextual information, environmental information, or other information not specifically targeted for processing as part of performing the function of the node) from one or more other nodes in its group (e.g., peer nodes, child nodes, or any suitable combination thereof) regarding the outputs of those one or more other nodes in the group.

[0021] According to the systems and methods described herein, a machine (e.g., an Al machine) may facilitate self-determination of nodes of a machine learning model. As the machine learning model is executed by the machine, the nodes of the machine learning model receive inputs, generate outputs, and take actions according to a sequence of time steps. Nodal self- determination can be facilitated by the machine by calculating a set of free energy measures for a node within the machine learning model. Each free energy measure calculated for the node corresponds to an action the mode can take (referred to as a nodal action) at an upcoming timestep. In some examples, the calculations are performed only with respect to information locally available to the node, without reference to global information about the network of nodes as a whole.

[0022] After the calculation of the free energy measures for the nodal actions, a nodal action is selected based on comparisons among the free energy measures. In some examples, the nodal action corresponding to a lowest free energy measure is selected. The selected nodal action is then performed by the node during the upcoming timestep. In the examples described herein, free energy is the basis used for selecting among thepossible nodal actions: if more than one nodal action is available to the node, the node will select the nodal action that has the most favorable (e.g., lowest) fee energy measure.

[0023] In some examples, nodal actions can include the node replicating itself within the executing machine learning model; the node deleting itself from the executing machine learning model; the node suspending itself for the upcoming timestep of the executing machine learning model; the node initiating reception of data from a further node within the executing machine learning model; the node terminating exchange of data with a further node within the executing machine learning model; and / or the node operating normally for the upcoming timestep of the executing machine learning model.

[0024] Self-determination of nodes of the machine learning model network may provide one or more beneficial effects. By using only local information available to the node, calculation of free energy measures can be greatly simplified relative to calculations that involve global information drawn from the machine learning model as a whole. This in turn reduces the computational costs and increases the speed of decision-making for selecting nodal actions. In some examples, nodes of the machine learning model can be implemented by parallel hardware capable of executing local regions of the machine learning model (e.g., individual nodes or clusters of nodes) independently and concurrently; in such configurations, reliance on local information for nodal action selection can enable efficient parallel execution of the regions of the model. The use of free energy as a decision-making metric for nodal action selection allows the machine learning model to learn and evolve in accordance with techniques of active inference, which can provide various benefits in the context of machine learning, as described in greater detail below.

[0025] FIG. 1 through FIG. 4 below illustrate examples of Al systems executing machine learning models that include groups of nodes used to learn and perform machine learning tasks. FIG. 5 through FIG. 10 illustrate examples of computational operations implementing active inference. Various nodal actions, and techniques for nodal self-determination based onfree energy, are described with reference to FIG. 11 through FIG. 13. FIG. 14 illustrates a machine suitable for implementation of example techniques described herein.

[0026] FIG. l is a network diagram illustrating a network environment 100 suitable for self-determination of machine learning model nodes based on free energy, according to some example embodiments. The network environment 100 includes an Al machine 102, a database 104, and devices 106 and 108, all communicatively coupled to each other via a network 110. The Al machine 102, with or without the database 104, may form all or part of a cloud 116 (e.g., a geographically distributed set of multiple machines configured to function as a single server), which may form all or part of a network-based system 118(e.g., a cloud-based server system configured to provide one or more network-based services to the devices 106 and 108). The Al machine 102, the database 104, and the devices 106 and 108 may each be implemented in a special-purpose (e.g., specialized) computer system, in whole or in part, as described below with respect to the machine 1400 of FIG. 14.

[0027] Also shown in FIG. 1 are users 112 and 114. One or both of the users 112 and 114 may be a human user (e.g., a human being), a machine user (e.g., a computer configured by a software program to interact with the device 106 or 108), or any suitable combination thereof (e.g., a human assisted by a machine or a machine supervised by a human). The user 112 is associated with the device 106 and may be a user of the device 106. For example, the device 106 may be a desktop computer, a vehicle computer, a home media system (e.g., a home theater system or other home entertainment system), a tablet computer, a navigational device, a portable media device, a smart phone, or a wearable device (e.g., a smart watch, smart glasses, smart clothing, or smart jewelry) belonging to the user 112. Likewise, the user 114 is associated with the device 108 and may be a user of the device 108. As an example, the device 108 may be a desktop computer, a vehicle computer, a home media system (e.g., a home theater system or other home entertainment system), a tablet computer, a navigational device, a portable media device, a smart phone, or a wearabledevice (e.g., a smart watch, smart glasses, smart clothing, or smart jewelry) belonging to the user 114.

[0028] Any of the systems or machines (e.g., databases and devices) shown in FIG. 1 may be, include, or otherwise be implemented in a specialpurpose (e.g., specialized or otherwise non-conventional and nongeneric) computer that has been modified to perform one or more of the functions described herein for that system or machine (e.g., configured or programmed by special-purpose software, such as one or more software modules of a special-purpose application, operating system, firmware, middleware, or other software program). For example, a special-purpose computer system able to implement any one or more of the methodologies described herein is discussed below with respect to the machine 1400 of FIG. 14, and such a special-purpose computer may accordingly be a means for performing any one or more of the methodologies discussed herein. Within the technical field of such special-purpose computers, a special-purpose computer that has been specially modified (e.g., configured by special-purpose software) by the structures discussed herein to perform the functions discussed herein is technically improved compared to other special-purpose computers that lack the structures discussed herein or are otherwise unable to perform the functions discussed herein. Accordingly, a special-purpose machine configured according to the systems and methods discussed herein provides an improvement to the technology of similar special-purpose machines.

[0029] As used herein, a “database” is a data storage resource and may store data structured in any of various ways, for example, as a text file, a table, a spreadsheet, a relational database (e.g., an object-relational database), a triple store, a hierarchical data store, a document database, a graph database, key-value pairs, or any suitable combination thereof. Moreover, any two or more of the systems or machines illustrated in FIG. 1 may be combined into a single system or machine, and the functions described herein for any single system or machine may be subdivided among multiple systems or machines.

[0030] The network 110 may be any network that enables communication between or among systems, machines, databases, and devices(e.g., between the Al machine 102 and the device 106). Accordingly, the network 110 may be a wired network, a wireless network (e.g., a mobile or cellular network), or any suitable combination thereof. The network 110 may include one or more portions that constitute a private network, a public network (e.g., the Internet), or any suitable combination thereof. Accordingly, the network 110 may include one or more portions that incorporate a local area network (LAN), a wide area network (WAN), the Internet, a mobile telephone network (e.g., a cellular network), a wired telephone network (e.g., a plain old telephone service (POTS) network), a wireless data network (e.g., a WiFi network or WiMax network), or any suitable combination thereof. Any one or more portions of the network 190 may communicate information via a transmission medium. As used herein, “transmission medium” refers to any intangible (e.g., transitory) medium that is capable of communicating (e.g., transmitting) instructions for execution by a machine (e.g., by one or more processors of such a machine), and includes digital or analog communication signals or other intangible media to facilitate communication of such software.

[0031] FIG. 2 is a block diagram illustrating some components of the Al machine, as configured for self-determination of machine learning model nodes based on free energy, according to some example embodiments. The Al machine 102 is shown as including a machine learning model 202. The machine learning model 202 includes a group 204 of nodes (e.g., each configured to perform a respectively corresponding task, such as to produce future outputs of that node from future inputs to that node based on a generative model of that node, and also to update its generative model based on past feedback received by that node in response to past outputs of that node produced from past inputs to that node). The group 204 of nodes includes multiple nodes (e.g., a plurality of nodes), such as a node 206 (e.g., a first node) and a node 208 (e.g., a second node), which may be a peer of the node 206 (e.g., the first node), a parent of the node 206 (e.g., the first node), a child of the node 206 (e.g., the first node), or a node of some otherrelationship with the node 206 (e.g., the first node) within the group 204 of nodes.

[0032] As shown in FIG. 2, the machine learning model 202 may form all or part of an app 210 (e.g., a mobile app, a server app, or other computer program) that is stored (e.g., installed) on the Al machine 102 (e.g., responsive to or otherwise as a result of data being received via the network 110, such as from the database 104 or from the device 106). Furthermore, one or more processors 212 (e.g., hardware processors, digital processors, or any suitable combination thereof) may be included (e.g., temporarily or permanently) in the app 210, the machine learning model 202, or any suitable combination thereof.

[0033] Any one or more of the components (e.g., modules) described herein may be implemented using hardware alone (e.g., one or more of the processors 212) or a combination of hardware and software. For example, any component described herein may physically include an arrangement of one or more of the processors 212 (e.g., a subset of or among the processors 212) configured to perform the operations described herein for that component. As another example, any component described herein may include software, hardware, or both, that configure an arrangement of one or more of the processors 212 to perform the operations described herein for that component. Accordingly, different components described herein may include and configure different arrangements of the processors 212 at different points in time or a single arrangement of the processors 212 at different points in time. Each component (e.g., module) described herein is an example of a means for performing the operations described herein for that component. Moreover, any two or more components described herein may be combined into a single component, and the functions described herein for a single component may be subdivided among multiple components. Furthermore, according to various example embodiments, components described herein as being implemented within a single system or machine (e.g., a single device) may be distributed across multiple systems or machines (e.g., multiple devices).

[0034] In some examples, the app 210 includes a node manager 214 for managing interactions within the group 204 of nodes of the machine learning model 202. The node manager 214 can be a software process or module that monitors certain aspects of the group 204 of nodes as a whole. Although some examples described herein implement nodal self-determination by relying entirely on local information available to a given node, in some cases a node manager 214 may be used to provide information about a subset of other non-local nodes in the group 204. For example, if a node needs to decide whether to form a connection to another node of the group 204, it may be helpful for a node manager 214 to maintain information about other remote nodes within the group 204 that may be suitable candidates for connection, as described in further detail below with reference to FIG. 12.

[0035] FIG. 3 illustrates a first example group 300 of nodes of the machine learning model 202. The group 300 includes a set of nodes, shown as first node 302, second node 304, and third node 306. The nodes are connected to each other by unidirectional connections, shown as a first connection 308 (from first node 302 to second node 304), a second connection 310 (from second node 304 to third node 306), and a third connection 312 (from third node 306 to first node 302). Each connection conveys or designates outputs of a transmitting node as inputs to a receiving node. The connections for which a given node is a transmitting node may be referred to as outgoing connections of that node; by the same token, those connections for which a node is the receiving node may be referred to as incoming connections of that node. In group 300, the transmitting node of first connection 308 is first node 302, and the receiving node of first connection 308 is second node 304; the transmitting node of second connection 310 is second node 304, and the receiving node of second connection 310 is third node 306; and the transmitting node of third connection 312 is third node 306, and the receiving node of third connection 312 is first node 302.

[0036] The nodes of the group 300 (and other networks or groups of nodes of machine learning models described herein) may be described herein as receiving inputs, processing data, and generating outputs. Similarly, the connections of the described groups of nodes may be described as conveying, transmitting, or providing data from one node to another node. Insome examples, these terms may refer to the operation of nodes and connections embodied as logical circuit components or other physical data processing or data communication components. For example, the nodes and connections of a network of a machine learning model as described herein may be implemented as circuit components of a field programmable gate array (FPGA) or another dynamically reconfigurable logical circuit. In some examples, the use of binary inputs and outputs for the nodes of the network enable the implementation of the network as a circuit comprising a plurality of logic gates.

[0037] In some examples, the nodes and connections may be implemented as data, for example, as data structures or instructions stored in a memory and manipulated by a physical processor. The machine 1400 described with reference to FIG. 14 below provides an example of a computing system suitable for such an implementation. In some examples, the processor executes instructions to perform operations of a system, such as an artificial intelligence (Al) system, that manipulates the data corresponding to the machine learning model (including the nodes and connections of the network) to perform the operations described herein. For example, a node may be implemented as a data structure associated with one or more parameters (e.g., prior probability distributions, also called priors, as described below), and processing the inputs of a node to generate the node's outputs may be performed by the physical processor processing the parameter values of the node and the inputs of the node to generate the outputs of the node. A connection may be implemented as data designating or identifying a relationship between a transmitting node and a receiving node, such that the outputs of the transmitting node are treated by the physical processor as inputs to the receiving node. In some examples, the nodes and connections may include instructions executed by the processor to perform these operations, whereas in other examples, the instructions are separate from the nodes and connections, and cause the processor to process the data corresponding to the nodes and connections. Thus, in some examples, each node may be said to include or be associated with parameter values, and these parameter values may be used to control the processing of the inputs of the node in order to generate the outputs of the node. It will beappreciated that all references herein to a node processing data, to a connection conveying data, and so on, are all intended to encompass all suitable implementations of the nodes and connections as any combination of hardware and software.

[0038] In some examples, as described below with reference to FIG. 13, the group 300 of nodes may be implemented in software or in reconfigurable hardware during some phases requiring reconfiguration of the network (e.g., adjustment of parameter values of the nodes), and the behavior of the network at a point in time may be fixed by mapping the behavior of the group 300 of nodes to a set of interconnected logic gates to create a design for a digital circuit, which may then be constructed to implement the behavior of the network as a physical digital circuit. In some examples, highly parallel execution of the machine learning model 202 can be implemented by a digital circuit, such as an FPGA or an application specific integrated circuit (ASIC).

[0039] In some examples, the nodes of the group 300 operate to process their inputs and generate their outputs at each of a sequence of discrete time steps. In some examples, all nodes of the group 300 operate at each time step; in other examples, some nodes may have different rates of operation such that some nodes of the group 300 may only process their inputs and generate (or update) their outputs at some time steps, such as every n time steps where n is a positive integer. As described below with reference to FIG. 12 and FIG. 13, some examples may selectively suspend operation of one or more nodes of the group 300 for one or more time steps based on nodal self-determined decision-making.

[0040] In some examples, a node generates its outputs at a current time step t by processing its inputs at a previous or prior time step t- . The inputs may be processed by applying various parameters (e.g., priors), such as matrices of priors, to the inputs, and computing one or more output values based on the processed parameters and inputs. In some examples, a node may generate only a single output value that is conveyed as an input to all connected nodes; in other examples, a node may generate multiple outputs to beconveyed to multiple connected nodes or as multiple output values conveyed to a single node.

[0041] In some examples, a single output generated by a node may be an activation value. An activation value may be regarded as a measure of how active the node is at the current time step based on its inputs from the previous time step. In some examples, the activation value of a given node is a continuous value, such as a floating-point value. In some examples, the activation value of a given node is a discrete value, such as a binary value (e.g., 0 or 1, corresponding to inactive or active, or “not firing” and “firing” in the context of an analogy of neuronal firing), or another member of a finite set of discrete values (e.g., a member of the set {0, 1, 2, 3} or a member of the set {00, 01, 10, 11 }). A discrete activation value may be determined by quantization, which may be either a stochastic quantization process or a deterministic quantization process. In a stochastic quantization process, for example, a distribution (e.g., a probability distribution) over all possible discrete output values may be generated by processing the inputs and priors, and a random or pseudo-random value may be used to select a discrete output value (or set of output values) from the probability distribution. In a deterministic quantization process, a single discrete output value (or set of output values) may instead be selected from the distribution of possible discrete output values, e.g., as the value or values having the highest probability or the highest prevalence.

[0042] In some examples, nodes may have connections to themselves, referred to as self-connections, in which they serve as both transmitting node and receiving node. In some such examples, a node may generate a single output for propagation to its connected nodes, and another output or set of outputs to itself, called self-outputs, via its self-connection(s). In some examples, a node may pass through its current inputs as self-outputs, thereby providing a one-time-step memory of its own inputs. This may enable the node to evaluate the accuracy of its own previous predictions of state transitions of the observed system, as described in greater detail below with reference to FIG. 5 through FIG. 10.

[0043] It will be appreciated that the three-node group 300 shown in FIG. 3 is intended as a simplified example, and that actual machine learning models may include networks having much larger number of nodes, such as thousands or millions of nodes. Thus, the group 300 may be regarded as a small section of a larger network, in which one or more of the illustrated nodes 302, 304, or 306 may have connections to or from one or more other unseen nodes of the larger network. In one such example, in the context of CNNs, first node 302 could be regarded as a neuron of a convolution layer, second node 304 could be regarded as a neuron of a rectified linear unit (ReLU) layer receiving an output activation map from the convolution layer, and third node 306 could be regarded as a neuron of a pooling layer receiving the ReLU layer output. The third connection 312 could be regarded as a back-connection to a previous layer as in a recurrent neural network (RNN), or the third connection 312 could be omitted (or set to a zero prior value by first node 302).

[0044] FIG. 4 illustrates a further example group 400 of nodes, including first node 402, second node 404, and third node 406. In group 400, the connections between nodes are in bidirectional pairs: first connection 408 from first node 402 to second node 404 and first connection back 414 from second node 404 to first node 402; second connection 410 from second node 404 to third node 406 and second connection back 416 from third node 406 to second node 404; and third connection 412 from third node 406 to first node 402 and third connection back 418 from first node 402 to third node 406.

[0045] The group 400 also shows first node 402 having a bidirectional pair of connections to an environment 420: a connection from environment 422, and a connection to environment 424. In some examples, the environment 420 as shown in FIG. 4 can represent a data source or other entity residing in an environment outside of the network, such that a node connected to the environment 420 (such as first node 402), also referred to as an environmentfacing node, acts as an input node and an output node for the group 400.

[0046] The connections to or from the environment 420 may act as connections to another node insofar as they convey inputs to, or receiveoutputs from, the environment-facing node. In some examples, the environment 420 can be regarded as an external data source (providing inputs to the network via the connection from environment 422) or a postprocessing stage for outputs of the network (e.g., generated data such as inference data). In some examples, multiple nodes of the network are environment-facing nodes, such that the inputs and outputs of the network at a given time step may comprise multiple values (e.g., multiple binary values for examples in which each node generates only binary outputs). Thus, for example, a network configured to perform an inference task on image data may have 1 million environment-facing nodes configured to each receive one bit of a 1000 x 1000 black and white pixel input image, or more environment-facing nodes for images having multi-bit encodings for each pixel (e.g., greyscale or multi-channel images). A network configured to categorize images data into two categories may only need as few as one environment-facing node to provide an output to the environment 420: an output of “0” might indicate categorization into the first category, whereas an output of “1” might indicate categorization into the second category. However, in many cases, using an active inference model (described below) for the behavior of the network may provide benefits with respect to inference tasks involving bidirectional communication with an environment 420, such as control systems using feedback from the system being controlled: in some such cases, the environment 420 is controlled at least in part by the outputs received via the connection to environment 424, and the environment 420 provides feedback on its own state as inputs via connection from environment 422, such that the network can process the feedback to generate desired control outputs. Thus, in some examples, the behavior of the network may provide benefits in the context of executing the machine learning model on a first machine and using the machine learning model to control the operation of, or performance of a task by, a second machine.

[0047] Thus, in some examples, the environment 420 may represent an entity configured to perform a task based on inputs received from one or more environment-facing nodes (e.g., first node 402) via connections (e.g., connection to environment 424).

[0048] In the illustrated example, each node 402, 404, 406 has a selfconnection 426, which is a connection in which the node is both the transmitting node and the receiving node of the connection. A selfconnection 426 may act as a one-time-step delay loop feeding a node's outputs back to itself. A self-connection 426 can also serve as a one-timestep inverter (e.g., generating a 1 in response to a 0) or a number of other operations: for example, in some cases, a node can process its self-signal from the last timestep and its environment signal from the last timestep according to logic dictated by its parameters (e.g., priors). In some examples, self-connections 426 are used in learning to predict the output of a data source (e.g., another node or the environment 420), as described below.

[0049] In some examples, a network with bidirectional pairs of connections, such as group 400, may be configured to perform inference according to the principles of active inference and the minimization of free energy - either variational free energy (VFE) or expected free energy (EFE)- against a generative model. Active inference is a framework for modeling Bayes-optimal behavior of an agent. In some examples herein using an active inference framework, an agent - such as a machine learning model, or even a node of a network driving the behavior of the machine learning model- acts to minimize an upper bound on the “surprise” resulting from the agent's observations. Surprise is a measure of the degree to which an agent’s current observations differ from its preferred observations and the informativeness of its observations. In some examples, VFE or EFE is a function of both reward and information gain, potentially in varying ratios, thereby biasing the node to both explore a parameter space and exploit its self-efficacy. In some examples herein using an active inference framework, agents seeking to minimize their surprise act on the observed system to solicit feedback that accords with the agent's desired observations: this activity of the agent is why the framework is referred to as “active” inference.

[0050] The active inference frameworks described herein propose that, even if an agent does not have the capability to directly minimize its surprise, it can at least seek to minimize a proxy for surprise, called variational free energy. The agent may minimize VFE OR EFE byperforming computation in response to, and in anticipation of, its observations. Example implementations of active inference described herein may thus rely on the free energy principle, by which an agent reduces its surprise by generating inferences using an internal model and updating its internal model to predict the observed (and acted-upon) system.

[0051] Examples of computational operations implementing active inference are described below with reference to FIG. 5 through FIG. 10. The examples of FIG. 5 through FIG. 10 may be implemented by nodes of the group 400.

[0052] In some examples, a network such as group 400 may be used to implement a machine learning model operating according to an active inference framework (also referred to as an active inference model). Each node of the network may be associated with parameters (e.g., priors) organized into one or more tensors (e.g., scalars, vectors, or matrices), also referred to as parameter tensors, each tensor modeling a different aspect of the node's behavior.

[0053] In some examples, the active inference behavior of the nodes of the network may be modeled as a partially observable Markov decision process (POMDP), characterized by the equation:POMDP equation wherein:• “T” indicates the time step, from a first time step T=1 to a final (or current) time step T=T,• “ot” indicates the observations o of the node (e.g., its inputs) at a given time step i=t,• “st” indicates the actual state 5 of the observed system (e.g., a hidden state that may differ from the observations of the node) at time step i=t,• "K" indicates a policy for the node, e.g., a possible sequence of actions (e.g., outputs to generate in response to inputs) that the node can take, and• “(|)” indicates an expected reward, e.g., a similarity between inputs received by the node and the node's preferred inputs.

[0054] The active inference behavior of a node may be implemented using three or more sets of priors organized into three or more tensors. Examples of tensors used to compute active inference behavior using a POMDP for a node of the network are illustrated and described with reference to FIG. 5.

[0055] FIG. 5 shows four tensors used by a POMDP of a node. The A tensor 502, B tensor 512 (consisting of the Bo subtensor 504 and Bi subtensor 506), C tensor 508, and D tensor 510 indicate four sets of priors maintained by the node. Each of these sets of priors is used to contribute to the performance of an active inference task by the network (e.g., group 400). The values of the priors shown in the example of FIG. 5 are initial prior values that may be assigned to a newly initialized node of the network (e.g., group 400), also referred to as an undifferentiated node or a stem node. The node may be deployed within the network to contribute to an inference task during a runtime period, during which the node may differentiate itself by adjusting its prior values to diverge from their initial values in response to interactions with other nodes of the network. A node that has undergone at least partial differentiation may be referred to as a differentiated node.

[0056] In some examples, the inference task performed by the network is an active inference task. In active inference, as described herein, a machine learning model acts upon a system and observes the system.

[0057] In some examples, the A tensor 502 may be regarded as modeling a probabilistic mapping of the node's observations (e.g., its inputs at a given time step) to an inferred hidden state of the observed system (e.g., a node or set of nodes). This mapping may be referred to as an observational model, denoted as:Observational Model

[0058] In the example shown in FIG. 5, the A tensor 502 is initialized as an identity matrix mapping observed inputs to exactly equivalent hidden states. In some examples, the inputs received by a node are equivalent to the state of the observed system; thus, there are no hidden states, and the A tensor 502 may remain unchanged during differentiation.

[0059] In some examples, the B tensor 512 may be regarded as modeling a probabilistic mapping of the node's observations (e.g., its inputs) at a given time step T-1 to a predicted set of observations at a subsequent time step T. Thus, the B tensor 512 may be referred to as a prediction parameter tensor. This mapping may be referred to as a transition model, denoted as:Transition Model

[0060] In some examples, the B tensor 512 includes a separate submatrix for each set of outputs of the node at the first time step: thus, the predicted inputs at the second time step are conditioned on the outputs of the node at the first time step. In the example shown in FIG. 5, the node has only a single output (e.g., a binary activation value) having only two possible states, 1 or 0. The B tensor 512 therefore consists of two submatrices: a Bo subtensor 504 corresponding to output 0, and a Bi subtensor 506 corresponding to output 1.

[0061] In the example shown in FIG. 5, Bo subtensor 504 and Bi subtensor 506 are both initialized as homogeneous matrices (e.g., all matrix elements set to a value of 0.5) mapping all observed inputs at the first time step with equal probabilities to predicted inputs at the second time step.

[0062] In some examples, the C tensor 508 may be regarded as modeling a prior probability distribution (also referred to simply as a prior) over the node's observations (e.g., its inputs). This prior over observations may be referred to as the node's preferences, prior preferences, or expectations, and is denoted as:Prior Over Observations

[0063] In the example shown in FIG. 5, the C tensor 508 is initialized as a zero vector mapping all observations to a zero reward. Thus, a node initialized according to the initial values shown in FIG. 5 has no preferred values for its inputs. In some examples, C is not 0 but is instead configured to help the node predict the output of another node: the first node (having the C tensor 708) prefers or expects to see its self-signal match the output of a second node. The second node may be a node that the first node is configured to predict, and in some cases, the second node may be referred to as a parent node of the first node.

[0064] In some examples, the D tensor 510 may be regarded as modeling a prior over an initial hidden state at initial time step T=1. This prior over the initial hidden state is denoted as:Prior Over Initial Hidden State

[0065] In the example shown in FIG. 5, the D tensor 510 is initialized as a homogeneous vector (e.g., all vector components set to a value of 0.5) weighting all initial state values with equal prior probability. Thus, a node initialized according to the initial values shown in FIG. 5 has no a priori beliefs about an initial state of the observed system.

[0066] In some examples, additional tensors may be used, whereas in other examples, one or more of the four tensors shown in FIG. 7 may be omitted (such as the D tensor).

[0067] In some cases, the system applies an epistemic term that pertains to knowledge, understanding, and information. The system can prioritize epistemic terms such that the machine learning model focuses on gaininginsight on the functions performed by the machine learning model and updating the model to learn from current inferences. For instance, the system increases the epistemic terms in the C tensor 508 such that the model focuses on certainty, uncertainty, and knowledge of the machine learning model.

[0068] In some examples, the B tensor 512 may be the primary or the only set of priors to be adjusted during differentiation of the node: in other words, each of the other sets of priors (the A tensor 502, C tensor 508, and D tensor 510) remains constant, whereas the transition model (implemented by the B tensor 512) is adjusted to attempt to predict state transitions by the observed system (e.g., other connected nodes), conditioned on the actions (e.g., outputs) of the node. However, in some examples, one or more of the other sets of priors may also be adjusted during differentiation.

[0069] FIG. 6 through FIG. 10 illustrate examples of how the priors of the B tensor 512 of a node may change during differentiation to adapt to different systems exhibiting different behaviors in response to the node's own actions.

[0070] FIG. 6 shows an initial Bo subtensor 504a, equivalent to the initial state of the Bo subtensor 504 shown in FIG. 5. The initial Bo subtensor 504a is the matrix of predictions for a future observation of the state of a system - in the illustrated example, the observed system is a single other node connected to the node associated with the initial Bo subtensor 504a, and the observed node generates a single output, having a one-bit binary value, at each time step. Similarly, the node associated with the initial Bo subtensor 504a also generates a single output, having a one-bit binary value, at each time step. Thus, the B tensor 512 of the node observing the system has two initial submatrices: a first submatrix, shown here in FIG. 6 as initial Bo subtensor 504a, conditioned on the output of the node having a “0” value, and a second submatrix, shown in FIG. 8 below as initial Bi submatrix 506a, conditioned on the output of the node having a “1” value.

[0071] The initial Bo subtensor 504a includes two columns of priors corresponding to two first time step inputs 602. The first time step inputs 602 correspond to the potential observation of two different states of the system (e.g., inputs to the node) at a previous time step T-1 (also referred toherein as the first time step). In this example, the node is assumed to see its inputs accurately; however, in other implementations, the agent may have some uncertainty as to whether its inputs are accurate or the result of noise (e.g., the A tensor 502 may not be an identity matrix). In this example, the node is assumed to see its inputs accurately; however, in other implementations, the agent may have some uncertainty as to whether its inputs are accurate or the result of noise (e.g., the A tensor 502 may not be an identity matrix). The states of the system may also be referred to as input states of the node observing the system. The priors for 0 input at first time step 603 are in the left column, and the priors for 1 input at first time step 604 are in the right column. The two columns may thus be regarded as corresponding to two input states of the node; in some examples, each column (and its associated set of priors, 603 or 604) is only used in computation of the node's behavior when the node observes the corresponding input state.

[0072] Returning to FIG. 6, the columns of the initial Bo subtensor 504a are split into two rows, the second time step inputs 601, corresponding to observation of two different states of the system at a current time step T (also referred to herein as the second time step). The priors for 0 input at second time step 605 are in the top row, and the priors for 1 input at second time step 606 are in the bottom row.

[0073] It will be appreciated that, for nodes observing systems with more complex outputs (e.g., multiple connected nodes, or nodes generating outputs more complex than a single bit per time step), the initial Bo subtensor 504a would include more columns and rows. For example, a node connected to three other nodes, each outputting a single bit at each time step, would need an initial Bo subtensor 504a with 23= 8 columns and 8 rows to map all previously observed system states to all predicted current system states. In some examples, a node may be able to determine that two or more of its inputs tend to be highly similar, and may have its B tensor 712 simplified by combining the two inputs into a single input value.

[0074] As described above, the initial Bo subtensor 504a is a probabilistic mapping of previous observations to predicted current observations,conditioned on the output of the node at time step T-1 having been the binary value 0. The Bi subtensor 506, by contrast, stores the prior values mapping previous observations to predicted current observations conditioned on the output of the node at time step T-1 having been the binary value 1. It will be appreciated that, for nodes having a more complex output than a single bit at each time step, a greater number of B submatrices would be required, equal to the number of possible output states (e.g., 8 submatrices for a 3 -bit binary output). However, in some examples, all inputs and all outputs may be treated as being independent of one another, and the B tensor 712 may include separate subtensors for the separate inputs, such that the 8 x 8 x 8 B tensor that would ordinarily be required (for 3 outputs and 3 dependent inputs) could be replaced by 3 separate subtensors of size 8 (2 x 2 x 2), each subtensor corresponding to a bidirectional connection with one other node.

[0075] Because the nodes of the network generate their outputs for a given time step T before receiving their neighboring nodes' outputs for time step T, the computations informed by the priors of the initial Bo subtensor 504a are performed after time step T-1 and before time step T. Thus, time step T may either be referred to as the “current” time step (meaning the time step for which the current outputs are being computed) or as the “next” or “future” time step (because the observations at time step T are not yet known).

[0076] During the runtime period, the node uses the prior values of initial Bo subtensor 504a to perform active inference, in which the initial Bo subtensor 504a predicts the state of the observed system, conditioned on the previous observed state of the system and conditioned on the node's own previous output. To the extent that observations diverge from the predicted observations of the second time step inputs 601, the node may adjust the priors of the B submatrices in an attempt to minimize future VFE or EFE resulting from the uncertainty of the predictions. Similarly, if the observations concur with the predicted observations of the second time step inputs 804, the node may adjust the priors of the B subtensors to increase the level of certainty of the expected observations. In this way, the initial Bo subtensor 504a may have its priors adjusted over time to predict the observed system, and thereby differentiate itself from its initial neutral, naive, undifferentiated state.

[0077] In some examples, the node selects its outputs to minimize VFE or EFE, as described above. VFE is a computation based on a generative model (e.g., a set of priors learned by a node or other agent) and observations (e.g., inputs to a node or other agent). VFE provides an upper bound on statistical surprise. There are various mechanisms to minimize VFE, including updating the generative model to more closely align with observations, or performing a computation of expected free energy (EFE) across possible actions and selecting one of the actions which leads to the lowest expected free energy. Operations involving the computation or minimization of EFE, as referred to herein, may be referred to as a computation or minimization of VFE.

[0078] To minimize EFE, a node may compute an expected distribution of free energy over multiple output states and select an output state that minimizes the expected free energy. The calculation of expected free energy may in some cases be affected by the node’s preferences (e.g., as represented by the C tensor 508), such that rows of the B tensor 512 corresponding to preferred observations of the node are expected to result in lower free energy than non-preferred rows of the B tensor 512. Thus, the node will calculate EFE distributed over all possible output states, then select an output that is at or near the minimum value of the EFE distribution. When operating in a deterministic mode, the node may select the output value that directly minimizes EFE. When operating in a stochastic mode, the node may weight various output values based on their corresponding EFE, or their proximity to the minimum value of the EFE distribution, and stochastically select an output value from the weighted distribution of output values.

[0079] Thus, the node’s preferences (e.g., the values of the parameters in the C tensor 508) may affect the degree to which the node operates to minimize VFE, EFE, or a combination of both VFE and EFE. A node with uniform zero parameter values in the C tensor 508 will have a close relationship between VFE and EFE, whereas a node with non-uniform parameter values in the C tensor 508 will tend to exhibit EFE behavior that diverges from that predicted solely from VFE.

[0080] In some cases, the node may switch between the stochastic mode and the deterministic mode based on the value of a stress parameter of the node that tracks the degree or frequency to which the node has predicted incorrectly over past time steps. The stress parameter may be adjusted downward on time steps when the node’s output predicts its target, and adjusted upward on time steps when the node’s output fails to predict its target. When the stress parameter value rises above a stress threshold, the node switches from the deterministic mode to the stochastic mode, and when the stress parameter value falls below the stress threshold, the node switches back from the stochastic mode to the deterministic mode. In some cases, two different stress thresholds may be used, such that a falling stress threshold for switching back to the deterministic mode is lower than a rising stress threshold for switching to the stochastic mode.

[0081] In some examples, the stress threshold (and / or the falling and rising stress thresholds) may be a further parameter stored by the node. In some examples, each stress threshold may be a learned parameter that is adjusted during the learning process to assist in minimizing VFE or EFE, as described herein with reference to adjustment of the priors of the B tensor.

[0082] The prior values of the B tensor 512 may be adjusted over a sequence of time steps to predict outputs of another node or a data source, such as the environment 420 external to the network. After the node makes observations (e.g., inputs are accessed or received by the node) at a given time step, the priors of the node’s B tensor 512 may be adjusted based on the currently observed input state, based on the observed input state at the previous time step, and based on the output generated by the node at the previous time step. As described above, a columnar slice of the B tensor 512 is selected based on the node’s inputs observed and its output generated at the last time step, and a column of priors of one of the B subtensors (e.g., Bo subtensor 504 or Bi subtensor 506) intersected by the columnar slice is selected based on the comparison of the priors of the various individual columns in the columnar slice. The priors of this selected column may then be adjusted in accordance with the currently observed input state, such that a higher prior is assigned to the prior in the row corresponding to the currentlyobserved input state, and all other priors in the row are decreased accordingly.

[0083] In some examples, the degree to which the priors of the column are adjusted is in proportion to the number of observations of each input state over time: thus, for example, if the input state of row 2 (e.g., the row of priors for 1 input at second time step 606 in the column of priors for 0 input at second time step 605) has been observed at 32% of time steps, it will be assigned a prior value of 0.32. In other examples, different degrees of adjustment may be used, such as a degree of adjustment proportional to a learning rate that may vary under different conditions.

[0084] In some cases, the node’s preferences may affect the degree to which the priors of the column are adjusted. When a preferred row is observed, in addition to increasing the prior in the observed row and decreasing the other rows accordingly, all of the priors of the column may be adjusted to sharpen the distribution of prior values, such that high prior values are raised and low prior values are lowered, rendering the priors of the column less uniform. In contrast, when a non-preferred row is observed, in addition to increasing the prior in the observed row and decreasing the other rows accordingly, all of the priors of the column may be adjusted to partially flatten the distribution of prior values, such that high prior values are lowered and low prior values are raised, rendering the priors of the column more uniform. The partial flattening of the prior values may result in an increase in the effective learning rate for priors in the column in future time steps, because it is easier for a new observation to stand out relative to other observations when all older observations are relatively uniform. This increase in the effective learning rate after observing a non-preferred row may increase the likelihood or speed with which the node will reconfigure the priors of the column, thereby potentially finding a more optimal set of priors that may assist in avoiding the non-preferred row in the future. By the same token, a slowed effective learning rate after observing a preferred row may reduce the likelihood or speed with which the node will reconfigure the priors of the column, thereby potentially locking the node more tightly into its current configuration.

[0085] It will be appreciated that, whereas some cases adjust only the priors of the column of the B tensor corresponding to the currently observed input state, in some examples only the priors of other columns of the B tensor may be adjusted. In some cases, this adjustment of the priors of non-observed states may yield comparable results in terms of increasing or decreasing the predictive certainty of the generative model being learned, adjusting the drive toward epistemic exploration, and the other techniques described herein.

[0086] FIG. 7 shows the initial Bo subtensor 504a after having undergone differentiation to predict an observed system - in this example, another node - that is configured to employ a “tit for tat” strategy in an iterated prisoner's dilemma task. The tit for tat strategy is one in which an agent will defect (in this example, output a “1”) immediately after an opponent defects, and will cooperate (in this example, output a “0”) immediately after an opponent cooperates. Thus, the output of the observed node is a deterministic function of this node's own previous action: in this example, if the node output a “1” at time step T-1, then the observed node will output a “1” at time step T, and if the node output a “0” at time step T-1, then the observed node will output a “0” at time step T.

[0087] Thus, after the node has undergone differentiation to predict the observed node employing the tit for tat strategy, the node will adjust the priors of its initial Bo subtensor 504a to the values shown in the differentiated Bo subtensor 702: the priors for 0 input at second time step 605 are all 1, indicating a 100% probability that an output of “0” will result in a next observation of a “0”, and the priors for 1 input at second time step 606 are all 0, indicating a 0% probability that an output of “0” will result in a next observation of a “1”. These prior values are set across both columns, indicating that they apply regardless of the observed state of the system.

[0088] By differentiating nodes to learn to predict their observed systems, some examples described herein may train the nodes of the network to generate inferences about the behavior of the rest of the system including (for environment-facing nodes) the environment outside of the machine learning model.

[0089] FIG. 8 and FIG. 9 show the same process of differentiation to predict a “tit for tat” observed node as the differentiation adjusts the prior values of the Bi subtensor 506. Initial Bi submatrix 506a has the same values as initial Bo subtensor 504a. The first time step inputs 602 are mapped to the second time step inputs 601 using priors split into two columns (priors for 0 input at first time step 703 and priors for 1 input at first time step 704) and two rows (priors for 0 input at second time step 705 and priors for 1 input at second time step 706). The mapping of initial Bi submatrix 506a is conditioned on the output of the node at time step T-1 being “1”.

[0090] As the node differentiates to predict the observed tit for tat strategy of its neighboring node, the initial Bi submatrix 506a adjusts its prior values to those shown in the differentiated Bi submatrix 902: the priors for 0 input at second time step 705 are all 0, indicating a 0% probability that an output of “1” will result in a next observation of a “0”, and the priors for 1 input at second time step 706 are all 1, indicating a 100% probability that an output of “1” will result in a next observation of a “1”. These prior values are set across both columns, indicating that they apply regardless of the observed state of the system.

[0091] FIG. 10 shows an initial B tensor 512a (equivalent to B tensor 512 shown in FIG. 5), and its potential to differentiate into multiple possible differentiated values depending on the behavior of the observed system. In this example, the observed system is a single node playing an iterated prisoner's dilemma task, as in the examples of FIG. 6 through FIG. 9.

[0092] If the observed node employs a tit for tat strategy, the initial B tensor 512a will differentiate over time into the tit-for-tat differentiated B tensor 512b, whose Bo and Bi submatrices are equivalent to differentiated Bo subtensor 702 and differentiated Bi submatrix 902 shown in FIG. 7 and FIG. 9. The tit-for-tat differentiated B tensor 512b exhibits absolute certainty in its predictions: the previous action of the node is determinative of the next observed state of the system, due to the nature of the tit for tat strategy.

[0093] If the observed node employs an “always defect” strategy, the initial B tensor 512a will differentiate over time into the defector differentiated B tensor 512c. In an always defect strategy, the observed node always outputs“1”, irrespective of previous outputs and irrespective of inputs. As a result, the node observing the defector node will have complete certainty (priors 0 and 1) about the next observed state: it can predict with 100% confidence that the next observed state will always be “1”. Notably, this prediction is premised on the previous observed state being “1” as well: the 1 and 0 prior values are only in the right column of the Bo subtensor 504 and Bi subtensor 506. Because the node has never observed a system state of “0”, the priors in the left columns are completely undifferentiated from their initial values of 0.5.

[0094] In the final illustrated example, if the observed node employs a “random” strategy, the initial B tensor 512a will differentiate over time into a form resembling the random differentiated B tensor 512d. In a random strategy, the observed node outputs “0” or “1” randomly, irrespective of previous outputs and irrespective of inputs. As a result, the node observing the random node will remain largely undifferentiated (priors all at approximately 0.5) about the next observed state: it cannot predict the next observed state with any confidence.

[0095] In some examples, a node may initially be generated as an undifferentiated node such as initial B tensor 512a, having a B tensor 512 with completely or largely uniform priors (e.g., 0.5 for all priors for two possible input states, 0.25 for all priors for four possible input states, and so on). Over a sequence of time steps, the parameters of the node (e.g., the priors of the B tensor 512) are adjusted such that the node learns a generative model configured to generate a value of a first output matching an expected value of a first input. In some examples, the node is configured to predict a data source (e.g., another node or the environment 420) that generates the first input, thereby learning to support a behavior (e.g., inference behavior) of the data source. Thus, for example, a data source may perform an inference task or other task based on its own output (designated as the first input to the first node), and first node has an objective function (e.g., the expected reward (]> of the POMDP equation above) configured to cause the first node to learn a generative model such that the first output of the first node contributes to and supports the task performed by the data source.

[0096] In some examples, the objective function may be in part a function of the C tensor 508, such that the first node minimizes its VFE or EFE in part by satisfying its preferences. For example, a first node may be configured to attempt to predict a first data source generating a first input. The first node may also have a self-input such that it can compare its own previous output with a current input: the C tensor 508 is configured to prefer input states in which the self-input (generated at time step t-1 but received at time step t) matches the currently observed first input at time step t. For example, the input state may have four possible values indicated as (selfinput received at time step t, first input received at time step t) = (0, 0)|(0, 1)|(1, 0)|(l, 1). Each input state may correspond to a row of the B tensor 512. The C tensor 508 will therefore be configured to prefer the first and fourth rows (0, 0) and (1, 1) and to not prefer the second and third rows (0, 1) and (1, 0). Thus, the C tensor 508 may be a vector with four values corresponding to these preference values for the four rows of the B tensor 512, such as (0.5, 0, 0, 0.5). In some examples, the C tensor 508 may be less completely configured to favor generation of outputs matching the predicted inputs from the prediction target, and the C tensor 508 may have more uniform priors in order to allow some exploration of non-preferred states: e.g., the C tensor 508 may have values (0.4, 0.1, 0.1, 0.4). The trade-off between exploration of non-preferred states and exploitation of self-efficacy to induce preferred states corresponds to the balance between a pragmatic term (encouraging exploitation) and an epistemic term (encouraging exploration) described below.

[0097] In some examples, the C tensor 508 may be uniform (e.g.: (0.25, 0.25, 0.25, 0.25)), indicating no preferred observations (e.g., no preferred rows of the B tensor 512), such that the objective function of the first node can be regarded as simply a minimization of surprise or expected surprise by minimizing VFE or EFE by improving the predictive certainty of the B tensor 512. Such a uniform configuration corresponds to a maximized epistemic term and a minimized pragmatic term.

[0098] As the first node learns to predict the data source, such that the first node can reliably generate the first output to match the first input at a given time step, the first node may be said to have learned a generative modelconfigured to configured to generate a value of the first output matching an expected value of the first input. Examples are described herein in which nodes of the network are generated and configured to predict other nodes or the environment 420, such that inference tasks or other tasks performed by the other node or environment 420 are supported by the output of the first node. As used herein, the term "prediction target" refers to a node, environment, or other data source or entity whose outputs the first node is configured to predict and, in some cases, mirror (by generating an identical or similar output at the same time). Similarly, the node configured to predict the other node, environment, or other data source or entity may be referred to herein as a “predicting node”. In some examples, a node may be added to the network such that the newly added node inherits certain characteristics of another existing node: in such cases, the new node may be referred to as the child (or child node) of the existing node, and the existing node may be referred to as the parent (or parent node) of the child node. In some examples, a child node may be configured to support its parent. In some examples, an environment-facing node may be created when the network is first generated, and this environment-facing node may be referred to as a seed node; seed nodes may be configured in some examples to support the environment 420.

[0099] Returning to FIG. 10, each of the differentiated states 512b, 512c, or 512d of the B tensor 512 of a node presents an example of a different degree of two capabilities of the node: predictive certainty, and self-efficacy. Predictive certainty refers to the degree of certainty in the node's predictions: the tit-for-tat differentiated B tensor 512b exhibits perfect predictive certainty, the defector differentiated B tensor 512c exhibits perfect predictive certainty only under some conditions (it is certain as long as the observed state is “1”, but would be completely uncertain at the next time step if it ever observed a “0”), and the random differentiated B tensor 512d has no predictive certainty.

[0100] The second capability of the node, self-efficacy, refers to a measure of the ability of the output of the node at the previous time step T-1 to affect the observations of the node at the current time step T. The self-efficacy of a node is high if the state of its previous outputs result in highly certaincurrent observations. Thus, the tit-for-tat differentiated B tensor 512b has very high self-efficacy, because its own action is completely determinative of the next observation, which will always be exactly equal to the node's own previous output. The self-efficacy of the defector differentiated B tensor 512c, by contrast, is very low, even though its predictive certainty is perfect for half of the possible observed previous states; the previous output of the node has no effect on its next observation. Finally, the self-efficacy of the random differentiated B tensor 512d is also very low, because its outputs also have no statistically meaningful effect on later observations.

[0101] Thus, predictive certainty and self-efficacy are separate metrics which may or may not co-occur.

[0102] In nodes having no preferences, such as the initial POMDP tensors described with reference to FIG. 5 above having zero values for the C tensor 508, high self-efficacy maps directly to low VFE or low EFE. However, in examples having preferences, minimizing VFE or EFE may require preferences to be satisfied, which requires not only self-efficacy but specifically self-efficacy with respect to soliciting preferred observations. For example, a node with preference (e.g., with non-zero values for the priors of the C tensor 508) may learn how to reliably solicit a current “1” observation under certain previous output and observation states; however, if the node prefers to observe “0” states of the observed system, and has not learned a way to reliably solicit “0” observations, the self-efficacy of the node will be very low.

[0103] In some examples, the C tensor 508 has values that can be used to guide the determination of VFE or EFE. For example, VFE or EFE can be determined based on a pragmatic term (e.g., "does the node get what is expected?") and an epistemic term (e.g., "does the node learn?"). The C tensor 508 can be used to adjust the balance between the pragmatic term and epistemic term. For example, the C tensor 508 can be set to 0 in some examples described herein, whereby the node is maximizing epistemic gain. In some examples, a higher epistemic term corresponds to weighting the predictive certainty of a node higher than the self-efficacy of the node within the C tensor 508.

[0104] In some cases, a pragmatic term is used to compute VFE or EFE. The pragmatic term may focus on the practical use or application of a concept, emphasizing its utility in real-world scenarios. The pragmatic term may be emphasized to enable the machine learning model to focus on action, decision-making, and achieving specific outcomes. The pragmatic term may be increased by changing the C tensor 508 such that the machine learning model focuses more on improving accuracy, efficiency, performance, and the like. In some examples, a higher pragmatic term corresponds to weighting the self-efficacy of a node higher than the predictive certainty of the node within the C tensor 508.

[0105] Thus, in some examples, in the POMDP equation above, the reward function (|) may change between assigning a higher weight to predictive certainty (e.g., the degree of certainty indicated by the B tensor 512) and assigning a higher weight to self-efficacy (e.g., the degree to which the policy Ti is likely to achieve the desired next observation) depending on the relative values of the epistemic term and the pragmatic term.

[0106] In some examples, nodes have internal states, and predictive accuracy could also or alternatively reflect not the ability of a node to correctly predict the output of another node, but instead to predict an internal state of the other node (e.g. whether the other node is operating in a stochastic or a deterministic mode, how strongly the other node holds its priors, etc.).

[0107] In some examples, predictive certainty and self-efficacy may be measured with respect to individual connections of a node. The certainty with which a node can predict inputs received via a given connection may be regarded as the node's predictive certainty with respect to that connection. Similarly, the extent to which a node's outputs via a given connection affect the state of one or more inputs subsequently observed may be regarded as the self-efficacy of that connection. In some examples, self-efficacy is computed for pairs of connections, such that the self-efficacy of a bidirectional pair of connections between a first node and a second node measures the degree to which the first node's output(s) to the second node affect the first node's subsequent observation of the second node's output(s).In some examples, the self-efficacy of a connection from the first node to the second node may instead be a measure of the degree to which the first node's output(s) to the second node affect all observations of the first node with respect to all of its connected nodes.

[0108] In some examples, the self-efficacy of each of a node's connections can be combined to generate a nodal self-efficacy measure. In some examples, the nodal self-efficacy of a node is computed as the sum of the self-efficacy of each of its outbound connections. In other examples, the nodal self-efficacy of a node is computed as an average (such as an arithmetic or geometric mean, a median, or another averaging function), or as some other combination of the self-efficacy measures of each of its outbound connections.

[0109] In some examples, the self-efficacy of a connection can be computed as the mean of the absolute difference between the sub-matrices conditioned on each action, e.g., Bo subtensor 504 and Bi subtensor 506. (For nodes having multiple connections, self-efficacy for a given connection may be computed by isolating the observations and predictions of the B tensor 512 for the connection being measured.) Thus, the self-efficacy for tit-for-tat differentiated B tensor 512b is the maximum value of 1 :Self-efficacy (S) for tit-for-tat differentiated B tensor 512b

[0110] In contrast, the self-efficacy for defector differentiated B tensor 512c is the minimum value of 0:Self-efficacy (S) for defector differentiated B tensor 512c[OHl] In some examples, additional factors may be used to compute VFE or EFE. A computational complexity metric may be included in the VFE or EFE calculation, representing the computational complexity of a node or asystem of nodes. The value of the computational complexity metric may roughly correspond to the computational costs involved in operating the node or system of nodes over one or more time steps. For example, a single node having connections to three other nodes may represent a level of computational complexity that is approximately 50 times as large as a node having a connection to only one other node, because free energy calculations on the B tensor 512 of a node scales exponentially with the number of connections of the node. Similarly, a set of two nodes, each having three connections, may have a computational complexity that is approximately ten times as large as a single node having two connections insofar as that connectivity requires more computational effort to compute free energy. Thus, the computational complexity of a set or system of nodes may reflect not only the number of nodes in the system but also the degree of connectivity of the nodes both inside and outside of the system.

[0112] In some examples, the computational complexity metric of the VFE or EFE calculation is balanced against the other metrics, such as predictive certainty and / or self-efficacy, to calculate VFE or EFE. The computational complexity metric can be weighted by a computational complexity factor, representing the sensitivity of the system to computational costs such as processor cycles, power, and storage relative to other considerations, such as predictive certainty and self-efficacy, represented by other terms such as the epistemic term and pragmatic term. Thus, in some examples, a higher computational complexity term can correspond to (negatively) weighting the computational complexity metric of a node to a greater extent than self- efficacy and predictive certainty.

[0113] In some examples, a network of nodes (e.g., group 400) may be restructured over a sequence of time steps. Beginning from a small set of nodes or a single node, the network may grow by adding nodes, shrink by removing nodes, increase connectivity by adding connections, or simplify connectivity by removing connections. In some examples, these restructuring actions can be nodal actions undertaken by a node on the basis of nodal self- determination, according to example techniques described herein.

[0114] FIG. 11 illustrates a sequence showing restructuring of a network of nodes of a machine learning model over time, according to techniques described herein. The network shown in FIG. 11 adds and removes nodes and connections over time. The connections shown in FIG. 11 may represent a pair of bidirectional connections, as in group 400.

[0115] The top row of FIG. 11, spanning from time to on the left to time ti on the right, begins on the left with a simple network at time to. A single first node 1102 has a connection to the environment, shown as first data source 1104.

[0116] Initially, the first data source 1104 provides input data to the first node 1102 of the machine learning architecture. The first data source 1104 can be any component, database, system, server, computing system, or the like, capable of providing or receiving data to the machine learning architecture. For example, the first data source 1104 may be a sensor that detects real-world information, a database that stores data for processing, a transmission module that receives data from a remote system, or any other component that can supply data to the architecture.

[0117] Moreover, in some cases, the machine learning model is part of or connected to another machine. As such, the output of the machine learning model changes the operation of the other machine. For example, the machine is connected to the first data source 1104, whereby characteristics of the machine are transmitted to the first data source 1104 and responses from the machine learning model are passed back to the first data source 1104. The responses can affect the operations of the connected machine.

[0118] The improved dynamic architecture of the machine learning model described herein dynamically restructures its internal architecture (e.g., the topology of the network) during runtime based on certain characteristics, such as certain characteristics of the network, environmental influences, system performance requirements and limitations, or changes thereof over time. In some cases, the machine learning model can test the restructured architecture using the same or different characteristics. As such, the machine learning model dynamically modifies its internal architecture to adapt to the needs and requirements of the machine learning model or the environment.

[0119] For example, the external first data source 1104 may be connected to a physical real-world machine (such as a robot) that sends sensory inputs into the machine learning model and adapts its movements and actions based on output of the machine learning model. Machine learning models can be connected to self-driving cars, enabling the machine learning model to interpret sensor data, make real-time decisions, and send responses to selfdriving cars on changing road conditions.

[0120] In some cases, machine learning models can be connected to a wide range of Internet of Things (loT) devices, including cameras, thermometers, and seismographs, enabling smart and adaptive automation in smart homes, cities, and industries.

[0121] In some cases, machine learning models can be connected to recommendation engines, providing personalized suggestions for products, content, or services. Machine learning models can be connected to computer vision applications, enabling object detection, facial recognition, and image understanding. Machine learning models can be connected to and providing instructions to gaming environments, providing adaptive gameplay, opponent Al, and personalized experiences.

[0122] The machine learning models described herein can interact with other systems and machines, and the model adapts to its new or changing environment dynamically by restructuring its internal architecture during runtime. The model can adapt its structure to better suit new data or tasks by adding / pruning nodes and layers.

[0123] In some cases, the model reduces its size and computational requirements by pruning redundant or unnecessary nodes and connections. This allows the model to run faster and on lower power budgets. The model can grow certain areas such as adding nodes to a layer to specialize on a complex task while shrinking other areas. This lets the model focus computation where more attention needed.

[0124] In some cases, the model can route around failed nodes or layers by restructuring the internal architecture of the network. This makes the system more robust and stable in the face of hardware and network faults or noise.In some cases, the model can rapidly adapt to new tasks by transferring and reusing previously learned features and nodes.

[0125] In some cases, the model dynamically grows areas and prunes others, reflecting the demands of the current task. The model can compress unimportant nodes and connections to reduce storage footprint and memory requirements. The model can be personalized for different users by learning unique sub-structures that specialize to each user.

[0126] After initialization, the first node 1102 may be configured to attempt to predict the first data source 1104, such that, over time, the parameters (e.g., priors) or first node 1102 are adjusted, as described above, to differentiate first node 1102 to learn a generative model matching the output (e.g., first output) of first node 1102 to the input (e.g., first input) received from first data source 1104. Thus, for example, if the first node 1102 is configured to play an iterated prisoner's dilemma game with first data source 1104, and the first data source 1104 is configured to always defect, then the B tensor 512 of first node 1102 will eventually differentiate to the form of defector differentiated B tensor 512c. However, if the first data source 1104 later changes its behavior and begins to exhibit a tit for tat strategy, then the first node 1102 may begin to fail to predict with the first data source 1104 due to its priors in the form of defector differentiated B tensor 512c. In response to detecting such failure to predict, the Al system may initiate the de-differentiation operation for first node 1102, e.g., by adjusting its C tensor 508 to emphasize the epistemic term relative to the pragmatic term, or by switching the first node 1102 from a deterministic mode into a stochastic mode. After the first node 1102 partially or fully de-differentiates, the first node 1102 may begin the re-differentiation operation, eventually re-learning to predict the first data source 1104 with priors approximating those of the tit-for-tat differentiated B tensor 512b.

[0127] During any of these operations of differentiation, de-differentiation, or re-differentiation, and in response to processing of inputs to generate outputs and inferences during the runtime period, the network may be restructured by adding two nodes, second node 1106 and third node 1108. The three nodes 1102, 1106, 1108 are all interconnected with each other,thereby form a first cluster 1110 of nodes in the network. Each added node may be configured to support an existing node of the first cluster 1110: for example, the first node 1102 may spawn the second node 1106, such that second node 1106 is the child of first node 1102 and is configured to support first node 1102 by mirroring the output of first node 1102. The third node 1108 may be spawned by either first node 1102 or second node 1106 and may be configured to mirror and support its parent node.

[0128] As time progresses further, just before time ti on the right side of the top row, the network may be further restructured to add another set of nodes (node 1114 and nodes 1116) and connections to form a second cluster 1112. The second node 1106 of the first cluster 1110 has a connection to a node 1114 of the second cluster 1112, potentially indicating that node 1114 is the child of second node 1106. Second node 1106 and node 1114 may be considered to be interface nodes providing inputs and outputs between the first cluster 1110 and second cluster 1112.

[0129] The bottom row of FIG. 11 spans from time ti on the left to time t2 on the right. On the left side of the bottom row, after time ti, the second cluster 1112 of the network is restructured to remove one of its nodes 1116.

[0130] On the right side of the bottom row, just before time t2, the second cluster 1112 is restructured again to add a further node 1116. It will be noted that, although the second cluster 1112 just before time ti and the second cluster 1112 just before time t2 each have five nodes, the pattern of interconnection of the nodes of the second cluster 1112 is different at the two different times. Furthermore, it may be the case that the parameters (e.g., the priors of the various tensors) of each node may be different over time.

[0131] In the various network configurations shown in FIG. 11, the various nodes may support each other's operations, and ultimately support a task of the first data source 1104 (such as an inference task or a task performed by a second machine). Nodes within a cluster may support the interface nodes of the cluster to learn to predict the interface nodes with other clusters or other data sources. The learned prediction and mirroring of the interface nodes (or environment-facing nodes) may support the operations of the interconnectedclusters or the environment 420 (e.g., first data source 1104). In some cases, the network may be rendered more accurate or more efficient by pruning nodes or connections between nodes; in other cases, accuracy may be improved by adding nodes or connecting previously-unconnected nodes. In some examples, added nodes may be connected to the environment 420 to improve support for the task performed by an external data source or other external entity.

[0132] By providing a network of a machine learning model that can be dynamically restructured during runtime, while performing an inference task for which it has been trained, an Al system may be provided that provides one or more benefits. The Al system may use a combined order of specific procedures that dynamically alters the structure of the machine learning model in real time and during inference. In response to certain input data (e.g., data from the environment 420), the Al system may both restructure the architecture of the machine learning model and perform an inference task on the restructured architecture. Not only do some examples and features of the Al system described herein eliminate the need for a separate training process from the inference process, but the dynamic nature of the Al system is different than the nature of traditional machine learning systems. The Al system does not simply automate traditional systems and concepts: instead, by leveraging dynamic architectural restructuring (e.g., by adding or removing connections, cycles, or nodes), the Al system naturally grows and prunes based on certain characteristics, such as certain characteristics of the network, environmental influences, system performance requirements and limitations, or changes thereof over time.

[0133] FIG. 12 illustrates a set of nodal actions that can be selected using nodal self-determination. In some examples, the set of nodal actions shown in FIG. 12 (or another set of nodal actions) constitute a predetermined set of nodal actions to be performed by a node of a machine learning model at an upcoming time step of execution of the machine learning model. At a current time step, the node can decide which of the nodal actions to perform at the upcoming time step by calculating a respective free energy measure for each potential nodal action, then selecting among the nodal actions based on comparisons among the free energy measures.

[0134] The connections shown in FIG. 12 may represent a pair of bidirectional connections, as in group 400. Self-connections are not shown for greater visual clarity. However, it will be appreciated that the nodes and groups of nodes shown in FIG. 12 are simplified versions of the types of nodes and groups of nodes that can be used for the nodal self-determination techniques described herein.

[0135] A current configuration 1200 is shown in the center of FIG. 12, representing the configuration of a node 1202 at a current time step. The node 1202 is the node that is making a decision about which nodal action to perform. In the current configuration 1200, the node 1202 is shown having a single existing connection 1210 to a single other node of the machine learning model, shown as local node 1204.

[0136] Surrounding the current configuration 1200 are five potential nodal actions: a first nodal action 1214, a second nodal action 1216, a third nodal action 1218, a fourth nodal action 1220, and a fifth nodal action 1222. Each of these nodal actions represents an action that the node 1202 can perform at an upcoming timestep. There is also a sixth nodal action 1224, representing a decision by the node 1202 to continue operating normally for the upcoming time step.

[0137] In order to select a nodal action from the set of nodal actions available, each potential nodal action is associated with a corresponding free energy measure. The calculation of free energy measure for a given nodal action may be performed based on the nature of each nodal action.

[0138] As a first example, the first nodal action 1214 corresponds to the node 1202 replicating itself within the executing machine learning model, such that a child node 1206 is added to the group of nodes. In the illustrated example, the child node 1206 has a connection 1212 to the node 1202 but no connection to the local node 1204; however, it will be appreciated that, in some examples, the child node 1206 may be added to the group such that it has initial connections to some or all of the nodes to which its parent node (in this case, node 1202) is connected.

[0139] To calculate the free energy measure for the first nodal action 1214, a combined free energy measure for a system of nodes is calculated. Thesystem of nodes is defined as the nodes within a container, shown as first container 1226. In this example, the system defined by the first container 1226 includes both the node 1202 and the child node 1206. The combined free energy measure can be calculated in some examples as a sum or other combined measure of the free energy of all the nodes within the first container 1226 and / or other metrics. For example, the predictive certainty and self-efficacy of the system defined by the first container 1226 can be calculated, as can the computational complexity metric of the system defined by the first container 1226. The combined free energy measure is used as a first free energy measure for the first nodal action 1214.

[0140] In some examples, adding a child node 1206 to the network will increase computational complexity (e.g., the computational complexity metric of the first free energy measure will tend to be greater than that of the node 1202 alone). However, adding the child node 1206 may in some cases also increase the predictive certainty and / or self-efficacy of the system defined by the first container 1226, thereby outweighing the increase in computational complexity to result in a lower value for the first free energy measure relative to the current configuration 1200.

[0141] As a second example, the second nodal action 1216 corresponds to the node 1202 deleting itself from the executing machine learning model, such that no nodes are present inside the second container 1228. In some examples, the second free energy measure corresponding to the second nodal action 1216 will reflect a computational complexity of zero, a predictive certainty, and / or a self-efficacy of zero. However, in cases where the self- efficacy and / or predictive certainty of node 1202 are low and / or where the computational complexity of node 1202 is high (e.g., due to a large number of connections), the second free energy measure may be improved (e.g., lower) relative to the current configuration 1200.

[0142] As a third example, the third nodal action 1218 corresponds to the node 1202 suspending itself for the upcoming timestep of the executing machine learning model. The suspended node 1202 is shown shaded to represent its inactivity. The calculation of a third free energy measure corresponding to the third nodal action 1218 is based on the single inactivenode 1202 in the third container 1230, and therefore may be similar to the calculation of the second free energy measure, insofar as both represent inactivity of the node 1202, either permanently (as a result of the second nodal action 1216) or temporarily (as a result of the third nodal action 1218). However, the second free energy measure may be weighted differently from the third free energy measure to distinguish between the permanent and temporary natures of the two nodal actions: for example, the computational complexity term used in calculating the third free energy measure could be smaller than the computational complexity term used in calculating the second free energy measure, thereby discounting the computational complexity savings due to its temporary nature. Alternatively, nodes may compute free energy over multiple future timesteps, in which case the second nodal action may correspond to a finding that free energy would be expected to be lower on all future timesteps, whereas the third nodal action may correspond to a finding that free energy would be expected to be lower on some but not all future timesteps.

[0143] It will be appreciated that calculation of the various free energy measures to enable nodal self-determination requires a certain amount of computational cost, which implies that even a node that has suspended operations will incur some computational cost and therefore computational complexity. In some examples, the node 1202 may be configured to “wake up" and perform a calculation of the free energy measures, either at every time step or periodically after some fixed or variable number of time steps. In some examples, the calculation of the free energy measures is performed by a suspended node by performing a “pre-computation” or approximation based on only a subset of the tensors of the node. For example, a suspended node may wake up every N time steps (N > 1), sample 1% of its B tensor 512, and perform an approximate pre-computation of the free energy measures for available nodal actions. The decision by the node of whether or not to resume regular operation can be made on the basis of this precomputation.

[0144] As a fourth example, the fourth nodal action 1220 corresponds to the node 1202 initiating reception of data from a further node within the executing machine learning model. In the illustrated example, initiatingreception of data from a further node corresponds to the addition of a connection 1212 between the node 1202 and a remote node 1208 of the machine learning model. By adding a connection 1212 to another node, the computational complexity of the node 1202 within the fourth container 1232 is likely to increase (e.g., the key dimension of its B tensor 512 is likely to double), but this may be offset by an increase in self-efficacy and / or predictive certainty enabled by the added connection 1212.

[0145] As described herein, self-determination involves the node 1202 deciding among nodal actions based on local information in some examples. However, this means that the node 1202 may be limited in its ability to make a decision about whether to connect to a remote node 1208 that it is not currently connected to. This limitation can be addressed in various ways in different examples.

[0146] In some cases, the node 1202 may retain local information pertaining to nodes to which it was previously connected. Based on this local information, the node 1202 can determine whether to re-connect to a previously-connected node, thereby re-initiating reception of data from a node from which the node 1202 previously terminated reception of data. For example, if the node 1202 observes a decline in its own self-efficacy or predictive certainty after disconnecting from another node, it may decide to re-connect to the other node based on the historical data about past performance and the present decline in performance.

[0147] In some cases, the node 1202 can initiate a connection 1212 to a remote node 1208 selected at random. For example, node 1202 may decide to create a new connection 1212 to a remote node 1208 chosen at random based on the observation that more connections tend to improve its free energy.

[0148] In some cases, a node manager 214 is used, as described above with reference to FIG. 2. The node manager 214 may maintain a list of remote nodes 1208 and data pertaining to characteristics of the remote nodes 1208 on the list. For example, remote nodes 1208 on the list maintained by the node manager 214 may be nodes that have determined that they should connect to a new remote node; thus, the node manager 214 can act as amatch-maker to pair nodes together when both nodes have decided to form a new connection.

[0149] In some examples, the data maintained by the node manager 214 about the nodes on its list may include data on characteristics that can be used to identify suitable remote nodes for establishing a new connection. The node manager 214 may act as a neutral provider of information; the node (e.g., node 1202) making the decision to connect can use this information to determine which remote node 1208 to connect to. Thus, the data provided to the node 1202 by the node manager 214 can be processed by the node 1202 in various different ways depending on the current state of the node 1202 and its learned tensor values, such that self-determination is preserved and the decision about which node to connect to is performed by the node 1202 itself based on its own learned behaviors. For example, the node 1202 could seek to connect to a remote node 1208 that is close by within the topology of the network; that is of a similar age; that is of a similar computational complexity, self-efficacy, and / or predictive certainty; that has a similar number of connections; that has similar timing and values of its outputs (e.g., using Hebbian learning); and / or any other suitable criteria for selecting a remote node 1208 for establishing a new connection.

[0150] In some examples, there may be multiple instances of the fourth nodal action 1220 included in the set of potential nodal actions, corresponding to multiple possible remote nodes 1208 to which the node 1202 may consider connecting.

[0151] As a fifth example, the fifth nodal action 1222 corresponds to the node 1202 terminating exchange of data with a further node (e.g., local node 1204) within the executing machine learning model. In the fifth nodal action 1222, the fifth free energy measure is calculated based on the node 1202 within the fifth container 1234, which now has one fewer connection and therefore, in some cases, approximately half the computational complexity. Thus, the fifth free energy measure may reflect a lower computational complexity metric, but may also reflect lowered self-efficacy and / or predictive certainty due to the lost connection to the local node 1204.

[0152] The determination of when a connection can be broken between two nodes can be made according to different rules in various examples. In some cases, both nodes (e.g., node 1202 and local node 1204) need to both determine that the connection should be broken. Thus, if the node 1202 and local node 1204 both determine that the exchange of data between node 1202 and local node 1204 should be terminated, then the node 1202 can terminate the exchange of data with the local node 1204. In some cases, the node 1202 may unilaterally terminate transmission of data to the local node 1204 (e.g., node 1202 may stop sending outputs to local node 1204), but may continue receiving inputs from local node 1204, resulting in a one-way connection from local node 1204 to node 1202. In some cases, the node 1202 may unilaterally terminate both transmission of data to, and reception of data from, the local node 1204.

[0153] The set of nodal actions shown in FIG. 12 also includes a sixth nodal action 1224, corresponding to the node 1202 operating normally for the upcoming timestep of the executing machine learning model. The sixth free energy measure corresponding to his sixth nodal action 1224 is calculated based on the node 1202 in the sixth container 1236 of the current configuration 1200.

[0154] FIG. 13 illustrates an example method 1300 method for nodal self- determination based on free energy. Although the example method depicts a particular sequence of operations, the sequence may be altered without departing from the scope of the present disclosure. For example, some of the operations depicted may be performed in parallel or in a different sequence that does not materially affect the function of the method. In other examples, different components of an example device or system that implements the method may perform functions at substantially the same time or in a specific sequence.

[0155] The method 1300 can be implemented by one or more processors executing instructions stored in a non-transitory, tangible computer readable storage medium. The tangible computer readable storage medium may also be used to store and retrieve the machine learning model.

[0156] The method 1300 will be described with reference to the example group 400 of FIG. 4. However, it will be appreciated that the method 1300 can be applied to various types of networks used in machine learning models, such as the examples described above with reference to FIG. 1 through FIG. 3.

[0157] Thus, in some examples, the machine learning model includes a network having a plurality of nodes and a plurality of connections between the nodes. Each node is configured to process one or more inputs to generate one or more outputs, and each connection is associated with a transmitting node and a receiving node and is configured to convey an output of the transmitting node as an input to the receiving node, at each time step of a sequence of discrete time steps. The network as a whole operates to generate inference data; for example, the environment 420 may receive outputs of the network as inference data, or the outputs of the network to the environment 420 may be further post-processed by other components of the machine learning model or an Al system using the machine learning model to generate the inference data. In some examples, the inference data may be generated on a slower time scale than the nodes of the network: for example, inference data may only be generated at every k time steps, or only when the network has generated certain patterns or quantities of output data. The inference data includes, or may be further processed to generate, an inference.

[0158] The instructions, when executed by the one or more processors, may cause the one or more processors to perform the operations shown in FIG. 13. The operations will be described as being performed by an Al system implemented by the instructions executed by the one or more processors.

[0159] According to some examples, the method 1300 includes calculating free energy measures for all nodal actions of a node (e.g., node 1202) at operation 1301. The free energy measures correspond to all nodal actions in the predetermined set of nodal actions that are performable by the node 1202 within the machine learning model during an upcoming timestep of the executing machine learning model, such as the set of nodal actions described above with reference to FIG. 12.

[0160] In some examples, as described above with reference to calculation of VFE and EFE generally, the calculation of the predicted free energy measures (e.g., EFE) of the node 1202 for the upcoming timestep involves calculating a difference between one or more preferred inputs of the node 1202 and one or more expected inputs received by the node 1202 at the upcoming timestep. Similarly, in some examples, a current free energy measure (e.g., VFE) of the node may be calculated for the current timestep by calculating a difference between one or more preferred inputs of the node and one or more actual inputs received by the node at the current timestep.

[0161] As described above, the set of nodal actions may include a first nodal action 1214 in which the node 1202 replicates itself within the machine learning model to generate a child node 1206. The first free energy measure corresponding to the first nodal action 1214 is calculated as a combined free energy measure of the node 1202 and the child node 1206 contained by first container 1226, as described above.

[0162] As described above, the set of nodal actions may include a second nodal action 1216 in which the node 1202 deletes itself from the machine learning model. The second free energy measure corresponding to the second nodal action 1216 is calculated as a free energy measure of the set of zero nodes contained by the second container 1228.

[0163] As described above, the set of nodal actions may include a third nodal action 1218 in which the node 1202 suspends itself for at least one timestep of the executing machine learning model. The third free energy measure corresponding to the third nodal action 1218 can be calculated based on the various metrics and terms described above with reference to VFE and EFE generally: for example, the calculation may involve a weighted balancing of predicted computational costs of operating the node for the at least one timestep (e.g., the computational complexity metric) and a predicted inferential accuracy of the node operating for the at least one timestep (e.g., the predictive certainty of the node 1202).

[0164] One or more other nodal actions (e.g., fourth nodal action 1220, fifth nodal action 1222, and sixth nodal action 1224) can also have theircorresponding free energy measures calculated at operation 1302, as described with reference to FIG. 12 above.

[0165] In some examples, the free energy measure calculated for each nodal action is a predicted free energy measure (e.g., EFE). In some examples, the free energy measure calculated for each nodal action is a predicted free energy measure (e.g., VFE). A current free energy measure may be used to select among nodal actions to be performed at the current time step rather than nodal actions to be performed at the upcoming time step.

[0166] According to some examples, the method 1300 includes selecting a nodal action based on comparisons of the free energy measures at operation 1302. Thus, the node 1202 selects a nodal action to be performed by the node 1202 during the upcoming timestep of the executing machine learning model. The selection is made from the predetermined set of nodal actions based on one or more comparisons among the free energy measures corresponding to the predetermined set of nodal actions. In some examples, the selected nodal action is the nodal action with the lowest free energy measure.

[0167] In some examples, as described above, a current free energy measure (e.g., VFE) is calculated for each nodal action instead of a predicted free energy measure (e.g., EFE). In such examples, the nodal action having the lowest corresponding current free energy measure may be selected.

[0168] According to some examples, the method 1300 includes the node 1202 performing the selected nodal action at the upcoming time step at operation 1304. In some examples, as described above, the nodal action (e.g., selected based on VFE instead of EFE) is performed at the current time step rather than at the upcoming time step.

[0169] In some examples, some or all of the operations of method 1300 may be performed with respect to each of a subset, or each of the full set, of nodes of the network, sequentially and / or in parallel, with respect to one or multiple time steps.

[0170] FIG. 14 is a diagrammatic representation of a machine 1400 within which instructions 1402 (e.g., software, a program, an application, an applet,an app, or other executable code) for causing the machine 1400 to perform any one or more of the methodologies discussed herein may be executed. For example, the instructions 1402 may cause the machine 1400 to execute any one or more of the methods described herein. The instructions 1402 transform the general, non-programmed machine 1400 into a special-purpose (e.g., specialized) machine 1400 programmed to carry out the described and illustrated functions in the manner described. The machine 1400 may operate as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1400 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1400 may comprise, but not be limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular telephone, a smartphone, a mobile device, a wearable device (e.g., a smartwatch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions 1402, sequentially or otherwise, that specify actions to be taken by the machine 1400. Further, while a single machine 1400 is illustrated, the term “machine" shall also be taken to include a collection of machines that individually or jointly execute the instructions 1402 to perform any one or more of the methodologies discussed herein. In some examples, the machine 1400 may comprise both client and server systems, with certain operations of a particular method or algorithm being performed on the server-side and with certain operations of the particular method or algorithm being performed on the client-side.

[0171] The machine 1400 may include processors 1404, memory 1406, and input / output I / O components 1408, which may be configured to communicate with each other via a bus 1410. In an example, the processors 1404 (e.g., a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) Processor, a Complex Instruction Set Computing (CISC) Processor, a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Radio-Frequency Integrated Circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 1412 and a processor 1414 that execute the instructions 1402. The term "processor" is intended to include multi-core processors that may comprise two or more independent processors (sometimes referred to as “cores”) that may execute instructions contemporaneously. Although FIG. 14 shows multiple processors 1404, the machine 1400 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiples cores, or any combination thereof.

[0172] The memory 1406 includes a main memory 1416, a static memory 1418, and a storage unit 1420, both accessible to the processors 1404 via the bus 1410. The main memory 1406, the static memory 1418, and storage unit 1420 store the instructions 1402 embodying any one or more of the methodologies or functions described herein. The instructions 1402 may also reside, completely or partially, within the main memory 1416, within the static memory 1418, within machine-readable medium 1422 within the storage unit 1420, within at least one of the processors 1404 (e.g., within the processor’s cache memory), or any suitable combination thereof, during execution thereof by the machine 1400.

[0173] The I / O components 1408 may include a wide variety of components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I / O components 1408 that are included in a particular machine will depend on the type of machine. For example, portable machines such as mobile phones may include a touch input device or other such input mechanisms, while a headless server machine will likely not include such a touch input device. It will be appreciated that the I / O components 1408 may include many other components that are not shown in FIG. 14. In various examples, the I / O components 1408 may include user output components 1424 and user input components 1426. The user output components 1424 may include visual components (e.g., a display such as a plasma display panel (PDP), a light-emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g.,speakers), haptic components (e.g., a vibratory motor, resistance mechanisms), other signal generators, and so forth. The user input components 1426 may include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo- optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or another pointing instrument), tactile input components (e.g., a physical button, a touch screen that provides location and force of touches or touch gestures, or other tactile input components), audio input components (e.g., a microphone), and the like.

[0174] Communication may be implemented using a wide variety of technologies. The I / O components 1408 further include communication components 1428 operable to couple the machine 1400 to a network 1430 or devices 1432 via respective coupling or connections. For example, the communication components 1428 may include a network interface component or another suitable device to interface with the network 1430. In further examples, the communication components 1428 may include wired communication components, wireless communication components, cellular communication components, Near Field Communication (NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and other communication components to provide communication via other modalities. The devices 1432 may be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a USB).

[0175] Moreover, the communication components 1428 may detect identifiers or include components operable to detect identifiers. For example, the communication components 1428 may include Radio Frequency Identification (RFID) tag reader components, NFC smart tag detection components, optical reader components (e.g., an optical sensor to detect onedimensional bar codes such as Universal Product Code (UPC) bar code, multi-dimensional bar codes such as Quick Response (QR) code, Aztec code, Data Matrix, Dataglyph™, MaxiCode, PDF417, Ultra Code, UCC RSS- 2D bar code, and other optical codes), or acoustic detection components(e.g., microphones to identify tagged audio signals). In addition, a variety ofinformation may be derived via the communication components 1428, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi® signal triangulation, location via detecting an NFC beacon signal that may indicate a particular location, and so forth.

[0176] The various memories (e.g., main memory 1416, static memory 1418, and memory of the processors 1404) and storage unit 1420 may store one or more sets of instructions and data structures (e.g., software) embodying or used by any one or more of the methodologies or functions described herein. These instructions (e.g., the instructions 1402), when executed by processors 1404, cause various operations to implement the disclosed examples.

[0177] The instructions 1402 may be transmitted or received over the network 1430, using a transmission medium, via a network interface device (e.g., a network interface component included in the communication components 1428) and using any one of several well-known transfer protocols (e.g., hypertext transfer protocol (HTTP)). Similarly, the instructions 1402 may be transmitted or received using a transmission medium via a coupling (e.g., a peer-to-peer coupling) to the devices 1432.

[0178] Various examples described herein may provide systems, methods, and computer-readable storage media for nodal self-determination of nodes of a machine learning model based on free energy. Various potential benefits may be realized by various examples described herein, as described throughout this specification.

[0179] Other technical features may be readily apparent to one skilled in the art from the figures, descriptions, and claims.

[0180] Example l is a method comprising: calculating, by a node within a machine learning model that is executing on one or more processors of a machine, free energy measures corresponding to all nodal actions in a predetermined plurality of nodal actions that are performable by the node within the machine learning model during an upcoming timestep of the executing machine learning model; selecting, by the node within the machine learning model that is executing on the one or more processors of the machine, a nodal action to be performed by the node during the upcomingtimestep of the executing machine learning model, the nodal action being selected from the predetermined plurality of nodal actions based on one or more comparisons among the free energy measures corresponding to the predetermined plurality of nodal actions during the upcoming timestep of the executing machine learning model; and performing, by the node within the machine learning model that is executing on the one or more processors of the machine, the nodal action selected based on the one or more comparisons of the free energy measures corresponding to the predetermined plurality of nodal actions during the upcoming timestep of the executing machine learning model.

[0181] In Example 2, the subject matter of Example 1 includes, wherein: the predetermined plurality of nodal actions includes at least one of: the node replicating itself within the executing machine learning model, the node deleting itself from the executing machine learning model, the node suspending itself for the upcoming timestep of the executing machine learning model, the node initiating reception of data from a further node within the executing machine learning model, or the node terminating exchange of data with a further node within the executing machine learning model.

[0182] In Example 3, the subject matter of Examples 1-2 includes, wherein: the calculating of the free energy measures includes at least one of: calculating a first free energy measure of the node replicating itself within the machine learning model, calculating a second free energy measure of the node deleting itself from the machine learning model, calculating a third free energy measure of the node suspending itself for at least one timestep of the executing machine learning model, calculating a fourth free energy measure of the node initiating reception of data from a further node within the executing machine learning model, calculating a fifth free energy measure of the node terminating exchange of data with a further node within the executing machine learning model, or calculating a sixth free energy measure of the node operating normally for the upcoming timestep of the executing machine learning model.

[0183] In Example 4, the subject matter of Examples 1-3 includes, wherein: the selecting of the nodal action to be performed by the node selects the nodal action with a lowest free energy measure among the free energy measures of the predetermined plurality of nodal actions for the upcoming timestep of the executing machine learning model.

[0184] In Example 5, the subject matter of Example 4 includes, wherein: the calculating of the free energy measures of the node for the upcoming timestep comprises calculating a difference between one or more preferred inputs of the node and one or more expected inputs received by the node at the upcoming timestep.

[0185] In Example 6, the subject matter of Examples 1-5 includes, wherein: the selecting of the nodal action to be performed by the node selects the nodal action with a lowest current free energy measure among current free energy measures of the predetermined plurality of nodal actions for a current timestep of the executing machine learning model.

[0186] In Example 7, the subject matter of Example 6 includes, wherein: the calculating of the current free energy measures of the node for the current timestep comprises calculating a difference between one or more preferred inputs of the node and one or more actual inputs received by the node at the current timestep.

[0187] In Example 8, the subject matter of Examples 3-7 includes, wherein: the node replicating itself within the machine learning model comprises: the node generating a child node within the machine learning model; and the calculating of the first free energy measure of the node replicating itself within the machine learning model comprises: calculating a combined free energy measure of the node and the child node.

[0188] In Example 9, the subject matter of Examples 3-8 includes, wherein: the calculating of the second free energy measure of the node deleting itself from the machine learning model comprises: calculating a free energy measure of a set of zero nodes.

[0189] In Example 10, the subject matter of Examples 3-9 includes, wherein: the calculating of the third free energy measure of the node suspending itself for at least one timestep of the executing machine learningmodel comprises: calculating predicted computational costs of operating the node for the at least one timestep; and calculating a predicted inferential accuracy of the node operating for the at least one timestep.

[0190] In Example 11, the subject matter of Examples 2-10 includes, wherein: the node initiating reception of data from a further node within the executing machine learning model comprises: the node re-initiating reception of data from a node from which the node previously terminated reception of data.

[0191] In Example 12, the subject matter of Examples 2-11 includes, wherein: the node initiating reception of data from a further node within the executing machine learning model comprises: the node initiating reception of data from a randomly selected node.

[0192] In Example 13, the subject matter of Examples 2-12 includes, wherein: the node initiating reception of data from a further node within the executing machine learning model comprises: the node initiating reception of data from a node selected by a node manager within the executing machine learning model.

[0193] In Example 14, the subject matter of Examples 2-13 includes, wherein: the node terminating exchange of data with a further node within the executing machine learning model comprises: determining that the further node has determined that exchange of data between the node and the further node should be terminated; and in response to the determining, terminating the exchange of data with the further node.

[0194] In Example 15, the subject matter of Examples 2-14 includes, wherein: the node terminating exchange of data with a further node within the executing machine learning model comprises: the node unilaterally terminating transmission of data to the further node.

[0195] In Example 16, the subject matter of Example 15 includes, wherein: the node terminating exchange of data with a further node within the executing machine learning model further comprises: the node unilaterally terminating reception of data from the further node.

[0196] Example 17 is a non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of amachine, cause the machine to perform first operations comprising: executing a machine learning model that includes, multiple nodes, a node among the multiple nodes being configured to perform second operations comprising: calculating, by the node within the machine learning model, free energy measures corresponding to all nodal actions in a predetermined plurality of nodal actions that are performable by the node within the machine learning model during an upcoming timestep of the executing machine learning model; selecting, by the node within the machine learning model, a nodal action to be performed by the node during the upcoming timestep of the executing machine learning model, the nodal action being selected from the predetermined plurality of nodal actions based on one or more comparisons among the free energy measures corresponding to the predetermined plurality of nodal actions during the upcoming timestep of the executing machine learning model; and performing, by the node within the machine learning model, the nodal action selected based on the one or more comparisons of the free energy measures corresponding to the predetermined plurality of nodal actions during the upcoming timestep of the executing machine learning model.

[0197] In Example 18, the subject matter of Example 17 includes, wherein: the predetermined plurality of nodal actions includes at least one of: the node replicating itself within the executing machine learning model, the node deleting itself from the executing machine learning model, the node suspending itself for the upcoming timestep of the executing machine learning model, the node initiating reception of data from a further node within the executing machine learning model, or the node terminating exchange of data with a further node within the executing machine learning model.

[0198] In Example 19, the subject matter of Examples 17-18 includes, wherein: the calculating of the free energy measures includes at least one of: calculating a first free energy measure of the node replicating itself within the machine learning model, calculating a second free energy measure of the node deleting itself from the machine learning model, calculating a third free energy measure of the node suspending itself for at least one timestep of the executing machine learning model, calculating a fourth free energy measureof the node initiating reception of data from a further node within the executing machine learning model, calculating a fifth free energy measure of the node terminating exchange of data with a further node within the executing machine learning model, or calculating a sixth free energy measure of the node operating normally for the upcoming timestep of the executing machine learning model.

[0199] In Example 20, the subject matter of Examples 17-19 includes, wherein: the selecting of the nodal action to be performed by the node selects the nodal action with a lowest free energy measure among the free energy measures of the predetermined plurality of nodal actions for the upcoming timestep of the executing machine learning model.

[0200] In Example 21, the subject matter of Example 20 includes, wherein: the calculating of the free energy measures of the node for the upcoming timestep comprises calculating a difference between one or more preferred inputs of the node and one or more expected inputs received by the node at the upcoming timestep.

[0201] In Example 22, the subject matter of Examples 17-21 includes, wherein: the selecting of the nodal action to be performed by the node selects the nodal action with a lowest current free energy measure among current free energy measures of the predetermined plurality of nodal actions for a current timestep of the executing machine learning model.

[0202] In Example 23, the subject matter of Example 22 includes, wherein: the calculating of the current free energy measures of the node for the current timestep comprises calculating a difference between one or more preferred inputs of the node and one or more actual inputs received by the node at the current timestep.

[0203] In Example 24, the subject matter of Examples 19-23 includes, wherein: the node replicating itself within the machine learning model comprises: the node generating a child node within the machine learning model; and the calculating of the first free energy measure of the node replicating itself within the machine learning model comprises: calculating a combined free energy measure of the node and the child node.

[0204] In Example 25, the subject matter of Examples 19-24 includes, wherein: the calculating of the second free energy measure of the node deleting itself from the machine learning model comprises: calculating a free energy measure of a set of zero nodes.

[0205] In Example 26, the subject matter of Examples 19-25 includes, wherein: the calculating of the third free energy measure of the node suspending itself for at least one timestep of the executing machine learning model comprises: calculating predicted computational costs of operating the node for the at least one timestep; and calculating a predicted inferential accuracy of the node operating for the at least one timestep.

[0206] In Example 27, the subject matter of Examples 18-26 includes, wherein: the node initiating reception of data from a further node within the executing machine learning model comprises: the node re-initiating reception of data from a node from which the node previously terminated reception of data.

[0207] In Example 28, the subject matter of Examples 18-27 includes, wherein: the node initiating reception of data from a further node within the executing machine learning model comprises: the node initiating reception of data from a randomly selected node.

[0208] In Example 29, the subject matter of Examples 18-28 includes, wherein: the node initiating reception of data from a further node within the executing machine learning model comprises: the node initiating reception of data from a node selected by a node manager within the executing machine learning model.

[0209] In Example 30, the subject matter of Examples 18-29 includes, wherein: the node terminating exchange of data with a further node within the executing machine learning model comprises: determining that the further node has determined that exchange of data between the node and the further node should be terminated; and in response to the determining, terminating the exchange of data with the further node.

[0210] In Example 31, the subject matter of Examples 18-30 includes, wherein: the node terminating exchange of data with a further node withinthe executing machine learning model comprises: the node unilaterally terminating transmission of data to the further node.

[0211] In Example 32, the subject matter of Example 31 includes, wherein: the node terminating exchange of data with a further node within the executing machine learning model further comprises: the node unilaterally terminating reception of data from the further node.

[0212] Example 33 is a system comprising: one or more processors; and a memory storing instructions that, when executed by at least one processor among the one or more processors, cause the system to perform first operations comprising: executing a machine learning model that includes, multiple nodes, a node among the multiple nodes being configured to perform second operations comprising: calculating, by the node within the machine learning model, free energy measures corresponding to all nodal actions in a predetermined plurality of nodal actions that are performable by the node within the machine learning model during an upcoming timestep of the executing machine learning model; selecting, by the node within the machine learning model, a nodal action to be performed by the node during the upcoming timestep of the executing machine learning model, the nodal action being selected from the predetermined plurality of nodal actions based on one or more comparisons among the free energy measures corresponding to the predetermined plurality of nodal actions during the upcoming timestep of the executing machine learning model; and performing, by the node within the machine learning model, the nodal action selected based on the one or more comparisons of the free energy measures corresponding to the predetermined plurality of nodal actions during the upcoming timestep of the executing machine learning model.

[0213] In Example 34, the subject matter of Example 33 includes, wherein: the predetermined plurality of nodal actions includes at least one of: the node replicating itself within the executing machine learning model, the node deleting itself from the executing machine learning model, the node suspending itself for the upcoming timestep of the executing machine learning model, the node initiating reception of data from a further node within the executing machine learning model, or the node terminatingexchange of data with a further node within the executing machine learning model.

[0214] In Example 35, the subject matter of Examples 33-34 includes, wherein: the calculating of the free energy measures includes at least one of: calculating a first free energy measure of the node replicating itself within the machine learning model, calculating a second free energy measure of the node deleting itself from the machine learning model, calculating a third free energy measure of the node suspending itself for at least one timestep of the executing machine learning model, calculating a fourth free energy measure of the node initiating reception of data from a further node within the executing machine learning model, calculating a fifth free energy measure of the node terminating exchange of data with a further node within the executing machine learning model, or calculating a sixth free energy measure of the node operating normally for the upcoming timestep of the executing machine learning model.

[0215] In Example 36, the subject matter of Examples 33-35 includes, wherein: the selecting of the nodal action to be performed by the node selects the nodal action with a lowest free energy measure among the free energy measures of the predetermined plurality of nodal actions for the upcoming timestep of the executing machine learning model.

[0216] In Example 37, the subject matter of Example 36 includes, wherein: the calculating of the free energy measures of the node for the upcoming timestep comprises calculating a difference between one or more preferred inputs of the node and one or more expected inputs received by the node at the upcoming timestep.

[0217] In Example 38, the subject matter of Examples 33-37 includes, wherein: the selecting of the nodal action to be performed by the node selects the nodal action with a lowest current free energy measure among current free energy measures of the predetermined plurality of nodal actions for a current timestep of the executing machine learning model.

[0218] In Example 39, the subject matter of Example 38 includes, wherein: the calculating of the current free energy measures of the node for the current timestep comprises calculating a difference between one or morepreferred inputs of the node and one or more actual inputs received by the node at the current timestep.

[0219] In Example 40, the subject matter of Examples 35-39 includes, wherein: the node replicating itself within the machine learning model comprises: the node generating a child node within the machine learning model; and the calculating of the first free energy measure of the node replicating itself within the machine learning model comprises: calculating a combined free energy measure of the node and the child node.

[0220] In Example 41, the subject matter of Examples 35-40 includes, wherein: the calculating of the second free energy measure of the node deleting itself from the machine learning model comprises: calculating a free energy measure of a set of zero nodes.

[0221] In Example 42, the subject matter of Examples 35-41 includes, wherein: the calculating of the third free energy measure of the node suspending itself for at least one timestep of the executing machine learning model comprises: calculating predicted computational costs of operating the node for the at least one timestep; and calculating a predicted inferential accuracy of the node operating for the at least one timestep.

[0222] In Example 43, the subject matter of Examples 34-42 includes, wherein: the node initiating reception of data from a further node within the executing machine learning model comprises: the node re-initiating reception of data from a node from which the node previously terminated reception of data.

[0223] In Example 44, the subject matter of Examples 34-43 includes, wherein: the node initiating reception of data from a further node within the executing machine learning model comprises: the node initiating reception of data from a randomly selected node.

[0224] In Example 45, the subject matter of Examples 34-44 includes, wherein: the node initiating reception of data from a further node within the executing machine learning model comprises: the node initiating reception of data from a node selected by a node manager within the executing machine learning model.

[0225] In Example 46, the subject matter of Examples 34-45 includes, wherein: the node terminating exchange of data with a further node within the executing machine learning model comprises: determining that the further node has determined that exchange of data between the node and the further node should be terminated; and in response to the determining, terminating the exchange of data with the further node.

[0226] In Example 47, the subject matter of Examples 34-46 includes, wherein: the node terminating exchange of data with a further node within the executing machine learning model comprises: the node unilaterally terminating transmission of data to the further node.

[0227] In Example 48, the subject matter of Example 47 includes, wherein: the node terminating exchange of data with a further node within the executing machine learning model further comprises: the node unilaterally terminating reception of data from the further node.

[0228] Example 49 is at least one machine-readable medium including instructions that, when executed by processing circuitry, cause the processing circuitry to perform operations to implement of any of Examples 1-48.

[0229] Example 50 is an apparatus comprising means to implement of any of Examples 1-48.

[0230] Example 51 is a system to implement of any of Examples 1-48.

[0231] Example 52 is a method to implement of any of Examples 1-48.

[0232] Unless the context clearly requires otherwise, throughout the description and the claims, the words “comprise,” “comprising,” and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense, i.e., in the sense of “including, but not limited to.” As used herein, the terms “connected,” “coupled,” or any variant thereof means any connection or coupling, either direct or indirect, between two or more elements; the coupling or connection between the elements can be physical, logical, or a combination thereof. Additionally, the words “herein,” “above,” “below,” and words of similar import, when used in this application, refer to this application as a whole and not to any particular portions of this application. Where the context permits, words using the singular or pluralnumber may also include the plural or singular number respectively. The word “or” in reference to a list of two or more items, covers all of the following interpretations of the word: any one of the items in the list, all of the items in the list, and any combination of the items in the list. Likewise, the term “and / or” in reference to a list of two or more items, covers all of the following interpretations of the word: any one of the items in the list, all of the items in the list, and any combination of the items in the list.

[0233] Although some examples, e.g., those depicted in the drawings, include a particular sequence of operations, the sequence may be altered without departing from the scope of the present disclosure. For example, some of the operations depicted may be performed in parallel or in a different sequence that does not materially affect the functions as described in the examples. In other examples, different components of an example device or system that implements an example method may perform functions at substantially the same time or in a specific sequence.

[0234] The various features, steps, and processes described herein may be used independently of one another, or may be combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of this disclosure. In addition, certain method or process blocks may be omitted in some implementations.

Claims

CLAIMSWhat is claimed is:

1. A method comprising: calculating, by a node within a machine learning model that is executing on one or more processors of a machine, free energy measures corresponding to all nodal actions in a predetermined plurality of nodal actions that are performable by the node within the machine learning model during an upcoming timestep of the executing machine learning model; selecting, by the node within the machine learning model that is executing on the one or more processors of the machine, a nodal action to be performed by the node during the upcoming timestep of the executing machine learning model, the nodal action being selected from the predetermined plurality of nodal actions based on one or more comparisons among the free energy measures corresponding to the predetermined plurality of nodal actions during the upcoming timestep of the executing machine learning model; and performing, by the node within the machine learning model that is executing on the one or more processors of the machine, the nodal action selected based on the one or more comparisons of the free energy measures corresponding to the predetermined plurality of nodal actions during the upcoming timestep of the executing machine learning model.

2. The method of claim 1, wherein: the predetermined plurality of nodal actions includes at least one of: the node replicating itself within the executing machine learning model, the node deleting itself from the executing machine learning model, the node suspending itself for the upcoming timestep of the executing machine learning model, the node initiating reception of data from a further node within the executing machine learning model, orthe node terminating exchange of data with a further node within the executing machine learning model.

3. The method of claim 1, wherein: the calculating of the free energy measures includes at least one of: calculating a first free energy measure of the node replicating itself within the machine learning model, calculating a second free energy measure of the node deleting itself from the machine learning model, calculating a third free energy measure of the node suspending itself for at least one timestep of the executing machine learning model, calculating a fourth free energy measure of the node initiating reception of data from a further node within the executing machine learning model, calculating a fifth free energy measure of the node terminating exchange of data with a further node within the executing machine learning model, or calculating a sixth free energy measure of the node operating normally for the upcoming timestep of the executing machine learning model.

4. The method of claim 1, wherein: the selecting of the nodal action to be performed by the node selects the nodal action with a lowest free energy measure among the free energy measures of the predetermined plurality of nodal actions for the upcoming timestep of the executing machine learning model.

5. The method of claim 4, wherein: the calculating of the free energy measures of the node for the upcoming timestep comprises calculating a difference between one or more preferred inputs of the node and one or more expected inputs received by the node at the upcoming timestep.

6. The method of claim 1, wherein:the selecting of the nodal action to be performed by the node selects the nodal action with a lowest current free energy measure among current free energy measures of the predetermined plurality of nodal actions for a current timestep of the executing machine learning model.

7. The method of claim 6, wherein: the calculating of the current free energy measures of the node for the current timestep comprises calculating a difference between one or more preferred inputs of the node and one or more actual inputs received by the node at the current timestep.

8. The method of claim 3, wherein: the node replicating itself within the machine learning model comprises: the node generating a child node within the machine learning model; and the calculating of the first free energy measure of the node replicating itself within the machine learning model comprises: calculating a combined free energy measure of the node and the child node.

9. The method of claim 3, wherein: the calculating of the second free energy measure of the node deleting itself from the machine learning model comprises: calculating a free energy measure of a set of zero nodes.

10. The method of claim 3, wherein: the calculating of the third free energy measure of the node suspending itself for at least one timestep of the executing machine learning model comprises: calculating predicted computational costs of operating the node for the at least one timestep; and calculating a predicted inferential accuracy of the node operating for the at least one timestep.

11. The method of claim 2, wherein:the node initiating reception of data from a further node within the executing machine learning model comprises: the node re-initiating reception of data from a node from which the node previously terminated reception of data.

12. The method of claim 2, wherein: the node initiating reception of data from a further node within the executing machine learning model comprises: the node initiating reception of data from a randomly selected node.

13. The method of claim 2, wherein: the node initiating reception of data from a further node within the executing machine learning model comprises: the node initiating reception of data from a node selected by a node manager within the executing machine learning model.

14. The method of claim 2, wherein: the node terminating exchange of data with a further node within the executing machine learning model comprises: determining that the further node has determined that exchange of data between the node and the further node should be terminated; and in response to the determining, terminating the exchange of data with the further node.

15. The method of claim 2, wherein: the node terminating exchange of data with a further node within the executing machine learning model comprises: the node unilaterally terminating transmission of data to the further node.

16. The method of claim 15, wherein: the node terminating exchange of data with a further node within the executing machine learning model further comprises:the node unilaterally terminating reception of data from the further node.

17. A machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform first operations comprising: executing a machine learning model that includes multiple nodes, a node among the multiple nodes being configured to perform second operations comprising: calculating, by the node within the machine learning model, free energy measures corresponding to all nodal actions in a predetermined plurality of nodal actions that are performable by the node within the machine learning model during an upcoming timestep of the executing machine learning model; selecting, by the node within the machine learning model, a nodal action to be performed by the node during the upcoming timestep of the executing machine learning model, the nodal action being selected from the predetermined plurality of nodal actions based on one or more comparisons among the free energy measures corresponding to the predetermined plurality of nodal actions during the upcoming timestep of the executing machine learning model; and performing, by the node within the machine learning model, the nodal action selected based on the one or more comparisons of the free energy measures corresponding to the predetermined plurality of nodal actions during the upcoming timestep of the executing machine learning model.

18. The machine-readable medium of claim 17, wherein: the predetermined plurality of nodal actions includes at least one of: the node replicating itself within the executing machine learning model, the node deleting itself from the executing machine learning model, the node suspending itself for the upcoming timestep of the executing machine learning model,the node initiating reception of data from a further node within the executing machine learning model, or the node terminating exchange of data with a further node within the executing machine learning model.

19. A system comprising: one or more processors; and a memory storing instructions that, when executed by at least one processor among the one or more processors, cause the system to perform first operations comprising: executing a machine learning model that includes multiple nodes, a node among the multiple nodes being configured to perform second operations comprising: calculating, by the node within the machine learning model, free energy measures corresponding to all nodal actions in a predetermined plurality of nodal actions that are performable by the node within the machine learning model during an upcoming timestep of the executing machine learning model; selecting, by the node within the machine learning model, a nodal action to be performed by the node during the upcoming timestep of the executing machine learning model, the nodal action being selected from the predetermined plurality of nodal actions based on one or more comparisons among the free energy measures corresponding to the predetermined plurality of nodal actions during the upcoming timestep of the executing machine learning model; and performing, by the node within the machine learning model, the nodal action selected based on the one or more comparisons of the free energy measures corresponding to the predetermined plurality of nodal actions during the upcoming timestep of the executing machine learning model.

20. The system of claim 19, wherein: the predetermined plurality of nodal actions includes at least one of:the node replicating itself within the executing machine learning model, the node deleting itself from the executing machine learning model, the node suspending itself for the upcoming timestep of the executing machine learning model, the node initiating reception of data from a further node within the executing machine learning model, or the node terminating exchange of data with a further node within the executing machine learning model.

Images