Autonomous machine with adaptive controller

Abstract

An autonomous machine arranged to provide an observable measure of mechanical energy consumed by the machine relative to mechanical energy stored by the machine such that a control system of the machine can make decisions based on the real energy stress of the machine. The autonomous machine has a control system with a predictive model for an internal environment and an external environment. Both predictive models are based on the same set of information representing the common energy basis of the machines. The set of information includes (i) a plurality of reciprocating signals indicative of direct interaction of the machine, and (ii) a plurality of non-reciprocating signals indicative of information available for the machine without consuming energy. The plurality of non-reciprocal signals include simulated signals, ensuring that a predictive model of the internal environment and the external environment is based on a set of equivalent parameters when needed.

Description

Technical Field

[0001] The present invention relates to a machine having an offline rechargeable power source (eg, a rechargeable battery, etc.) and a control system configured to autonomously maintain a positive energy state of the machine by monitoring the machine's interaction with its internal and external environments. Background Art

[0002] A machine is an assembly of one or more parts intended to achieve some mechanical or other advantage in performing work. Autonomous machines (also referred to herein as "automatons") typically operate in a defined environment using a control system. Autonomous machines typically include off-line energy storage capacity and a means to restore it, commensurate with the limited capacity of the off-line storage and the autonomous functionality of the machine.

[0003] It is desirable for a machine to limit its energy losses by maximizing its energy efficiency. Especially for automata, maximizing efficiency not only facilitates maximizing the time period required for the machine to restore its energy reserves, but also a functioning automaton must store enough energy to facilitate recovery.

[0004] There are many ways to implement a machine controller, with its complexity somewhat reflecting the complexity of the machine it controls. A controller can be a simple mechanical device or a complex electronic processing device, which can also use multiple electromechanical or other transducers as sensors or as mechanical actuators.

[0005] A machine controller may employ feedback in which a measure of the machine output serves as a driver for its input. Within the range of motion permitted by the mechanical capabilities of the machine, sufficient and stable gain within this feedback system gives the machine performance limited primarily by the accuracy of its sensing.

[0006] Machine controllers may also employ feedforward, where previously acquired knowledge embedded in a model of the machine or its environment enables the (nominally) optimal machine action to be explicitly or otherwise predicted via an estimate of the most likely outcome (perhaps from a "Markov chain").

[0007] Rather than employing previously acquired, hard-coded information, an “adaptive” machine controller can exploit learning capabilities, where the controller updates its predictive model based on events and outcomes measured by the machine, thereby causing the model, and therefore the machine, to “adapt” to its environment.

[0008] Adaptive model optimization via gradient ascent / descent methods is often commensurate with the "InfoMax principle", where minimizing prediction error is used to maximize accuracy, while minimizing predictor complexity is used to maximize its efficiency (in the controller and resulting actions), driven by overall negative feedback [1]. However, the convergence provided by the inherent negative feedback of machine controllers may achieve local optimal actions that are not necessarily consistent with the global optimal result. The necessary modulation of this convergence via additional positive feedback to allow the search for better non-local solutions is called "wandering".

[0009] More complex adaptive machine controllers may employ so-called "deep learning" approaches, in which one or more "hidden layers" build intermediate correlations between controller inputs and outputs. For example, a Helmholtz machine may employ a circuitous "neural network" somewhat analogous to the flow of information in a central nervous system [2], in which an adaptive "recognition" element models the inputs (sensations) of the machine controller, while an adaptive "generation" element models the outputs (actions) of the machine.

[0010] A superset of control system analysis applicable to machine controllers is the "Free Energy Principle" [3], which describes the minimization of "free energy" in a feedback system [4]. Note, however, that "free energy" is a theoretical measure, not real thermodynamic energy. Controllers based on the "Free Energy Principle" can produce Bayesian inference networks [5], in which posterior (output) predictions are developed from conditional probabilities of prior predictions, and in which free energy provides a measure of the "unexpected" difference between the controller's predictions and its sensed reality.

[0011] In an ideal inference network, the recognition or generative model is updated, for example, to minimize its information redundancy (or reduce the difference from the maximally efficient representation), so that well-defined modeled objects produce data “clusters” that provide the basis for higher-level representations [6].

[0012] Constrained by the sensing modality and its acuity, such a controller is able to identify or infer clusters of data from information events at the machine interface, providing the basis for the subsequent development of a hierarchical model in which the modeled components are distinguished by so-called “Markov Blankets” [7]. The Markov Blanket provides a “scale” for the model, where lower-level model components are incorporated into higher-level model components and assigned a certain “value”, called a “reward” or “utility” (or a complementary “loss” or “cost”), and the machine controller accordingly takes an action determined at that particular scale. Summary of the invention

[0013] Most generally, the present invention provides an autonomous machine (e.g., a computer-controlled device) that is arranged to provide observable measurements of mechanical energy consumed by the machine relative to mechanical energy stored by the machine so that decisions of the machine's control system can be based on the actual energy stress of the machine.

[0014] Specifically, the present invention can provide an autonomous machine with a control system having predictive models for an internal environment and an external environment, wherein both predictive models are based on the same set of information that represents a common energy basis for the machine. The set of information can include: (i) a plurality of reciprocating signals transmitted on a reciprocating channel, wherein the plurality of reciprocating signals indicate direct interactions of the machine ("near field"), and (ii) a plurality of non-reciprocating signals transmitted on a non-reciprocating channel, wherein the plurality of non-reciprocating signals indicate information that the machine can obtain without consuming energy ("far field"), and wherein the plurality of non-reciprocating signals include simulated signals, ensuring, when necessary, that the predictive models for the internal and external environments are based on a set of equivalent parameters. The control system can also provide a dedicated parallel path for common-mode signals within the machine's internal environment. This ensures that the signals within the parameter set associated with the common mode are orthogonal to the differential-mode signals in both predictive models, thereby enabling the predictive models to distinguish between internal and external power dissipation, and thus provide a true picture indicating energy stress on the machine.

[0015] The present invention differs from prior art techniques utilizing high-order representations because the high-order components of the prediction models are not "isolated" from the low-order components by a Markov blanket. Through the architecture described herein, only the portions (e.g., layers) of the prediction models that have errors will cause flux in the feedback loops between these models, thereby ensuring that the true energy basis of the mechanical interactions of the machine permeates to the relevant machine controller portions and presents errors at the interface of the machine.

[0016] The present invention may be implemented on any autonomous machine that is capable, through training or otherwise, of maintaining itself in its environment, wherein the machine includes offline energy storage capacity rather than a permanently engaged power source, and wherein the machine is capable of restoring its offline energy storage by mechanically interacting with its environment.

[0017] Therefore, according to the present invention, there is provided a machine capable of autonomous operation, comprising: a rechargeable off-line power source; a physical interface through which the machine interacts with an external environment, the physical interface comprising: a reciprocating interface operable in differential mode and common mode, wherein the reciprocating interface is configured to provide motion via a differential mode output and to provide motion cancellation of a common mode output; and a non-reciprocating interface configured to passively interact with the external environment via a non-reciprocating input (e.g., a sensor, etc.); a first control element configured to control internal and external operations of the machine and maintain a positive energy state of the machine; and a second control element disposed between the first control element and the physical interface to mediate information exchange therebetween, wherein the second control element is configured to communicate with the physical interface via a first set of reciprocating information channels and to communicate with the first control via a second set of reciprocating information channels. control element communications, wherein a first set of reciprocating information channels includes an input channel configured to transmit information from a non-reciprocating input into the machine, and a simulated output channel that forms a reciprocating pair with the input channel, wherein a second control element includes a first prediction model arranged to predict communications on the first set of reciprocating information channels, and a second prediction model arranged to predict communications on the second set of reciprocating information channels, wherein both the first prediction model and the second prediction model are adaptive models that are combined in a feedback arrangement to minimize an error energy flux between the first set of reciprocating information channels and the second set of reciprocating information channels, and wherein the first control element is configured to maintain a common-mode bias at the physical interface and includes a common-mode regulator arranged to establish independent parallel common-mode paths that appear in an internal environment of the machine.

[0018] The control system architecture defined herein provides particular advantages for the ability of the predictive model to distinguish between clusters of information representing different aspects of the internal and external environments that the machine can infer. Specifically, the combination of (i) ensuring that the common-mode signal is orthogonal to other signals on both the input and output sides of the second control element, and (ii) using channel simulation to establish a full reciprocating information channel on both sides of the second control element means that the predictive model has a common energy basis, where clusters of information associated with the "near field" portion (associated with the reciprocating interface) can be distinguished from clusters of information associated with the "far field" portion (associated with the non-dissipative interface), and where clusters of information associated with internal and external effects can be distinguished within the near field portion. This ability to distinguish manifests itself in the increased sensitivity of the predictive model to internal and external effects.

[0019] The rechargeable off-line power source may be a battery or other suitable portable power source. In some examples, the power source may include a substrate forming part of the machine, for example, a consumable portion of a housing or other body of the machine.

[0020] The physical interface may include any suitable structure for applying a force to the environment to achieve a physical effect. Specifically, the reciprocating interface may be configured to enable the machine to move within the external environment, for example, into a location where a rechargeable offline power source can be charged. In this way, the reciprocating interface establishes a real energy basis between the first control element and the physical interface. For example, the reciprocating interface may include a pair of reciprocating servo motors. The reciprocating interface may be configured to provide motion from a differential mode output and provide motion cancellation from its common mode output. Each dimension in which motion is required requires a separate set of reciprocating elements.

[0021] The non-reciprocating interface can be any suitable passive sensor for detecting the characteristics of the external environment. The sensor can be an image sensor or a scanner, which is configured to detect information about the environment surrounding the machine. Alternatively or in addition, it can include an environmental sensor, for example, for measuring temperature, humidity, etc. The non-reciprocating interface can also include a (nominal) output, such as a lighting source. However, it is understandable that this output is non-dissipative from the perspective of mechanical energy transferred to the environment. The non-dissipative nature of the non-reciprocating interface means that it provides a non-reciprocating information channel, such as indicating the far field of the machine, which is converted into a reciprocating pair by a simulated output channel, and the simulated output channel itself is essentially non-dissipative. If the non-reciprocating interface also includes a (nominal) output, the first group of reciprocating information channels also includes a simulated input channel, which forms a reciprocating pair with an output channel that transmits information to the non-reciprocating interface. In the architecture proposed in this article, each non-reciprocating input or output forms a reciprocating pair. Therefore, the machine can have multiple non-reciprocating channels, each of which originates from a first control element or a physical interface, and therefore has multiple corresponding simulated channels.

[0022] The non-reciprocating channel and its corresponding simulation channel are copied into the second set of reciprocating information channels so that the first prediction model and the second prediction model operate on the same set of information. However, in some cases, the simulation channel does not need to completely "run through" between the first control element and the physical interface. Instead, they can terminate at the second control element, thereby forming a short-cut channel. An example of a short-cut channel may be an output from a sequence memory in the first control element. This output may affect future actions, but does not need to be fed into the physical interface. Another example of a short-cut channel may be an input from a color detector in the physical interface. This input may form part of the far-field information available to the machine, but does not need to be fed to the first control element.

[0023] The first control element may be configured to regulate the positive energy state of the machine, for example using an appropriately configured feedback loop. The feedback loop for the positive energy state of the machine may include one or more sub-feedback loops. For example, the first control element may employ a first feedback loop to control the internal environment of the machine and employ a second feedback loop to cause the motion required to restore its offline power supply. The first control element may be implemented using a control model that associates actions or sequences thereof with results. For example, a model may encode motion instructions that can control a physical interface to enable the machine to operate in its environment. In one example, the control model may include a memory arranged to store one or more action sequences, each action sequence being associated with a result that records the effect of the sequence on the objectives of the feedback loop.

[0024] The model may have goals or drivers that are used to prioritize available actions in a given scenario. For example, a base driver may be related to regulating a machine's positive energy state. Thereafter, other drivers may be arranged in a cascade, forming a hierarchy that reflects their decreasing priority. Having cascaded drivers in the first control element may also allow one or more drivers to be suspended until the machine has gained enough knowledge to ensure its long-term energy stability.

[0025] The energy state of the machine may be related to its operating state. For example, the machine may be arranged to terminate operation when the stored energy becomes zero, causing the machine to stop running. The first control element may be configured to control the cascade drive on this basis.

[0026] The first control element may also include an adaptive learning module arranged to update its model. For example, the model may be adjusted using Pavlovian learning.

[0027] The common mode bias maintained by the first control element may be arranged such that the reciprocating interface output has a positive internal power dissipation.Thus, the common mode regulator may be configured to regulate the internal dissipation via a feedback or feed-forward arrangement relative to some reference.

[0028] In some embodiments, internal dissipation from the common mode bias can be used to provide heating to simulate or create a heat generating machine, where a thermal sensor is utilized to adjust the reference of the common mode regulator to compensate for changes in internal temperature.

[0029] The independent parallel common mode paths may be physically defined within the machine, or may be simulated paths formed within the first control element.

[0030] Both the first set of reciprocating information channels and the second set of reciprocating information channels may include a plurality of reciprocating channel pairs to maintain a real energy basis established in the physical interface at the second control element. When the input to the second control element is non-reciprocating (e.g., because it is associated with a non-reciprocating interface), the second control element is configured to simulate a corresponding output to create a reciprocating pair. Each simulated reciprocating channel is arranged to be nominally dissipative.

[0031] The first prediction model is configured to generate an output including signals on one or more of the first set of reciprocating information channels. The second control element can be arranged so that the prediction it forms from its first model (which can represent the external environment), even from a single sensed input, is projected to all sensed inputs included in the physical interface. Similarly, the second prediction model is configured to generate an output including signals on one or more of the second set of reciprocating information channels. Therefore, error flux that only appears on a subset of the relevant reciprocating information channels can still be mapped to information on all channels.

[0032] If the predictive models are sufficiently accurate that no error is perceived by either model, the second control element operates in a converged state and there is no information exchange flux through the second control element between the first control element and the physical interface. The benefit of this situation is that the model (and therefore the machine) becomes optimally adapted to its environment. Only when an error is perceived in either predictive model does information flux flow within the second control element and a local negative feedback loop becomes evident where the model impedances are driven to equality. This arrangement can be used to adjust the predictive models so that new model components can be identified and added during machine operation.

[0033] In one embodiment, the convergence path in the feedback loop formed within the second control element may be driven in part by the error component on the common mode output.

[0034] At the moment of defining the convergence path, the control system architecture described in this article gives the machine its own relative energy model, which is presented in parallel and indistinguishably from the sensed energy reality, forming its near-field interaction projected within the external environment of the far-field model. This means that the prediction model can be adjusted based on the information of the difference between the machine's near field and its far field, where the near field is the field in direct contact with the machine and dissipating real mechanical power (internal or external). In addition, the setting of the common mode regulator makes it possible to distinguish between the internal and external power dissipation of the machine, which in turn provides information indicating the energy stress on the machine.

[0035] When the second control element operates in the divergence phase (i.e., when there are errors in one or both prediction models), the model errors in the second control element can be observed at the physical interface or the reciprocating channel simulation point (if different). These errors can be effectively presented at the physical interface in the appropriate sensing mode.

[0036] The first prediction model and the second prediction model may each adopt a layered or hierarchical model. When arranged as discussed above, all layers below the error source are substantially cancelled out, so that the energy information presented on the physical interface is caused by the higher layers in the converged model.

[0037] The first prediction model and the second prediction model and the control model in the optional first control element may be modular models comprising a plurality of interchangeable units, such as layers in a hierarchical model that may be switched alternately as required.

[0038] In some embodiments, the convergence path in the second control element can be modulated so that the overall convergence contains shorter divergence periods, i.e., "wandering". For example, in an embodiment in which the first control element includes a memory of sequences and results, each result is associated with a positive or negative score for each driver unit, and the results can be used to modulate the behavior of the second control element, such as by limiting the range of the Bayesian search for local minima. In another example, the second control element can be modulated based on the available energy, which information can be obtained from the first control element. The machine can be configured to suppress wandering until the model in the first control element operates in a stable manner.

[0039] The machine may be part of a group of machines having the same control architecture. In some embodiments, the second control element may be arranged to introduce a perturbation in the first prediction model and / or the second prediction model in order to introduce a change in the model of the group of machines. Specifically, the perturbation may be introduced into a layer of the first prediction model or the second prediction model when in a converged state. In this case, the perturbation may become a permanent part of the learned model.

[0040] The machine may be arranged in a distributed manner, for example on one or more physical subcomponents interconnected in a suitable manner (eg via a wireless network), and wherein the predictive model is established on all physical subcomponents and interfaces therein.

[0041] In one embodiment, the first control element itself may be configured to influence the learning process within the second control element by providing a non-linear response on one or more of the second set of reciprocating information channels.

[0042] A machine may operate in both physical (real) and virtual (eg, simulated) environments, and references herein to the "external environment" of a machine should be interpreted accordingly.

[0043] The first control element and the second control element may be implemented as a computer control unit, for example as software or firmware running on a processor within a machine.

[0044] The present invention includes combinations of the described aspects and preferred features except where such combinations are expressly not permitted or are expressly avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Embodiments and experiments illustrating the principles of the present invention will now be discussed with reference to the accompanying drawings, in which:

[0046] Figure 1 is a schematic diagram showing functional components of an autonomous machine as an embodiment of the present invention;

[0047] Figure 2 is a schematic diagram showing the interrelationship of prediction models used in a control system of an autonomous machine according to the present invention;

[0048] Figure 3 is a schematic diagram showing energy flux when a predictive model of a control system operates in a converged state; and

[0049] Figure 4 is a schematic diagram showing energy flux when the predictive model of the control system operates in a divergent state. DETAILED DESCRIPTION

[0050] Various aspects and embodiments of the present invention will now be discussed with reference to the accompanying drawings. Other aspects and embodiments will be apparent to those skilled in the art. All documents mentioned herein are incorporated herein by reference.

[0051] The arrangement described herein provides two adaptive predictors arranged between an interface of a machine controller and its operating environment. A first predictor predicts events sensed in the machine environment; a second predictor predicts a response of the machine controller.

[0052] The invention calibrates both predictors via a measure of some common mode output orthogonal to the differential mode signal causing or measuring the differential power output of the machine.In the described arrangement, the adaptive predictor may be viewed as controlling the (nominal mechanical) impedance presented to the machine interface and controller.

[0053] The predictors are connected in local loops, such as to minimize the information (energy) flowing therein (nominally via near-simultaneous convergence of the individual predictors, but positive feedback may also be used to promote learning via initial divergence). When the predictors employ hierarchical memory networks (such as Bayesian hierarchical networks), convergence occurs in a step-wise manner.

[0054] The information flowing in the loop during these steps is derived from the residual error energy in the prediction. Impedance cancellation allows the machine controller to react to these errors (and therefore any new information) relative to their impact (prediction) on the machine energy state (or some function thereof).

[0055] When the predictor is arranged to drive a "one-to-many" (nominally all) output, it is understood in the machine's interface (what the machine controller "sees") that its energy itself is located at the center of its sensed environment and is projected as an object with energy value to the machine. This model itself is orthogonal to the environment and is always present during divergence in the loop.

[0056] The invention also provides a machine comprising one or more interfaces, possibly in different environments, wherein the advantages of the invention are present at the interfaces of all machines. The invention can also be configured to accommodate more than one machine (controller), each having its own energy state and drive to nominally maintain that state.

[0057] Figure 1 1 is a schematic diagram showing the functional components of a machine 100 as an embodiment of the present invention. The machine 100 includes an off-line rechargeable power source 102, a control system 104, and a physical interface 110 representing the ability of the machine to interact with its external environment 112, such as by providing the mobility required to enable it to interface with an external energy source for power recharging.

[0058] The control system 104 includes a first control element 106 and a second control element 108 that interact to maintain a positive energy state of the machine via regulating the machine and its internal environment 114 and controlling motion and interaction with its external environment 112 .

[0059] The internal environment of a given machine is usually well understood and is usually represented in this paper by a discrete matrix α. In contrast, the external environment is likely to be constantly changing and is defined by a more abstract matrix, .

[0060] In the following discussion, all time-related indicators are omitted for clarity. In addition, the specific manner in which information is encoded in the control system 104 is not discussed in detail, as it is understood that those skilled in the art can implement the following teachings using known techniques. In fact, the general machine architecture disclosed herein is applicable to a wide variety of implementations and to autonomous machines suitable for use in many different applications.

[0061] In a general sense, it can be appreciated that the physical interfaces 110 of the machine interact with the external environment 112 in a variety of different ways, which can be viewed as multiple interfaces. For example, the output |F> of the machine at these interfaces can be represented as a force exerted on the external environment. Similarly, the input |U> of the machine at these interfaces can represent a measure of speed, etc., from which a measure of physical resistance can be derived, for example.

[0062] Interfaces can be divided into two categories: (i) reciprocating interfaces, and (ii) non-reciprocating interfaces. A reciprocating interface is an interface where the machine interacts directly with the external environment and consumes energy. In this embodiment, the control system is configured to operate the reciprocating interface in common mode and differential mode.

[0063] For m reciprocating interface, for differential mode output,

[0064] |F m >=Z α |U m >+Z β |U m >=|Z α +Z β |U m >=Z m |U m >

[0065] |U m >=Y m ||F m >

[0066]

[0067] Where Z α and Z β Represents the mechanical impedance of the internal and external environments. m and Y m are matrices representing the impedance and admittance, respectively, relating the reciprocating output to the reciprocating input. m Indicates the power (or energy) consumed by the machine through the reciprocating interface.

[0068] At the same time, for common-mode output, due to Z β |U0>=0,

[0069] |F0>=Z α |U0>

[0070] |U0>=Y α |F0>

[0071]

[0072] Therefore, the power dissipated in the internal impedance can be expressed as

[0073]

[0074] As for the external impedance,

[0075]

[0076] The mechanical efficiency of a machine when performing a certain mechanical action can be expressed by η m Indicates that

[0077]

[0078] And where η m Inversely proportional to the energy stress on the machine.

[0079] A non-reciprocating interface is an interface that transmits information in a manner that does not substantially lose energy to the external environment. A non-reciprocating interface includes one or more non-reciprocating outputs and one or more non-reciprocating inputs. For example, a non-reciprocating input may represent a sensor reading or other passive observation of the external environment.

[0080] In a general sense, the machine 100 can be understood as having a total of n non-reciprocating interfaces, consisting of p non-reciprocating output interfaces and q non-reciprocating input interfaces. By definition, |F q >=0 and |U p >=0, which means that the power consumed by n non-reciprocating interfaces is zero, that is,

[0081] W p = <F p |U p >=0

[0082] W q = <F q |U q >=0

[0083] W n =W p +W q = <F n |U n >=0

[0084] Considering that the machine 100 has a total of σ interfaces, where σ=m+n, the following relationship can be expressed regarding the total power (energy) consumed by the machine:

[0085] W σ =W0+W m =W α +W β

[0086] The physical interface 110 may be understood as a functional entity that converts between the interaction |F>, |U> with the external environment on the interface discussed above and the control signals |f>, |u> exchanged on multiple information channels with the control system 104. In a general sense, the information channels may thus be classified as reciprocating and non-reciprocating, even though there is no one-to-one correspondence between information channels and interfaces.

[0087] It can be understood from the above that m reciprocating channels can be energy-conserving, so that

[0088] <f m |u m >= <F m |U m >

[0089] <f0|u0> =<F0|U0>

[0090] It is also possible that energy conservation requires that the number of reciprocating channels be smaller than the number of reciprocating interfaces. For motion in one spatial dimension, at least two reciprocating interfaces are required, with one additional interface required for each additional dimension.

[0091] For the convenience of the following explanation, assume that the p non-reciprocating outputs are |F p >=|f p >, and the q non-reciprocating inputs are |U q >=|u q >.

[0092] More generally, all input channels |u l > and all output channels |f k >, where k = m + p and l = m + q, the matrix representing the physical interface can be Based on the understanding that the orthogonal common-mode signal and differential-mode signal are added in m reciprocating channels, the expression is:

[0093]

[0094] in has units of mechanical admittance or maneuverability and represents the relationship between the input channel |u> and the output channel |f> due to the combination of the physical interface and the external environment and is defined herein as

[0095]

[0096] It can be noted that there is a dissipated power W in the internal environment and control system Ξ , which has nothing to do with the power consumed by the machine in question, but is a component of the differential equation that describes the overall energy state E of the machine, where

[0097]

[0098] The control system 104 (which may also be referred to herein as a "controller") includes a first control element 106, which functions to regulate the internal environment 114, such as by employing a first feedback loop to control the internal environment of the machine, and employing a second feedback loop to cause the movement required to restore the offline power source when necessary. For example, the first control element can provide a main feedback loop that embeds sufficient positive feedback to promote energy consumption, such as locating and engaging the power source (required to restore power) required, while providing overall negative feedback that promotes energy efficiency and strives to maintain a positive energy state of the machine. Therefore, the first control element 106 includes the required logic to instruct the operation of the physical interface 110 to achieve the desired goal, that is, to enable the machine to operate in the external environment.

[0099] Stated in a general sense, the first control element 106 can be understood as interacting with the internal environment 114 through a set of J internally facing input channels |w> and i internally facing output channels |h>. Similarly, the first control element 106 can be understood as interacting with the physical interface through a set of externally facing input channels |v> and externally facing output channels |g>. In the present invention, the set of externally facing input channels |v> and externally facing output channels |g> can be mapped to input channels |u> and output channels |f> associated with the physical interface, and are therefore considered to include k and l channels, respectively, as discussed above. The above-mentioned interaction of the first control element with the internal environment and the external environment can be expressed as follows:

[0100] |W j > = α|h i >

[0101] and

[0102]

[0103] It can be seen that for some components that control motion

[0104] |w j > = α[I-αβ 22 ] -1 β 21 |v1>

[0105] And therefore

[0106] |g k >=β 11 |v1>+β 12 |w j >=β 11 +β 12 α[I-αβ22 ] -1 β 21}|v1>=θ|v1>

[0107] Therefore, considering again that the orthogonal common-mode signal and differential-mode signal are added in m reciprocating channels, the following relationship can be obtained:

[0108]

[0109] Here θ is a matrix with mechanical impedance units and represents the relationship between the first control element's externally facing input channel |v> and externally facing output channel |g> due to the combination of the first control element and the internal environment, and the nominal impedance on the reciprocating channel is defined as

[0110]

[0111] As discussed above, the control system is configured to operate the reciprocating interface in common mode and differential mode. In this embodiment, the first control element applies a common mode bias, which is arranged so that the reciprocating interface has a positive internal power dissipation. The regulation of this internal dissipation is achieved via a common mode regulator 116 (e.g., a common mode servo amplifier), which is arranged in parallel with the internal environment 114 and is configured to operate in a feedback arrangement or a feedforward arrangement (as a suitable component of α or β22) relative to a reference. The reference can be a persistent energy source formed in part by a positive feedback element, which also serves as the basic source of the overall positive energy state of the machine.

[0112] A common-mode regulator is added to maintain the common-mode power output W0 and pass the β 22 The defined path directs the common-mode signal away from β11. By defining θ = A + B, where

[0113] A=β 12 α[I-αβ 22 ] -1 β 21

[0114] and

[0115] B=β 11

[0116] Further subdivision allows the definition of

[0117]

[0118] in

[0119]

[0120] and

[0121]

[0122] For p non-reciprocating outputs |v p > = 0 and q non-reciprocating inputs | g q >=0, where the internal power consumed is

[0123]

[0124] And the same is true for n non-reciprocating channels and σ total channels

[0125]

[0126] Due to the action of the common mode regulator 116,

[0127]

[0128] And therefore

[0129]

[0130] The control system 104 also includes a second control element 108 disposed between the first control element 106 and the physical interface 110. The second control element 108 mediates between the first control element 106 and the physical interface 110 with the goal of offloading the physical interface from the first control element and vice versa by using two predictive models discussed below.

[0131] In order to establish a realistic energy basis for operating the prediction model within the second control element 108, the second control element 108 is configured to create a simulated reciprocating channel for each non-reciprocating channel it processes. Four types of simulated reciprocating channels can be created in this way:

[0132] ■ a simulated reciprocating output channel provided in the set of outward-facing output channels |v> for each non-reciprocating channel in the set of outward-facing input channels |g>,

[0133] ■ A simulated reciprocating output channel provided in the set of output channels |u> for each non-reciprocating channel in the set of input channels |f>.

[0134] ■ a simulated reciprocating input channel provided in the set of outward-facing input channels |g> for each non-reciprocating channel in the set of outward-facing output channels |v>,

[0135] ■ a simulated reciprocating input channel, provided in the set of input channels |f) for each non-reciprocating channel in the set of output channels |u>,

[0136] A common real energy basis is established in the second control element 108 through the energy conserving link with the external environment in the physical interface 110 discussed above.

[0137] To take into account the simulated channels, it is necessary to recalculate the k outputs and l inputs for the total number of interfaces σ in a more general formula:

[0138] On the output side of the second control element:

[0139]

[0140] If the non-reciprocating channel is expressed as

[0141]

[0142] and

[0143]

[0144] but

[0145]

[0146] Among them, it is worth noting

[0147]

[0148] and The rest of the . is a matrix representing the external world seen by the second control element 108. It is referred to as the external matrix hereinafter.

[0149] A similar approach can be taken on the input side of the second control element:

[0150]

[0151] If the non-reciprocating channel is expressed as

[0152]

[0153] and

[0154]

[0155] but

[0156]

[0157] Among them, it is worth noting

[0158]

[0159] and The rest of the . is a matrix representing the internal world seen by the second control element 108. It is referred to as the internal matrix hereinafter.

[0160] The simulation of reciprocating channels provides the opportunity to create quasi-reciprocating channels that are (nominally) suitable non-dissipative interface source or load terminals, i.e., where

[0161] |u p >=I|f p >

[0162] and

[0163] |g q >=I|v q >

[0164] And|v p > and |f q ) is “open.” Note that the simulated non-reciprocating inputs or outputs used by the second control element 108 do not necessarily continue to the first control element 106 or the physical interface 110. These non-returned channels can be understood as “stubbed” channels of the second control element 108.

[0165] As described above, the second control element 108 provides two different prediction models that independently predict the σ >、|v σ > and |f σ >、|u σ > Modeling the information presented. The predictive models may be implemented as feed-forward or feedback elements using conventional techniques, or rearranged relative to each other in any known manner to achieve an equivalent effect.

[0166] Figure 2 is a schematic diagram showing the interrelationship of the prediction models 120, 122 used in the second control element 108 in an embodiment of the present invention. Figure 2 For illustration purposes only, the implementation of the concepts is depicted in a simplified analog circuit with a single channel.

[0167] On the output side, develop or select an external matrix The purpose of the model Y122 is to minimize the first energy ε (or alternatively the input error vector |u ε > ε |u ε >), where

[0168] ε= <f σ |u ε >

[0169] ​

[0170] ε= <f σ |u ε >= <f σ |Y ε |f o >

[0171] and

[0172]

[0173] On the input side, develop or select an internal matrix Model The goal is to minimize the second energy ε (or alternatively, the input error vector |g ε > <g ε |g ε >), where

[0174] ε= <v σ |g ε )

[0175]

[0176] and

[0177]

[0178] exist Figure 2 In the diagram, for simplicity, a pair of operational amplifiers 124, 126 are depicted as each receiving a single channel. These elements provide a virtual ground "current" summing node, and two negative converters 130, 132 (e.g., negative impedance converters) are used to invert the outputs of the models 120, 122 to achieve a cancellation effect. Each of the models 120, 122 can be understood as a passive impedance, but they represent a time-varying pulse impedance function for modulating the impedance presented by the feedback loop set within the second control element 108.

[0179] Each of the models 120, 122 may be generated by utilizing a Bayesian hierarchical network. In general, the models are arranged such as to present a negative impedance to the associated channel, or in a manner similar to that effect. The second control element 108 is arranged such that the prediction it forms from the model 122 of its external environment (even from a single sensed input) is projected onto nominally all sensed inputs comprised by the physical interface 110. Figure 2 As shown, the predictive models 120 , 122 are combined in such a way that in the absence of the model (ie, without the second control element), the machine defaults to the first control element 106 directly controlling the physical interface 110 .

[0180] However, by implementing adaptive learning techniques in the second control element 108, the models 120, 122 can adapt to gradually offload the physical interface 110 from the first control element 106, and vice versa. When the models 120, 122 have sufficient fidelity so that no error energy ε is discerned, the models will operate in a converged state, such as Figure 3 At this point, there is no flux in the combination of models 120 and 122 because

[0181]

[0182] and

[0183]

[0184] When errors occur in the models 120, 122 (i.e., when an unknown perturbation occurs), the combined flux establishes a feedback loop in the second control element 108, such as Figure 4 As shown. The combined flux will again drive the models 120, 122 towards convergence - either by selecting a higher fidelity model or extracting (and possibly learning) a new model. Since the error energy ε is necessarily the same in both prediction models,

[0185]

[0186] Make

[0187]

[0188] Figure 3 and Figure 4 The effect shown is that the well-modeled portions of |g> and |u> cancel each other out, appearing as an infinite impedance in parallel with the error impedances |gε> and |uε> formed by the unmodeled portions. Therefore, only the current due to the modeling errors will flow in Figure 4 Flow occurs in the local loop shown, and thus the configuration shown provides "renormalization" of the error.

[0189] The most effective models include hierarchical hierarchies, such as the Bayesian hierarchical networks mentioned above. Convergence in this configuration is step-wise, as the layers are selected and convergence occurs in gradual stages in the local feedback loops. When convergence requirements allow and the overall control loop produces sufficient positive feedback, convergence may also require further divergence ("wandering") before converging to some non-local equilibrium point.

[0190] In the diffuse state ( Figure 4), the model error in the second control element can be observed at the physical interface or at the reciprocating channel simulation point. The error shown can be effectively presented at the physical interface in an appropriate sensing mode. When the second control element adopts a layered or hierarchical model, all layers below the error source are substantially offset so that the energy information presented in the physical interface is caused by the convergence of the higher level layers in the model at the moment of defining the convergence path within the local feedback loop in the second control element.

[0191] More specifically, during the divergence period in the second control element (i.e. when Y ε and is non-zero), even if W m =0, will also be non-zero because the common-mode servo action ensures that W α >0. Therefore, when there is mechanical power output (via ) or prediction of mechanical power output (via ) or the prediction of the effect of mechanical power output on the external environment (via ) or (more obviously) if there are disturbances in the machine's internal environment, the model error The amount It will be obvious.

[0192] The layered model provides a lower layer section with no perceived error and a higher layer section with more significant divergence. Due to the effective impedance cancellation provided by the prediction model, only the higher layers will appear in the effective load presented on the second control element.

[0193] Thus, the second control element provides error "renormalization." More specifically, energy errors perceived in the external environment model are adjusted by the requirements determined by the corresponding parts of the model of the first control element and then presented in possibly all interfaces within the second control element.

[0194] The selection of appropriate model components when assembling the prediction models 120, 122 can develop into a Markov blanket formed around information clusters that can be fully described in a higher-level, more efficient manner. The model components can be formed adaptively or established in some pre-programmed form.

[0195] In one example, the second control element can be configured to force predetermined degradation terms in the first prediction model and the second prediction model to decay gradually over time. In this scheme, only those cross-words that are strengthened by their frequent use will be retained, and "memories" that are no longer relevant will be discarded.

[0196] The architecture of the control system discussed above, and in particular the provision of common mode regulators and reciprocating channel simulation, enables the predictive model to distinguish between various information clusters. At the basic level of the model, and for the convergence step of the renormalization, in use, there will be a cluster of information in the combined flux in the second control element, which represents the measurement of the physical energy machine itself and its action relative to its energy state, and there will be clusters of information (or objects) related to the mechanical power output or its prediction (defining the near field of the machine), which can be distinguished from information that is not related to the mechanical power output of the machine or its prediction (forming the sensed far field of the machine).

[0197] For the information cluster defining the near field, the further differentiation provided by the common mode servo and the correlation of the common mode and differential mode power outputs provides a relative measure of the internal power dissipation or a prediction thereof, and the measure represents the external mechanical power dissipation or a prediction thereof, so that an information cluster representing the machine itself is inferred, which is different from its far field according to the boundaries inherent in its mechanical interaction and is subject to energy stress, on which energy stress there is obviously some degree of agency, where the agency includes determining a convergence step within the second control element, determining the action caused by the first control, and where the renormalized error is effectively presented, such as providing a focusing window of the entire sensed reality of the machine based on the relative energy value of that portion of the sensed reality determined by the renormalization of the output of the first control element relative to the machine.

[0198] It may be noted that the presentation of the relative energy measure is only apparent after the model error is identified and until convergence is reached. Thus, the presentation may be asynchronous and occur at discrete moments in real time.

[0199] The first control element can be configured to operate relative to a reference, which can indicate a positive energy state of the machine. In one example, the machine can simulate a heat-generating machine by configuring the reference to indicate an internal temperature of the machine. For example, the reference can be provided by an output from a common-mode regulator having one or more thermometers that sense the internal temperature of the machine. Thus, the machine can be configured to control its own heating via modulating the common-mode power output.

[0200] The features disclosed in the foregoing description or in the appended claims or in the drawings, expressed in their specific form or as means for performing the disclosed functions or as methods or processes for obtaining the disclosed results, may, where appropriate, be used alone or in any combination of such features to realize the invention in its various forms.

[0201] Although the present invention has been described in conjunction with the exemplary embodiments described above, many equivalent modifications and variations will be apparent to those skilled in the art when this disclosure is given. Therefore, the exemplary embodiments of the present invention set forth above are considered to be illustrative rather than restrictive. Various changes may be made to the described embodiments without departing from the spirit and scope of the present invention.

[0202] For the avoidance of any doubt, any theoretical explanations provided herein are provided to enhance the reader's understanding. The inventors do not wish to be bound by any of these theoretical explanations.

[0203] Any section headings used herein are for organizational purposes only and are not to be construed as limiting the subject matter described.

[0204] Throughout the specification, including the appended claims, unless the context requires otherwise, the words "comprise" and "include" and variations such as "comprises", "comprising" and "including" will be understood to imply the inclusion of stated integers or steps or groups of integers or steps but not the exclusion of any other integers or steps or groups of integers or steps.

[0205] It must be noted that, as used in this specification and the appended claims, the singular forms "a", "an", and "the" include plural referents unless the context clearly indicates otherwise. Ranges may be expressed herein as from "about" one particular value and / or to "about" another particular value. When such a range is expressed, another embodiment includes from the one particular value and / or to the other particular value. Similarly, when a value is expressed as an approximation by using the antecedent "about", it should be understood that the particular value forms another embodiment. The term "about" in relation to a numerical value is optional and means, for example, + / - 10%.

[0206] References

[0207] [1]Linsker (1988) "Self-organization in a perceptual network". IEEEComputer, 21(3), 105-117

[0208] [2]Dayan(1995) "The Helmholtz Machine".Neural Comput.7, 889-904

[0209] [3]Friston(2006)“A Free Energy Principle for lhe Brain”.J.Physi0l.Paris 100,70-87

[0210] [4]Feynman(1972)“Statistical Mechanics:A Set of Lectures”.BenjaminISBN 9-7800805-325093

[0211] [5]Mackay(2003)“Information Theory,Inference,and LearningAlgorithms”.Cambridge University Press ISBN 9-780521-642989

[0212] [6]Tishby(1999)″The Information BoEleneck Method″.37th AllertonConference on Communication,Control,and Computing 368-377

[0213] [7]Friston(2008)“Hierarchical Models in the Brain”.PLOS Comput.Biol.4(11)

Claims

1. A machine capable of autonomous operation, the machine comprising: Rechargeable offline power supply; A physical interface, through which the machine interacts with an external environment, the physical interface includes: a reciprocating interface capable of operating in differential mode and common mode, wherein the reciprocating interface is configured to provide motion via a differential mode output and to provide motion cancellation of a common mode output; and a non-reciprocating interface configured to passively interact with the external environment via a non-reciprocating input; a first control element configured to control internal and external operations of the machine and maintain a positive energy state of the machine; and a second control element, the second control element being disposed between the first control element and the physical interface to mediate information exchange therebetween, wherein the second control element is configured to communicate with the physical interface via a first set of reciprocating information channels and to communicate with the first control element via a second set of reciprocating information channels, wherein the first set of reciprocating information channels includes an input channel configured to transmit information from the non-reciprocating input to the machine and an emulated output channel forming a reciprocating pair with the input channel, wherein the second control element comprises a first prediction model arranged to predict communications on the first set of reciprocating information channels and a second prediction model arranged to predict communications on the second set of reciprocating information channels, wherein both the first prediction model and the second prediction model are adaptive models that are combined through a feedback arrangement to minimize the error energy flux between the first set of reciprocating information channels and the second set of reciprocating information channels, and Wherein the first control element is configured to maintain a common mode bias at the physical interface and comprises a common mode regulator arranged to establish an independent parallel common mode path within the internal environment of the machine.

2. The machine of claim 1 , wherein the differential mode output of the reciprocating interface is configured to move the machine within the external environment, and wherein the reciprocating interface includes at least one pair of independent reciprocating elements in each dimension of motion.

3. The machine of claim 1 or 2, wherein the first control element is configured to adjust a positive energy state of the machine.

4. The machine of claim 3 , wherein the first control element is implemented by a control model encoding instructions capable of controlling the physical interface to enable the machine to perform actions in its environment, wherein the control model is configured to operate based on an internal reference indicative of the energy state of the machine, and wherein the control model employs a first feedback loop to control the internal environment of the machine and employs a second feedback loop to cause the motion required to restore the offline power source.

5. The machine of claim 4, wherein the control model comprises a plurality of driver units arranged to prioritize a set of available actions, and wherein the plurality of driver units comprises a base driver followed by a sequence of sub-drivers.

6. The machine of claim 5, wherein the base drive is configured to maintain a positive energy state of the machine.

7. A machine according to claim 4 or 5, wherein the first control element is configured to pause one or more sub-drives during initial operation.

8. A machine according to any one of claims 4 to 7, wherein the first control element comprises an adaptive learning module arranged to update the control model.

9. A machine as claimed in any preceding claim, wherein the common mode regulator is driven by an internal reference signal indicative of a positive energy state of the machine.

10. A machine according to any preceding claim, wherein the common mode bias maintained by the first control element is arranged to cause the common mode output of the reciprocating interface to have a positive internal power dissipation, wherein the common mode regulator is configured to regulate the internal power dissipation to control the internal temperature of the machine.

11. A machine according to any preceding claim, wherein the first set of reciprocating information channels and the second set of reciprocating information channels each include a plurality of reciprocating channel pairs, the plurality of reciprocating channel pairs establishing a real energy basis in the second control element.

12. A machine according to any preceding claim, wherein the non-reciprocating interface includes a non-reciprocating output, and wherein the first set of reciprocating information channels also includes output channels that transmit information to the non-reciprocating output and simulated input channels that form reciprocating pairs with the output channels.

13. A machine according to any preceding claim, wherein the first set of reciprocating information channels and the second set of reciprocating information channels each include a plurality of non-reciprocating channel pairs, each channel pair consisting of a non-reciprocating channel originating from the first control element or the physical interface and a corresponding simulation channel.

14. The machine of claim 13, wherein one or more of the non-reciprocating channel pairs terminate prior to the first control element or the physical interface, thereby forming a stub channel.

15. A machine according to any preceding claim, wherein the first prediction model is configured to generate an output comprising signals on all channels in the first set of reciprocating information channels, and the second prediction model is configured to generate an output comprising signals on all channels in the second set of reciprocating information channels.

16. A machine according to any preceding claim, wherein the first prediction model and the second prediction model operate towards a convergence state, in which the energy flux between the first set of reciprocating information channels and the second set of reciprocating information channels is minimized, and wherein the second control element is configured to control a path to the convergence state in a step-by-step manner.

17. A machine according to claim 16, wherein in a divergent state, the second control element is arranged to identify an observable representation of the energy flux between the first reciprocating information channel and the second set of reciprocating information channels at the physical interface, and to control the path to the convergent state based on the identified representation.

18. The machine of claim 17, wherein the second control element is configured to modulate the path to the converged state.

19. A machine as claimed in any preceding claim configured in a distributed manner over a plurality of physical sub-components.

20. A machine according to any preceding claim, wherein the first control element is configured to generate a non-linear response in one or more of the second set of reciprocating information channels.

21. A machine according to any preceding claim, wherein the second control element is arranged to introduce a disturbance in the first predictive model and / or the second predictive model.

22. A method of operating an autonomous machine according to any preceding claim, wherein the method comprises: determining an error energy flux between the first set of reciprocating information channels and the second set of reciprocating information channels based on an observable characteristic of the physical interface; One or both of the first prediction model and the second prediction model are adjusted based on the determined error energy flux.

23. The method of claim 22, wherein the first prediction model and the second prediction model each comprise a layered hierarchical model, and wherein adjusting one or both of the first prediction model and the second prediction model comprises selecting a layer to add to each hierarchical model or making changes in each hierarchical model to reduce the error energy flux.

24. A computer program product comprising computer readable instructions stored on a non-transitory carrier, wherein the computer readable instructions are executable by a computer to perform the method according to claim 22 or 23.

Images