Method and apparatus for an analog neural network

Abstract

A method and an apparatus for processing an analog input signal using an analog neural network and generating a corresponding analog output signal are disclosed. An analog input interface is configured to supply a task input signal and a compensation input signal. One or more analog neuron circuits are connected to the analog input interface. An analog parameter storage array modifies task input signals and supplies these to an analog computation circuit that generates an output signal based on a combination of contributions from the modified task input signals and the compensation input signal.

Description

[0001] METHOD AND APPARATUS FOR AN ANALOG NEURAL NETWORK

[0002] CROSS REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of priority of Canadian Patent Application No. 3,247,742 filed on July 8, 2024, the disclosure of which is incorporated herein by reference.

[0004] BACKGROUND

[0005] 1. Field of the Technology

[0006] The present application relates to receiving an analog input signal and generating a corresponding analog output signal. In particular, it relates to generating analog output signals in an analog neural network.

[0007] 2. Description of the Related Art

[0008] Digitally-implemented neural networks are increasingly used due to advances in algorithms and hardware. However, their power consumption, especially for large language models, is so high that entire power stations are sometimes required to supply the necessary electricity. Smaller digital neural networks consume less power, but still too much for many applications that absolutely require the lowest possible power consumption. Such applications might include monitoring environmental conditions in a remote region, using solar power to charge a battery. In remote monitoring applications, such as detecting the presence of a particular bird species to infer weather or soil quality over time, power availability is often severely limited.

[0009] However, analog neural networks are subject to variations in process, temperature and voltage, often referred to as PVT. The operating conditions defined by process, temperature and other factors greatly limit the usability of such circuits. Process variations, for example, must be tightly controlled, in order to avoid the need to in-situ retraining of the analog neural network. Such in-situ training is time-consuming and makes manufacture much more expensive. Temperature variations, at the individual component level, may be compensated by well-known analog circuit architectures, but these require greatly increased numbers of components and high power consumption.

[0010] The potential for low power neural processing offered by analog neural networks is undermined by their susceptibility to operating conditions, whether fixed during manufacture, or variable during the operation of the circuit. There is a need for at least one of a method and an apparatus that will overcome at least one of the above-identified limitations.

[0011] BRIEF SUMMARY

[0012] It is an object of the present technology to ameliorate at least some of the inconveniences present in the prior art. One or more implementations of the present technology may provide and / or broaden the scope of approaches to and / or methods of achieving the aims and objects of the present technology.

[0013] It will be appreciated that the technology disclosed herein offers significant advantages by enabling physically small and energy-efficient processing of neural network workloads in applications for which available size and energy limitations would otherwise prevent the required level and speed of computation.

[0014] According to a broad aspect of the technology, there is disclosed an analog neural network apparatus for operating in the presence of one or more operating conditions affecting a plurality of analog components associated with the analog neural network apparatus used to generate an output of the analog neural network, comprising an analog input interface configured to supply a task input signal and supply a compensation input signal, a plurality of analog neurons connected to the analog input interface, each analog neuron comprising an analog parameter storage array storing a plurality of parameters of the analog neural network, the analog parameter storage array having a plurality of analog inputs and comprising a plurality of analog weighting elements, each configured to store an analog weighting factor and to modify a task input signal supplied from the analog input interface, an analog computation circuit, configured to generate an analog output signal based on a combination of contributions by one or more of the modified task input signals and the compensation input signal, wherein a plurality of the analog weighting elements store analog weighting factors that define the relative contribution of the compensation input signal, such that variations in the one or more operating conditions are compensated for in the analog output signal generated by at least one of the analog neurons.

[0015] According to one or more implementations of the technology, the compensation input signal is applied to an accumulator in the analog computation circuit without being weighted by an analog weighting element.

[0016] According to one or more implementations of the technology, the analog neural network apparatus further comprises an analog compensation input register for generating the compensation input signal as a fixed signal maintained constant during multiple inference operations of the analog neural network.

[0017] According to one or more implementations of the technology, the analog compensation input register comprises a plurality of compensation input elements for generating the compensation input signal as a plurality of analog fixed signals.

[0018] According to one or more implementations of the technology, the compensation input elements comprise resistive elements.

[0019] According to one or more implementations of the technology, the analog neural network apparatus further comprises a programming circuit for the resistive elements. According to one or more implementations of the technology the resistive elements comprise memristors.

[0020] According to one or more implementations of the technology, the operational characteristics of the compensation input elements are defined in response to ex-situ training of a model of the apparatus to improve the apparatus's compensation for variations in the one or more operating conditions.

[0021] According to one or more implementations of the technology, the analog neural network apparatus further comprises a temperature sensor for supplying the compensation input signal.

[0022] According to one or more implementations of the technology, a plurality of the analog neurons are configured to form a multi-stage analog signal processing path from the task input signal to the corresponding analog output signal, and wherein the compensation input signal influences at least two analog computation circuits located at different stages along the signal processing path.

[0023] According to one or more implementations of the technology, each analog weighting factor is stored as a resistance representing a learned network parameter.

[0024] According to one or more implementations of the technology, the analog neural network apparatus further comprises an analog delay line for modifying the task input signal supplied to at least one of the analog parameter storage arrays, the analog delay line comprising a plurality of memory cells for propagating the task input signal through the memory cells in a time-sequenced manner under control of a propagation signal, the propagated task input signal comprising a plurality of propagated task input signals, each of the propagated task input signals being supplied from a respective memory cell of the analog delay line. According to one or more implementations of the technology, the analog neural network apparatus is further configured to supply a reference task input signal from the analog input interface during a calibration operation, wherein the reference task input signal is propagated through an analog delay line during the calibration operation.

[0025] According to one or more implementations of the technology, the analog neural network apparatus further comprises an compensation input register configured for storing an analog output of the apparatus during a calibration operation of the apparatus and for supplying the stored analog output as a compensation input signal during a subsequent inference operation of the apparatus.

[0026] According to one or more implementations of the technology, the analog neural network apparatus is configured to supply a reference task input signal from the analog input interface during a calibration operation.

[0027] According to one or more implementations of the technology, at least one of the analog computation circuits comprises a rectified linear unit (ReLU).

[0028] According to one or more implementations of the technology, the analog weighting factors have been determined through training in response to simulated variation of the one or more operating conditions.

[0029] According to a broad aspect of the technology, there is disclosed a method for processing an analog input signal and generating a corresponding analog output signal based on stored analog neural network parameters in the presence of variations in one or more operating conditions affecting a plurality of the analog components used to generate the analog output signal, the method comprising providing an analog task input signal, supplying an analog compensation input signal, generating a plurality of weighted signals by modifying the task input signal in response to a plurality of analog weighting factors previously stored in one or more analog parameter storage arrays, generating the corresponding analog output signal based on a combination of contributions by the weighted signals derived from the task input signal and the compensation input signal, and providing the generated analog output signal.

[0030] According to one or more implementations of the technology the compensation input signal is applied to an accumulator in the analog computation circuit without being weighted by an analog weighting element.

[0031] According to one or more implementations of the technology the compensation input signal is maintained constant across multiple inference operations.

[0032] According to one or more implementations of the technology, the compensation input signal comprises a plurality of analog fixed signals.

[0033] According to one or more implementations of the technology, the method includes supplying the compensation input signal from a temperature sensor.

[0034] According to one or more implementations of the technology, the task input signal is propagated through an analog delay line in a time-sequenced manner.

[0035] According to one or more implementations of the technology, the compensation input signal is based on an analog output signal generated by the method during a calibration operation performed prior to an inference operation.

[0036] According to one or more implementations of the technology, the method includes supplying a reference task input signal to the analog neural network during a calibration operation, and generating the compensation input signal based on a resulting output signal.

[0037] According to one or more implementations of the technology, the analog weighting factors were obtained through training of a model of the analog neural network under simulated variation of the one or more operating conditions.

[0038] According to one or more implementations of the technology, the same compensation input signal is used across a plurality of different operating conditions.

[0039] According to one or more implementations of the technology, the method comprises generating the compensation input signal in response to detecting a change in an operating condition.

[0040] According to one or more implementations of the technology, generating the analog output signal comprises applying a non-linear combination of the contributions.

[0041] According to one or more implementations of the technology the non-linear combination comprises a rectified linear unit (ReLU) function.

[0042] Implementations of the present technology each have at least one of the above-mentioned objects and / or aspects, but do not necessarily have all of them. It should be understood that some aspects of the present technology that have resulted from attempting to attain the above-mentioned object may not satisfy this object and / or may satisfy other objects not specifically recited herein.

[0043] Additional and / or alternative features, aspects and advantages of implementations of the present technology will become apparent from the following description, the accompanying drawings and the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] For a better understanding of the present technology, as well as other aspects and further features thereof, reference is made to the following description which is to be used in conjunction with the accompanying drawings, where:

[0045] Figure 1 is a diagram which shows a voice-activated solar-powered light, including an analog neural network using one or more implementations of the technology;

[0046] Figure 2 is a flowchart which details processing steps performed when training and using the analog neural network disclosed in Figure 1 , including a processing step of training network parameters, a processing step of storing corresponding parameters and a processing step of detecting a spoken activation phrase;

[0047] Figure 3 is a block diagram which illustrates the components of the voice- activated solar-powered light shown in Figure 1 ;

[0048] Figure 4 is a diagram which illustrates an embodiment of the analog neural network shown in Figure 1 , including analog signal convolution blocks and an analog signal classifier block;

[0049] Figure 5 is a diagram which illustrates details of an analog signal convolution block of the kind shown in Figure 4;

[0050] Figure 6 is a diagram which illustrates details of one implementation of the analog signal convolution block shown in Figure 5;

[0051] Figure 7 is a diagram which illustrates details of one implementation of the analog signal classifier shown in Figure 4; Figure 8 is a flowchart which details the processing step of training network parameters shown in Figure 2;

[0052] Figure 9 is a flowchart which details the processing step of storing parameters shown in Figure 2;

[0053] Figure 10 is a diagram which illustrates an embodiment of the analog neural network shown in Figure 1 , including a compensation input register and an analog neuron circuit;

[0054] Figure 11 is a diagram which details the compensation input register shown in Figure 10;

[0055] Figure 12 is a diagram which details the analog neuron circuit shown in Figure 10;

[0056] Figure 13 is a diagram which illustrates an embodiment of the analog neural network shown in Figure 1 ;

[0057] Figure 14 is a flowchart which details the processing step of detecting a spoken activation phrase shown in Figure 2;

[0058] Figure 15 is a block diagram which shows an implementation of an analog neural network for operating in the presence of one or more operating conditions affecting a plurality of analog components; and

[0059] Figure 16 is a flowchart which shows an implementation of a method for operating an analog neural network for operating in the presence of one or more operating conditions affecting a plurality of its components. DESCRIPTION OF EXAMPLE EMBODIMENTS

[0060] The examples and conditional language recited herein are principally intended to aid the reader in understanding the principles of the present technology and not to limit its scope to such specifically recited examples and conditions. It will be appreciated that those skilled in the art may devise various arrangements which, although not explicitly described or shown herein, nonetheless embody the principles of the present technology and are included within its spirit and scope.

[0061] Furthermore, as an aid to understanding, the following description may describe relatively simplified implementations of the present technology. As persons skilled in the art would understand, various implementations of the present technology may be of a greater complexity.

[0062] In some cases, what are believed to be helpful examples of modifications to the present technology may also be set forth. This is done merely as an aid to understanding, and, again, not to define the scope or set forth the bounds of the present technology. These modifications are not an exhaustive list, and a person skilled in the art may make other modifications while nonetheless remaining within the scope of the present technology. Further, where no examples of modifications have been set forth, it should not be interpreted that no modifications are possible and / or that what is described is the sole manner of implementing that element of the present technology.

[0063] Moreover, all statements herein reciting principles, aspects, and implementations of the present technology, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof, whether they are currently known or developed in the future. Thus, for example, it will be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative circuitry embodying the principles of the present technology. Similarly, it will be appreciated that any flowcharts, flow diagrams, state transition diagrams, pseudo-code, and the like represent various processes which may be substantially represented in computer-readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0064] Now referring to Figure 1 , there is shown an apparatus for receiving an analog input signal and generating a corresponding analog output signal. The apparatus is a solar-powered light 101 that is switched on in response to a spoken command 102, which is the phrase "lights on". The solar-powered light 101 includes a solar cell 103 that charges a battery inside the solar-powered light 101 during the day. It will be appreciated that the amount of power received by the solar cell 103 depends on its area and efficiency, and a low cost, low area solar cell 103 is preferred. A microphone 104, always on, generates an analog input signal in response to environmental sounds. Digital systems for detecting keywords or phrases are well established, but typically consume hundreds of milliwatts of power, exceeding the sustainable energy budget of the solar- powered light 101 , and rendering the light 101 useless within a couple of hours of darkness, during which time it is most needed.

[0065] The solar-powered light 101 comprises an analog neural network integrated circuit 105 that performs keyword and phrase detection at greatly reduced power in comparison with digital systems. The reduced power required by the analog neural network 105 enables it to be used in low power, low cost, battery-operated applications, including the solar-powered light switch 101. As a result of signal processing performed by the analog neural network 105, a Light Emitting Diode (LED) array 106 is activated when the microphone picks up the spoken phrase "lights on". A support post 107 allows the solar-powered light 101 to be placed beside a footpath, and in other places where a mains supply for the light is unavailable, and it must be operated on solar power received by the solar cell 103. The solar-powered light 101 also comprises a Printed Circuit Board (PCB) 108 upon which various components are mounted, including the analog neural network integrated circuit 105. Now referring to Figure 2, there are illustrated processing steps for the preparation and use of the solar-powered light 101 shown in Figure 1. It will be appreciated by the skilled addressee that some of the preparation steps, performed during the product development phase of the solar-powered light 101 require use of a general purpose digital computer 200. The skilled addressee will appreciate that the general purpose digital computer 200 may be of any type.

[0066] Two kinds of training may be performed when training the analog neural network and defining a set of neural network parameters for detection of the spoken phrase 102. A first kind of training is ex-situ training, where the characteristics of the analog neural network 105 are fully simulated in a digital emulation running on the general purpose computer 200.

[0067] A second kind of training is in-situ training, where training is performed by optimizing analog parameters with the help of the general purpose computer 200, but taking account of the measured errors and characteristics of the analog neural network 105. Both types of training may be performed prior to production, to produce a set of parameters that can be programmed into any particular manufactured instance of the analog neural network integrated circuit 105. Alternatively, or in addition, training may be performed individually for each analog neural network 105, prior to, or even after its inclusion in a specific product such as the solar-powered light 101 .

[0068] According to processing step 201 a question is asked as to whether ex- situ training is to be performed. If required, ex-situ training is performed at processing step 202 using a digital model of the analog neural network circuit 105 running on the general-purpose computer 200, to define the parameters of the analog neural network 105. According to processing step 203, neural network parameters are stored as analog values in the analog neural network 105.

[0069] According to processing step 204, a question is asked as to whether in- situ training is to be performed. If required, in-situ training is performed at processing step 205 under control of the general-purpose computer 200 while connected to the analog neural network 105.

[0070] According to processing step 205, parameters are defined and / or refined by programming analog values stored in the analog neural network 105.

[0071] According to processing step 206, the analog neural network integrated circuit and its associated circuits are placed in the solar-powered light 101. According to processing step 207, the solar-powered light 101 is deployed at its location. It will be appreciated that the solar-powered light 101 remains operational for as long as sufficient charge remains in its internal battery.

[0072] According to processing step 208, the spoken activation phrase 102 is detected and the LED array 106 is switched on.

[0073] According to processing step 209, the LED array 106 is switched off after five minutes. It will be appreciated by the skilled addressee that while five minutes is disclosed, the LED array 106 can be switched off after any defined number of minutes.

[0074] It will be appreciated by the skilled addressee that processing steps 208 and 209 are then repeated indefinitely, over many months and possibly years.

[0075] Now referring to Figure 3, there is shown an implementation of the solar- powered light switch 101 shown in Figure 1. It will be appreciated by the skilled addressee that various alternative embodiments may be provided for the solar- powered light switch 101.

[0076] The solar-powered light switch 101 comprises the microphone 104, a preamplifier 301 and an anti-aliasing filter 302. The microphone 104 supplies analog signals to the pre-amplifier 301 which supplies amplified analog signals to the anti-aliasing filter 302. The skilled addressee will appreciate that the microphone 104 and the pre-amplifier 301 may be of various types as known to the skilled addressee.

[0077] It will be further appreciated that the anti-aliasing filter 302 comprises a low pass filter that removes frequency components above 6kHz and enables the analog neural network to sample audio signals at 12kHz, which keeps current consumption low while capturing the essential characteristics of spoken commands.

[0078] The solar-powered light switch 101 further comprises the analog neural network 105, a comparator 305 and an ultra-low power microcontroller 307.

[0079] The output of the anti-aliasing filter 302 is supplied as the input 303 of the analog neural network 105. The analog neural network 105 has an output 304 in the form of a voltage that exceeds a threshold when the spoken command 102 is detected. The comparator 305 detects when the threshold is exceeded and supplies a detection signal 306 to the ultra-low power microcontroller 307. The skilled addressee will appreciate that the comparator 305 and the ultra-low power microcontroller 307 may be of various types. In one or more embodiments, the ultra-low power microcontroller 307 is STM32L432KC manufactured by STMicroelectronics NV (Chemin du Champ-des-Filles 39, 1228 Plan-les-Ouates, Switzerland). The skilled addressee will appreciate that various alternative implementations may be possible.

[0080] The analog neural network 105 will be described in more details hereinbelow.

[0081] The solar-powered light switch 101 further comprises a semiconductor light switch 309 and the LED array 106. The ultra-low power microcontroller 307 supplies an activation signal 308 to the semiconductor light switch 309, which then supplies power to the LED array 106. The skilled addressee will further appreciate that the semiconductor light switch 309 and the LED array 106 may be of various types. For instance, and in one or more embodiments, the LED array 106 is Cree XP G3, manufactured by CreeLED, Inc (4001 E Hwy 54, Suite 2000. Durham, NC 27713, USA). The light switch 309 is IRLZ44NPBF manufactured by Infineon Technologies AG (IFAG CP CC Am Campeon 1-15 85579 Neubiberg Germany). The skilled addressee will appreciate that various alternative implementations may be possible.

[0082] After five minutes, in one or more embodiments, the ultra-low power microcontroller 307 switches off the semiconductor light switch 309, unless another activation command 102 has been detected.

[0083] The solar-powered light switch 101 further comprises an in-circuit programming interface 310.

[0084] It will be appreciated that the in-circuit programming interface 310 facilitates an optional connection to the general-purpose computer 200 for programming or training the analog neural network 105. It will be appreciated by the skilled addressee that this enables locale-specific parameters to be programmed into the analog neural network 105, via a Serial Protocol Interface (SPI) 311 , in one or more embodiments, so that an activation command 102 in a different language can be detected. The in-circuit programming interface 310 also facilitates in-situ training 205 to be performed even after the solar-powered light has been manufactured, should this be necessary. The SPI 311 is also used to support normal operation of the analog neural network, including initialization. The skilled addressee will appreciate that the in-circuit programming interface 310 may be of various types.

[0085] The solar-powered light switch 101 further comprises a charging circuit 321 , the solar cell 103 and a battery 313. The charging circuit 312 receives power from the solar cell 103 and uses this to charge the battery 313, which will provide power for the LED array 106 and the entire circuit of the solar-powered light 101. The skilled addressee will appreciate that the charging circuit 312, the solar cell 103 and the battery 313 may be of various types. For instance, and in one or more implementations, the charging circuit 321 is BQ25570 manufactured by Texas Instruments (12500 T I Blvd, Ste T, Dallas, TX 75243, USA), the solar cell 103 is KS-Q103G manufactured by China Solar Ltd, (Fushen Builing, Huaxia Road, Futian District, Shenzhen, Guangdong, 518000, China) and the battery 313 is INR-18650-P26A, manufactured by E-One Moli Energy (Canada) Limited (20000 Stewart Crescent, Maple Ridge, BC, Canada, V2X 9E7).

[0086] Now referring to Figure 4, there is shown an implementation of the analog neural network 105 shown in Figure 3. It will be appreciated that in one or more embodiments, the analog neural network 105 is implemented as a monolithic silicon integrated circuit and includes an input interface and multiple analog processing blocks.

[0087] An analog input interface 401 receives the analog input signal 303 and provides a high impedance buffer 402 to ensure the input signal 303 is not unintentionally modified. The low impedance output of the buffer 402 supplies an analog task input signal 403.

[0088] The analog input interface 401 also includes a temperature sensor 404 that generates an analog compensation input signal 405. The analog task input signal 403 and the analog compensation input signal 405 are supplied to a first analog signal convolution block 406. A second analog signal convolution block 407 also receives the task input signal 403 and the compensation input signal 405.

[0089] The first analog signal convolution block 406 receives the task input signal 403 and provides this to an analog delay line 408. The analog delay line 408 provides multiple output signal, each of which represents the task input signal 403 delayed by a different amount of time. The outputs from the delay line 408 are supplied to a first analog neuron circuit 409. The compensation input signal 405 is also as an input to the first analog neuron circuit 409. Outputs from the delay line 408 and the compensation input signal 405 are also supplied as inputs to three additional neuron circuits 410, 411 and 412.

[0090] The analog delay line 408 comprises a plurality of memory cells for propagating the task input signal through the memory cells in a time-sequenced manner under control of a propagation signal 505. The propagated task input signal comprises a plurality of propagated task input signals, each of the propagated task input signals being supplied from a respective memory cell of the analog delay line 408.

[0091] The inputs to the first neuron circuit 409 are supplied to a parameter storage array 413 which modifies the inputs according to respective weighting factors. The resulting modified signals are supplied to an analog computation circuit 414 which includes an accumulator 415 and an analog activation function 416. The activation function 416 generates and provides the output of the first analog neuron circuit 409. The other analog neuron circuits 410 to 412 operate in accordance with the description provided above for neuron circuit 409. The activation function 416 generates the analog output signal 421 by applying a nonlinear combination of the contributions supplied from the differential amplifier 415.

[0092] The first analog signal convolution block has four neurons, 409 to 412, each of which provides an output signal. The second analog signal convolution block has the same structure, and also provides four analog outputs. In total, there are eight analog outputs 417 to 424 provided by the analog signal convolution blocks 406 and 407.

[0093] The neuron outputs 417 to 424 are supplied as inputs to an analog signal classifier block 425, which also receives the compensation input signal 405. The analog signal classifier block 425 provides the output 304 of the analog neural network 105.

[0094] The skilled addressee will appreciate that various alternative implementations may be possible.

[0095] The temperature sensor 404 is responsive to the operating temperature of the analog neural network, as indicated by the thermal coupling path 429.

[0096] An SPI circuit 426 communicates with the microcontroller 307 via the SPI connections 311. The SPI circuit comprises an interface 427 and SPI registers 428 for controlling initialization and programming of the analog neural network 105.

[0097] Now referring to Figure 5, there is shown an embodiment of the analog signal convolution block 401 shown in Figure 4. It will be appreciated by the skilled addressee that Figure 5 also describes the structure of the other analog convolution block 407. However, for the purpose of clarity, the description will be provided for the analog convolution block 406 only.

[0098] The analog signal convolution block 406 includes the analog delay line 408 that comprises multiple memory cells 501 , 502, 503 and 504. It will be appreciated by the skilled addressee that while only four memory cells are shown in Figure 5, many more are provided. The analog task signal 403 to the input of the delay line 408 is propagated from the input 403 to the first memory cell 501 in a time- sequenced manner under control of a propagation signal 505 generated by a propagation state machine 506. If, for example, an analog value of 0.5V is provided as the task input 403, this will propagate to the first memory cell 501 in response to the propagation signal 505. The task input 403 may then change to 0.3V, and, in response to the next propagation signal, an electrical charge representing the analog value of 0.5V is passed to the second memory cell 502, and the first memory cell 501 now holds the value of 0.3V. Stored task signals, or samples, are propagated through the defined sequence of the memory cells 501 to 504 in discrete time steps controlled by the propagation signal 505.

[0099] The memory cells 501 to 504 provide analog outputs 507 to 510 to the first neuron 409 and its analog parameter storage array 413 which is programmable and non-volatile. The analog parameter storage array 413 comprises multiple analog weighting elements 511 to 515, each providing a respective weighting factor. Each analog weighting factor is stored as a resistance representing a learned network parameter defined during training 202 and / or 205. The first weighting element 511 is associated with the compensation input signal 405. The other weighting elements 512 to 515 are associated with memory cells 501 to 504. The analog parameter storage array 413 also comprises an analog bias element 516 for storing an analog bias value.

[0100] The weighting element 511 modifies the compensation input signal 405 and the weighting elements 512 to 515 modify the correspondingly delayed task input signal stored in or transferred from their respective associated memory cell 501 to 504. The weight values and bias value are stored as resistance states in memristors in the circuit of the analog parameter storage array 413.

[0101] The neuron circuit 409 further comprises the analog computation circuit 414 that includes the accumulator 415 and the activation function 416. The accumulator 415 provides an analog sum-of-products with bias signal, where each product is the result of modifying a corresponding signal stored in or transferred from a memory cell or the compensation input signal 405. For example, if the voltage on memory cell 501 is A volts, and the weighting element 512 implements a factor of B, their product is AB volts. All such products are added together by the accumulator 415. The bias 516 is also included in this sum, as is the product of the compensation input signal 405 with its respective weight 511.

[0102] The activation function 416 implements a Rectified Linear Unit (ReLU) nonlinear transfer function. The output from the activation function 416 provides the analog output 421 previously shown in Figure 4.

[0103] The analog signal convolution block 406 also comprises a second neuron 410 that includes its own parameter storage array 517 comprising weighting elements 518 to 522 and bias element 523. For example, if the weighting element 519 implements a factor of C, the corresponding product is AC, because weighting element 519 receives the signal 507 from the first memory cell 501 .

[0104] The second neuron 410 also comprises its own analog computation circuit 524, including an accumulator 525 and an activation function 526. The activation function 526 provides the output 422 previously shown in Figure 4. Third and fourth neurons 411 and 412 have been omitted for the sake of clarity, and generate outputs 423 and 424, respectively.

[0105] The analog parameter storage arrays 409 and 517 are programmable and store analog values for weights and bias in non-volatile memory elements. A programming circuit 527 supplies programming signals 528 to the analog parameter storage arrays 409 and 517 shown in Figure 4, and to those omitted from Figure 4 for the sake of clarity.

[0106] While it is disclosed that the delay line 408 comprises memory cells 501- 504, it will be appreciated that the plurality of memory cells may comprise any number of memory cells suitable with a given application.

[0107] In one or more implementations, the analog delay line 408 comprises a one-dimensional arrangement of memory cells. In one or more alternative embodiments, the analog delay line 408 comprises a two-dimensional arrangement of memory cells. In fact, it will be appreciated that any arrangement of memory cells may be used.

[0108] While it is disclosed that the compensation input signal 405 provides a voltage indicative of the temperature of the analog neural network, it will be appreciated that the compensation input signal 405 may comprise any number of signals for which compensation is required. For example, it may comprise two signals, the first being temperature-related with the second being related to the voltage of the power supply for the analog neural network 105.

[0109] Now referring to Figure 6, there is shown an implementation of key elements of the analog neural network 105 shown in Figure 4.

[0110] The propagation signal 505 comprises two parts, 505a and 505b. In one or more implementations, each propagation signal part is a non-overlapping pulse waveform which causes an analog input signal to be propagated from the input into the first memory cell 501 , then to the second memory cell 502, through a defined sequence of memory cells 501 to 504 in discrete time steps defined by the non-overlapping waveforms of the propagation signal 505.

[0111] While 505a is high, 505b is low, causing transistor 601 to charge capacitor 602 to the voltage of the task input 403. During this time, transistor 603 is off, so that the stored charge on capacitor 604 is stable and can be used to perform analog computation. Transistors 601 and 603 are controllable transfer elements. An isolation transistor 605 improves linearity of charge storage and transfer. In the next phase of propagation, 505a is low and 505b goes high. Stored charge is passed to the next memory cell in response to the propagation signal 505. The capacitor 604 receives charge from the capacitor 602 via the transistor 603, so that the voltage previously present at the task input 403 is now stored in the first memory cell 501 . The same process repeats along the analog delay line 408. Each memory cell has a buffer so that the voltage on its capacitor can be supplied for computation without discharging the capacitor and interfering with the propagation of charge in its defined sequence. For example, the first memory cell 501 has a buffer 606. This facilitates output to analog weighting elements, including 512 and 513. The defined sequence of charge transfer takes place in discrete time steps under control of the propagation signal 505. Analog weighting element 512 comprises two memristors, which can be programmed to a particular resistance. A first memristor 607 is used to define a negative weighting factor and a second memristor is used to define a positive weighting factor. Usually, one of the two memristors will be programmed for a very high resistance, so that only the desired positive or negative weighting factor is produced, although in an embodiment both memristors can be programmed to conduct. The analog bias element 516 is similarly provided by a pair of memristors 609 and 610, enabling a positive or negative bias value to be provided as input to the accumulator 415.

[0112] The temperature sensor 404 includes a current mirror formed by two transistors 611 and 612 with a common resistor 613. These supply the same current to transistors 614 and 615, which are biased into partial conduction by the same gate voltage, but are implemented with different physical lengths and widths, so that their resistances are different. The differing geometries of transistors 614 and 615 result in different temperature sensitivities, and therefore different voltages presented to the two inputs of a differential amplifier 616. The output of the differential amplifier provides the compensation input signal 405, which is supplied to the weighting element 511. The weighting element 511 comprises two memristors 617 and 618, which provide positive and negative contributions to the positive and negative inputs 619 and 620 of the accumulator 415.

[0113] The activation function 416 is a Rectified Linear Unit (ReLU) implemented by limiting a linear function to the magnitude of the power supply of the analog neural network 105. The output of the ReLU 416 provides the first output 412 of the first analog signal convolution block 406.

[0114] It will be appreciated that the transistors, capacitors and other electronic components shown in Figure 6 each have subtly different responses to temperature. Without compensation, these responses combine to limit operation of the analog neural network to an impractically narrow range of temperatures. The skilled addressee will be familiar with techniques for temperature stabilization in analog circuits, and will recognize that known methods impose prohibitive complexity upon the achievable temperature-stabilized structures of useful analog neural networks.

[0115] By providing the compensation input signal 405, the analog neural network 105 can be trained 202 ex-situ to reliably operate and perform inferencing across a wide range of temperatures. Temperature is one of the most important operating conditions of an analog neural network. Another important operating condition is its power supply voltage. However, in practice it is possible to regulate the supply voltage to a high degree of precision, in a way that is not possible with temperature.

[0116] The skilled addressee will recognize that the implementation shown in Figure 6 is indicative of structures repeated similarly or identically in multiple parts of the circuit of the analog neural network 105.

[0117] The programming circuit 527 shown in Figure 5 comprises components shown in Figure 6 that enable the memristors to be programmed with their respective resistance values. These resistance values define the neural network parameters represented by weighting elements such as 511 to 515 and bias element 516.

[0118] It will be appreciated that the programming is performed by setting both propagation signal components 505a and 505b to zero volts and switching off a pull-up transistor 621 for the bias element 516. Next, a forward or reverse voltage is supplied to the weighting memristor being programmed. For example, the weighting element 512 is programmed by activating a tri-state buffer 62 and supplying a programming voltage P0 to its input. In order to program the first memristor 607, another tri-state buffer 623 is activated and supplied with a programming voltage PM. Alternatively, the second memristor 608 is programmed by activating a different tri-state buffer 624 and supplying it with a programming voltage PP.

[0119] It will be appreciated that such programming is performed incrementally and by testing the resulting effect on the circuit after programming. A similar process is performed to program the analog bias element 516. Programming of the weighting element 511 associated with the compensation input signal 405 is performed similarly, although these programming components have been omitted for the sake of clarity.

[0120] From the schematic shown in Figure 6, the skilled addressee will be able to implement the additional analog parameter storage array 517 and analog computation circuit 524 shown in Figure 5, including the associated elements of the programming circuit 527.

[0121] Now referring to Figure 7, there is shown an implementation of the analog signal classifier block 425, shown in Figure 4, which takes the form of an analog neuron circuit. Outputs 417 to 424 from the analog signal convolution blocks 406 and 407 are supplied as inputs to an analog parameter storage array 701. This comprises analog weighting elements 702 to 710, and an analog bias element 711 . An analog computation circuit 712 comprises an accumulator 713 to combine the weighted input signals and a ReLU activation function 714 that provides the output 304 of the block 425 and also of the analog neural network 105. The skilled addressee will recognize Figure 7 as an analog neuron, with an additional input for the compensation input signal 405.

[0122] The skilled addressee will be immediately able to reuse elements of the circuit shown in Figure 6 to implement the analog signal classifier block 425. The computation circuit 712 in the classifier block 425 operates identically to the computation circuits 415 and 524 in the convolution blocks 406 and 407, reusing identical circuit structures for summation and activation.

[0123] In some embodiments the compensation input signal 405 is applied to the accumulator 712 directly without being weighted by an analog weighting element 702.

[0124] Now referring to Figure 8, there is shown an embodiment for performing ex-situ training of the analog neural network 105 using a digital model of the analog neural network 105 running on the general-purpose computer 200.

[0125] According to processing step 801 , hyperparameters are defined for training the neural neural network 105. Hyperparameters include the range and rate of variation of operating conditions simulated during training.

[0126] According to processing step 802, a first training sample is selected. This will be a recording of a person saying the activation phrase, selected from hundreds or even thousands of recordings, spoken by hundreds of different people.

[0127] According to processing step 803, operating conditions are specified. When training for the embodiment of Figure 4, the operating conditions will be one of a randomly selected temperature at which the neural network 105 is expected to operate reliably.

[0128] Operating conditions also include variations in manufactured characteristics of the components of the analog neural network. These operating conditions do not change at all during the operation of the chip, or change extremely slowly. For example, it is known that transistors, such as transistor 601 , will vary in their characteristics, from chip to chip, and even in similar circuits on the same chip. These operating conditions are varied during training, so minimize the network's sensitivity to them. However, it will be appreciated that the ability of a network to exhibit invariance to such manufacturing variations is limited. Subsequent embodiments, described below, provide a solution to this known issue. According to processing step 804, inference is performed using a digital model of the analog neural network 105. The digital model includes modeling of responses to temperature and process variations that are expected to be exhibited by the components of the analog neural network.

[0129] According to processing step 805, the parameters of the neural network 105 are revised using backpropagation. This results in an incremental change in the ability of the analog neural network to perform its task reliably.

[0130] According to processing step 806, a test is performed in order to determine whether an additional training iteration should be performed. This question will be answered according to observations of the rate of improvement and accuracy of the modeled neural network during the inference performed at step 804. If the accuracy is low, or the rate of improvement is high, control is directed back to step 802.

[0131] Eventually, control will be directed to processing step 807, according to which a question is asked as to whether hyperparameters need further refinement. This question will also be answered according to observations of the rate of improvement and accuracy of the modeled neural network during inference performed at step 804. In this case, however, the criteria are more strict. If needed, control is directed back to step 801 , for further refinement of hyperparameters. Eventually, this will not be necessary, and training will be complete.

[0132] The skilled addressee will recognize that the steps of Figure 8 represent a fairly simple iterative approach to training a neural network. In practice, sophisticated statistics may be combined to guide adjustment of hyperparameters, as is known, and the steps of Figure 8 are necessarily simplified to show the essential steps when training a model of the analog neural network 105. The skilled addressee will also recognize that inference step 804 may be performed, at step 205, on the actual neural network itself, while connected to the general-purpose computer 200. Backpropagation, performed at step 805, is carried out in the digital domain, based upon measurements of signals in the analog neural network 105. This hybrid approach allows training to be customized for individual devices, should this be necessary.

[0133] Now referring to Figure 9, there is shown an embodiment for storing neural network parameters in the analog neural network 105, shown at step 203 in Figure 2.

[0134] According to processing step 901 , a first analog weighting element or bias element memristor is selected for programming. According to processing step 902, a test signal is propagated through the respective analog delay line 408 so the respective memristor's resistance can be measured.

[0135] According to processing step 903, a measurement is performed between the memristor's resistance and the target required to correctly represent the neural net parameter weight or bias.

[0136] According to processing step 904, a test is performed in order to determine as to whether there is no significant difference. If so, the memristor has been successfully programmed with the required parameter value. If not, control is directed to processing step 905, where the memristor's resistance is adjusted by applying a programming voltage to the memristor.

[0137] It will be appreciated that the programming voltage is applied using the tristate buffers 622, 623 and 624 shown in Figure 6, and / or other tri-state buffers in other parts of the analog neural network 105. The parameters of the analog neural network 105 are provided by the memristors in weighting elements and bias elements previously described, including weighting elements 511 to 515 and bias element 516. For example, memristors 607 and / or 608 define one parameter of the analog neural network 105. Memristors 609 and 610 define another parameter of the analog neural network 105. After processing step 905, control is directed back to processing step 903, to determ ine whether further adjustment of the selected weighting element or bias element is required. Eventually, control will be directed to processing step 906, where a test is performed to determine as to whether additional weighting elements or a biasing element require programming. If so, control is directed to processing step 901 . Alternatively, that completes the processing steps required for programming the parameters of the analog neural network 105.

[0138] Now referring to Figure 10, there is shown an alternative embodiment in which the compensation input signal 405 is provided by a compensation input register 1001. The compensation input register comprises six non-volatile analog storage elements 1002 to 1007 which provide the compensation input signal to the analog signal convolution blocks 406 and 407. The same compensation input signal 405 is used across multiple different operating conditions, including when the operating conditions change in real time and when the operating conditions remain the same over time.

[0139] The skilled addressee will recognize that a component such as transistor 621 shown in Figure 6 may be fabricated on a silicon chip in such a way as to malfunction. Such malfunction is considered an operating condition of the neural network 105 that requires compensation, and may be advantageously compensated by suitable training of the neural network 105 when such training includes the compensation input register 1001. Other compensated operating conditions may include, but are not limited to, broken memristors, capacitors, resistors, amplifiers, buffers and signal connections within the analog neural network 105.

[0140] The analog delay line 408 of the first analog signal convolution blocks has the same size as shown in Figure 5. The analog neuron circuit 409 comprises a larger parameter storage array 1008 which is required due to the compensation input signal 405 comprising six component signals. A larger analog computation circuit 1009 includes an accumulator 1010 and the same activation function 416 that has been previously described. The analog signal classifier block 425 is similarly modified to process the multiple components of the compensation input signal 405.

[0141] The compensation input register 1001 stores fixed values, which have been determined during training to enable the analog neural network to automatically compensate for temperature and other variations, without temperature, or the other variations, being explicitly measured. The network's learned intelligence modifies the contribution of the unchanging compensation input signal 405, such that variations in one or more operating conditions are compensated for in the analog output signal 304 or one or more of the internal analog output signals 417 to 424.

[0142] The ability of the analog neural network 105 to perform such compensation is the result of the optimization of the fixed voltages produced by the compensation input register 1001. The fixed voltages of the compensation input signal 405 are optimized as a hyperparameter at step 801 in Figure 8, thereby causing the analog neural network to compensate for variations in operating conditions such as temperature and manufacturing tolerances that vary from chip- to-chip.

[0143] Now referring to Figure 11 , there is shown an implementation of the compensation input register 1001 which supplies the compensation input signal 405 comprising six component signals 1101 to 1106. The first non-volatile analog storage element 1002 comprises resistive elements 1107, 1108 arranged as a potential divider whose output is supplied to a buffer 1109. The resistive elements are implemented as memristors 1107, 1108. The other non-volatile analog storage elements 1003 to 1007 are similarly configured. A programming circuit 1110 is provided for programming the memristors 1107, 1108 and the memristors of the other non-volatile analog storage elements 1003 to 1007. The skilled addressee will be able to design a suitable programming circuit 1110 given the example of programming buffers 622, 623 and 624 in Figure 6 and their operation described with reference to the steps shown in Figure 9. In other embodiments, the non-volatile analog storage elements 1002 to 1007 are implemented using charge trap transistors (CCTs).

[0144] Now referring to Figure 12, there is shown an implementation of the neuron 409 for the embodiment shown in Figure 10. The components 1101 to 1106 of the compensation input signal 405 are supplied to respective analog weighting elements 1201 to 1207 whose outputs are provided as inputs to the accumulator 1010. Task input values, provided in their delayed form by the delay line 408, are supplied as inputs 507, 510 to respective analog weighting elements 1207, 1208. A bias element 1209 provides a bias output to the accumulator 1010.

[0145] It will be appreciated that the analog signal transfer block 425 will have a structure very similar to that shown for the neuron 409 in Figure 12, including receiving the components 1101 to 1106 of the compensation input signal 405.

[0146] Now referring to Figure 13, there is shown an alternative embodiment in which the compensation input signal 405 is provided by a compensation input register 1301 updated from an output 304 of the network 105. During a calibration phase, a reference task input 1302 is provided as the task input signal 403 via a task input switch 1303. An inference is then performed by the network 105, based on the reference task input 1302. The resulting output 304 is then used to update the compensation input signal 405 supplied by the compensation input register 1301.

[0147] The purpose of this is best explained by an analogy. Imagine that the reference task input 1302 provides an image, and the analog neural network 105 has been trained to recognize this image. If the image is recognized perfectly, the output 304 is set to one volt. No compensation is required. The neural network has been trained to consider a compensation input signal of one volt as indicative that no compensation is required. Consider next that the neural network 105 is at a high temperature of thirty- five degrees, enough to cause significant degradation of the inference process. The output 304 is now reduced to half a volt, due to the image not being properly recognized. However, the compensation input register 1301 is now set to provide a compensation input signal 405 of half a volt, and the network has been trained to provide compensation accordingly.

[0148] Subsequently, when the task input switch 1303 has been set to select live input signals 303 for the task input 403, temperature compensation will be performed.

[0149] In some embodiments, the input 1304 for the compensation input register 1301 is obtained from an internal output, such as one of the outputs 417 to 424. In some embodiments, the input 1304 for the compensation input register 1301 has multiple components, and the compensation input signal 405 also has multiple components.

[0150] The task reference input 1302 provides a sequence of signal variations over a period of time so that the analog delay line will be able to generate a suitable series of delayed outputs. In an embodiment, the signal variations are supplied simultaneously from the reference task input to the parameter storage array, thereby bypassing the analog delay line 408 and avoiding the need to provide a time-sequenced series of voltages.

[0151] A temperature sensor 1305 is provided so that the microcontroller 307 can determine when calibration needs to be performed.

[0152] In an embodiment, the ability of the analog neural network 105 to perform such compensation has been enhanced by training and optimization of the signal produced by the reference task input 1202. The signal produced by the reference task input is optimized as a hyperparameter at step 801 in Figure 8, thereby causing the analog neural network to optimize its ability to compensate for variations in operating conditions such as temperature and manufacturing tolerances.

[0153] Now referring to Figure 14, there is shown an embodiment for the step 208 of detecting the spoken activation phrase using the embodiment shown in Figure 13.

[0154] According to processing step 1401 , a question is asked as to whether the temperature indicated by the sensor 1305 has changed significantly. If the temperature has changed by more than one degree Celsius, control is directed to step 1402.

[0155] According to processing step 1402 the reference task input 1202 is selected by the task input switch 1303.

[0156] According to processing step 1403 a reference task input signal is generated and supplied to the analog delay line 408. This takes place over a period of time, such that the outputs of the delay line 408 (and other delay line outputs elsewhere in the neural network 105) are fully updated with the required reference task input.

[0157] According to processing step 1404, an inference is performed, based on the reference task input 1302.

[0158] According to processing step 1405, the compensation input signal 405 is updated in response to feedback from an output 304 of the neural network.

[0159] According to processing step 1406, the temperature, given by the sensor 1305, is recorded by the microcontroller 307.

[0160] According to processing step 1407 the task input is selected using the switch 1303 to connect to the output of the buffer 402. According to processing step 1408, an inference is then performed, based on the task input 403.

[0161] According to processing step 1409, a question is then asked as to whether the activation phrase has been detected. If not, control is directed back to step 1401. The steps are repeated, with calibration being performed by steps 1402 to 1406 whenever the temperature has been observed to change. Eventually, the activation phrase is detected, completing the step 208 of detecting a spoken activation phrase.

[0162] Now referring to Figure 15, there is shown a summary of the invention. An analog input interface 401 is configured to supply a task input signal 403 and a compensation input signal 405. The compensation input signal 405 is provided by a compensation input source 1501 . The compensation input source 1501 may non-exclusively be one of a temperature sensor 404, a compensation input register 1001 holding fixed values 1002 to 1007 or a compensation input register 1301 holding temporarily stored values. One or more analog neurons 409 to 412 are connected to the analog input interface, directly via the compensation input signal 405 and indirectly, via the analog delay line 408. Each analog neuron comprises an analog parameter storage array 413 storing a plurality of parameters 511 to 515 of the analog neural network, the analog parameter storage array having a plurality of analog inputs 405, 507 to 510 and comprising a plurality of analog weighting elements 511 to 515, each configured to store an analog weighting factor and to modify one of the task input signals supplied from the analog input interface. Each analog neuron further comprises an analog computation circuit 414, configured to generate an analog output signal 421 based on a combination of contributions by one or more of the modified task input signals and the compensation input signal.

[0163] A plurality of the analog weighting elements 511 to 515 store analog weighting factors that define the relative contribution of the compensation input signal 405, such that variations in the one or more operating conditions 429 are compensated for in the analog output signal generated by at least one of the analog neurons.

[0164] In some embodiments, the stored analog parameters define a neural network structure; in others, they may define a different parameterized analog computation architecture capable of performing similar functions.

[0165] In some embodiments, the compensation input source 1501 comprises one or more additional analog computation circuits, which may comprise one or more neurons in one or more neural layers, configured for generating the compensation input signal 405 from one or more input sources, such as a temperature sensor, a voltage monitor, or other representation of one or more operating conditions of the analog neural network 105, including broken or malfunctioning components such as one or more memristors 607, 608.

[0166] Now referring to Figure 16, there is shown a summary of the method of operating the analog neural network 105.

[0167] According to processing step 1601 an analog task input signal is provided.

[0168] According to processing step 1602 an analog compensation input signal is provided.

[0169] According to processing step 1603, weighted signals are generated by modifying the task input signal 403 using respective analog weighting factors 512 to 515.

[0170] According to processing step 1604, a corresponding analog output 421 is generated based on a combination of contributions from the weighted signals derived from the task input signal 403 and the compensation input signal 405.

[0171] According to processing step 1605, the output signal generated at step 1604 is provided to another neuron 425 or to the comparator 305.

[0172] The skilled addressee will appreciate that the operating conditions of the analog neural network include fixed operating conditions, such as the geometries of individual components manufactured as part of a silicon chip, as well as unfixed operating conditions, such as temperature and supply voltage. Other unfixed operating conditions include radiation, which is an important factor in telecommunications satellites. Fixed and unfixed operating conditions can be compensated by one or more of the embodiments that have been described, provided suitable training is performed in accordance with the steps of Figure 8, their equivalent, or any version of the training process that has been described.

Claims

CLAIMS:1 . An analog neural network apparatus for operating in the presence of one or more operating conditions affecting a plurality of analog components associated with the analog neural network apparatus used to generate an output of the analog neural network, comprising: an analog input interface configured to: supply a task input signal; and supply a compensation input signal; a plurality of analog neurons connected to the analog input interface, each analog neuron comprising: an analog parameter storage array storing a plurality of parameters of the analog neural network, the analog parameter storage array having a plurality of analog inputs and comprising a plurality of analog weighting elements, each configured to store an analog weighting factor and to modify a task input signal supplied from the analog input interface; an analog computation circuit, configured to generate an analog output signal based on a combination of contributions by: one or more of the modified task input signals; and the compensation input signal; wherein a plurality of the analog weighting elements store analog weighting factors that define the relative contribution of the compensation input signal, such that variations in the one or more operating conditions are compensated for in the analog output signal generated by at least one of the analog neurons.

2. The apparatus of claim 1 , wherein the compensation input signal is applied to an accumulator in the analog computation circuit without being weighted by an analog weighting element.

3. The apparatus of any one of claims 1 or 2, further including an analog compensation input register for generating the compensation input signal as a fixed signal maintained constant during multiple inference operations of the analog neural network.

4. The apparatus of claim 3, wherein the analog compensation input register comprises a plurality of compensation input elements for generating the compensation input signal as a plurality of analog fixed signals.

5. The apparatus of claim 4, wherein the compensation input elements comprise resistive elements.

6. The apparatus of any one of claims 1 to 5, further comprising a programming circuit for the resistive elements.

7. The apparatus of any one of claims 5 or 6, wherein the resistive elements comprise memristors.

8. The apparatus of any one of claims 4 to 7, wherein the operational characteristics of the compensation input elements have been defined in response to ex-situ training of a model of the apparatus to improve the apparatus's compensation for variations in the one or more operating conditions.

9. The apparatus of any one of claims 1 to 8, further including a temperature sensor for supplying the compensation input signal.

10. The apparatus of any one of claims 1 to 9, wherein a plurality of the analog neurons are configured to form a multi-stage analog signal processing path from the task input signal to the corresponding analog output signal, and wherein the compensation input signal influences at least two analog computation circuits located at different stages along the signal processing path.11 . The apparatus of any one of claims 1 to 10, wherein each analog weighting factor is stored as a resistance representing a learned network parameter.

12. The apparatus of any one of claims 1 to 11 , further comprising an analog delay line for modifying the task input signal supplied to at least one of the analog parameter storage arrays, the analog delay line comprising a plurality of memory cells for propagating the task input signal through the memory cells in a time- sequenced manner under control of a propagation signal, the propagated task input signal comprising a plurality of propagated task input signals, each of the propagated task input signals being supplied from a respective memory cell of the analog delay line.

13. The apparatus of claim 12, further configured to supply a reference task input signal from the analog input interface during a calibration operation, wherein the reference task input signal is propagated through the analog delay line during the calibration operation.

14. The apparatus of any one of claims 1 to 13, further including an compensation input register configured for storing an analog output of the apparatus during a calibration operation of the apparatus and for supplying the stored analog output as a compensation input signal during a subsequent inference operation of the apparatus.

15. The apparatus of claim 14, further configured to supply a reference task input signal from the analog input interface during the calibration operation.

16. The apparatus of any one of claims 1 to 15, wherein at least one of the analog computation circuits comprises a rectified linear unit (ReLU).

17. The apparatus of any one of claims 1 to 16, wherein the analog weighting factors have been determined through training in response to simulated variation of the one or more operating conditions.

18. A method for processing an analog input signal and generating a corresponding analog output signal based on stored analog neural network parameters in the presence of variations in one or more operating conditions affecting a plurality of the analog components used to generate the analog output signal, the method comprising: providing an analog task input signal; supplying an analog compensation input signal; generating a plurality of weighted signals by modifying the task input signal in response to a plurality of analog weighting factors previously stored in one or more analog parameter storage arrays; generating the corresponding analog output signal based on a combination of contributions by: the weighted signals derived from the task input signal; and the compensation input signal; and providing the generated analog output signal.

19. The method of claim 18, wherein the compensation input signal is applied to an accumulator in the analog computation circuit without being weighted by an analog weighting element.

20. The method of any one of claims 18 or 19, wherein the compensation input signal is maintained constant across multiple inference operations.21 . The method of any one of claims 18 to 20, wherein the compensation input signal comprises a plurality of analog fixed signals.

22. The method of any one of claims 18 to 21 , including supplying the compensation input signal from a temperature sensor.

23. The method of any one of claims 18 to 23, wherein the task input signal is propagated through an analog delay line in a time-sequenced manner.

24. The method of any one of claims 18 to 23, wherein the compensation input signal is based on an analog output signal generated by the method during a calibration operation performed prior to an inference operation.

25. The method of any one of claims 18 to 24, further comprising supplying a reference task input signal to the analog neural network during a calibration operation, and generating the compensation input signal based on a resulting output signal.

26. The method of any one of claims 18 to 25, wherein the analog weighting factors were obtained through training of a model of the analog neural network under simulated variation of the one or more operating conditions.

27. The method of any one of claims 18 to 26, wherein the same compensation input signal is used across a plurality of different operating conditions.

28. The method of any one of claims 18 to 27, further comprising generating the compensation input signal in response to detecting a change in the operating condition.

29. The method of any one of claims 18 to 28, wherein generating the analog output signal comprises applying a non-linear combination of the contributions.

30. The method of claim 29, wherein the non-linear combination comprises a rectified linear unit (ReLU) function.

Images