Physical random value generation using a resistive memory device

Abstract

Physical random value generation using a resistive memory device The present disclosure relates to a device configured to generate a random value, the device comprising: a resistive memory device (102) capable of being incrementally programmed; and a control circuit (104) configured to: - perform a first programming operation of the resistive memory device with first programming parameters in order to generate a first conductance variation of a first polarity in the resistive memory device; - perform a second programming operation of the resistive memory device with second programming parameters in order to generate a second conductance variation of a second polarity in the resistive memory device, the second polarity being the opposite polarity to the first polarity; and - generate the random value by reading a conductance value of the resistive memory device following the first and second programming operations.

Description

DESCRIPTIONTITLE : Physical random value generation using a resistive memory deviceTechnical field

[0001] The present disclosure relates generally to the field of electronic devices , and in particular to physical random value or number generation, and to machine learning using such physical random value or number generation .Background art

[0002] Random number generation is employed in diverse applications from cryptography to machine learning . Some solutions to random number generation are based on physical devices having variable behavior . However, a di f ficulty with physical devices that have been proposed in the past for random number generation is that they tend to lack scalability for applications in which a relatively large number of random numbers should be generated in parallel , and / or they have limitations in terms of tuning the parameters of the probability distribution of the generated random numbers , such as the mean and standard deviation of the probability distribution .

[0003] There is thus a need in the art for a solution for physical random value or number generation that at least partially addresses one or more drawbacks in the prior art .Summary of Invention

[0004] According to one aspect , there is provided a device configured to program a random value , the device comprising : a resistive memory device capable of being incrementally programmed; and a control circuit configured to program the resistive memory device with the random value , the control circuit beingconfigured to :- perform a first programming operation of the resistive memory device with the first programming parameters in order to generate a first conductance variation of a first polarity in the resistive memory device ;- perform a second programming operation of the resistive memory device with the second programming parameters in order to generate a second conductance variation of a second polarity in the resistive memory device , the second polarity being the opposite polarity to the first polarity; and wherein either :- a magnitude of the first programming parameters is di f ferent to a magnitude o f the second programming parameters , and the control circuit being further configured to read the random value programmed in the resistive memory device by reading a programmed conductance value of the resistive memory device following the first and second programming operations ; or- a magnitude of the first programming parameters is the same as a magnitude of the second programming parameters , the control circuit being further configured to perform a third programming operation of the resistive memory device before the first programming operation or after the second programming operation, the random value programmed in the resistive memory device being read by reading a conductance value of the resistive memory device following the first , second and third programming operations .

[0005] According to another aspect , there is provided a device configured to generate a random value , the device comprising : a resistive memory device capable of being incrementally programmed; and a control circuit configured to : perform a first programming operation of the resistive memory device with first programming parameters in order to generate a first conductance variation of a first polarity inthe resistive memory device ; perform a second programming operation of the resistive memory device with second programming parameters in order to generate a second conductance variation of a second polarity in the resistive memory device , the second polarity being the opposite polarity to the first polarity; and generate the random value by reading a conductance value of the resistive memory device following the first and second programming operations .

[0006] According to one embodiment , the resistive memory device is capable of being incrementally programmed to have a conductance level in a range from a minimum conductance level to a maximum conductance level , and wherein the conductance value of the resistive memory device following the first and second programming operations is greater than said minimum conductance level and less than said maximum conductance level , the first and second conductance variations each for example being less than a di f ference between the minimum and maximum conductance levels .

[0007] According to one embodiment , the control circuit is further configured to perform analog to digital conversion on the read conductance value in order to generate a digital random value , the digital random value for example having a bit-length of at least two .

[0008] According to one embodiment , a magnitude of the first programming parameters is di f ferent to a magnitude of the second programming parameters .

[0009] According to one embodiment , the resistive memory device is a domain-wall device , and a pulse-width, or a number of pulses , applied to the domain-wall device during the first programming operation is di f ferent from a pulse-width, or a number of pulses , applied to the domain-wall device during the second programming operation .

[0010] According to one embodiment , the control circuit is configured to implement a Bayesian model update operation, wherein the magnitude of one of the first or second programming operations is based on a magnitude and sign of a Langevin gradient .

[0011] According to one embodiment , the magnitude of the first programming parameters is di f ferent to the magnitude of the second programming parameters , and the magnitude of said one of the first or second programming operations is based on, and for example proportional to a programming magnitude equal to :

[0012] [Math 1 ]

[0013] whereis a Langevin gradient , gtis a learning rate , and (p is a function that converts the noise introduced by the application of the Langevin gradient into a programming magnitude .

[0014] According to one embodiment , a magnitude of the first programming parameters is the same as a magnitude of the second programming parameters , the control circuit further being configured to perform a third programming operation of the resistive memory device before the first programming operation or after the second programming operation, the random value programmed in the resistive memory device being read by reading a conductance value of the resistive memory device following the first , second and third programming operations .

[0015] According to one embodiment , the control circuit is configured to implement a Bayesian model update operation, wherein the magnitude of said third programming operation is based on a magnitude of a Langevin gradient , and the polarityof third programming operation is based on a sign of the Langevin gradient .

[0016] According to one embodiment , the control circuit is configured to :- perform a fourth programming operation of the resistive memory device with fourth programming parameters in order to generate , with respect to the programmed conductance , a fourth conductance variation of a fourth polarity in the resistive memory device , the fourth polarity being equal to the first or second polarity;- perform a fi fth programming operation of the resistive memory device with fi fth programming parameters in order to generate a fi fth conductance variation of a fi fth polarity in the resistive memory device , the fi fth polarity being the opposite polarity to the fourth polarity; and wherein either : a magnitude of the fourth programming parameters is di f ferent to a magnitude of the fi fth programming parameters , and the control circuit is further configured to read the random value programmed in the resistive memory device by reading a programmed conductance value of the resistive memory device following at least the first , second, fourth and fi fth programming operations ; or- a magnitude o f the fourth programming parameters is the same as a magnitude of the fi fth programming parameters , the control circuit being further configured to perform a sixth programming operation of the resistive memory device before the fourth programming operation or after the fi fth programming operation, the random value programmed in the resistive memory device being read by reading a conductance value of the resistive memory device following at least the first , second, fourth, fi fth and sixth programming operations

[0017] According to one embodiment , the device further comprises a second resistive memory device capable of beingincrementally programmed, wherein the control circuit is further configured to program the second resistive memory device with a second random value , the control circuit being configured to : perform a first programming operation of the second resistive memory device with seventh programming parameters in order to generate a first conductance variation of a seventh polarity in the second resistive memory device , the seventh polarity being equal to the first or second polarity; perform a second programming operation of the second resistive memory device with eighth programming parameters in order to generate a second conductance variation of an eighth polarity in the second resistive memory device , the eighth polarity being the opposite polarity to the seventh polarity; and wherein either : a magnitude of the seventh programming parameters is di f ferent to a magnitude of the eighth programming parameters , and the control circuit is further configured to read the second random value programmed in the resistive memory device by reading a programmed conductance value of the second resistive memory device following the first and second programming operations of the second resistive memory device ; or- a magnitude of the seventh programming parameters is the same as a magnitude of the eighth programming parameters , the control circuit being further configured to perform a third programming operation of the second resistive memory device before the first programming operation or after the second programming operation, the second random value programmed in the second resistive memory device being read by reading a conductance value of the second resistive memory device following the first , second and third programming operations of the second resistive memory device .

[0018] According to a further aspect , there is provided a method of programming and reading a random value , the method comprising : programming, by a control circuit , a resistive memory device with the random value , the res istive memory device being capable of being incrementally programmed, wherein programming the resistive memory device comprises :- performing, by the control circuit , a first programming operation of a resistive memory device with the first programming parameters in order to generate a first conductance variation of a first polarity in the resistive memory device ; and- performing, by the control circuit , a second programming operation of the resistive memory device with the second programming parameters in order to generate a second conductance variation of a second polarity in the resistive memory device , the second polarity being the opposite polarity to the first polarity; and wherein either :- a magnitude of the first programming parameters is di f ferent to a magnitude of the second programming parameters , the method further comprising reading the random value programmed in the resistive memory device by reading, by the control circuit , a conductance value of the resistive memory device following the first and second programming operations ; or- a magnitude of the first programming parameters is the same as a magnitude of the second programming parameters , the method further comprising performing a third programming operation of the resistive memory device before the first programming operation or after the second programming operation, the random value programmed in the resistive memory device being read by reading, by the control circuit , aconductance value of the resistive memory device following the first , second and third programming operations .

[0019] According to a further aspect , there is provided a method of random value generation, the method comprising : performing, by a control circuit , a first programming operation of a resistive memory device with first programming parameters in order to generate a first conductance variation of a first polarity in the resistive memory device , wherein the resistive memory device is capable of being incrementally programmed; performing, by the control circuit , a second programming operation of the resistive memory device with second programming parameters in order to generate a second conductance variation of a second polarity in the resistive memory device , the second polarity being the opposite polarity to the first polarity; and generating the random value by reading, by the control circuit , a conductance value of the resistive memory device following the first and second programming operations .

[0020] According to one embodiment , the resistive memory device is capable of being incrementally programmed to have a conductance level in a range from a minimum conductance level to a maximum conductance level , and wherein the conductance value of the resistive memory device following the first and second programming operations is greater than said minimum conductance level and less than said maximum conductance level , the first and second conductance variations each for example being less than a di f ference between the minimum and maximum conductance levels .

[0021] According to one embodiment , the method further comprises performing analog to digital conversion on the read conductance value in order to generate a digital random value ,the digital random value for example having a bit-length of at least two.

[0022] According to one embodiment, a magnitude of the first programming parameters is different to a magnitude of the second programming parameters.

[0023] According to one embodiment, the resistive memory device is a domain-wall device, and a pulse-width, or a number of pulses, applied to the domain-wall device during the first programming operation is different from a pulse-width, or a number of pulses, applied to the domain-wall device during the second programming operation.

[0024] According to one embodiment, the method further comprises :- performing a fourth programming operation of the resistive memory device with fourth programming parameters in order to generate, with respect to the programmed conductance, a fourth conductance variation of a fourth polarity in the resistive memory device, the fourth polarity being equal to the first or second polarity;- performing a fifth programming operation of the resistive memory device with fifth programming parameters in order to generate a fifth conductance variation of a fifth polarity in the resistive memory device, the fifth polarity being the opposite polarity to the fourth polarity; and wherein either: a magnitude of the fourth programming parameters is different to a magnitude of the fifth programming parameters, and the method further comprising reading the random value programmed in the resistive memory device by reading a programmed conductance value of the resistive memory device following at least the first, second, fourth and fifth programming operations; or- a magnitude of the fourth programming parameters is thesame as a magnitude of the fi fth programming parameters , the method further comprising performing a sixth programming operation of the resistive memory device before the fourth programming operation or after the fi fth programming operation, the random value programmed in the resistive memory device being read by reading a conductance value of the resistive memory device following at least the first , second, fourth, fi fth and sixth programming operations .

[0025] According to one embodiment , the device further comprises a second resistive memory device capable of being incrementally programmed, wherein the method further comprises programming the second resistive memory device with a second random value , the method comprising :- performing a first programming operation of the second resistive memory device with seventh programming parameters in order to generate a first conductance variation of a seventh polarity in the second resistive memory device , the seventh polarity being equal to the first or second polarity;- performing a second programming operation of the second resistive memory device with eighth programming parameters in order to generate a second conductance variation of an eighth polarity in the second resistive memory device , the eighth polarity being the opposite polarity to the seventh polarity; and wherein either : a magnitude of the seventh programming parameters is di f ferent to a magnitude of the eighth programming parameters , and the method further comprises reading the second random value programmed in the resistive memory device by reading a programmed conductance value of the second resistive memory device following the first and second programming operations of the second resistive memory device ; or- a magnitude of the seventh programming parameters is the same as a magnitude of the eighth programming parameters , andthe method further comprises performing a third programming operation of the second resistive memory device before the first programming operation or after the second programming operation of the second resistive memory device , the second random value programmed in the second resistive memory device being read by reading a conductance value of the second resistive memory device following the first , second and third programming operations of the second resistive memory device .

[0026] According to a further aspect , there is provided a method of implementing a Bayesian model update operation comprising the above method, wherein the magnitude of one of the first or second programming operations is based on a magnitude and sign of a Langevin gradient .

[0027] According to one embodiment , a magnitude of the first programming parameters is the same as a magnitude of the second programming parameters , the method further comprising, performing, by the control circuit , a third programming operation of the resistive memory device before the first programming operation or after the second programming operation, the random value programmed in the resistive memory device being read by reading a conductance value of the resistive memory device following the first , second and third programming operations .

[0028] According to yet a further aspect , there is provided a method of implementing a Bayes ian model update operation comprising the above method, wherein the magnitude of said third programming operation is based on a magnitude of a Langevin gradient , and the polarity of third programming operation is based on a sign of the Langevin gradient .Brief description of drawings

[0029] The foregoing features and advantages , as well as others , will be described in detail in the followingdescription of speci fic embodiments given by way of illustration and not limitation with reference to the accompanying drawings , in which :

[0030] Figure 1 schematically illustrates a random value generation device according to an example embodiment of the present disclosure ;

[0031] Figure 2 shows graphs illustrating an example of conductance variations of a resistive memory device of the device of Figure 1 ;

[0032] Figure 3 is a graph illustrating an example of a probability distribution for a change in position of a domainwall of the resistive memory device of Figure 1 ;

[0033] Figure 4 is a flow diagram illustrating operations in a method of generating a random value according to an example embodiment of the present disclosure ;

[0034] Figure 5 shows graphs illustrating an example of a probability distribution for a change in position of a domainwall in the case that programming operations of opposite polarities are applied to the resistive memory device ;

[0035] Figure 6 shows graphs illustrating another example of a probability distribution for a change in position of a domain-wall in the case that programming operations of opposite polarities are applied to the resistive memory device ;

[0036] Figure 7 shows graphs illustrating another example of a probability distribution for a change in position of a domain-wall in the case that programming operations of opposite polarities , and a further programming operation, are applied to the resistive memory device ;

[0037] Figure 8 shows graphs illustrating another example of a probability distribution for a change in position of adomain wall in the case that programming operations of opposite polarities and of di f ferent durations are applied to the resistive memory device ;

[0038] Figure 9 is a graph illustrating an example of an SGLD ( Stochastic Gradient Langevin Dynamics ) algorithm involving random sampling;

[0039] Figure 10 schematically illustrates a device capable of machine learning according to an example embodiment of the present disclosure ;

[0040] Figure 11A is a flow diagram illustrating operations in a method of implementing an SGLD model update operation according to an example embodiment of the present disclosure ;

[0041] Figure 11B shows graphs illustrating an SGLD model update operation according to an example embodiment of the present disclosure ;

[0042] Figure 12 is a graph representing examples of the Langevin gradient and of Gaussian samples according to an example embodiment of the present disclosure ;

[0043] Figure 13 schematically illustrates an array of domain-wall resistive memory devices according to an example embodiment of the present disclosure ;

[0044] Figure 14 is a graph representing probability distributions of discrete levels of a domain wall device ;

[0045] Figure 15A is a flow diagram illustrating operations in a machine learning algorithm based a MCMC according to a speci fic example in which it is implemented using a domainwall device ;

[0046] Figure 15B is a flow diagram illustrating operations in a machine learning algorithm based a MCMC according to an alternative example of that of Figure 15A, again implemented using a domain-wall device ;

[0047] Figure 16 is a flow diagram illustrating a method of generating a prediction according to an example embodiment of the present disclosure;

[0048] Figure 17 illustrates an example of in and out of distribution data;

[0049] Figure 18 is a graph showing results as a function of the precision of a DW device;

[0050] Figure 19 is a graph plotting the effect of an array size; and

[0051] Figure 20 is a graph illustrating the performance of in / out-of-distribution detection of a model.Description of embodiments

[0052] Like features have been designated by like references in the various figures. In particular, the structural and / or functional features that are common among the various embodiments may have the same references and may dispose identical structural, dimensional and material properties.

[0053] Unless indicated otherwise, when reference is made to two elements connected together, this signifies a direct connection without any intermediate elements other than conductors, and when reference is made to two elements coupled together, this signifies that these two elements can be connected or they can be coupled via one or more other elements .

[0054] In the following disclosure, unless indicated otherwise, when reference is made to absolute positional qualifiers, such as the terms "front", "back", "top", "bottom", "left", "right", etc., or to relative positional qualifiers, such as the terms "above", "below", "higher", "lower", etc., or to qualifiers of orientation, such as"horizontal", "vertical", etc., reference is made to the orientation shown in the figures.

[0055] Unless specified otherwise, the expressions "around", "approximately", "substantially" and "in the order of" signify within 10 %, and preferably within 5 %.

[0056] Figure 1 schematically illustrates a random value generation device 100 according to an example embodiment of the present disclosure. The device 100 for example comprises a resistive memory device 102 and a control circuit 104 (CTRL CIRCUIT) .

[0057] The resistive memory device 102 is for example of the type that is capable of being incrementally programmed. This means in particular that a programming operation can be used to modify the conductance from an initial conductance value, and the resulting conductance is a function of the magnitude and polarity of a given parameter of the programming operation and also of the initial conductance value. Examples of resistive memory devices having this property include the following :1) magnetic domain-wall (DW) devices, as described for example in the publications: Kumar, Durgesh, et al. "Domain wall memory: Physics, materials, and devices." Physics Reports 958 (2022) : 1-35; and Sbiaa, R., and S. N. Piramanayagam. "Multilevel domain wall memory in constricted magnetic nanowires." Applied Physics A 114 (2014) : 1347-1351; and2) Phase Change Memory as described in the publication: La Barbera, Selina, et al. "Narrow Heater Bottom Electrode-Based Phase Change Memory as a Bidirectional Artificial Synapse", Adv. Electron. Mater. 2018, 4, 1800223.

[0058] The resistive memory device 102 is for example capable of being incrementally programmed to have a conductance level in a range from a minimum conductance level to a maximumconductance level , the minimum and maximum conductance levels being respectively the lowest and higher conductance levels of the device 102 that can be programmed . For example , the minimum and maximum conductance levels are characteristics of the device 102 , and correspond to inherent limits of the device 102 that will not be exceeded whatever the programming operations that are applied to the device 102 . Furthermore , the resistive memory device 102 is capable of being programmed to one or more conductance levels that fall between the minimum and maximum conductance levels , that is one or more values that are higher than the minimum conductance level and lower than the maximum conductance level . In some embodiments , the resistive memory device 102 is capable of being programmed to have any of one or more discrete conductance levels falling between the minimum and maximum conductance levels , such that the programmed conductance level represents more than 1 bit of data . In other embodiments , the resistive memory device 102 is capable of being programmed to have any analog conductance falling between the minimum and maximum conductance levels .

[0059] Figure 1 illustrates an example in which the device 102 is a DW device , which for example comprises two stacked magnetic layers 106 and 108 . A DW device is a non-volatile memory, that may be multi-level . The device has three electrodes 110 , 112 , 114 , each of which is for example coupled to the control circuit 104 .

[0060] The electrode 110 is in contact with the upper "pinned" layer 106 , which comprises only magnetic spin up, as represented by upward pointing arrows in the layer 106 . For example , the electrode 110 contacts an upper surface of the layer 106 .

[0061] The electrodes 112 , 114 contact the bottom " free" layer 108 , and for example respectively contact the lateral extremities of the bottom layer 108 . The bottom layer 108 comprises a portion having magnetic spin up, as represented by upward pointing arrows in the layer 108 , and a portion having magnetic spin down, as represented by downward pointing arrows in the layer 108 . A domain-wall ( DW) corresponds to the interface between the spin up and spin down portions . In the example of Figure 1 , the magnetic spin up portion extends from the electrode 112 to the domain-wall DW, and the magnetic spin down portion extends from the electrode 114 to the domain wall DW .

[0062] By passing, using the control circuit 104 , at least one electrical current pulse between the electrodes 112 and 114 , in one direction or the other, the position of the domain-wall DW between magnetic spin up and magnetic spin down portions can be moved in a linear fashion towards the electrode 112 , or towards the electrode 114 , symmetrically . In particular, the polarity of this at least one current pulse determines the direction in which the domain-wall DW moves . For example , during a write operation, the control circuit 104 is configured to couple the electrode 114 to ground, to leave the electrode 110 floating or at high impedance , and to apply at least one positive or negative write current pulse to the electrode 112 .

[0063] By applying, using the control circuit 104 , a voltage between one of the electrodes 112 , 114 and the top electrode 110 , a current that passes through the DW device can be read, this current depending on the extent of tunneling magnetoresistance ( TMR) between the two layers 106 , 108 . The TMR itsel f depends on the ratio of spin UP in the upper fixed layer 106 and spin UP in the bottom free layer 108 . Therefore , by moving the position of the domain-wall DW, the conductanceof the device can be modified in a linear fashion. This allows the device to store data in a non-volatile fashion. For example, to perform a read operation, the control circuit 104 is configured to couple the top electrode 110 to ground, to leave the electrode 114 floating or at high impedance, and to apply a read pulse to the electrode 112. The conductance of the device between the electrode 112 and the top electrode 110 is then for example proportional to the read current.

[0064] For example, assuming that the domain-wall DW is in an initial position represented by a dashed line in Figure 1, the application of a current pulse between the electrodes 112, 114 for example causes a shift AX of the domain-wall, and this shift can be positive or negative depending on the polarity of the current. For example, a positive current pulse applied through the layer 108 from the electrode 112 to the electrode 114 causes the domain-wall DW to shift towards the electrode 114, and a negative current pulse applied through the layer 108 from the electrode 112 to the electrode 114, which is equivalent to a positive current pulse applied from the electrode 114 to the electrode 112, causes the domainwall DW to shift towards the electrode 112.

[0065] Figure 2 is a graph illustrating an example of conductance (g) variations of a resistive memory device of the device of Figure 1 as a function of time t, and in view of current pulses applied to the device.

[0066] For example, the conductance g of the device 102 is initially at a level gl . At a time tl, a negative current pulse pl is applied to electrode 112, causing the domain-wall DW to shift towards the electrode 112, and thus the conductance to decrease to a level g2. At a time t2, a negative current pulse p2, which is similar in magnitude and width to the pulse pl, is applied to electrode 112, causing the domainwall DW to shi ft again towards the electrode 112 , and thus the conductance to decrease to a level g3 . At a time t3 , a positive current pulse p3 is applied to electrode 112 , causing the domain wall DW to shi ft towards the electrode 114 , and the conductance to increase . For example , the pulse p3 has a similar magnitude to the pulses pl and p2 , but twice the width, and thus the conductance increases to a level close to the initial conductance level gl .

[0067] However, the change in the position of the domain-wall DW in response to a current pulse is subj ect to a variety of variability sources . Therefore , it is more accurate to describe the position change , and thus the change in conductance , using a probability distribution, as will now be described with reference to Figure 3 .

[0068] Figure 3 is a graph illustrating an example of a probability distribution p (AX) for a change in position AX of the domain wall DW of the resistive memory device 102 of Figure 1 in response to a positive current pulse of a given pulse-width and magnitude . The distribution may for example extend between a minimum displacement AXmin and a maximum displacement Xmax , and has a single peak, at a displacement Xpeak .

[0069] It has been found that the shape of the probabil ity distribution is impacted by the pulse-width and magnitude of the programming current pulse . For example , for a given magnitude of programming current , the greater the pulse-width, the greater the width of the probability distribution, in other words the distance between AXmin and AX max .

[0070] It has also been found that the probability distribution is symmetric as regards the polarity of the current pulse . In other words , the same probability distribution will be obtained for a negative pulse as for apositive pulse , the di f ference being that the displacement of the domain-wall will be by a negative rather than positive of fset .

[0071] Figure 4 is a flow diagram illustrating operations in a method 400 of generating a random value , centered on a current value , according to an example embodiment of the present disclosure . The method of Figure 4 is for example implemented by the control circuit 104 coupled to the resistive memory device 102 .

[0072] The control circuit 104 is for example configured to program the resistive memory device 102 with the random value by setting at least first and second programming parameters based on a desired probability distribution of the random value , and then performing first and second programming operations based on the first and second programming parameters , as detailed below . For example , the magnitude of at least the first and second programming parameters , and in some cases also of third programming parameters of a third programming operation, are used to control the probability distribution . For example , the magnitudes of the first and second programming parameters , and in some cases also the third programming parameters , result in a selection of the probability distribution from which the random value is generated .

[0073] In an operation 401 ( FIRST PROGRAMMING OPERATION OF FIRST POLARITY) , the first programming operation of the resistive memory device 102 is performed based on the first programming parameters in order to create a conductance variation of a first polarity between first and second electrodes of the device 102 , for example between the electrodes 110 and 114 in the example of Figure 1 . The term "polarity" is used when referring to the conductance variationin order to designate a conductance increase or a conductance decrease . For example , in the case of the domain-wall device , the programming parameters define the pulse-width, magnitude and polarity of the current pulse , and in some cases the number of current pulses , applied between the electrodes 112 and 114 of the device , and the conductance variation is the result of the position variation of the domain wall . In other technologies of resistive memory, the programming parameters may define other variables , such as a voltage magnitude .

[0074] In an operation 402 ( SECOND PROGRAMMING OPERATION OF SECOND POLARITY) , the second programming operation of the resistive memory device 102 is performed based on the second programming parameters in order to create a conductance variation of a second polarity between the first and second electrodes of the device 102 , for example between the electrodes 110 and 114 in the example of Figure 1 . The second polarity is opposite to the first polarity . In other words , i f the conductance variation of the first polarity is a conductance increase ( corresponding to a decrease in the resistance ) , the conductance variation of the second polarity is a conductance decrease ( corresponding to an increase in the resistance ) , and vice versa .

[0075] In some embodiments , the first and second programming parameters are equal and opposite , such that the probability distributions will also be equal and opposite . For example , in the case of a DW device , both of the first and second programming operations are based on current pulses of the same pulse-width and same magnitude , but one of the pulses is positive , and the other is negative . In this case , a third programming operation is for example applied to generate a third conductance variation, for example in order to achieve an intended bias between the conductance variations , in otherwords to apply a further a delta amount 6 to the resulting probability distribution .

[0076] Alternatively, as described in more detail below, in some embodiments there may be an o f fset between the first and second programming parameters in order to achieve the intended bias between the conductance variations .

[0077] In an operation 403 (READ RANDOM VALUE PROGRAMMED BY RES ISTIVE MEMORY DEVICE ) , the random value is read by reading the programmed conductance between the first and second electrodes of the resistive memory device 102 , for example between the electrodes 110 and 114 in the example of Figure 1 . In some embodiments , the read random value is converted from an analog value , such as a voltage or current level , into a digital value by an analog to digital converter (not illustrated) that for example forms part of the control circuit 104 . In some embodiments , the digital random value has a bit-length of at least two bits , and preferable of at least 6 or 8 bits .

[0078] For example , the conductance value of the resistive memory device 102 following the first , second, and optionally third, programming operations is greater than the minimum conductance level of the device 102 and less than the maximum conductance level of the device . Furthermore , the first and second, and optionally third, conductance variations are each for example less than a di f ference between the minimum and maximum conductance levels , in other words these operations do not simply result in a change of the conductance of the device 102 between binary states .

[0079] In some embodiments , the operations 401 to 402 are repeated one or more times in order to program the resistive memory device with new random values prior to the read operation 403 . During each repetition of the operation 401 ,402 , the programming parameters are for example di f ferent from those applied during the first pair of programming operations , such that the new conductance value is based on a di f ferent probability distribution to that of the first pair of programming operations . Furthermore , in each repetition, the resistive memory device is incrementally programmed, meaning that the conductance variation is a variation with respect to the latest conductance that was programmed .

[0080] In further embodiments , the operations 401 to 403 are repeated for one or more further resistive memory devices that are similar to the device 102 , in order to program the one or more further resistive memory devices with corresponding random values . The programming parameters applied during the operations 401 and 402 , and optionally during a third programming operation, for each of the one or more further resistive memory devices , are for example di f ferent from those applied during the programming operations of the resistive memory device 102 , such that the programmed conductance value of each of the one or more further resistive memory devices is based on a di f ferent probability distribution to that of the programming operations of the first device .

[0081] Advantageously, the random value programmed to the resistive memory device 102 will occur as a function of a probability distribution having the shape of a normal distribution, in other words a Gaussian distribution, having a mean value and standard deviation that can be independently controlled using the first and second programming parameters , as will now be described in more detail with reference to Figures 5 to 8 .

[0082] Figures 5 to 8 illustrate graphs of resulting probability distributions based on the example of programming a domain-wall memory device with a current pulse of varying pulse-width . It will be apparent to those skilled in the art that similar ef fects could be achieved using other types of resistive memory devices capable of being incrementally programmed, such as a PCM device as referenced above .

[0083] Figure 5 shows graphs illustrating an example of a probability distribution for a change in position of a domainwall in the case that programming operations of opposite polarities are applied to the resistive memory device . Furthermore , the programming parameters are substantially equal and opposite , and in particular the current pulse-widths and magnitudes are substantially the same . In particular, Figure 5 shows a graph 502 of the probability density associated with the first programming operation based on a positive current pulse , which results in a positive displacement (AX + ) of the domain-wall , a graph 504 of the probability density associated with the second programming operation based on a negative current pulse , which results in a negative displacement ( AX — ) of the domain-wall , the negative pulse being of the same pulse-width and magnitude as the positive current pulse , and a graph 506 showing the resulting probability distribution of the two programming operations , which corresponds to the normal summing law . In particular, this probability distribution p (AX) is centered around zero , implying that the mean displacement is the initial position of the domain-wall .

[0084] For example , the sampled domain-wall position x can be expressed as :

[0085] [Math 2 ]

[0086] where x+and x_ are the means of the normal distributions of the positive and negative displacement of the domain wall respectively, <J(T) is a is a function giving the standard deviation of the normal dependent on the pulsewidth T , and x0is the original domain wall position .

[0087] By varying the pulse-widths T that is used for the positive and negative pulse pairs , the standard deviation of the probability distribution used to generate the random number can be determined . Importantly, this standard deviation can be defined independently of the distribution mean .

[0088] Furthermore , for small values of T , it is likely that the ratio of the device conductance range , divided by the minimum achievable standard deviation that can be achieved with the shortest pulse-width, will be relatively high . This for example allows for tighter di stributions , with a smaller standard deviation, which is useful for some applications . For example , according to the SGLD ( Stochastic Gradient Langevin Dynamics ) algorithm described below, the optimal ratio is generally relatively low .

[0089] Figure 6 shows graphs illustrating another example of a probability distribution for a change in position of a domain-wall in the case that programming operations of opposite polarities are applied to the resistive memory device Graphs 602 , 604 and 606 of Figure 6 are similar respectively to the graphs 502 , 504 and 506 of Figure 5 , except that the positive and negative current pulses have a larger pulsewidth in the example of Figure 6 , and thus the resulting probability distribution p (AX) is also wider, with a mean that is still equal to zero .

[0090] In some embodiments , it may be desired that the random value is selected from a probabi lity distribution having a mean of fset with respect to the initial value . Solutions for generating such an of fset are described in relation with Figures 7 and 8 .

[0091] Figure 7 shows graphs illustrating another example of a probability distribution for a change in position of a domain-wall in the case that programming operations of opposite polarities , and a further programming operation, are applied to the resistive memory device . In particular, graphs 702 and 704 illustrate probability distributions for a positive current pulse and a negative current pulse respectively, similar to the graphs 502 and 504 respectively of Figure 5 . Furthermore , after performing the first and second programming operations , a third programming operation is performed, represented by a graph 706 , and involving the application of a further current pulse that of fsets by a delta amount 8 the resulting probability distribution . In the example of Figure 7 , the third programming operation involves applying a further negative current pulse . As shown by a graph 708 , the resulting probability distribution is the sum of the probability distributions of the three graphs 702 , 704 and 706 , and thus the of fset 6 ( OFFSET 6 ) displaces the probability distribution in the negative direction .

[0092] Figure 8 shows graphs illustrating another example of a probability distribution for a change in position of a domain-wall in the case that programming operations of opposite polarities and of di f ferent durations are applied to the resistive memory device . In the example of Figure 8 , only two programming operations are applied, but the pulse widths during the first and second programming operations are di f ferent . Figure 8 illustrates in particular an example of a current pulse 802 ( PUSH) applied during the firstT1 programming operation, and an example of a current pulse 804 ( PULL ) applied during the second programming operation . The current pulses 802 and 804 for example have the same magnitude . The current pulse 802 for example has a pulse-width of E, and the current pulse 804 for example has a pulse-width of E + L . The part of the pulse width E that is common to both of the pulses 802 and 804 for example results in a symmetrical distribution, centered on the current value ( for example a normal probability distribution p (a ) ) represented by graph 806 . The additional part L of the pulse 804 for example results in a probability distribution p ( L ) represented by a graph 808 , which will shi ft the overall probability distribution in a similar fashion to what is illustrated in the graph 708 of Figure 7 .

[0093] The possibility o f defining the standard deviation of the probability distribution independently of the distribution mean renders the solution described herein particularly advantageous in Markov Chain Monte Carlo methods of machine learning . This is because the probability distribution is the same regardless of current domain-wall position . Examples of the application of the described solution to MCMC methods will now be described in more detail with reference to Figures 9 to 15 .

[0094] Figure 9 is a graph illustrating an example of the SGLD algorithm, which involves random sampling . In particular, this algorithm is a type of Markov chain Monte Carlo (MCMC ) sampling approach widely applied in machine learning . It is a probabilistic alternative to approaches involving the fully gradient-based learning of maximum likelihood models .

[0095] According to gradient-based learning, the algorithm is based on a loss function, and the aim is to determine an optimal model that minimi zes the loss . A maj or drawback ofloss-based approaches is that , in the case of low-precision model parameters or parameter updates , the algorithm may not converge towards the optimal model . This is often the case when learning using a resistive memory based model .

[0096] In contrast , according to MCMC learning, the aim is to estimate the probability density of a posterior distribution . In particular, as represented by Figure 9 , MCMC learning is based on determining an approximation of the posterior distribution 902 of the probability density ( PROBABILITY DENS ITY) of the model 0 . To do thi s , from an initial model ( INITIAL MODEL ) , a chain of random samples is generated using locali zed random j umps and guided by the Langevin gradient . Figure 9 illustrates an example of a first random sampling operation in which a calculated weight gradient (WEIGHT GRADIENT ) is added to the initial model , and the weight sample (WEIGHT SAMPLE ) corresponds to a random sampling around the new model (NEW MODEL ) . This process is repeated, for example based on an update equation of the SGLD algorithm, until a posterior approximation of the posterior distribution 902 has been generated . In some cases , the approximation is further aided by rej ection steps according to the Metropolis-adj usted Langevin algorithm (MALA) .

[0097] An advantage of the approach represented by Figure 9 is that the resulting posterior approximation can be used to make accurate predictions , in particular with a small amount of noisy training data, in addition to providing well- calibrated uncertainty estimates related to those predictions

[0098] Figure 10 schematically illustrates a device 1000 capable of Bayesian machine learning according to an example embodiment of the present disclosure . For example , the device 1000 comprises a resistive memory array 1002 (RES ISTIVE MEMORY ARRAY) 1002 , which is an array of resistive memory devices102 as described herein . The array 1002 is for example coupled to a processing device 1004 ( PROCESS ING DEVICE ) , which is configured to control read and write operations to the resistive memory devices of the array 1002 . The processing device 1004 is also for example coupled to a further memory 1006 (MEMORY) , implemented by one or more non-volatile memory devices , such as a hard disk and / or Flash memory, and / or by one or more volatile memory devices , such as RAM (Random Access Memory) , for example SRAM ( Static RAM) or DRAM ( Dynamic RAM) . For example , the processing device 1004 comprises one or more processors configured to execute instructions stored in the memory 1006 and perform calculations such as calculating the partial derivative of a parameter and converting this into a pulse-width, which can be used to program a device . Additionally or alternatively, the functions of the processing device 1004 could at least partially be implemented by hardware , for example by one or more AS ICs (Application Speci fic Integrated Circuit ) .

[0099] The memory 1006 is also for example configured to store a training dataset 1008 ( TRAINING DATASET ) used for training a Bayesian neural network model 1010 .

[0100] As known by those skilled in the art , the parameters of a Bayesian neural network model are represented by a set of N parameter or weight samples having a probability density corresponding to the trained model .

[0101] Parameters of the Bayesian neural network model 1010 resulting from the training operations , such as random samples of the weights of the model 1010 (BAYES IAN NN MODEL (WEIGHTS ) ) , are for example generated via learning and stored in the resistive memory array 1010 . For example , for each parameter of the model 1010 , the resistive memory array 1002 stores N random samples of the parameter generated, for example , asdescribed above , where N is for example equal to at least two , and typically to 50 or more . Furthermore , other parameters of the Bayesian NN model , such as activation vectors calculated by applying activation functions to the inner-product currents from the array or activation mean and standard deviations used to perform batch normali zation, are stored in the memory 1006 (BAYES IAN NN MODEL ( OTHER PARAMETERS ) ) .

[0102] According to another embodiment , the resistive memory array 1002 is used to generate the samples for the model , and then an ADC is used to store these samples in the memory 1006 .

[0103] In some embodiments , the processing device 1004 is also coupled to a control interface 1012 ( CTRL INTERFACE ) . For example , following training, the device 1000 is configured to implement an inference using the trained Bayesian NN model 1010 . The control interface 1012 is for example coupled to one or more sensors (not illustrated) and / or to one or more actuators ( also not illustrated) . The one or more sensors for example comprise one or more image sensors , depth sensors , heat sensors , microphones , or any other type of sensor . For example , the one or more sensors 320 comprise an image sensor having a linear or 2 -dimensional array of pixels . The image sensor is for example a visible light image sensor, an infrared image sensor, an ultrasound image sensor, or an image depth sensor, such as a LIDAR ( Light Detection And Ranging) image sensor . In this case , input data samples captured by the sensors and provided to the processing device 1004 are images , and the processing device 1004 is configured to perform image processing on the images in order to determine one or more actions to be applied via the actuators . The one or more actuators for example comprise a robotic system, such as a robotic arm trained to pull up weeds , or to pick ripe fruit from a tree , an automatic steering or breaking systems in a vehicle , or an electronicactuator, which is for example configured to control the operation of one or more circuits , such as waking up a circuit from sleep mode , causing a circuit to enter into a sleep mode , causing a circuit to generate a text output , to perform a data encoding or decoding operation, etc . According to a further example , the one or more sensors are interior and / or exterior temperature sensors of a building heating and / or cooling system, comprising for example a heat pump as the main energy source . In such a case , the one or more actuators are for example activation circuits that activate the heating or cooling systems .

[0104] While Figure 10 illustrates an example in which training and inference are implemented by a same device , in alternative embodiments these functions could be implemented by separate devices , a first device being used for training in order to generate the random samples of the NN model 1010 stored in the res istive memory array 1002 , and a second device receiving the trained NN model 1010 , and performing inference based on this model including the programmed random samples of the parameters stored in the resistive memory array 1002 , or in a resistive memory array of the second device to which the random samples have been trans ferred .

[0105] Figure 11A is a flow diagram illustrating operations in a method 1100 of implementing an SGLD model update operation according to an example embodiment of the present disclosure . The method 1100 is for example implemented by the device 1000 of Figure 10 .

[0106] According to embodiments described herein, an SGLD model update operation is for example implemented based on the following equation :

[0107] [Math 3 ]

[0108] where 0t+1is the updated model , 0tis the current model ,is a Langevin gradient of the model , calculated, for example , from a batch of data with respect to a set of labels . The term VgL^^O^ is typically the sum of a likelihood term and the model prior distribution . For a classi fication problem, the likelihood could be cross-entropy over a minibatch of labelled data and the prior could be set as a Gaussian distribution . The Langevin gradient is also commonly multiplied by a factor when used in the mini-batch setting, for example equal to N / n, where N is the number of data points in a training set and n is the mini-batch si ze . In practice , these quantities are for example calculated in the log-domain, which turns multiplications into sums and prevents numerical underflow issues . The symbol rjtis a learning rate , for example equal to between le-5 and le-3 , and Etis a unit Gaussian sample . In certain embodiments , the learning rate , and therefore the standard deviation of the Gaussian samples , is made to reduce in an exponential fashion . Alternatively, the standard deviation of the Gaussian sample s is maintained at a fixed value or is controlled to cycle between lower and higher values following a Cosine function .

[0109] In an operation 1101 , the Langevin gradient g Ot) is for example calculated as the derivative of the following equation :

[0110] [Math 4 ] logp(0t)

[0111] where the hyper-parameter ysgacts to scale the contribution of the likelihood in order to compensate for the noise in the stochastic gradients , logp(y 0t) is the likelihood of the model 0tover the data in a mini-batch y^ of thetraining dataset , which for example comprises mini-batches containing d data-points , and logp(0t) is the prior over the model ' s parameters .

[0112] After operation 1101 , a parameter update operation is performed on each parameter of the model based on the Langevin gradient calculated in operation 1101 . For example , this is performed using three programming steps 1103 , 1104 and 1105 performed on each parameter .

[0113] In the programming step 1103 , a resistive memory device 102 of the array 1002 storing the parameter is programmed based on first programming parameters in order to create a conductance variation of a first magnitude that is proportional to the Langevin gradient , and of a first polarity that is a function of the sign of the Langevin gradient . For example , in the case of the domain-wall device , the magnitude is defined by the pulse-width, and the polarity of the current pulse applied between the electrodes 112 and 114 of the device defines the sign .

[0114] In the programming step 1104 , after the programming step 1103 , a second programming operation of the resistive memory device 102 is performed based on second programming parameters in order to create a second conductance variation of a second polarity and a second magnitude . For example , the second polarity may be the same as or the opposite of the first polarity .

[0115] In the programming step 1105 , after the programming step 1104 , a third programming operation of the resistive memory device 102 is performed based on third programming parameters in order to create a conductance variation of a third polarity opposite to the second polarity . In other words , i f the conductance variation of the second polarity is a conductance increase , the conductance variation of the thirdpolarity is a conductance decrease , and vice versa . The second and third programming parameters are for example equal and opposite , such that the probability distributions will also be equal and opposite . For example , in the case of a DW device , both of the second and third programming operations are based on current pulses of the same pulse-width and same magnitude , but one of the pulses is positive , and the other is negative .

[0116] The second and third programming steps 1104 and 1105 are for example used in order to apply the sample noiseGt of the SGLD model update operation . For example , in the case of programming a domain-wall memory device with a current pulse of varying pulse-width, together the magnitude of the pulse-widths of the second and third programming operations are for example proportional to / 2r]t. In some embodiments , it would also be possible to reduce the pulse-widths of the second and third programming operations by the noise part pf the pulse-width of the first programming step 1103 . In other words , assuming that the total noise part would correspond to a pulse-width E , i f the magnitude of the Langevin gradient part will incur a noise corresponding to a pulse-width of 1 , then the pulse-widths applied during the second and third programming steps will generate a noise contribution equal to (E-l ) . This is illustrated in more detail Figure 11B .

[0117] Figure 11B shows graphs illustrating an SGLD model update operation according to an example embodiment of the present disclosure .

[0118] The principle of SGLD is shown by the graph on the left-hand side of Figure 11B showing the probability density p ( g) of a model g, whereby samples ( crosses ) are generated by moving in the direction of a Langevin gradient ( arrow L ) , and then perturbing this update with a sample from a unit noise distribution ( distribution E ) . The right-hand graphs ofFigure 11B illustrate how this update can be achieved through a pair of push-pull ("push" , "pull" ) current pul ses , which achieve a sample from the distribution E and also the Langevin gradient VgL^^O^ . In particular, in the example of Figure 11B, the push current pulse implements the second programming step 1104 , and the pull pulse implements the first and third programming steps 1103 , 1105 in the form of a single pull current pulse . For example , the pulse duration Tpushof the push current pulse is equal to :

[0119] [Math 5 ]

[0120] where (p is a function that converts the noise variance introduced by the application of the Langevin gradient into a pulse-width, and this part |Ve£D(0t) represents thatamount 1 by which the pulse-widths can be shortened . It is assumed that the quantityis directly related to the pulse-width, but alternatively it would be possible to apply a conversion operation to convert this quantity into a corresponding pulse-width . The pulse duration Tpuu of the pull current pulse is for example equal to TpuU= Tpush+ TL, where TLis proportional to the Langevin gradient VgL^^O^ .

[0121] More generally, whether or not the programming steps are performed by current pulses or other means , one of the programming operations for example has a magnitude that is proportional to a programming magnitude equal to :

[0122] [Math 6 ]

[0123] where <p is now a function that converts the noise introduced by the application of the Langevin gradient into a corresponding programming magnitude .

[0124] Referring again to Figure 11A, in some embodiments , a random value stored by the resistive memory device 102 is read by reading the programmed conductance of the resistive memory device 102 following the programming step 1105 . Furthermore , this random value is for example converted from an analog value , such as a voltage or current level , into a digital value by an analog to digital converter (not illustrated) that for example forms part of the control circuit 104 .

[0125] In some embodiments , the operations 1103 to 1105 are repeated one or more times according to the sample generation approach that is adopted .

[0126] In alternative embodiments , it would be possible to perform the first programming step 1103 after the second and third programming steps 1104 , 1105 , or after the second programming step 1104 and before the third programming step 1105 and / or to combine the first programming step with the second or third programming step .

[0127] An advantage of the method of Figure 11A for performing SGLD is that any lack of precision available in the programming of a resistive memory devices , such as a DW device , is ef fectively drowned out by the random sampling processing or even absorbed into the noise generated by the two programming steps , which are equal and opposite . For example , assuming that the learning factor rjtis less than 1 , which is typically the case , the square root operation will ensure that the Gaussian noise contributes more to the update on average than the Langevin gradient . This is illustrated in Figure 12 .

[0128] Figure 12 is a graph representing examples of the Langevin gradient termand of Gaussian samples Etaccording to an example embodiment of the present disclosure . In particular, Figure 12 illustrates examples based on a typical SGLD update step, and the weight update histogram for all synapses in a layer are shown . It can be observed that , for the most part , the noise of the Gaussian sampling dominates over the deterministic updates , typically equal to 95% of cases . For the remaining cases , typically equal to around 5% of cases , although the Langevin update is greater than the Gaussian noise update , because the gradient is in fact so large , it is easy to obtain an update step in the correct direction despite the variability associated with device programming .

[0129] Figure 13 schematically illustrates an array 1300 of domain-wall resistive memory devices 102 according to an example embodiment of the present disclosure . This DW memory array 1300 for example forms part of the resistive memory array 1002 of Figure 10 .

[0130] The DW memory array 1300 comprises M columns (M COLUMNS ) , where M is equal to 4 in the example of Figure 13 , and N rows (N ROWS ) , where N is equal to 3 in the example of Figure 13 . In some embodiments , each row of the DW memory array 1300 acts as a sample of the parameters ( in the present example M=4 parameters ) of a neuron in a Bayesian neural network . A fully-connected neural network layer could be composed of a number of such Bayesian neurons in a layer and then composed of several such layers . In this embodiment , each Bayesian neuron is for example stored and sampled using a separate array . Alternatively, in another embodiment , Bayesian neurons could share the same physical array and N arrays are for example used in order to store N samples of these neurons . In such a case , di f ferent neurons use di f ferentranges of row indexes to store their samples. Alternatively, multiple arrays like the array 1300 could be used as multiple samples of a full layer in the neural network model. The following description is based on the example in which each Bayesian neuron is stored and sampled using a separate array.

[0131] Each device 102 is part of a cross-point cell in the array 1300 that further comprises a transistor 1302 and a transistor 1304.

[0132] An input voltage line Vin[0] to Vin[M-l] is for example associated with each column, and is for example coupled via a corresponding control line to the electrode 112 of each of the devices 102 of its corresponding column. The electrode 114 of each of the devices 102 of a given column is for example coupled, via the main conducting nodes of its corresponding transistor 1302, to a programming voltage Vprg[0] to Vprg[M-l] provided on a corresponding column line by a pulse generator unit (PULSE GEN) . The gates of the transistors 1302 of each row are for example coupled to a corresponding row select programming signal Vsp[0] to Vsp [N- 1] provided on a corresponding control line. The top electrode 110 of each of the devices 102 of a given row is for example coupled, via the main conducting nodes of its corresponding transistor 1304, and the main conducting nodes of a row output transistor 1306, to a corresponding row current output line providing a corresponding output current iout[0] to iout [N- 1] . The gates of the transistors 1304 of a given column are for example coupled to a corresponding column select read signal Vsr[0} to Vsr[M-l] provided on a corresponding control line. The gates of the transistors 1306 are for example controlled by a corresponding select N signal Vsn[0] to Vsn[N- 1] provided on a corresponding control line. The control lines Vsn and the Vsp are for example operated out of phase, suchthat a device is never selected for programming and reading simultaneously .

[0133] The example of Figure 13 will be described assuming unsigned parameters , i . e . , a single positive conductance per parameter, and an array configuration that does not support multiplication and accumulation . In the case of signed parameters , i . e . , the subtraction of two positive conductance per parameter, two such arrays could be used and the currents produced by each could be subtracted, for example using a simple circuit based on two stacked current mirrors . A person skilled in the art will understand how these principles can be extended to signed cross-points , with a pair of devices , and MAC capable arrays .

[0134] In the case that the array configuration does not support multiplication and accumulation, and of the Bayesian neuron where one array stores the samples of its input weights , each column of the array 1300 for example stores all of the samples of one parameter, N samples for example being generated per parameter . In a read mode , the voltage Vin is for example applied to one of the columns . This voltage for example corresponds to an input data feature . The select read signal Vsr is applied to the gates of the transistors 1304 in this column, resulting in a current , iout , flowing out of the right-hand side of the array corresponding to the multiplication of the input data value by each of the N samples of one parameter . By iterating over all of the columns , it is possible to access the stored samples of all N rows . Alternatively, it would be possible to read one device from one row at a time - this is what would be done during the training phase of the model . This can for example be achieved by selecting only one of the rows using the select N signal Vsn .

[0135] In other cases , such as the example of Figure 13 , according to a first mode , all devices in one of the M-columns are for example read simultaneously by selecting one column using the Vsr signals , for example by setting one of the Vsr signals HIGH and the rest LOW, and selecting all of the rows , for example by setting all of the Vsn signals HIGH . This permits , for example , having acces s to the full di stribution of a given parameter in a single instant via all of the output currents from all N rows . According to a second mode , a MAC operation is applied between the presented voltage input data vectors Vin and one of the n samples stored in the array . For example , a row of the array is selected, for example by setting all of the Vsr signals HIGH and only one of the Vsn signals of the N rows HIGH and the rest low, the output current of the nth row will be equal to the MAC between one samples parameters and the input data . According to a third mode , a PWM-based reading can be achieved where a constant reference voltage is applied to all Vin and the data is encoded via the pulse-width applied to each of the Vsr columns . This for example involves the use of an ADC that converts the current into a digital value to integrate the current , instead of simply sensing the current as in the second mode . A person skilled in the art of Memristor array design will aware of many methods of ADC designs for current sensing and for current integration . In a fourth mode , is would also be possible to read only a single device conductance in one row by setting the appropriate Vsr line HIGH, setting all others LOW and then setting only one of the Vsn lines HIGH .

[0136] A person skilled in the art will be aware there are many possible array designs for two and three terminal devices that can support the sequence of operations for implementing the described random number generation method and Langevin sampling methods . For example , the line Vsr could be removed,which would remove the possibility of performing the first mode of reading.

[0137] During the writing phase, i.e., the training of the model by the SGLD algorithm, the pulse generator unit is for example coupled to one of the N rows corresponding to the current sample by setting the appropriate select programming signal Vsp. In this case, the input voltage Vin acts as a reference voltage or a virtual ground, depending on the polarity of the pulses. The pulse generator unit is for example configured to calculate, for each of the M parameters, the sequence of programming pulses to be applied, where the pulse-width and the pulse polarity for example determine to what extent the domain wall will be moved, and therefore what the probability distribution is sampled from. In some embodiments, the pulses are applied in parallel for all M parameters for the array row under programming. All other select signals are grounded.

[0138] In some embodiments, an online sample thinning procedure is possible. According to this procedure, instead of advancing one row at a time, many samples are produced at each row before moving onto the next. Therefore, only the last sample generated at each row is stored in the array - the others have been "thinned out" in an online fashion. When passing from each row to the next, a calibration step is for example performed in order to bring the devices of row n as close as possible to the corresponding devices in row n-1. This may be done, for example, by reading a device in row n- 1 and applying a pulse to the device in row n to reduce the difference between their conductances, and repeating these steps until the conductance difference is within an acceptable tolerance. This may introduce a slight error between the last sample of row n-1 and the initial sample of row n. However, in practice, this perturbation has been found to help theperformance of the model trained by SGLD, as it facilitates a better exploration of the posterior - in particular if it has many modes.

[0139] In some embodiments, for DW-SGLD to operate continuously, instead of stopping when the process arrives at the final row of the array, it is possible to wraparound to the first row, whereby the sample that was previously stored in the first row is overwritten. This is appealing from the point of view of continual learning and domain distribution shifts .

[0140] Figure 14 is a graph representing probability distributions (Probability, p(AX|T, P) ) of discrete domain-wall changes (Domain-wall change, AX( / zm)) of a domain wall device. In the example of Figure 14, a case is illustrated in which a single domain wall device 102 is capable of being programmed with 10 different pulse-widths (with positive and negative pulses) of durations of T = 10, 20, 30, 40 and 50 ns. This gives an example of the different probability distributions that can be sampled from by adapting this programming parameter. In practice, it has been found that the domainwall device 102 capable of being programmed to have at least any one of 256 discrete distinguishable levels, and thus it is possible to generate at least 256 non-overlapping Gaussians - corresponding to an 8-bit memory element.

[0141] Figure 15A is a flow diagram illustrating operations in a machine learning algorithm 1500 based on MCMC according to a specific example in which it is implemented using by the device 1000 of Figure 10 and the domain-wall array 1300 of Figure 13.

[0142] In an operation 1501, a variable n is for example set to 0, and a count value C is set equal to an initial value thinning_ratio corresponding to a thinning ratio, equal forexample to 256 or another value. The thinning ratio is used to reduce the total number of samples, and improves "mixing" of the chain. In particular, instead of storing all the samples, thinning is used whereby, for example, every 256th sample is stored.

[0143] In an operation 1502, for example, for each epoch e of one or more epochs, and for example, for each batch b of one or more batches, an operation 1503 is implemented. Operation 1503 involves performing, for each data-point d in the batch, a series of operations 1504 to 1511.

[0144] The operation 1504 for example involves, for each column c of set of columns of the resistive memory array, activating the select N signal Vsn[n] and the select read signal Vsr[c] , setting lout [n] =Vin [c] *G [n, c] , where G[n,c] is the single device selected for reading that may have been previously written to. The output current Iout[n,c] is for example read by an ADC and other peripheral circuity known to a person skilled in the art of resistive memory array design.

[0145] The operation 1505 for example involves calculating the Langevin gradientfor G[n] .

[0146] The operation 1506 for example involves converting the gradients to pulse-widths T ( Ve£D(0t) ) and pulse polarities sign (V0L) .

[0147] The operation 1507 for example involves deactivating the select n signal Vsn[n] and the select read signal Vsr[c] .

[0148] The operation 1508 for example involves activating the select programming signal Vsp[n] and applying the calculated pulses to the M devices of the row of the array in parallel.

[0149] The operation 1509 for example involves applying the same pair of positive and negative polarity pulses T (<J) to each of the M devices.

[0150] The operation 1510 for example involves decrementing the count value C (C=C-1) .

[0151] The operation 1511 for example involves determining whether C is equal to 0 (C==0) , and if so, then if n is equal to the maximum row number MAX_ROWS, in other words it is the last row of the array, then the variable n is set back to zero, else n is incremented, and C is reset to thinning_ratio, and a calibration of row n is performed. For example, if moving onto row n+1 from row n, a device in row n+1 would be read by activating the corresponding Vsr column. Based on the difference between the conductances of these two devices, a pulse-width, or number of fixed pulses, is calculated, which should move the device of the row n+1 to that in row n. This pulse is for example applied by selecting the correct row with the Vsp signal, and then applying the calculated pulse to the device. This is for example performed for each device until a maximum number of iterations has been reached or the conductance difference is within a tolerable error. Alternatively, all devices in a row are read, and then they are programmed in parallel.

[0152] Figure 15B is a flow diagram illustrating operations in a machine learning algorithm 1500' based a MCMC according to an alternative example of that of Figure 15A, again implemented using a domain-wall device. The algorithm of Figure 15B is similar to that of Figure 15A, except that the operation 1504 is replaced by a MAC operation.

[0153] The operation 1504 of Figure 15B for example involves selecting one of Vsn lines (that of the current sample) and performing a read, based for example on the second or third reading mode described above. The resulting current lout is then for example used to evaluate the likelihood function of this sample. If a prior distribution is being used, the fourthreading mode is for example used to read the individual conductance values in each row, although in some cases a Uni form prior is used, and in such cases the reading of each device is for example not performed .

[0154] In operation 1505 of Figure 15B, the Langevin gradientfor G [n] is for example calculated using the likelihood evaluated thanks to the reading performed in the previous step 1504 .

[0155] Furthermore , in operation 1506 of Figure 15B, in the case that a pair of programming operations are applied with opposite polarities and pulse-widths of di f ferent magnitudes , the pulse-widths of the pulses are for example determined by the polarity of the Langevin gradient . I f the gradient is negative , the first pulse is for example shorter in duration than the second pulse .

[0156] Figure 16 is a flow diagram illustrating a method 1600 of generating a prediction implemented using by the device 1000 of Figure 10 and the domain-wall array 1300 of Figure 13 , according to an example embodiment of the present disclosure .

[0157] In an operation 1601 , all of the M column select read signals Vsr are for example activated, and all of the select programming signal Vsp are for example deactivated .

[0158] In an operation 1602 after the operation 1601 , for each row n, the row select signal Vsn [n] of the nth row is for example activated, an output current lout [n] is for example set as Vin [ : ] *G [n, : ] , and the output current Iout [ n] is for example stored, for example after analog to digital conversion . This value for one row for example corresponds to one sample of the prediction distribution of one parameter . The prediction distribution of each parameter is for examplecomposed of N such samples, generated by performing the operation 1602 for each row n.

[0159] In an operation 1603, statistics are for example generated based on the set of n output currents lout [ : ] generated on the operation 1602, such as the distribution means (for predictions) and entropy.

[0160] In the case where the model is a single neuron, e.g. a logistic regression model, the statistics are for example calculated in iout directly. In other cases, an evaluation is for example made as to how the currents sum into activation vectors and how these activation vectors propagate forwards in the model, for example generating voltages for other arrays that store the parameters of these subsequent neurons and layers. For example, the entropy of the output Bayesian neurons can be calculated and used to quantify the model uncertainty .

[0161] Figures 17 to 20 illustrates results that have been obtained using the method described herein. In particular, the described algorithm was implemented using a device model that limits the conductance range, and models the Langevin updates as Gaussian samples within a given range of bit precision, i.e., two 4-bit devices mean that an update can land anywhere within a Gaussian distribution that is 1 / 32 of the conductance range wide (i.e., at three standard deviations, the value is 1 / 32 of the conductance range) . The task addressed was MNIST (Mixed National Institute of Standards and Technology) and KMNIST out of distribution detection whereby the model is evaluated on its ability to correctly classify unseen MNIST digits as well as detect the presence of digits that do not correspond to the classes (those fromKMNIST) observed in its training set - an example of in and out of distribution data is shown in Figure 17.

[0162] Figure 18 shows the results (score - "SCORE / l.O" as a function of the number of bits per device - "NO. BITS PER DEVICE") as the precision of the DW device is swept from 3 bits to 11 bits, and quantifies the impact of the Langevin update precision. The results show the impact on the accuracy (curve 1802) and the out-of-distribution detection (curve 1804) as the DW precision is swept from 11 bits to 3 bits. Above 3-bits, the impact on the accuracy is negligible whereas the lower precision actually appears to benefit the out of distribution detection capabilities. The region 1806 of the curve shows where the zero-skipping of gradient updates (i.e., not programming the device if the update was below a certain threshold) was important for the sampler to converge.

[0163] Figure 19 shows the score ("SCORE / l.O") as a function of the number of samples ("NO. SAMPLES") , and plots the effect of the array size (i.e., how many samples are stored - corresponding the either the number of rows, or the number of arrays if an array per layer sample is used) for a thinning ratio of 256 (curve 1902) and a thinning ratio of 50 (curve 1904) . The diminishing benefit of more samples is clearly seen as well as the importance of large online thinning ratios for the ability of the model to detect out of distribution data. The accuracy is not plotted here, but the effect of the number of samples did not have a large impact above 8 samples.

[0164] Figure 20 shows sensitivity vs. specificity curves representing the accuracy of in / out-of-distribution detection (IN DIST. ACC. / OUT DIST. ACC.) , as the entropy threshold is increased, of the model with 128 samples and 4-bit precision Langevin updates relative to a floating-point precision deterministic model. The curve 2002 represents a Bayesianneural network EPI (Epistemic uncertainty) , while the curve 2004 represents a Det ( Deterministic Model ) neural network . Here , a comparison is made for floating-point 32-bit precision floating-point model , which demonstrates that the SGLD model , even at 4-bits , is signi ficantly better at recogni zing out of distribution data than a deterministic model trained through stochastic gradient descent . A point 2006 on the curve 2006 represents a reading AUG LAMA = 0 . 974 , where AUG is the Area Under Curve , and a point 2108 on the curve 2002 represents a reading AUG det = 0 . 915 . The epistemic uncertainty of each unseen data-point is calculated and compared against a threshold value . This curve is generated by sweep values for the threshold value then by summing up the area under the curve it is possible to determine the number of AUG .

[0165] An advantage of the embodiments described herein is that the solution based on resistive memory devices for random value generation is easily scalability for applications in which a relatively large number of random numbers should be generated in parallel . Furthermore , it is possible to tune the parameters of the probability distribution of the generated random numbers , such as the mean and standard deviation of the probability distribution, and in particular, it is possible to decorrelate the tuning of the standard deviation of the probability distribution from the mean of the probability distribution .

[0166] Furthermore , embodiments described herein provide a Langevin sampling technique permitting to generate and store (via learning) parameters at the same time in a resistive memory array .

[0167] Various embodiments and variants have been described . Those skilled in the art will understand that certain features of these embodiments can be combined and other variants willreadily occur to those skilled in the art. For example, domain wall updates could be realized based on pulse numbers instead of pulse width. In such a case, a fixed pulse is for example applied, with a certain polarity, a certain number of times and a vast multitude of array organizations and reading and writing strategies could be employed.

[0168] Furthermore, thinning could be achieved in a different manner than sampling down counting. For example, a metric combining the sample likelihood with the Euclidean distance between internal model representation, i.e., at the penultimate layer of the model, could be used that may permit to reduce the total number of samples to be stored.

[0169] Furthermore, instead of using SGLD, the Metropolis- adjusted Langevin algorithm (MALA) could be used, according to which an acceptance / re ection step is added after each sample. This acceptance is for example calculated based on the product of the ratios of the likelihoods and priors of the proposed and previously accepted models. However, in MALA, sampling is made each time from the same starting point, whereas certain resistive memory technologies, such as DW devices, do not offer a means of storing this value, and therefore it will be forgotten after the first random number generation, and the more the model samples after rejection steps, the more it will drift further and further from the starting point. Therefore, a mitigation strategy could be to limit the number of samples where, for example, the probability threshold is scaled to a smaller number as the number of rejections increases.

[0170] It should also be noted that the calculation of the prior is a relatively expensive process since it requires each parameter to be read at each mini-batch. It may also, in some cases, lead to a high rejection rate that may negativelyimpact the ef ficiency of the algorithm . In any case , the physical limitations in conductance naturally impose a prior on the parameters to be a truncated uni form . In the case of such a uni form prior, since all parameters are equally likely, the prior is does not actually required to be calculated . Therefore , in some embodiments it may be preferable to use a uni form prior ( ef fectively no prior at all ) on the model i f this is not important for the task . Importantly, this will for example avoid the time consuming process of reading all of the device conductances one by one to evaluate the prior probability .

[0171] Furthermore , non-symmetric pulse pairs could be applied in order to account for a non-linear conductance changes , which may occur in technologies such as PCM, and by bringing asymmetry in this manner, it ensures that the noise distribution is symmetrical and centered on the current value .

[0172] Finally, the practical implementation of the embodiments and variants described herein is within the capabilities of those skilled in the art based on the functional description provided hereinabove .

Claims

CLAIMS1 . A device configured to program a random value , the device comprising : a resistive memory device ( 102 ) capable of being incrementally programmed; and a control circuit ( 104 ) configured to program the resistive memory device ( 102 ) with the random value , the control circuit being configured to :- perform a first programming operation of the resistive memory device with the first programming parameters in order to generate a first conductance variation of a first polarity in the resistive memory device ;- perform a second programming operation of the resistive memory device with the second programming parameters in order to generate a second conductance variation of a second polarity in the resistive memory device , the second polarity being the opposite polarity to the first polarity; and wherein either : a magnitude of the first programming parameters is di f ferent to a magnitude of the second programming parameters , and the control circuit ( 104 ) being further configured to read the random value programmed in the resistive memory device by reading a programmed conductance value of the resistive memory device following the first and second programming operations ; or- a magnitude of the first programming parameters is the same as a magnitude of the second programming parameters , the control circuit ( 104 ) being further configured to perform a third programming operation of the resistive memory device be fore the first programming operation or after the second programming operation, the random value programmed in the resistive memory device being read by reading a conductance value of the resistive memory devicefollowing the first , second and third programming operations .2 . The device of claim 1 , wherein the resistive memory device ( 102 ) is capable of being incrementally programmed to have a conductance level in a range from a minimum conductance level to a maximum conductance level , and wherein the conductance value of the resistive memory device following the first and second programming operations is greater than said minimum conductance level and less than said maximum conductance level , the first and second conductance variations each for example being less than a di f ference between the minimum and maximum conductance levels .3 . The device of claim 1 or 2 , wherein the control circuit ( 104 ) is further configured to perform analog to digital conversion on the read conductance value in order to generate a digital random value , the digital random value for example having a bit-length of at least two .4 . The device of any of claims 1 to 3 , wherein the resistive memory device is a domain-wall device , and a pulse-width, or a number of pulses , applied to the domain-wal l device during the first programming operation is di f ferent from a pulse-width, or a number of pulses , applied to the domainwall device during the second programming operation .5 . The device of any of claims 1 to 4 , wherein the magnitude of the first programming parameters is di f ferent to the magnitude of the second programming parameters , and the control circuit ( 104 ) is configured to implement a Bayes ian model update operation, wherein the magnitude of one of the first or second programming operations is based on a magnitude and sign of a Langevin gradient .6 . The device of claim 5 , wherein the magnitude of said one of the first or second programming operations is based on,and for example proportional to, a programming magnitude equal to:[Math 7]whereis a Langevin gradient, gtis a learning rate, and (p is a function that converts the noise introduced by the application of the Langevin gradient into a programming magnitude .

7. The device of any of claims 1 to 6, wherein the control circuit (104) is configured to implement a Bayesian model update operation, wherein the magnitude of said third programming operation is based on a magnitude of a Langevin gradient, and the polarity of third programming operation is based on a sign of the Langevin gradient.

8. The device of any of claims 1 to 7, wherein:- the first conductance variation of the first polarity is present between first and second electrodes (110, 112, 114) of the resistive memory device;- second conductance variation of the second polarity is present between the first and second electrodes (110, 112, 114) of the resistive memory device; and- reading the random value comprises reading a programmed conductance value between the first and second electrodes (110, 112, 114) of the resistive memory device.

9. The device of any of claims 1 to 8, wherein the control circuit (104) is configured to:- perform a fourth programming operation of the resistive memory device with fourth programming parameters in order to generate, with respect to the programmed conductance, a fourth conductance variation of a fourth polarity in the resistive memory device, the fourth polarity being equalto the first or second polarity;- perform a fi fth programming operation of the resistive memory device with fi fth programming parameters in order to generate a fi fth conductance variation of a fi fth polarity in the resistive memory device , the fi fth polarity being the opposite polarity to the fourth polarity; and wherein either : a magnitude of the fourth programming parameters is di f ferent to a magnitude of the fi fth programming parameters , and the control circuit ( 104 ) is further configured to read the random value programmed in the resistive memory device by reading a programmed conductance value of the resistive memory device following at least the first , second, fourth and fi fth programming operations ; or- a magnitude of the fourth programming parameters is the same as a magnitude of the fi fth programming parameters , the control circuit ( 104 ) being further configured to perform a sixth programming operation of the resistive memory device before the fourth programming operation or after the fi fth programming operation, the random value programmed in the resistive memory device being read by reading a conductance value of the resistive memory device following at least the first , second, fourth, fi fth and sixth programming operations .10 . The device of any of claims 1 to 9 , further comprising a second resistive memory device ( 102 ) capable o f being incrementally programmed, wherein the control circuit ( 104 ) is further conf igured to program the second resistive memory device with a second random value , the control circuit being configured to :- perform a first programming operation of the second resistive memory device with seventh programming parameters in order to generate a first conductance variation of aseventh polarity in the second res istive memory device , the seventh polarity being equal to the first or second polarity;- perform a second programming operation of the second resistive memory device with eighth programming parameters in order to generate a second conductance variation of an eighth polarity in the second resistive memory device , the eighth polarity being the opposite polarity to the seventh polarity; and wherein either :- a magnitude of the seventh programming parameters is di f ferent to a magnitude of the eighth programming parameters , and the control circuit ( 104 ) is further configured to read the second random value programmed in the resistive memory device by reading a programmed conductance value of the second resistive memory device following the first and second programming operations of the second resistive memory device ; or- a magnitude of the seventh programming parameters is the same as a magnitude of the eighth programming parameters , and the control circuit ( 104 ) is further configured to perform a third programming operation of the second resistive memory device before the first programming operation or after the second programming operation of the second resistive memory device , the second random value programmed in the second resistive memory device being read by reading a conductance value of the second resistive memory device following the first , second and third programming operations of the second resistive memory device .11 . A method of programming and reading a random value , the method comprising : programming, by a control circuit ( 104 ) , a resistive memorydevice ( 102 ) with the random value , the resistive memory device being capable of being incrementally programmed, wherein programming the resistive memory device comprises : performing, by the control circuit ( 104 ) , a first programming operation of a resistive memory device ( 102 ) with the first programming parameters in order to generate a first conductance variation of a first polarity in the resistive memory device ; and - performing, by the control circuit ( 104 ) , a second programming operation of the resistive memory device ( 102 ) with the second programming parameters in order to generate a second conductance variation of a second polarity in the resistive memory device , the second polarity being the opposite polarity to the first polarity; and wherein either : a magnitude of the first programming parameters is di f ferent to a magnitude of the second programming parameters , the method further comprising reading the random value programmed in the resistive memory device by reading, by the control circuit ( 104 ) , a conductance value of the resistive memory device following the first and second programming operations ; or - a magnitude of the first programming parameters is the same as a magnitude of the second programming parameters , the method further comprising performing a third programming operation of the resistive memory device before the first programming operation or after the second programming operation, the random value programmed in the resistive memory device being read by reading, by the control circuit ( 104 ) , a conductance value of the resistive memory device following the first , second and third programming operations .12 . The method of claim 11 , wherein the resistive memory device ( 102 ) is capable of being incrementally programmed to have a conductance level in a range from a minimum conductance level to a maximum conductance level , and wherein the conductance value of the resistive memory device following the first and second programming operations is greater than said minimum conductance level and less than said maximum conductance level , the f irst and second conductance variations each for example being less than a di f ference between the minimum and maximum conductance levels .13 . The method of claim 11 or 12 , further comprising performing analog to digital conversion on the read conductance value in order to generate a digital random value , the digital random value for example having a bitlength of at least two .14 . The method of any of claims 11 to 13 , wherein the resistive memory device is a domain-wall device , and a pulse-width, or a number of pulses , applied to the domainwall device during the first programming operation is di f ferent from a pulse-width, or a number of pulses , applied to the domain-wall device during the second programming operation .15 . A method of implementing a Bayesian model update operation comprising the method of any of claims 11 to 14 , wherein the magnitude of one of the first or second programming operations is based on a magnitude and sign of a Langevin gradient .16 . A method of implementing a Bayesian model update operation comprising the method of any of claims 11 to 15 , wherein the magnitude of said third programming operation is based on a magnitude of a Langevin gradient , and thepolarity of third programming operation is based on a sign of the Langevin gradient .

Images