Powder reduction apparatuses, systems, and methods thereof

Abstract

The present disclosure provides a power reduction apparatus, a system, and a method thereof. The apparatus may comprise a pull up network and a pull down network both coupled to an output of the apparatus. The pull up network includes one or more parallelly coupled pull up branches, each pull up branch having a pull up capacitor and a pull up module. The pull down network may also include one or more parallelly coupled pull down branches, each pull down branch having a pull down capacitor and a pull down module. The configuration of capacitors in the apparatus may help to improve power efficiency. Further, the apparatus may be applied to a neuron model based artificial neural networks.

Description

[0001] POWDER REDUCTION APPARATUSE S, SYSTEMS, AND METHODS THEREOF

[0002] CROSS-REFERENCE TO RELATED APPLICATION

[0003] The present disclosure claims priority to Singapore Patent Application No. 10202401487S, filed on 27 May 2024, which is incorporated by reference herein.

[0004] TECHNICAL FIELD

[0005] The present disclosure relates to apparatuses, systems and methods for power reduction and the uses thereof, especially the uses in artificial neural networks.

[0006] BACKGROUND

[0007] The power consumption in modern Very Large-Scale Integration (VLSI) of integrated circuits (ICs) is high, causing many of their applications expensive or even infeasible.

[0008] Artificial neural networks are currently used for a lot of artificial intelligence (Al) applications such as natural language processing, computer vision, robotics, autonomous vehicles and so on. Implementations of many of these Al applications involve circuitry such as VLSI ICs. Due to high computational complexity, execution of such Al applications also consumes a significant amount of power, which make the things worse.

[0009] Therefore, there is a need to improve the energy efficiency and reduce power consumptions for, including but not limited to, circuitry (e.g., ICs), apparatus, and artificial neural network system.

[0010] SUMMARY

[0011] In an aspect of the present disclosure, it provides an apparatus which is helpful to reduce power consumption and improve power efficiency. The apparatus comprises a pull up network and a pull down network both of which are coupled to an output of the apparatus. When the apparatus is in operation, the pull up network may be coupled between a first voltage and the output, and the pull down network may be coupled between a second voltage lower than the first voltage and the output. The pull up network may include one or more parallelly coupled pull up branches, each pull up branch having a pull up capacitor with one end coupled to the output. The pull down network including one or more parallelly coupled pull down branches, each pull down branch having a pull down capacitor with one end coupled to the output. Furthermore, the pull up branch may further comprise a pull up module coupled in series with the pull up capacitor in the same pull up branch Alternatively or additionally, the pull down branch may further comprise a pull down module coupled in series with the pull down capacitor in the same pull down branch.

[0012] The pull up module may be a P-type module, which, for example, may comprise one or more P- type Metal-Oxide-Semiconductor (PMOS) transistors. Meanwhile, the pull down module may be a N-type module, which for example, may comprise one or more N-type Metal-Oxide- Semiconductor (NMOS) transistors.

[0013] In an aspect of the disclosure, it provides an apparatus which may function as an enhanced inverter circuit. In an aspect of the disclosure, it provides an apparatus which may function as an enhanced multi-input NAND logical circuit. In an aspect of the disclosure, it provides an apparatus which may function as an enhanced multi-input NOR logical circuit.

[0014] In another aspect of the disclosure, it provides an apparatus which is a combination of multiple aforementioned apparatus. The multiple aforementioned apparatus may be same or different. In an example, different aforementioned apparatuses are in parallel connection with one another to form the combination. In another example, two same aforementioned apparatuses are combined by using the input of one as the output of the other apparatus, and vice the versa.

[0015] In still another aspect of the disclosure, the apparatus is used in artificial neutral networks. The output of the apparatus, being a function of the weighted sum of logical input to each of the pull up branch in the apparatus and the weighted sum of logical input to each of the pull down branch in the apparatus, is used as an output of at least one neuron in the artificial neutral networks. Since the pull up network or pull down network may respectively be replaced by a pull up capacitive element or a pull down capacitive element, the output of the apparatus may also be a function of the weighted sum of logical input to each of the pull up branch in the apparatus or the weighted sum of logical input to each of the pull down branch in the apparatus.

[0016] In yet another aspect of the disclosure, it provides a system including the apparatus and an activation function module. The activation function module receives output signal of the apparatus as its input, and processes the received signal to generate an output used as an output of at least one neuron in the artificial neutral networks. Furthermore, the system may also comprise a separate pull up capacitor coupled between the first voltage and the output of the apparatus Alternatively and additionally, the system may also comprise a separate pull down capacitor coupled between the output of the apparatus and the second voltage.

[0017] In yet another aspect of the disclosure, it provides a method for improving power efficiency by the apparatus or system. In yet still another aspect of the disclosure, it provides a method to apply the apparatus or the system in artificial neutral networks.

[0018] These and other features and advantages will become apparent from the following detailed description of the presently embodiment(s), taken in conjunction with the accompanying drawings.

[0019] BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings referenced here are meant as illustrative of only some embodiments, and not of all embodiments, unless otherwise explicitly indicated. The word “exemplary” is used herein to mean “serving as an instance, example or illustration.”

[0021] Fig 1 shows a schematical diagram illustrating an exemplary apparatus according to various embodiments of the present disclosure.

[0022] Fig. 2 shows an exemplary apparatus with P-type module(s) and N-type module(s) according to various embodiments of the present disclosure.

[0023] Fig. 3A, Fig.3B, Fig.3C and Fig.3D respectively illustrate an exemplary circuitry implementation of an apparatus according to an embodiment of the present disclosure.

[0024] Fig. 4 shows an exemplary circuitry implementation of an apparatus, which is a combination of several apparatus according to an embodiment of the present disclosure.

[0025] Fig 5 A and Fig.5B each shows a block diagram of a configurable capacitor and its corresponding implementation in an exemplary circuitry.

[0026] Fig. 6 schematically shows an exemplary model for artificial neural networks to which exemplary embodiments of the present disclosure may be applied.

[0027] Fig. 7 shows a schematical diagram illustrating an exemplary system for artificial neural networks according to various embodiments of the present disclosure.

[0028] Fig. 8 shows an exemplary implementation of the system in Fig. 7.

[0029] Fig .9 shows an exemplary flow chart of a method according to various embodiments of the present disclosure.

[0030] Tn the drawings, like reference numerals will be used for like elements unless stated otherwise The drawings referenced here are meant as illustrative of only some embodiments, and not of all embodiments, unless otherwise explicitly indicated. For elements of same category but may have individually varying properties, reference numerals in the format of “x-n” are used, where x detonates all elements belonging to the same category and is same for all the elements in the category, where n varies with each element, and x-n denotes a specific element in the category so as to distinguish the elements one from another.

[0031] DETAILED DESCRIPTION

[0032] It will be readily understood that the components of the embodiments, as generally described and illustrated in the Figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the apparatus, system, method of the present embodiments, as presented in the Figures, is not intended to limit the scope of the embodiments.

[0033] Reference throughout this specification to “an embodiment,” “an exemplary embodiment,” or “selected embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, appearances of the phrases “in an exemplary embodiment,” “in one embodiment,” “in various embodiments,” or “in exemplary embodiments” or “in an embodiment” in various places throughout this specification are not necessarily referring to the same embodiment.

[0034] The illustrated embodiments will be best understood by reference to the drawings, wherein like parts are designated by like numerals throughout. The following description is intended only by way of example, and simply illustrates certain selected embodiments of apparatuses, circuits, systems, methods, and processes that are consistent with the embodiments as claimed herein.

[0035] The term “coupled” may refer to a direct or indirect connection, wireless, wire, and all link measures suitable for the application of the embodiment. The term “capacitor” may refer to any capacitor or any capacitive element, their variants, or derivatives without departing from the scope of the disclosure. The term “signal” may refer to voltage signal, current signal, light signal, magnetic signal, data signal, control signal, digital signal (interchangeably with logical signal in the description), analog signal, or RF signal. The meaning of “a”, “an” and “the” may include single or plural reference. The meaning of “in” may include “in” and “on”. The term “transistors” may refer to any field effect transistors or the variant thereof including but not limited to metal oxide semiconductor (MOS) field effect transistor (FET), complementary metal-oxide- semiconductor (CMOS) transistors, BiCMOS transistors, Bipolar junction transistors, their variants, or derivatives without departing from the scope of the disclosure. The term “P-type module” may refer to PMOS transistors, P-type solar cells, P-type semiconductor diodes, P-type thermoelectric modules, P-type Thin Film Transistors (TFTs), P-type light Emitting Diodes (LEDs), their variants or derivatives without departing from the scope of the disclosure. The term “N-type module” may refer to NMOS transistors, N-type solar cells, N-type semiconductor diodes, N-type thermoelectric modules, N-type Thin Film Transistors (TFTs), N-type light Emitting Diodes (LEDs), their variants or derivatives without departing from the scope of the disclosure.

[0036] The term “is configured to” may refer to that a component, device, apparatus, element, system, module, network, branch, part, or their variants are arranged, designed, or adapted to perfonn a specific function or achieve a particular result. Unless otherwise specified, “first”, “second”, and “third”, etc. are not intended to imply that the defined objects should be in a given sequence, either priority, temporally, spatially, ranking, and so on.

[0037] Fig. 1 shows a schematical diagram illustrating an exemplary apparatus 100 according to various embodiments of the present disclosure.

[0038] In Fig. 1, the apparatus 100 is shown comprising two parts coupled: a pull up network 110, and a pull down network 120. For the pull up network 110, one end of which may be coupled to a high power supply voltage (e.g., VDD) 101 when the apparatus 100 is in operation, and the other end may be coupled to or function as an output 107 of the apparatus 100. The pull up network 110 may include one or more pull up branches (e.g., pull up branch 1 to i, where i is an integer equal to or greater than 1) coupled in parallel between the VDD 101 and the output 107, for example, there are i pull up branches are shown in Fig.l. Each pull up branch may have a pull up capacitor 105 (i.e., capacitor 105-1, 105-2, ..., 105-i respectively for pull up branch 1, 2, ..., i and having a corresponding capacitance Cui, Cu2, . . ., Cui) with one end coupled to the output 107 Further, each capacitor 105 may be coupled in series with a pull up module 103(i.e., 103-1, 103-2, ..., 103-i). For example, as shown in Fig.l, the capacitor 105-1 is coupled with a pull up module 103-1 between the VDD and the output 107; the capacitor 105-2 is coupled with a pull up module 103-2 between the VDD and the output 107; and the capacitor 105-i is coupled with a pull up module 103-i between the VDD and the output 107. When the apparatus 100 is in operation, each pull up module 103 may be provided with a pull up input set and each pull up input set may include one or more inputs. For example, the pull up module 103-1 is provided with an input set 121-1 ; the pull up module 103-2 is provided with an input set 121-2; and the pull up module 103-i is provided with an input set 121-i. The input sets 121-1, 121-2, ... ,121-i may be coupled to or function as input(s) of the apparatus 100, and may be same or different. For the latter case, it will be better understood from the example shown in Fig. 4.

[0039] For the pull down network 120, one end of which may be coupled to a low power supply voltage (e.g., VSS) 102 when the apparatus 100 is in operation, and the other end may also be coupled to or function as the output 107. The pull down network 120 may include one or more capacitors 106 coupled in parallel between the output 107 and the VSS 102. Each capacitor 106 may be coupled in series with a pull down module 104. The pull down network 120 may include one or more pull down branches (e.g., pull down branch 1 to j, where j is an integer equal to or greater than 1, and j may be same or different than i) coupled in parallel between the output 107 and the VSS 102, for example, there are j pull down branches are shown in Fig.1. Each pull down branch may have a pull down capacitor 106 (i.e., capacitor 106-1, 106-2, ...,106-j respectively for pull down branch 1, 2, ..., j and having a corresponding capacitance Cai, Cd2, ..., Cdj) with one end coupled to the output 107. Further, each capacitor 106 may be coupled in series with a pull down module 104(i.e., 104-1 , 104-2, . . ., 104-j). For example, as shown in Fig.1 , capacitor 106-1 is coupled with a pull down module 104-1 between the output 107 and the VSS 102; capacitor 106- 2 is coupled with a pull down module 104-2 between the output 107 and the VSS 102; and the capacitor 106-j is coupled with a pull down module 104-j between the output 107 and the VSS 102. When the apparatus 100 is in operation, each pull down module 104 may be provided with a pull down input set and each pull down input set may include one or more inputs For example, the pull down module 104-1 is provided with an input set 122-1; the pull down module 104-2 is provided with an input set 122-2; and the pull down module 104-j is provided with an input set 122-j . The input sets 122-1, 122-2, ... , 122-j may be coupled to or function as the input(s) of the apparatus 100, and may be same or different. For the latter case, it will be better understood from the example shown in Fig. 4.

[0040] Though in Fig.l the total number of pull up modules 103 is shown as same as the total number of the capacitors 105, i.e. i, it will be appreciated that the total number of pull up modules 103 may be equal to or less than i. That is, one or more pull up branches may only include a capacitor without a pull up module. Similarly, in Fig.l the total number of pull down modules 104 is shown as same as the total number of the capacitors 106, i.e. j, but it will be appreciated that the total number of pull down modules 104 may be equal to or less than j. That is, one or more pull down branches may only include a capacitor without a pull down module. Also, the total number of pull up modules 103 may be same (as shown in Fig.l) or different (not shown) from the total number of pull down modules 104. Further, the pull up input set and the pull down input set may be same or different. For example, in an apparatus implemented with CMOS transistors, the pull up input set and pull down input set for the pair of PMOS and NMOS transistors constructing the CMOS are same.

[0041] In Fig. l, the pull up modules 103-1, 103-2, ..., 103-i and / or the pull down modules 104-1, 104- 2, ..., 104-j may be configured to implement same or different functions. Further, the apparatus 100 may have the pull up modules 103(e.g., modules 103-1 to 103-i) together with the pull down modules 104 (e , modules 104-1 to 104-j) at the same time or only have either the pull up modules 103 or the pull down modules 104 The pull up network 110 is used to pull the output 107 up to a higher voltage in the direction to the high power supply voltage VDD 101, while the pull down network 120 is used to pull the output 107 down to a lower voltage in the direction to the low power supply voltage VSS 102.

[0042] Here, provided Uxto indicate the logical output (not shown) of the pull up module 103-x, where x is an integer between 1 and i, e.g., Ui indicates the logical output of the module 103-1, and Ui indicates the logical output of the module 103-i. Cuxis used to indicate the capacitance of the capacitor 105-x, wherein x is an integer between 1 and i. For example, Cui indicates the capacitance of the capacitor 105-1, and Cui indicates the capacitance of the capacitor 105-i. When the module 103-x is configured to provide an electrically ON path between the VDD 101 and the respective capacitor 105-x, that is to say the path between the VDD 101 and the respective capacitor 105-x is electrically enabled by the module 103-x, the pull up module 103-x may be configured to output a logical 1, i.e., Ux=l . When the pull up module 103-x is configured to provide an electrically OFF path between the VDD 101 and the respective capacitor 105-x, the path between the VDD 101 and the respective capacitor 105-x is electrically disabled by the module 103-x, i.e., Ux=0.

[0043] Here, provided Dyto indicate the logical output (not shown) of the pull down module 104-y, wherein y is an integer between 1 and j, e.g., Di indicates the logical output of the module 104-1, and Dj indicates the logical output of the module 104-j . Cay is used to indicate the capacitance of the capacitor 106-y, wherein y is an integer between 1 and j. For example, Cai indicates the capacitance of the capacitor 106-1 , and Caj indicates the capacitance of the capacitor 106-j. When the module 104-y is configured to provide an electrically ON path between the VSS 102 and the respective capacitor 106-y, that is the path between the VSS 102 and the respective capacitor 106- y is electrically enabled by the module 104-y, the pull down module 104-y may be configured to output a logical 1, i.e., Dy=l. When the pull down module 104-y is configured to provide an electrically OFF path between the VSS 102 and the respective capacitor 106-y, the path between the VSS 102 and the respective capacitor 106-y may be electrically disabled by the module 104-y, i.e., Dy=0.

[0044] Thus, without consideration of parasitic parameters, the total equivalent capacitance Cute of the pull up network 110 may be calculated according to equation 1, and the total equivalent capacitance Cate of the pull down network 120 may be calculated pursuant to equation 2: Equation 1 where: i is the total number of the pull up capacitors used in the pull up network 110. Here, the total number of pull up modules 103 also equals to i, but it may be less than i. It should be noted in case that a pull up branch only includes a pull up capacitor without a pull up module, the pull up branch is enabled by default, i.e., the U for the pull up branch is 1. Equation 2 where: j is the total number of the pull down capacitors used in the pull down network 120. Here, the total number of modules 104 also equals j, but it may be less than j. It should be noted in case that a pull down branch only includes a pull down capacitor without a pull down module, the pull down branch is enabled by default, i.e., the Dyfor the pull down branch is 1.

[0045] Provided that the capacitance Cuxof the capacitor 105-x is used as the weight of the logical input for the pull up branch x (here, x is an integer and xE [1, i]), the logical input being the logical output of the pull up module 103-x when there is a pull up module in the pull up branch, and being logical 1 when there is no pull up module in the pull up branch. In other words, the capacitance Cux of the capacitor 105-x is the weight of the logical output of the pull up module 103-x when it exists, or the weight of logical 1 in the absence of pull up module 103-x. The total equivalent capacitance Cute of the pull up network 1 10 is the weighted sum of logical input to each of the paralleled pull up branches.

[0046] Alternatively or additionally, provided that the capacitance Cdy of the capacitor 106-y is used as the weight of the logical input for the pull down branch y (here, y is an integer and yE[l, j]), the logical input for the pull down branch y being the logical output of the pull down module 104-y when there is a pull down module in the pull down branch, and being logical 1 when there is no pull down module in the pull down branch. In other words, the capacitance Cdy of the capacitor 106-y is the weight of the logical output of the pull down module 104-y when it exists, or the weight of logical 1 in the absence of pull down module 104-y. The total equivalent capacitance Cate of the pull down network 120 is the weighted sum of logical input to each of the paralleled pull down branches.

[0047] All pull up capacitors in the pull up network 110 as a whole (the total equivalent capacitance Cute) may be deemed as in series connection with all pull down capacitors in the pull down network 120 as a whole (the total equivalent capacitance Cate), electrically. Without consideration of parasitic parameters, the voltage at the output node 107 can be calculated pursuant to equation 3 :

[0048] Voutput Equation 3

[0049] From equation 3, the voltage at the output node 107 in Fig.1 is determined by the capacitance ratio of the total equivalent capacitance of all pull up capacitors in the pull up network 110 (Cute) to the sum of the total equivalent capacitance of all pull up capacitors in the pull up network 110 and the total equivalent capacitance of all pull down capacitors in the pull down network 120 (Cute + Cate). It should note, controlling the exact capacitance of a capacitor may be challenging, especially in ICs, but it is relatively easier to control the capacitance ratio. The logical output of the apparatus may be derived from the voltage at the output 107.

[0050] For an apparatus implemented in circuits, e.g., various ICs, the output 107 may be followed by an output stage (not shown in Fig.l), for example, an inverter or a buffer functioning as an output stage. Other output stages may also be added to the output 107 according to applications. The output from the output stage may be configured as a logical 1 or 0.

[0051] Fig. 2 shows an exemplary apparatus 200 with P-type module and N-type module according to various embodiments of the present disclosure. Fig.2 is similar to Fig. 1, except that the pull up modules 103-1, 103-2, ..., 103-i in Fig. 1 are respectively implemented as P-type modules 203-1, 203-2, ..., 203-i and the pull down modules 104-1, 104-2, ..., 104-j in Fig. 1 are respectively implemented as N-type modules 204-1, 204-2, ..., 204-j in Fig. 2.

[0052] Accordingly, the VDD 101, the pull up network 110, pull up capacitors 105-1, 105-2, ..., 105-i and pull up input sets 121-1, 121-2, ..., 121-i in Fig. 1 may be applied similarly in Fig.2 as VDD 201, the pull up network 210, pull up capacitors 205-1, 205-2, ..., 205-i and pull up input sets 221-

[0053] 1, 221-2, . . . , 221-i in Fig. 2, respectively. Alternatively or additionally, the VSS 102, the pull down network 120, pull down capacitors 106-1, 106-2, ..., 106-j, and pull down input sets 122-1, 122-

[0054] 2, ..., 122-j in Fig. 1 may be applied similarly in Fig.2 as VSS 202, the pull down network 220, pull down capacitors 206-1 , 206-2, ..., 206-j, and pull down input sets 222-1 , 222-2, ..., 222-j in Fig. 2, respectively.

[0055] Also, the output of the pull up network 210 and the output of the pull down network 220 may be coupled to or function as the output 207 of apparatus 200. For simplification, same or similar parts between Fig. 2 and Fig 1 are not repeated but a person with skills in the art would have the ability to apply the teaching of Fig 1 to similar contents related to Fig .2.

[0056] Fig.3A shows an exemplary apparatus 310 according to an embodiment of the disclosure. In Fig. 3 A, the apparatus 310 may include a PMOS transistor 313 and a NMOS transistor 314, the gates of which are commonly connected to a common input 317 of the apparatus 310. When the apparatus 310 is in operation, the source of the PMOS transistor 313 may be coupled to a high power supply voltage (e.g., VDD) 311 and the source of the NMOS transistor 314 may be coupled to a low power supply voltage (e g , VSS) 312. The drain of the PMOS transistor 313 is coupled to the output 318 of the apparatus 310 via a pull up capacitor 315, and the drain of the NMOS transistor 314 is coupled to the output 318 via a pull down capacitor 316. It can be seen that the PMOS transistor 313 and the NMOS transistor 314 in Fig. 3Amay respectively function as one P- type module in at least one pull up branch and as one N-type module in at least one pull down branch in the apparatus 200 of Fig. 2, and the transistors 313, 314 together with the capacitors 315, 316 may form an enhanced CMOS structure, which enable the apparatus 310 to be used as an enhanced CMOS inverter.

[0057] Generally, the leakage current flowing through the dielectric material of a capacitor is small. As illustrated in Fig.3A, when the enhanced CMOS inverter is in a static or steady state with no switching of the gate input 317, the pull up capacitor 315 may reduce the leakage power from the VDD 311 through the PMOS transistor 313 to the output 318; and the pull down capacitor 316 may reduce the 1 eakage power from the output 318 through the NMO S transi st or 314 to the VS S 312. When the enhanced CMOS inverter is in a dynamic state with the gate input 317 switching (e.g., the gate input 317 is switched either from logic 1 to logic 0, or from logic 0 to logic 1), the PMOS transistor 313 and the NMOS transistor 314 are simultaneously conducted for a short duration of time, resulting in a short-circuit power. This short-circuit power may be furtherly reduced by the pull up capacitor 315 and the pull down capacitor 316. In this way, the static power of a transistor (pull up module and / or pull down module) and short-circuit power of the branch (corresponding pull up branch and pull down branch) including the transistor (pull up module and / or pull down module) may be controllably reduced As a result, the total power of the apparatus 310 is reduced.

[0058] As PMOS transistor 313 and the NMOS transistor 314 in Fig. 3A may respectively function as a P-type module in at least one pull up branch and a N-type module in at least one pull down branch in the apparatus 200 of Fig. 2, or respectively function as a pull up module in at least one pull up branch and a pull down module in at least one pull down branch in the apparatus 100 of Fig. 1, the above scheme of power reduction illustrated with reference to Fig. 3A is also applicable to any one or any combination of the apparatus 100, 200 and all other apparatus hereinafter (including but not limited to apparatus 310, 330, 350, 370, 400, 750 and 850) according to the present disclosure and the variants thereof not departing from the principle of the disclosure. It will be appreciated that, when the apparatus is implemented with VLSI circuits with millions or even more transistors, the aforementioned power reduction will be even more significant.

[0059] Fig.3B shows an exemplary apparatus 330 according to an embodiment of the disclosure. In Fig. 3B, the apparatus 330 may include two PMOS transistors 333, 334 and two NMOS transistors 335, 336 One input 339 of the apparatus 330 is coupled to the gates of the PMOS transistor 333 and the NMOS transistor 336, and the other input 340 is coupled to the gates of the PMOS transistor 334 and the NMOS transistor 335. The sources of the PMOS transistors 333, 334, when the apparatus 330 is in operation, may be coupled to a high power supply voltage (e.g., VDD) 331, and the drains of PMOS transistors 333, 334 are coupled together with one end of a pull up capacitor 337. The drain of the NMOS transistor 335 is coupled to an end of a pull down capacitor 338, and the source of the NMOS transistor 335 is coupled to the drain of the other NMOS transistor 336. The source of the NMOS transistor 336, when the apparatus 330 is in operation, may be coupled to a low power supply voltage (e.g.VSS) 332. Herein, the phrase “the apparatus is in operation” may refer to a status that the apparatus is powered, for example, being coupled between a higher voltage and a lower voltage. The other ends of the pull up capacitor 337 and pull down capacitor 338 are commonly coupled to the output 341 of the apparatus 330. It can be seen that, the PMOS transistors 333, 334 as a whole may function as one pull up module / P-type module in one pull up branch, and the NMOS transistors 335, 336 as a whole may function as one pull down module / N-type module in corresponding one pull down branch in the apparatus 100 of Fig.1 or apparatus 200 in Fig. 2. Both the pull up input set for the pull up module / P-type module and the pull down input set for the pull down module / N-type module include two inputs 339 and 340. Also, the PMOS transistors 333, 334, NMOS transistors 335, 336 together with the capacitors 337, 338 may form an enhanced CMOS structure, which enable the apparatus 330 to be used as an enhanced two-input NAND gate.

[0060] Fig 3C shows an exemplary apparatus 350 according to an embodiment of the disclosure. In Fig 3C, the apparatus 350 may include two PMOS transistors 353, 354 and two NMOS transistors 355, 356. One input 359 of the apparatus 350 is coupled to the gates of the PMOS transistor 353 and the NMOS transistor 355, and the other input 360 is coupled to the gates of the PMOS transistor 354 and the NMOS transistor 356. The source of the PMOS transistor 353, when the apparatus is in operation, may be coupled to a high power supply voltage (e g., VDD) 351. The drain of PMOS transistors 353 is coupled to the source of the other PMOS transistor 354, the drain of which in turn is coupled to an end of a pull up capacitor 357. The drains of the two NMOS transistor 355, 356 are all coupled to one end of a pull down capacitor 358. The sources of the NMOS transistor 355, 356, when the apparatus is in operation, may be coupled to a low power supply voltage (e.g., VSS) 352. The other ends of the pull up capacitor 357 and pull down capacitor 358 are coupled together to the output 361 of the apparatus 350. Similar to Fig.3A, the static power and short- circuit power of the apparatus 350 may be controllably reduced by the pull up capacitor 357 and pull down capacitor 358 as illustrated in Fig.3C.

[0061] It can be seen that, the PMOS transistors 353, 354 as a whole may function as one pull up module / P-type module in at least one pull up branch, and the NMOS transistors 355, 356 as a whole may function as one pull down module / N-type module in at least one pull down branch in the apparatus 100 of Fig. 1 or apparatus 200 in Fig. 2. Both the pull up input set for the pull up module / P-type module and the pull down input set for the pull down module / N-type module include two inputs 359 and 360. Also, the PMOS transistors 353, 354, NMOS transistors 355, 356 together with the capacitors 357, 358 may form an enhanced CMOS structure, which enable the apparatus 350 to be used as an enhanced two-input NOR gate.

[0062] Though two inputs are shown in Figs. 3B and 3C, a person skilled in the art will appreciate that the number of inputs is not limited to two, but can be any number suitable for the specific application of the apparatus. Similar to apparatus 310 in Fig.3A, the static power and short-circuit power of the apparatus 330, 350 may be controllably reduced by the respective pull up capacitors and pull down capacitors. Thus, compared with the traditional NAND gate and NOR gate implemented by MOSFET, the apparatus 330 (may function as an enhanced two-inputNAND gate) and 350 (may function as an enhanced two-input NOR gate) may realize corresponding logical function with improved power efficiency.

[0063] Fig.3D shows an exemplary apparatus 370 according to an embodiment of the disclosure. In Fig.3D, the apparatus 370 may include two PMOS transistors 373, 374 and two NMOS transistors 375, 376. The gates of the PMOS transistor 373 and the NMOS transistor 375 may be coupled to a node 382 of the apparatus 370, and the gates of another PMOS transistor 374 and NMOS transistor 376 may be coupled together to another node 381 of the apparatus 370. The sources of the PMOS transistors 373, 374, when the apparatus 370 is in operation, may be coupled to a high power supply voltage (e.g., VDD) 371. The drain of the PMOS transistor 373 is coupled with an end of a pull up capacitor 377, and the drain of the other PMOS transistor 374 is coupled with an end of another pull up capacitor 378. The other end of the pull up capacitor 377 and the other end of the pull up capacitor 378 are coupled to the nodes 381, 382, respectively. The drains of the NMOS transistors 375 and 376 are respectively coupled with an end of a pull down capacitor 379 and an end of another pull down capacitor 380, the other end of which in turn are separately coupled to the node 381, and the node 382 The sources of the NMOS transistors 375 and 376, when the apparatus 370 is in operation, may be coupled to a low power supply voltage (e.g., VSS) 372.

[0064] It can be seen that, the PMOS transistor 373, NMOS transistors 375, pull up capacitor 377, pull down capacitor 379 may respectively correspond to the PMOS transistor 313, NMOS transistor 314, pull up capacitor 315, pull down capacitor 316 in one apparatus 310. Similarly, the PMOS transistor 374, NMOS transistor 376, pull up capacitor 378, pull down capacitor 380 may respectively correspond to the PMOS transistor 313, NMOS transistors 314, pull up capacitor 315, pull down capacitor 316 in another apparatus 310. Thus, the apparatus 370 may be deemed as a combination of two apparatus 310, wherein the input of one apparatus 310 function as the output of the other apparatus 310, and vice the versa. The apparatus 370 may be used in latches, flip-flops, memories, among others. Similar to Fig.3A, the static power and short-circuit power of the apparatus 370 may be controllably reduced by the pull up capacitors 377, 378 and pull down capacitors 379, 380 as illustrated in Fig.3D. Thus, the power efficiency for the apparatuses in the present disclosure are improved.

[0065] It should be noted that logical functions used in circuits (e g., ICs) may be formed by any one or any combination of the apparatus in Fig.3 A to Fig.3D and the variants thereof without departing from the spirit and scope of the disclosure. Thus, the apparatus according to embodiments of the present disclosure is applicable to various scenarios including but not limited to circuits, ICs, VLSI ICs, chips, semiconductor devices, and so on.

[0066] Fig. 4 shows an apparatus 400 according to an embodiment of the present disclosure. Similar to apparatus 100 and 200, the apparatus 400 may include a pull up network 410 and a pull down network 420 that are coupled between a high power supply voltage (e.g., VDD) 401 and a low power supply voltage (e.g., VSS) 402. The apparatus 400 may also include one or more branches coupled in parallel between the VDD 401 and the VSS 402. Each branch may include a pull up branch and / or a pull down branch Further, each branch may be implemented as any of the apparatus 310, 330, 350 and their variants. For example, as illustrated in Fig.4, there are multiple branches, wherein the first branch 430 is implemented in a form of an enhanced CMOS inverter 310 as shown in Fig. 3A with similar input 421, the second branch 440 is implemented in a form of an enhanced two-input NAND gate 330 as shown in Fig. 3B with similar inputs 423 and 424, and the last branch 490 is implemented in a form of an enhanced two-input NOR gate 350 as shown in Fig. 3C with similar inputs 427, 428 (the branches between 440 and 490 are not shown). All the branches may have a common output 407, which is also the output of the apparatus 400.

[0067] When the apparatus 400 is in operation, each of the pull up capacitors 405-1, 405-2, . . . , 405-i may be used to reduce the leakage power from the VDD 401 through the corresponding PMOS transistors coupled with it to the output 407. Alternatively or additionally, each of the pull down capacitors 406-1, 406-2, . . 406-j may be used to reduce the leakage power from the output 407 through the corresponding NMOS transistors coupled with it to the VSS 402. In Fig.4, i equals to j. Further, the short-circuit power of the apparatus 400 is also reduced similar to Fig.3A, 3B, and 3C. Consequently, the power of the apparatus 400 is controllably reduced.

[0068] It should note that, the capacitances of all capacitors in Figs 1 to 4, 7, and 8 and other apparatus according to the present disclosure may be constant or variable. Figs. 5A and 5B respectively show an exemplary configurable capacitor with a variable capacitance and its exemplary circuitry implementation. Other implementations of variable capacitances for capacitors are also applicable for any of the capacitors in the present disclosure.

[0069] It will be appreciated though some embodiments of apparatus in the present disclosure are implemented with electronic circuits such as Fig. 3 A-3D to Fig. 4, it is just for illustration without any intention to limit. The apparatuses according to embodiments of the present disclosure may be implemented, or at least a part thereof may be implemented in other measures not departing from the principle and scope of the disclosure, including but not limited to acoustic device, optical device, magnetic device, wireless device, among others.

[0070] Fig.5A shows an example for a configurable capacitor 511 which is applicable to any pull up capacitor in pull up networks in Figs. 1 to 4, 7, and 8. The capacitor 511 has two ends 531 and 532, and its capacitance may be configured to vary according to requirements. The capacitor 511 may be implemented as illustrated in a virtual block 510, which includes one or more branches coupled in parallel between the two ends 531 and 532. For each branch, a PMOS transistor among the PMOS transistors 512-1, 512-2, ..., 512-k and a pull up capacitor among the capacitors 513-1, 513-2, ..., 513-g is electrically coupled in series between the ends 531 and 532. Here, g is an integer equal or greater than 1 and g is no less than k. However, to make the capacitance of the capacitor 511 variable, g is preferably greater than 1 When a branch between the ends 531 and 532 only includes a pull up capacitor, two ends of this capacitor are directly coupled to 531 and 532. A controlling logic (not shown) may be used to control the ON / OFF of each of the PMOS transistor 512-1 to 512-k. By configuring the ON / OFF (conducting / non-conducting) of each PMOS transistor and thus the branch in which the PMOS transistor exists, the total capacitance of the block 510, i.e., the capacitor 511, is variable.

[0071] Fig 5B shows an example for a configurable capacitor 521 which is applicable to any pull down capacitor in pull down networks in Figs. 1 to 4, 7, and 8. The capacitor 521 has two ends 541 and 542, and its capacitance may be configured to vary according to requirements. The capacitor 521 may be implemented as illustrated in a virtual block 520, which includes one or more branches coupled in parallel between the two ends 541 and 542. For each branch, a pull down capacitor among the capacitors 523-1, 523-2, . . ., 523-h and a NMOS transistor among the NMOS transistors 522-1, 522-2, ..., 522-q is electrically coupled in series between the ends 541 and 542. Here, h is an integer equal or greater than 1 and h is no less than q. However, to make the capacitance of the capacitor 521 variable, h is preferably an integer greater than 1. When a branch between the ends 541 and 542 only includes a pull down capacitor 523, two ends of this capacitor are directly coupled to 541 and 542. A controlling logic (not shown) may be used to control the ON / OFF of each of the NMOS transistor 522-1 to 522-q. By configuring the ON / OFF (conducting / non- conducting) of each NMOS transistor and thus the branch in which the NMOS transistor exists, the total capacitance of the block 520, i.e., the capacitor 521, is variable.

[0072] The pull up capacitor and / or pull down capacitor in the apparatus according to the present disclosure may be helpful for power reduction of the apparatus. Further, the capacitance of pull up capacitor or pull down capacitor may be configured greater than the parasitic capacitor to an extent that the capacitance of the parasitic capacitor(s) can be ignored in Equations 1-3 and 5-8. It will be appreciated that g and h may be same or different. Thus, any capacitor in the pull up networks and / or pull down networks of the apparatus may be configured with a suitable capacitance according to requirements, causing the flexibility of the apparatus greatly improved.

[0073] Fig. 6 schematically shows an exemplary model 600 (e.g., a neuron) used in artificial neural networks. The model 600 decides its output y according to equation 4,

[0074] Equation 4 where: xiis the 1thinput of the model, wi is the weight corresponding to the xi, b is a bias, f is a function, and n indicates the total number of the inputs or the total number of the weights, and n is an integer greater than 0 and preferably greater than 1.

[0075] With reference to Fig. 6, xi, X2 to Xnare inputs of the model 600 and weights wi, W2 to Wn are in correspondence to xi, X2 to xn, respectively. A bias b and the sum of xi*wi are used as inputs of the function f, to get the output y of the model 600. For example, the function may be a sigmoid function, a rectified linear unit (ReLU) function, a logistic function, or other functions suitable for applications of the model. Artificial neural networks may include multiple neurons as illustrated in Fig. 6, each of which for example may be constructed with a series of multiply-accumulate (MAC) blocks followed by a nonlinear function. The series of MAC blocks and nonlinear function may be implemented in circuits such as ICs. However, such circuits will consume significant power and chip area, causing the implementation expensive or even infeasible. By applying the apparatus and / or method according to the present disclosure to such artificial neural networks, the issues can be alleviated or resolved.

[0076] It can be seen that the logical output of the pull up module, Ux, and the capacitance of the pull up capacitor, Cux, in an apparatus (e.g., apparatus 100, 200, etc.) according to present disclosure may respectively correspond to the input Xi and weight wi in Equation 4. In some examples such as apparatus as explicitly shown in Figs. 1 to 4, the bias b may be regarded as 0. In some examples, the bias b may be provided by other separate elements including but not limited to capacitors which are parallelly coupled with the pull up network of the apparatus. Also, the Equation 3 used to calculate the voltage at the output of the apparatus according to the present disclosure may correspond to at least a part of the function f in Equation 4. The logical output of Equation 4 may be derived from the said output voltage of the apparatus with known measures.

[0077] Alternatively or additionally, the logical output of the pull down module, Dy, and the capacitance of the pull down capacitor, Cdy, in an apparatus according to present disclosure may respectively correspond to the input Xi and weight wi in Equation 4. For the apparatus as explicitly shown in Figs.1 to 4, the bias b may be regarded as 0. In some examples, the bias b may be provided by other separate elements including but not limited to capacitors which are parallelly coupled with the pull down network of the apparatus. Also, the Equation 3 used to calculate the voltage at the output of the apparatus according to the present disclosure may correspond to at least a part of the function f in Equation 4.

[0078] Thus, an apparatus according to various embodiments of the present disclosure, for example, by any one or any combination of the apparatus 100, 200, 310, 330, 350, 400 or any variant thereof not departing from the principle of the present disclosure, may be used to construct at least a part of the neuron model 600 up to artificial neutral networks including multiple such neurons.

[0079] Fig. 7 shows an exemplary system 700 according to various embodiments of the present disclosure.

[0080] The system 700, which is mainly intended to be used in artificial neural networks, may include an apparatus 750 according to an embodiment of the present disclosure, and an activation function module 730. The activation function module 730 is configured to receive signal(s) from an output node 709 of the apparatus 750, process the signal(s), and output the processed signal as an output 731 of the activation function module 730, which is also coupled to or function as an output of the system 700.

[0081] In Fig. 7, pull up modules 703-1, 703-2, ...,703-i may respectively correspond to the pull up modules 103-1, 103-2, ...,103-i in Fig.l; and pull up capacitors 705-1, 705-2, ...,705-i may respectively correspond to the pull up capacitors 105-1, 105-2, ...,105-i in Fig.l. Further, pull down modules 704-1, 704-2, . . . ,704-j may respectively correspond to the pull down modules 104- 1 , 104-2, . . ,,104-j in Fig. l ; and pull down capacitors 706-1, 706-2, ...,706-j may respectively correspond to the pull down capacitors 106-1 106-2, . . . , 106-j in Fig. 1. For simplification, contents same or similar to that of Fig.1 are not repeated here but a person skilled in the art would have the ability to understand the details of apparatus 750 in view of the teaching regarding Figs. 1 to 6.

[0082] The system 700, when in operation, may further comprise a separate pull up capacitor 707 (with a capacitance Cbu) with one end coupled to a high power supply voltage (e.g., VDD) 701 and the other end coupled to an output 709 of the apparatus 750, and / or a separate pull down capacitor 708 (with a capacitance Cbd) with one end coupled to the output 709 and the other end coupled to a low power supply voltage (e.g., VSS) 702. In this case, the separate pull up capacitor 707 together with pull up modules 703, pull up capacitors 705 in apparatus 750, and the connections among them form a pull up network 710 of the system 700. Alternatively and additionally, the separate pull down capacitor 708 together with pull down modules 704, pull down capacitors 706 in apparatus 750, and the connections among them form a pull down network 720 of the system 700.

[0083] Though the apparatus 750 is shown in the form of the apparatus 100 illustrated in in Fig. l, it will be appreciated that the apparatus 750 may be in the form of any one or any combination of the apparatuses or any variant thereof not departing from the principle of the present disclosure. Also, though in Fig.7 the separate pull up capacitor 707 and separate pull down capacitor 708 are illustrated as being separate from the apparatus 750, each of them may be a part of the apparatus 750 or be integrated into the apparatus 750. For example, the separate pull up capacitor 707 may be implemented with a pull up capacitor in a pull up branch without pull up module, and / or the separate pull down capacitor 708 may be implemented with a pull down capacitor in a pull down branch without pull down module.

[0084] Similar to Fig.l, here, Cux indicates the capacitance of the capacitor 705-x, and Ux indicates the logical output of the pull up module 703-x connected in series with the capacitor 705-x in the same pull up branch, wherein x is an integer between 1 and i. When the module 703-x is configured to provide an electrically ON path between the VDD 701 and the respective capacitor 705-x, that is to say the path between the VDD 701 and the respective capacitor 705-x is electrically enabled by the module 703-x, the pull up module 703-x may be configured to output a logical 1, i.e., Ux=l. When the pull up module 703-x is configured to provide an electrically OFF path between the VDD 701 and the respective capacitor 705-x, the path between the VDD 701 and the respective capacitor 705-x is electrically disabled by the module 703-x, i.e., Ux=0.

[0085] Also, Cdy is used to indicate the capacitance of the capacitor 706-y, and Dyis used to indicate the logical output of the pull down module 704-y connected in series with the capacitor 706-y in the same pull down branch, wherein y is an integer between 1 and j. When the module 704-y is configured to provide an electrically ON path between the VSS 702 and the respective capacitor 706-y, that is to say the path between the VSS 702 and the respective capacitor 706-y is electrically enabled by the module 704-y, the pull down module 704-y may be configured to output a logical 1, i.e., Dy=l. When the pull down module 704-y is configured to provide an electrically OFF path between the VSS 702 and the respective capacitor 706-y, the path between the VSS 702 and the respective capacitor 706-y may be electrically disabled by the module 704-y, i.e., Dy=0.

[0086] Similarly, a pull up capacitor may be used to reduce the leakage power from the VDD 701 through the pull up module in the same pull up branch to the output 709 of the apparatus 750, and a pull down capacitor may be used to reduce the leakage power from the output 709 through the pull down module in the same pull down branch to the VSS 702. Further, the short-circuit power of the apparatus 750 is also reduced by the pull up capacitors and pull down capacitors as mentioned above.

[0087] Without consideration of parasitic parameters, the total equivalent capacitance Cut of the pull up network 710 of the system 700 can be calculated by equation 5, which differs from Equation 1 by an additional term, Cbu, which is the capacitance of the separate pull up capacitor 707, i.e., the total equivalent capacitance Cut of the pull up network 710 of the system 700 being equivalent to the sum of the total equivalent capacitance of the pull up network of the apparatus 750 and the capacitance of the separate pull up capacitor 707.

[0088] Equation 5

[0089] Similarly, without consideration of parasitic parameters, the total equivalent capacitance Cat of the pull down network 720 of the system 700 can be calculated by equation 6, which differs from Equation 2 by an additional term, Cbd, which is the capacitance of the separate pull down capacitor 708, i.e., the total equivalent capacitance Cdt of the pull down network 720 of the system 700 being equivalent to the sum of the total equivalent capacitance of the pull down network of the apparatus 750 and the capacitance of the separate pull down capacitor 708.

[0090] Equation 6

[0091] It should note that each pull up module 703-x may receive a pull up input set 721-x and generate a logical output Ux, e g., 1 or 0, for connecting or disconnecting the pull up branch including the corresponding pull up capacitor 705-x with the pull up network. And each pull down module 704- y may receive a pull down input set 722-y and generate a logical output Dy, e.g., 1 or 0, for connecting or disconnecting the pull down branch including the corresponding pull down capacitor 706-y with the pull down network.

[0092] It can be seen that the logical output of the pull up module 703-x, Ux, and the capacitance of the pull up capacitor 705-x, Cu , may respectively correspond to the input xi and weight wi in Equation 4 Further, the capacitance of the separate pull up capacitor 707, Cbu, may correspond to the bias b in Equation 4. Alternatively or additionally, the logical output of the pull down module 704-y, Dy, the capacitance of the pull down capacitor 706-y, Cdy, and the capacitance of the separate pull down capacitor 708, Cbd, may respectively correspond to the input Xi, weight wi and bias b in Equation 4, too.

[0093] The voltage at the node 709 in Fig.7 is determined by the capacitance ratio of the Cut to the sum of Cut and Cdt. Similarly to equation 3, without considering of parasitic parameters, the output voltage at the node 709 can be calculated pursuant to equation 7:

[0094] Equation 7 where, Cut is calculated according to equation 5 and Cat is calculated according to equation 6.

[0095] From above it can be understood that, the Equation 7 used to calculate the voltage at the output 709 of the apparatus 750 may correspond to at least a part of the function f in Equation 4. That is to say, even without the activation function module 730, the apparatus 750 may be configured to function as or apply to the neuron model 600 up to artificial neutral networks including multiple such neurons. However, to make the apparatus 750 more flexible and applicable to more applications especially in artificial neutral networks, the apparatus 750 is followed by the activation function module 730 to form the system 700. The activation function module 730 may be configured to further compose the Equation 7(or Equation 3 when the bias is 0) to form a composition function z used as the function f in Equation 4, so as to extend the applications of the system 700. That is, the function z implemented by the activation function module 730 may take a signal (e.g., the voltage signal) at the output 709 of the apparatus 750 as its input, and use z(Voutput) as the function f in Equation 4 to produce the output 731 of the system 700. This allows the apparatus 750 or system 700 to perform complex operations that a single function such as Equation 3 or 7 cannot achieve alone, making it adaptable to different applications and scenarios. The activation function module 730 may be implemented by hardware, software, firmware, or any combination thereof. Since the activation function module 730 is separated from the apparatus, the area of the apparatus may be reduced and the flexibility of the apparatus may be increased

[0096] Further, each of the capacitors 705-1 to 705-i, 706-1 to 706-j, 707 and 708 may be a configurable capacitor (e.g., a configurable capacitor as shown in Fig. 5A or Fig. 5B) with variable capacitance or a capacitor with constant capacitance. Thus, in equation 7, the respective weight wi and the bias b, may be implemented by configurable capacitors, which enable the weights and / or bias to be adjusted / updated during the operation of neural networks, further increasing the flexibility of the apparatus and / or system according to the present disclosure.

[0097] The voltage at the output node 709 may be adjusted by Cut and / or Cat, for example, the voltage at the output node 709 will increase when Cut increases, and the voltage at the output node 709 will decrease when Cat increases. By introducing two groups of weighted sums, i.e., Cut calculated as Equation 5 and Cat calculated as Equation 6, the computation throughput may be increased. Equation 5 or 6 respectively augments Equations 1 or 2 with a bias, which enhance the flexibility of the apparatus and system. Thus, the neural network system 700 may increase the throughput of computations for processing input data and weight data, which further reduces the power consumption and hardware complexity, allowing the feasibility of implementation in hardware for some applications.

[0098] Fig. 8 shows an exemplary system 800 of the system in Fig. 7. In Fig.8, the pull up network 810 of the system 800 only includes one pull up branch with one pull up capacitor 805(the number of pull up capacitors in pull up network of the apparatus 850 equals to 1, i.e., i=l). That is to say, there is neither a separate pull up capacitor as indicated by 707 in Fig. 7 nor a pull up module (the number of pull up modules in pull up network of the apparatus 850 equals to 0) connected with the pull up capacitor 805 in the same pull up branch. For the pull down network 820, it is similar to the pull down network 720 in Fig. 7. For example, pull down capacitors 806-1 to 806-j, pull down module 804-1 to 804-j, inputs 822-1 to 822-j, separate pull down capacitor 808 may respectively correspond to the pull down capacitors 706-1 to 706-j, pull down module 704-1 to 704-j, inputs 722-1 to 722-j , separate pull down capacitor 708 in Fig. 7. Thus, the details regarding same or similar contents between Fig. 7 and Fig. 8 are not repeated here for simplification.

[0099] Referring back to Equations 5, 6 and 7, the output voltage at the node 809 of the apparatus 850 can be calculated as equation 8:

[0100] Equation 8 where Cuis the capacitance of the pull up capacitor 805 of the apparatus 850, and Cat is the total equivalent capacitance of all pull down capacitors in the pull down network 820 of the system 800, which can be calculated by equation 6 similarly.

[0101] From above it can be understood that, the Equation 8 used to calculate the voltage at the output 809 of the apparatus 850 may correspond to at least a part of the function f in Equation 4. Thus, the apparatus 850 may be configured to function as or apply to the neuron model 600 up to artificial neutral networks involving multiple such neurons. However, to make the apparatus 850 more flexible and applicable to more applications especially in artificial neutral networks, the apparatus 850 is followed by the activation function module 830. The activation function module 830 may be configured to further compose the Equation 8 to form a composition function z used as the function f in Equation 4, so as to extend the applications of the system 800.

[0102] Compared with the general system 700, the system 800 is simplified by only introducing one group of weighted sum, which facilitates reducing the complexity, area space of the apparatus, which in turn further reduce the power consumption for some applications. The system 800 can be used for the applications that only need one group of weighted sum (with or without a bias). Though the system 800 in Fig.8 is illustrated with one pull up branch in the pull up network of the system, it will be appreciated it may be replaced with a system having pull up networks as shown in Fig. 7 but including a pull down network only having one pull down branch without a pull down module and / or a parallel connected separate pull down capacitor.

[0103] Fig.9 shows an exemplary flow chart of a method 900 according to various embodiments of the present disclosure.

[0104] In step 910, an apparatus receiving one or more inputs. The apparatus may be any one or any 1 combination of the apparatus 100, 200, 310, 330, 350, 400, 750, 850 or any variant thereof not departing from the principle of the present disclosure.

[0105] In step 920, using a pull up capacitor to reduce the power of a pull up branch where the pull up capacitor exists in the apparatus; and / or using a pull down capacitor to reduce the power of a pull down branch where the pull down capacitor exists in the apparatus. Due to the reasons mentioned with respect to Figs. l to 8, the power efficiency of the apparatus may be improved.

[0106] Optionally, in step 930, using capacitance of each pull up capacitor as weight for a logical output generated by corresponding pull up module connected in series with the pull up capacitor, or as a weight of logical 1 in the absence of pull up module; and / or using capacitance of each pull down capacitor as weight for a logical output generated by corresponding pull down module connected in series with the pull down capacitor, or as a weight of logical 1 in the absence of pull down module.

[0107] Optionally, in step 940, applying the apparatus to a neural network system and using the logical output of the apparatus as the output of the neural network system. The logical output of the apparatus is derived from the output voltage of the apparatus.

[0108] Optionally, in step 950, applying the apparatus to a neural network system and further applying an activation function implemented by an activation function module to the output voltage of the apparatus to produce an output as the neural network system. The neural network system may include the apparatus according to the present disclosure and the activation function module which may further proceed composition function to the voltage output of the apparatus. For example, the neural network system may be implemented with the system 700, 800 or any variant thereof without departing from the spirit and scope of the disclosure.

[0109] According to the present disclosure, various examples as following are provided.

[0110] Example 1. An apparatus, comprising: a pull up network and a pull down network both of which are coupled to an output of the apparatus, when the apparatus is in operation, the pull up network is coupled between a first voltage and the output, the pull up network including one or more parallelly coupled pull up branches, each pull up branch having a pull up capacitor with one end coupled to the output, and the pull down network is coupled between a second voltage which is lower than the first voltage and the output, the pull down network including one or more parallelly coupled pull down branches, each pull down branch having a pull down capacitor with one end coupled to the output, wherein each pull up branch and / or each pull down branch, when the apparatus is in operation, is configured to receive one or more inputs to the apparatus. Example 2. The apparatus according to Example 1, wherein at least one pull up branch further comprises a pull up module coupled in series with the other end of the pull up capacitor in the respective pull up branch, and / or at least one pull down branch further comprises a pull down module coupled in series with the other end of the pull down capacitor in the respective pull down branch.

[0111] Example 3. The apparatus according to Example 2, wherein the pull up module, is configured to receive a pull up input set from the one or more inputs and generate a logical value to enable / disable the pull up branch in which the pull up module exists; and / or the pull down module is configured to receive a pull down input set from the one or more inputs and generate a logical value to enable / disable the pull down branch in which the pull down module exists.

[0112] Example 4. The apparatus according to Example 3, wherein the pull up branch is enabled when the logical value generated by the pull up module is logical 1 or when the pull up branch does not include a pull up module, and / or the pull down branch is enabled when the logical value generated by the pull down module is logical 1 or when the pull down branch does not include a pull down module.

[0113] Example 5. The apparatus according to any one of Examples 1 to 4, wherein the one or more pull up branches are individually and correspondingly coupled to the one or more pull down branches

[0114] Example 6. The apparatus according to any one of Examples 1 to 5, wherein each pull up branch of the one or more pull up branches comprises one pull up module and one pull up capacitor coupled in series between the first voltage and the output of the apparatus, and / or each pull down branch of the one or more pull down branches comprises one pull down module and one pull down capacitor coupled in series between the second voltage and the output of the apparatus.

[0115] Example 7. The apparatus according to any one of Examples 2 to 6, wherein the pull up module is a P-type module, and the pull down module is a N-type module.

[0116] Example 8. The apparatus according to Example 7, wherein the P-type module comprise one or more P-type Metal-Oxide-Semiconductor (PMOS) transistors, and the N-type module comprises one or more N-type Metal-Oxide-Semiconductor (NMOS) transistors.

[0117] Example 9. The apparatus according to any one of Examples 5 to 8, wherein the pull up module in one pull up branch is a PMOS transistor with its source coupled to the first voltage and drain coupled to the other end of the pull up capacitor, and the pull down module in the corresponding one pull down branch is a NMOS transistor with its source coupled to the second voltage and drain coupled to the other end of the pull down capacitor, wherein a common gate of the PMOS and NMOS transistors receives one input to the apparatus. Example 10 The apparatus according to any one of Examples 5 to 8, wherein the pull up module in one pull up branch comprises n parallelly coupled PMOS transistors with their sources all coupled to the first voltage and drains coupled to the other end of the pull up capacitor, and the pull down module in corresponding one pull down branch comprises n NMOS transistors coupled in series between the other end of the pull down capacitor and the second voltage, wherein n is an integer larger than one, and each PMOS transistor of the n PMOS transistors and each NMOS transistor of the n NMOS transistors form a pair to receive one input of n inputs to the apparatus from a common gate of the pair

[0118] Example 11. The apparatus according to any one of Examples 5 to 8, wherein the pull up module in one pull up branch comprises n PMOS transistors connected in series between the first voltage and the pull up capacitor, and the pull down module in corresponding one pull down branch comprises n NMOS transistors parallelly connected between the pull down capacitor and the second voltage, wherein n is an integer larger than one, and each PMOS transistor of the n PMOS transistors and each NMOS transistor of the n NMOS transistors form a pair to receive one input of n inputs to the apparatus from a common gate of the pair.

[0119] Example 12. The apparatus according to any one of Examples 5 to 8, wherein the apparatus includes multiple pull up branches and same number of pull down branches, wherein at least one pull up branch and its corresponding pull down branch is configured as an apparatus as illustrated in Example 9, at least one pull up branch and its corresponding pull down branch is configured as an apparatus as illustrated in Example 10, and / or at least one pull up branch and its corresponding pull down branch is configured as an apparatus as illustrated in Example 11.

[0120] Example 13. The apparatus according to any one of Examples 5 to 8, wherein the apparatus includes two connected parts each of which is configured as an apparatus as illustrated in Example 9, wherein an input of one part is coupled to an output of the other part, and vice the versa.

[0121] Example 14. The apparatus according to any one of Examples 1 to 12, wherein a capacitance of each pull up capacitor is used as a weight of a logical input for each pull up branch where the pull up capacitor exists, the logical input for the pull up branch being a logical value generated by the pull up module in same pull up branch or logical 1 in the absence of the pull up module, the total equivalent capacitance of all pull up branches being the weighted sum of logical input to each pull up branch of the apparatus, and / or a capacitance of each pull down capacitor is used as a weight of a logical input for each pull down branch where the pull down capacitor exists, the logical input for the pull down branch being a logical value generated by the pull down module in same pull down branch or logical 1 in the absence of the pull down module, the total equivalent capacitance of all pull down branches being the weighted sum of logical input to each pull down branch of the apparatus.

[0122] Example 15. A use of the apparatus according to Example 14 in artificial neutral networks, wherein the output voltage of the apparatus, being a function of the total equivalent capacitance of all pull up branches of the apparatus, and / or the total equivalent capacitance of all pull down branches of the apparatus, is used as an output of at least one neuron in the artificial neutral networks.

[0123] Example 16. An artificial neutral networks system comprising: an apparatus as illustrated in Example 14, and an activation function module, which in operation, is configured to: receive a signal from the output of the apparatus as an input of the activation function module, and process the received signal to generate an output of the system such that the output of the system is a function of a weighted sum of logical input to each pull up branch in the apparatus and / or a weighted sum of logical input to each pull down branch in the apparatus.

[0124] Example 17. An artificial neutral networks system comprising: an apparatus as illustrated in Example 14; a separate pull up capacitor coupled between the first voltage and the output of the apparatus, and / or a separate pull down capacitor coupled between the output of the apparatus and the second voltage; and an activation function module, which in operation, is configured to: receive a signal from the output of the apparatus as an input of the activation function module, process the received signal to generate an output of the system such that the output of the system is a function of: a sum of weighted sum of logical input to each pull up branch in the apparatus and the capacitance of the separate pull up capacitor, and / or a sum of weighted sum of logical input to each pull down branch in the apparatus and the capacitance of the separate pull down capacitor.

[0125] Example 18. The system according to Example 16 or 17, wherein at least one up to all capacitors used in the system are configurable capacitors having variable capacitance.

[0126] Example 19. A method, comprising: an apparatus as illustrated in Example 14 receiving one or more inputs; using each pull up capacitor to reduce the power of each pull up branch where the pull up capacitor exists; and / or using each pull down capacitor to reduce the power of each pull down branch where the pull down capacitor exists.

[0127] Example 20. A method, comprising: an apparatus as illustrated in Example 14 receiving one or more inputs; using capacitance of each pull up capacitor as a weight for a logical output generated by the pull up module coupled in series with the pull up capacitor, or as a weight of logical 1 in the absence of pull up module, and / or using capacitance of each pull down capacitor as a weight for a logical output generated by the pull down module connected in series with the pull down capacitor or as a weight of logical 1 in the absence of pull down module; applying the apparatus to a neural network system; and applying an activation function to the output voltage of the apparatus to generate an output as the neural network system.

[0128] It should note, all capacitors in the apparatus, system, or method as illustrated in above Examples may have fixed capacitance or variable capacitance.

[0129] While particular embodiments of the present disclosure have been shown and described, it will be appreciated by those skilled in the art that, based upon the teachings herein, changes, variations and modifications may be made without departing from the spirit and scope of the disclosure. Accordingly, it is intended to embrace all such alternatives, modification and variations that fall with the sprit and broad scope of the appended claims.

Claims

Claims1. An apparatus, comprising: a pull up network and a pull down network both of which are coupled to an output of the apparatus, when the apparatus is in operation, the pull up network is coupled between a first voltage and the output, the pull up network including one or more parallelly coupled pull up branches, each pull up branch having a pull up capacitor with one end coupled to the output, and the pull down network is coupled between a second voltage which is lower than the first voltage and the output, the pull down network including one or more parallelly coupled pull down branches, each pull down branch having a pull down capacitor with one end coupled to the output, wherein each pull up branch and / or each pull down branch, when the apparatus is in operation, is configured to receive one or more inputs to the apparatus.

2. The apparatus according to claim 1, wherein at least one pull up branch further comprises a pull up module coupled in series with the other end of the pull up capacitor in the respective pull up branch, and / or at least one pull down branch further comprises a pull down module coupled in series with the other end of the pull down capacitor in the respective pull down branch.

3. The apparatus according to claim 2, wherein the pull up module, is configured to receive a pull up input set from the one or more inputs and generate a logical value to enable / disable the pull up branch in which the pull up module exists; and / or the pull down module is configured to receive a pull down input set from the one or more inputs and generate a logical value to enable / disable the pull down branch in which the pull down module exists.

4. The apparatus according to claim 3, wherein the pull up branch is enabled when the logical value generated by the pull up module is logical 1 or when the pull up branch does not include a pull up module, and / or the pull down branch is enabled when the logical value generated by the pull down module is logical 1 or when the pull down branch does not include a pull down module.

5. The apparatus according to any one of claims 1 to 4, wherein the one or more pull up branches are individually and correspondingly coupled to the one or more pull down branches.6 The apparatus according to any one of claims 1 to 5, wherein each pull up branch of the one or more pull up branches comprises one pull up module and one pull up capacitor coupled in series between the first voltage and the output of the apparatus, and / or each pull down branch of the one or more pull down branches comprises one pull down module and one pull down capacitor coupled in series between the second voltage and the output of the apparatus.

7. The apparatus according to any one of claims 2 to 4, wherein the pull up module in one pull up branch is a PMOS transistor with its source coupled to the first voltage and drain coupled to the other end of the pull up capacitor, and the pull down module in the corresponding one pull down branch is a NMOS transistor with its source coupled to the second voltage and drain coupled to the other end of the pull down capacitor, wherein a common gate of the PMOS and NMOS transistors receives one input to the apparatus.

8. The apparatus according to any one of claims 2 to 4, wherein the pull up module in one pull up branch comprises n parallelly coupled PMOS transistors with their sources all coupled to the first voltage and drains coupled to the other end of the pull up capacitor, and the pull down module in corresponding one pull down branch comprises n NMOS transistors coupled in series between the other end of the pull down capacitor and the second voltage, wherein n is an integer larger than one, and each PMOS transistor of the n PMOS transistors and each NMOS transistor of the n NMOS transistors form a pair to receive one input of n inputs to the apparatus from a common gate of the pair.9 The apparatus according to any one of claims 2 to 4, wherein the pull up module in one pull up branch comprises n PMOS transistors connected in series between the first voltage and the pull up capacitor, and the pull down module in corresponding one pull down branch comprises n NMOS transistors parallelly connected between the pull down capacitor and the second voltage, whereinn is an integer larger than one, and each PMOS transistor of the n PMOS transistors and each NMOS transistor of the n NMOS transistors form a pair to receive one input of n inputs to the apparatus from a common gate of the pair.

10. The apparatus according to any one of claims 2 to 4, wherein the apparatus includes multiple pull up branches and same number of pull down branches, wherein at least one pull up branch and its corresponding pull down branch is configured as an apparatus as claimed in claim 7, at least one pull up branch and its corresponding pull down branch is configured as an apparatus as claimed in claim 8, and / or at least one pull up branch and its corresponding pull down branch is configured as an apparatus as claimed in claim 9.

11. The apparatus according to any one of claims 2 to 4, wherein the apparatus includes two connected parts each of which is configured as an apparatus as claimed in claim 7, wherein an input of one part is coupled to an output of the other part, and vice the versa.12 The apparatus according to any one of claims 1 to 10, wherein a capacitance of each pull up capacitor is used as a weight of a logical input for each pull up branch where the pull up capacitor exists, the logical input for the pull up branch being a logical value generated by the pull up module in same pull up branch or logical 1 in the absence of the pull up module, the total equivalent capacitance of all pull up branches being the weighted sum of logical input to each pull up branch of the apparatus, and / or a capacitance of each pull down capacitor is used as a weight of a logical input for each pull down branch where the pull down capacitor exists, the logical input for the pull down branch being a logical value generated by the pull down module in same pull down branch or logical 1 in the absence of the pull down module, the total equivalent capacitance of all pull down branches being the weighted sum of logical input to each pull down branch of the apparatus.

13. Ause of the apparatus according to claim 12 in artificial neutral networks, wherein the output voltage of the apparatus, being a function of the total equivalent capacitance of all pull up branches of the apparatus, and / or the total equivalent capacitance of all pull down branches of the apparatus, is used as an output of at least one neuron in the artificial neutral networks14 An artificial neutral networks system comprising: an apparatus as claimed in claim 12; and an activation function module, which in operation, is configured to: receive a signal from the output of the apparatus as an input of the activation function module, and process the received signal to generate an output of the system such that the output of the system is a function of a weighted sum of logical input to each pull up branch in the apparatus and / or a weighted sum of logical input to each pull down branch in the apparatus.

15. An artificial neutral networks system comprising: an apparatus as claimed in claim 12; a separate pull up capacitor coupled between the first voltage and the output of the apparatus, and / or a separate pull down capacitor coupled between the output of the apparatus and the second voltage; and an activation function module, which in operation, is configured to: receive a signal from the output of the apparatus as an input of the activation function module, process the received signal to generate an output of the system such that the output of the system is a function of: a sum of weighted sum of logical input to each pull up branch in the apparatus and the capacitance of the separate pull up capacitor, and / or a sum of weighted sum of logical input to each pull down branch in the apparatus and the capacitance of the separate pull down capacitor.

16. The system according to claim 14 or 15, wherein at least one up to all capacitors used in the system are configurable capacitors having variable capacitance.

17. A method, comprising: an apparatus as claimed in claim 12 receiving one or more inputs; using each pull up capacitor to reduce the power of each pull up branch where the pull up capacitorexists; and / or using each pull down capacitor to reduce the power of each pull down branch where the pull down capacitor exists.

18. A method, comprising: an apparatus as claimed in claim 12 receiving one or more inputs; using capacitance of each pull up capacitor as a weight for a logical output generated by the pull up module coupled in series with the pull up capacitor, or as a weight of logical 1 in the absence of pull up module, and / or using capacitance of each pull down capacitor as a weight for a logical output generated by the pull down module connected in series with the pull down capacitor or as a weight of logical 1 in the absence of pull down module; applying the apparatus to a neural network system; and applying an activation function to the output voltage of the apparatus to generate an output as the neural network system.

Images