Capacitor-based synapse network structure with metal shielding between outputs

Abstract

A neural network device includes a first plurality of synaptic network capacitors, where synaptic network capacitors of the first plurality of synaptic network capacitors share a first output terminal. The neural network device also includes a second plurality of synaptic network capacitors, where synaptic network capacitors of the second plurality of synaptic network capacitors share a second output terminal. Still further, the neural network device includes a metal shield disposed between the first output terminal and the second output terminal. The neural network device can be used as part of an artificial intelligence system.

Description

Background Technology

[0001] Deep neural networks (DNNs) are examples of artificial neural networks (ANNs), which are structures consisting of multiple layers (e.g., an input layer, multiple hidden or intermediate layers, and an output layer). Each layer comprises network nodes (represented as "neurons") operatively coupled via connectors weighted according to the relative importance of connections (represented as "synapses") in different configurations. DNNs have many hidden or intermediate layers (e.g., on the order of tens, hundreds, or even thousands, hence the term "depth") and are designed to mimic the decision-making functions of the biological human (or animal) brain.

[0002] Therefore, DNNs have been implemented in many technological fields, including but not limited to artificial intelligence systems, which include machine or deep learning algorithms implemented in one or more neuromorphic or neural network devices. Examples of applications of neural network-based artificial intelligence systems include, but are not limited to, image or object recognition. In image or object recognition, a type of DNN called a convolutional neural network (CNN) is used (e.g., a CNN performs mathematical convolution operations to generate one or more image classifications).

[0003] The underlying computational hardware used to implement DNNs (CNNs or any other form of ANN) can include, but is not limited to, a central processing unit (CPU), a graphics processing unit (GPU), or an application-specific integrated circuit (ASIC) based on standard complementary metal-oxide-semiconductor (CMOS) logic. Resistor-based networks (e.g., memristors) have commonly been used to implement the synapses of DNNs. Summary of the Invention

[0004] Embodiments of the present invention provide a capacitor-based synaptic network structure with metal shielding between outputs.

[0005] In one illustrative embodiment, a neural network device includes a first plurality of synaptic network capacitors, wherein the synaptic network capacitors of the first plurality of synaptic network capacitors share a first output terminal. The neural network device also includes a second plurality of synaptic network capacitors, wherein the synaptic network capacitors of the second plurality of synaptic network capacitors share a second output terminal. Furthermore, the neural network device includes a metallic shield disposed between the first output terminal and the second output terminal.

[0006] Further illustrative embodiments are provided, respectively, in the form of a method for manufacturing a neural network device having a capacitor-based synaptic network (with a metal-shielded output) and an artificial intelligence system having a neural network device (with a capacitor-based synaptic network having a metal-shielded output).

[0007] These and other features and advantages of the embodiments described herein will become more apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0008] Figure 1 A block diagram of a neural network computing environment is depicted, which can be used to implement one or more illustrative embodiments.

[0009] Figure 2 A schematic circuit diagram of a neural network device is depicted, which can be used to implement one or more illustrative embodiments.

[0010] Figure 3 A schematic circuit diagram is depicted for a single-output capacitor-based structure used to implement a positive synaptic network.

[0011] Figure 4 A schematic circuit diagram is depicted for a single-output capacitor-based structure used to realize a negative synaptic network.

[0012] Figure 5 A schematic circuit diagram is depicted to illustrate the coupling capacitance problem in a multi-output capacitor-based structure for implementing synaptic networks.

[0013] Figure 6 An isometric structural diagram is depicted of a multi-output capacitor-based structure with metal shielding between outputs, used to implement a synaptic network, according to an illustrative embodiment.

[0014] Figure 7 A schematic circuit diagram is depicted showing a capacitor-based structure with metal shielding between outputs for implementing a synaptic network, according to an illustrative embodiment.

[0015] Figure 8A A flowchart is depicted according to an illustrative embodiment of a method for manufacturing a multi-output capacitor-based structure with metallic shielding between outputs for implementing a synaptic network.

[0016] Figure 8B Another flowchart depicts a method for manufacturing a multi-output capacitor-based structure with metallic shielding between outputs for implementing a synaptic network, according to an illustrative embodiment.

[0017] Figure 8C A block diagram of an artificial intelligence system having one or more neural network devices according to an illustrative embodiment is depicted, the one or more neural network devices having a capacitor-based synaptic network with a metal-shielded output.

[0018] Figure 9 A diagram depicts an exemplary processor system according to an illustrative embodiment.

[0019] Figure 10 A diagram depicts a cloud computing environment according to an illustrative embodiment.

[0020] Figure 11 A diagram depicting an abstract model layer according to an illustrative embodiment is provided. Detailed Implementation

[0021] As mentioned in the background section above, the underlying computational hardware used to implement DNNs (CNNs or any ANNs) can include, but is not limited to, CPUs, GPUs, and ASICs. For example, resistive random access memory (ReRAM) or phase-change memory (PCM) devices (called crosspoint structures) have been used to perform computations associated with DNNs, where resistance / conductance is used to represent synapses. ReRAM is a type of non-volatile RAM that uses changes in resistance across a dielectric solid material called a memristor to store data. PCM is a type of non-volatile RAM that uses the resistance comparison between a highly conductive crystalline phase and a low-conductivity amorphous phase of a phase-change material to store data.

[0022] In addition to the resistor-based schemes mentioned above for implementing synapses in DNNs, it has been recognized that capacitors can be used to represent synapses. Capacitor-based synaptic networks provide similar neuronal connection functionality to resistor-based synaptic networks, but without consuming static power. However, a drawback of neural networks with capacitor-based synapses is that output nodes may interact through unwanted coupling capacitance.

[0023] Before further explaining the drawbacks associated with existing capacitor-based synaptic networks and how the illustrative embodiments overcome these and other drawbacks, an example of an artificial intelligence system having a set of neural network devices that can be used to implement the illustrative embodiments will be described.

[0024] More specifically, Figure 1 A neural network computing environment 100 including a neural network device 102 is depicted, which can be used to implement one or more illustrative embodiments. As shown, the neural network device 102 includes a neural network 104 and receives one or more input signals and generates one or more output signals.

[0025] As an example only, in image or object recognition applications, the input signal may include an image, and a neural network device implements a CNN to generate one or more image classifications as output signals. A CNN uses convolutional layers of neurons connected via synapses, and features are extracted from the image and processed by the network. The final layer of the CNN provides the computed classification associated with the input image, resulting in a trained neural network with weights and feature detectors. The trained neural network is then used for real-time image recognition based on the target image.

[0026] Therefore, in Figure 1 In this context, neural network device 102 is configured to train neural network 104, and then use the trained neural network 104 to perform image or object recognition. As a simple example, images of cars can be used to train the CNN, and subsequent images can then be applied to the trained CNN to determine whether any image in the subsequent images includes a car. Using, for example... Figure 1 Image recognition using the illustrated neural network devices can be used in a variety of applications, including but not limited to engineering, security, and medicine, to name just a few. Multiple neural network devices 102 can be used together to form an artificial intelligence system. Furthermore, in some embodiments, a single neural network device 102 may include multiple neural networks 104. Additionally, in some embodiments having multiple neural network devices 102, one or more of the neural network devices 102 may have a single neural network 104, while one or more other neural network devices 102 may have multiple neural networks 104.

[0027] Figure 2 A schematic circuit diagram of a neural network device 200 is described, which can be used to implement one or more exemplary embodiments. It should be understood that the neural network device 200 is... Figure 1 The example implementation of the neural network device 102 with neural network 104 in the example should be understood to be related to the detailed description of neural network device 102 and neural network 104 in the illustrative embodiments. Figure 2 As described in the text, neural network devices and neural networks can have... Figure 2 One or more other components and elements not explicitly shown in the document.

[0028] As shown in the figure, the neural network device 200 includes multiple word lines 210 (WL1, WL2, WL3, WL4, ..., WLN) and multiple bit lines 220 (BL1, BL2, BL3, ..., BLP). The word lines 210 and bit lines 220 are not directly connected, but instead have synaptic elements connected between them at each intersection (i.e., physical locations where the word lines and bit lines are physically close to each other but not physically connected). The synaptic elements... Figure 2The synaptic elements are represented as synaptic elements 231, 232, 233, 234, ..., 23Q. Each bit line 220 also has an integrator 240 and a threshold detector 242 connected as shown. The plurality of synaptic elements 231, 232, 233, 234, ..., 23Q, integrator 240, and threshold detector 242 collectively constitute a neural network portion 230 (e.g., part of neural network 104 of neural network device 102). Each neural network portion 230 is configured to change its electrostatic capacitance in response to one or more input signals. Furthermore, it should be understood that each individual synaptic element 231, 232, 233, 234, ..., 23Q is addressable via its corresponding word line, and the plurality of synaptic devices forming each neural network portion 230 are addressable via their corresponding bit lines. In the illustrative embodiment, the neural network portion 230 is implemented using semiconductor manufacturing techniques to produce one or more semiconductor devices. Note that although memory device type terms such as “word line” and “bit line” are used illustratively herein, it is understood that these control signal lines are used more generally to access each individual synaptic element and may not necessarily be referred to as word lines and bit lines in every neuromorphic embodiment.

[0029] Figure 3 A schematic circuit diagram of a single-output capacitor-based structure 300 for implementing a positive synaptic network is depicted. More specifically, the single-output capacitor-based structure 300 is... Figure 2 Examples of multiple synaptic elements 231, 232, 233, 234, ..., 23Q in the neural network portion 230. As shown in the figure, the single-output capacitor-based structure 300 includes multiple capacitors C1, C2, C3, ..., C... n Each capacitor has a capacitor specifically designed for C1, C2, C3, ..., C n Coupled to input voltage sources V1, V2, V3, ..., V n The first terminal is coupled to a single common output V. o The second terminal. Note that the inputs are V1, V2, V3, ..., V n Corresponding to Figure 2 The word (access) lines are WL1, WL2, WL3, ..., WLN, and the output is V. o Corresponding to Figure 2 One of the access lines BL1, BL2, BL3, ... or BLP. Each capacitor C1, C2, C3, ..., C n Configured such that its electrostatic capacitance can be based on its corresponding input voltage V1, V2, V3, ..., V n And is adjusted to one of several levels. It should be understood that, for simplicity, the integrator 240 and threshold detector 242 are not... Figure 3As shown in the diagram; however, it should be understood that integrator 240 receives instructions for each capacitor C1, C2, C3, ..., C n The signal representing the electrostatic capacitance level is used to generate an integrated signal. This integrated signal is then compared to a given threshold in threshold detector 242. More specifically, the total electrostatic capacitance C... tot Defined as:

[0030]

[0031] And therefore, the voltage output V o Defined as:

[0032]

[0033] It is obvious that V o It is a linear combination of all input voltages. Each input voltage V i The synaptic value is C i / C tot V o The comparison is made against a given threshold, and based on the comparison result, the neural network part 230 generates a decision.

[0034] Figure 4 A schematic circuit diagram of a single-output capacitor-based structure 400 for implementing a negative synaptic network is depicted. More specifically, the single-output capacitor-based structure 400 is... Figure 2 Another example of multiple synaptic elements 231, 232, 233, 234, ..., 23Q in the neural network portion 230. As shown, the single-output capacitor-based structure 400 includes multiple capacitors C. 11 C 12 C 21 C 22 ..., C n1 C n2 Each capacitor has a specific design for C. 11 C 12 C 21 C 22 ..., C n1 C n2 Coupled to input voltage sources V1, -V1, V2, -V2, ..., V n -V n The first terminal is coupled to a single common output V. o The second terminal of each capacitor C. 11 C 12 C 21 C 22 ..., C n1 C n2Configured such that its electrostatic capacitance is based on its corresponding input voltage V1, -V1, V2, -V2, ..., V n -V n It can be adjusted to one of multiple levels. It should be understood that, for the sake of simplifying the accompanying drawings, the integrator 240 and the threshold detector 242 are... Figure 4 Not shown in the diagram; however, it should be understood that integrator 240 receives instructions for each capacitor C. 11 C 12 C 21 C 22 ..., C n1 C n2 The signal represents the electrostatic capacitance level, and thus generates an integral signal. The integral signal is compared to a given threshold in threshold detector 242. More specifically, it is similar to the definitions in equations (1) and (2) above:

[0035]

[0036] And therefore:

[0037]

[0038] If C i1 <C i2 If so, the synapse is considered negative. Again, V o The comparison is made against a given threshold, and based on the comparison result, the neural network part 230 generates a decision.

[0039] Figure 5 A schematic circuit diagram depicts the coupling capacitance problem in a multi-output capacitor-based structure 500 for implementing a synaptic network. As shown, the multi-output capacitor-based structure 500 includes a first plurality of capacitors C. 11 C 21 C 31 ..., C n1 Second and multiple capacitors C 12 C 22 C 32 ..., C n2 The first plurality of capacitors C 11 C 21 C 31 ..., C n1 Each capacitor in the circuit has a capacitor coupled to an input voltage source V1, V2, V3, ..., V... n The first terminal is coupled to a single common output V. o1 The second terminal. The second plurality of capacitors C. 12 C 22 C 32 ..., C n2Each capacitor in the circuit has a capacitor coupled to an input voltage source V1, V2, V3, ..., V... n The first terminal is coupled to a single common output V. o2 The second terminal. Although Figure 5 Each of the two groups of capacitors in the middle is connected to Figure 3 The capacitors shown are identical, but each can be alternatively matched with... Figure 4 The multiple capacitors shown are configured identically, or in some other configuration. The key point is that, due to the parasitic capacitive coupling effect caused by the electric field, one signal near another causes interference in the form of noise in that other signal. Therefore, as... Figure 5 As shown, the output V o1 Interference output V o2 Conversely, the parasitic capacitive coupling effect between the two outputs is represented as the coupling capacitance C. x Because of C x The existence of V o1 and V o2 They influence each other, therefore they cannot be defined by the direct formulas of equations (2) or (4) above. Output V o1 and V o2 This unwanted coupling capacitance between them causes problems for the neural network because it can introduce errors in the integration (integrator 240) and threshold detection (threshold detector 242) stages, and thus lead to incorrect judgments in the entire artificial intelligence algorithm.

[0040] The illustrative embodiments overcome the above and other disadvantages associated with capacitor-based synaptic networks by providing a structure with metal shielding between outputs.

[0041] Figure 6 An isometric structural diagram of a multi-output capacitor-based structure 600 with metallic shielding between outputs for implementing a synaptic network, according to an illustrative embodiment, is depicted. More specifically, as shown, the multi-output capacitor-based structure 600 includes three groups of multiple capacitors 602-1, 602-2, and 602-3, each group comprising three capacitors, each labeled 610. Each capacitor 610 (represented for simplicity by the capacitor in the upper right corner of the figure) includes a first terminal 612 and a second terminal 614, wherein a dielectric material 616 is disposed between the two terminals 612 and 614. The first terminal 612 of each capacitor 610 is operatively coupled to an input voltage V1, V2, or V3, respectively. The second terminal 614 of each capacitor 610 is operatively coupled to an output voltage, in this case V... o3 (That is, as shown in the figure, capacitors in the same group of multiple capacitors are operatively coupled to the same output V.) o1 V o2 or Vo3 Dielectric material 616 is a high-k material, such as, but not limited to, hafnium dioxide (HfO2), zirconium dioxide (ZrO2), and combinations thereof.

[0042] Furthermore, each of the plurality of capacitors 602 has an associated metal shield 620. In an illustrative embodiment, the metal shield 620 is made of materials such as, but not limited to, titanium nitride (TiN), tungsten (W), copper (Cu), and combinations thereof. In an illustrative embodiment, the size of the metal shield 620 can range from a few nanometers to hundreds of nanometers, depending on the size scaling of the capacitors. As shown, the metal shield 620 is disposed between each of the plurality of capacitors 620 shown. The metal shield 620 disposed in this manner prevents interference from adjacent outputs to a given output, such as V. o1 and V o2 They are capacitively decoupled from each other, while V o2 and V o3 They are capacitively decoupled from each other. Figure 6 In the illustrated embodiment, the metal shield 620 is grounded; however, the metal shield 620 has an added C tot This will be in Figure 7 Further explanation will be provided in the context of this.

[0043] Figure 7 A schematic circuit diagram is depicted of a multi-output capacitor-based structure 700, which has metallic shielding between outputs and is used to implement a synaptic network according to an illustrative embodiment. It should be understood that the multi-output capacitor-based structure 700 is similar to... Figure 5 The multi-output capacitor-based structure 500, but assuming metal shielding 620, is as follows. Figure 6 The configuration shown is between multiple capacitors 702-1 and 702-2, such that the output V o1 It has a separate shielding capacitor C x1 And the output V o2 It has a separate shielding capacitor C x2 .

[0044] Therefore, through the example of multiple capacitors 702-1, C tot Now defined as:

[0045]

[0046] And therefore:

[0047]

[0048] For multiple capacitors 702-2 C tot and V o2It is changed in the same way as shown in equations (5) and (6), but with the addition of C. x2 Instead of C x1 .

[0049] Figure 8A A flowchart depicts a method 800 for manufacturing a multi-output capacitor-based structure with metallic shielding between outputs for implementing a synaptic network, according to an illustrative embodiment. It will be understood that method 800 can be used, for example, to manufacture... Figure 1 Method 800 is performed as part of the semiconductor manufacturing process of the neural network device 102 in the example; however, in an alternative embodiment, method 800 may be performed by other manufacturing techniques configured to perform the steps of the method.

[0050] Step 802 forms at least two groups of multiple capacitors, wherein multiple capacitors in the same group share a common output while having individual inputs (e.g., see...). Figure 6 Or 7). It should be understood that the semiconductor fabrication of the at least two sets of multiple capacitors can be performed in a variety of ways. By way of example only, the capacitor fabrication technique described in U.S. Patent No. 10,505,108 entitled "Memcapacitor, Neuro Device, and Neural Network Device" and issued on December 10, 2019, is suitable for fabricating the at least two sets of multiple capacitors in step 802.

[0051] Step 804 involves forming a metallic shielding layer between the outputs of each pair of at least two groups of multiple capacitors as described herein.

[0052] In at least one embodiment based on semiconductor manufacturing, reference ( Figure 6 (The) multi-output capacitor-based structure 600, Figure 8B A method 810 is described as being performed to form a metallic shielding layer between the outputs of each pair of multiple capacitors 602.

[0053] First, in step 811, the metal wire (V) of the capacitor output terminal is formed. o1 V o2 or V o3Next, in step 812, a high-k dielectric material 616 is formed and patterned. In step 813, a first interlayer dielectric (ILD) layer is formed and a chemical mechanical planarization (CMP) operation is applied. Next, in step 814, trenches are formed in the first ILD layer and metal is deposited to form shielding metal lines. In step 815, a second ILD layer is formed and CMP is applied. In step 816, via structures are etched to reach the high-k dielectric material 616, and then the via structures are metallized. In step 817, capacitor input metal lines (V1, V2, or V3) are then formed. It should be understood that... Figure 8B The manufacturing process described herein is merely an example, and alternative embodiments are expected to include additional or alternative manufacturing steps and techniques, and in different order, to form a multi-output capacitor-based structure with metal shielding between outputs.

[0054] Figure 8C A block diagram of an artificial intelligence system 820 with one or more neural network devices according to an illustrative embodiment is depicted. The one or more neural network devices have a capacitor-based synaptic network with a metal-shielded output. In one or more illustrative embodiments, it is possible to... Figure 8A Method 800 and / or Figure 8B Method 810 is used to manufacture capacitor-based synaptic networks with metal-shielded outputs.

[0055] As shown in the figure, the artificial intelligence system 820 includes one or more neural network devices 822 having capacitor-based synaptic networks with metal-shielded outputs 824. In one exemplary embodiment, the one or more neural network devices 822 of the artificial intelligence system 820 with capacitor-based synaptic networks are implemented by one or more application-specific integrated circuits (ASICs) having metal-shielded outputs 824. An ASIC is an integrated circuit (IC) chip or device customized for a specific purpose, including logic (e.g., circuits, processors, memory, etc.) programmed with executable program code (e.g., instruction code, computer program code, etc.) or otherwise configured for that purpose. In this exemplary case, the specific purpose is to implement and execute an artificial intelligence system (e.g., machine learning algorithms), and more specifically, to implement a synaptic network that serves as part of a neural network. An ASIC is also considered a system-on-a-chip (SoC).

[0056] It should also be understood that the artificial intelligence system 820 and its components can be implemented in alternative circuit / processor-based technologies, such as those including one or more multi-core central processing units (CPUs), one or more graphics processing units (GPUs), and one or more field-programmable gate arrays (FPGAs). In some embodiments, the artificial intelligence system 820 can be implemented as a combination of two or more circuit / processor-based technologies (e.g., ASICs, CPUs, GPUs, FPGAs, etc.).

[0057] As described in this article, Figures 1-8C The techniques described herein may also include providing a system comprising different software modules, each contained on a tangible computer-readable and recordable storage medium. For example, all modules (or any subset thereof) may be on the same medium, or each may be on a different medium. Modules may include any or all of the components shown in the figures and / or described herein. In embodiments of the invention, modules may, for example, run on a hardware processor. The method steps can then be performed using the different software modules of a system executed on a hardware processor as described above. Furthermore, the computer program product may include a tangible computer-readable and recordable storage medium having code adapted to be executed to perform at least one method step described herein, including providing the different software modules to the system.

[0058] in addition, Figures 1-8C The techniques described herein can be implemented via a computer program product that may include computer-usable program code stored in a computer-readable storage medium in a data processing system, wherein the computer-usable program code is downloaded from a remote data processing system via a network. Furthermore, in embodiments of the invention, the computer program product may include computer-usable program code stored in a computer-readable storage medium in a server data processing system, wherein the computer-usable program code is downloaded to a remote data processing system via a network for use by the remote system on the computer-readable storage medium.

[0059] Embodiments of the present invention or elements thereof may be implemented in the form of an apparatus including a memory and at least one processor coupled to the memory and configured to perform exemplary method steps.

[0060] Furthermore, embodiments of the present invention can utilize software running on a computer or workstation. (See reference) Figure 9Such an implementation may employ, for example, a processor 902, a memory 904, and an input / output interface, such as a display 906 and a keyboard 908. The term "processor" as used herein is intended to include any processing device, such as a processing device including a multi-core CPU, GPU, FPGA, and / or other forms of processing circuitry such as one or more ASICs. Furthermore, the term "processor" may refer to more than one individual processor. The term "memory" is intended to include memory associated with a processor (e.g., CPU, GPU, FPGA, ASIC, etc.), such as, for example, RAM (Random Access Memory), ROM (Read-Only Memory), fixed memory devices (e.g., hard disk drives), removable memory devices (e.g., disk drives), flash memory, etc. Additionally, the phrase "input / output interface" as used herein is intended to include, for example, mechanisms for inputting data to the processing unit (e.g., a mouse) and mechanisms for providing results associated with the processing unit (e.g., a printer). The processor 902, memory 904, and input / output interfaces such as the display 906 and keyboard 908 may be interconnected, for example, via a bus 910, as part of a data processing unit 912. For example, appropriate interconnections via bus 910 can also be provided to network interface 914, such as a network card, which can be provided to interface with a computer network, and to media interface 916, such as a disk or CD-ROM drive, which can be provided to interface with media 918.

[0061] Therefore, computer software including instructions or code for performing methods as described in embodiments of the invention herein can be stored in an associated memory device (e.g., ROM, fixed or removable memory) and, when ready for use, partially or entirely loaded (e.g., loaded into RAM) and implemented by a CPU. Such software may include, but is not limited to, firmware, resident software, microcode, etc.

[0062] A data processing system suitable for storing and / or executing program code will include at least one processor 902 directly or indirectly coupled to a memory element 904 via a system bus 910. The memory element may include local memory used during the actual implementation of the program code, a mass storage device, and a cache memory that provides temporary storage for at least some of the program code to reduce the number of times code must be retrieved from the mass storage device during implementation.

[0063] Input / output or I / O devices (including but not limited to keyboard 908, display 906, indicator devices, etc.) can be coupled to the system directly (such as via bus 910) or through an intermediate I / O controller (omitted for clarity).

[0064] Network adapters such as the Network Interface 914 can also be coupled to the system, enabling the data processing system to couple to other data processing systems or remote printers or storage devices via an intermediate private or public network. Modems, cable modems, and Ethernet cards are just a few of the currently available types of network adapters.

[0065] As used herein (including the claims), "server" includes a physical data processing system that runs server programs (e.g., such as...). Figure 9 (See system 912 shown). It is understood that such a physical server may or may not include a monitor and keyboard.

[0066] This invention can be a system, method, and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to perform aspects of the invention.

[0067] Computer-readable storage media can be tangible devices capable of retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or recessed structures with instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0068] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device, or via a network, such as the Internet, a local area network (LAN), a wide area network (WAN), and / or a wireless network, to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the respective computing / processing device.

[0069] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages ​​such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, to perform aspects of this invention, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions to personalize the electronic circuits by utilizing the status information of the computer-readable program instructions.

[0070] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0071] These computer-readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and / or other devices to operate in a particular manner, such that the computer-readable storage medium in which the instructions are stored includes an article of writing comprising instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0072] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions, which execute on the computer, other programmable apparatus or other device, perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0073] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than indicated in the figures. For example, two blocks shown consecutively may actually be implemented as a single step, executed simultaneously, substantially simultaneously, with partial or complete time overlap, or these blocks may sometimes be executed in reverse order, depending on the functions involved. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0074] It should be noted that any method described herein may include additional steps of providing a system comprising different software modules implemented on a computer-readable storage medium; modules may include any or all of the components detailed herein. The method steps may then be performed using the different software modules and / or submodules of the system as described above, which execute on hardware processor 902. Furthermore, the computer program product may include a computer-readable storage medium having code suitable for implementation to perform at least one method step described herein, including providing different software modules to the system.

[0075] In any case, it should be understood that the components shown herein can be implemented in various forms, including hardware, software, or combinations thereof, such as application-specific integrated circuits (ASICs), functional circuits, suitably programmed digital computers with associated memory, etc. Given the teachings of the invention provided herein, those skilled in the art will be able to conceive of other implementations of the components of the invention.

[0076] It should be understood that although this disclosure includes a detailed description of cloud computing, the implementation of the teachings set forth herein is not limited to a cloud computing environment. Rather, embodiments of the invention can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0077] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with service providers. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0078] The features are as follows:

[0079] On-demand self-service: Cloud consumers can unilaterally and automatically provide computing power, such as server time and network storage, as needed, without requiring manual interaction with the service provider.

[0080] Wide Area Network (WAN) Access: Capabilities are available on the network and accessed through standard mechanisms that facilitate the use of heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0081] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. There is a sense of location independence because consumers typically do not control or know the exact location of the resources provided, but can specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0082] Rapid and flexible: Features can be delivered quickly and flexibly, and in some cases, automatically, to scale out quickly and release quickly to scale in. For consumers, the available capabilities typically appear unlimited and can be purchased at any time and in any quantity.

[0083] Measurement services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the service type (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both service providers and consumers.

[0084] The service model is as follows:

[0085] Software as a Service (SaaS): The capability offered to consumers is the ability to use the provider's applications running on cloud infrastructure. Applications can be accessed from various client devices through a thin client interface such as a web browser (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating system, storage, or even the individual application capabilities, with possible exceptions of limited user-specific application configuration settings.

[0086] Platform as a Service (PaaS): This provides consumers with the ability to deploy applications created or acquired by the consumer onto cloud infrastructure using programming languages ​​and tools supported by the provider. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they have control over the deployed applications and the configuration of possible application hosting environments.

[0087] Infrastructure as a Service (IaaS): This provides consumers with the capability to deliver processing, storage, networking, and other basic computing resources that enable them to deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but they do have control over the operating system, storage, deployed applications, and possibly limited control over chosen networking components (e.g., host firewalls).

[0088] The deployment model is as follows:

[0089] Private cloud: Cloud infrastructure operated solely by an organization. It can be managed by the organization or a third party, and can exist internally or externally.

[0090] Community cloud: A cloud infrastructure shared by several organizations and supporting a specific community with shared concerns (e.g., tasks, security requirements, policies, and compliance considerations). It can be managed by an organization or a third party and can exist internally or externally.

[0091] Public cloud: Cloud infrastructure available to the general public or large industrial groups and owned by organizations that sell cloud services.

[0092] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain a single entity but are bound together by standardized or proprietary technologies that enable data and applications to be ported together (e.g., cloud bursting for load balancing between clouds).

[0093] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is the infrastructure of a network of interconnected nodes.

[0094] Now for reference Figure 10 The diagram illustrates an illustrative cloud computing environment 1050. As shown, the cloud computing environment 1050 includes one or more cloud computing nodes 1010 to which local computing devices used by cloud consumers can communicate, such as personal digital assistants (PDAs) or cellular phones 1054A, desktop computers 1054B, laptop computers 1054C, and / or automotive computer systems 1054N. The nodes 1010 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment 1050 to provide infrastructure, platform, and / or software as a service, without requiring cloud consumers to maintain resources on their local computing devices. It should be understood that... Figure 10The types of computing devices 1054A-N shown are for illustrative purposes only, and computing node 1010 and cloud computing environment 1050 can communicate with any type of computerized device via any type of network and / or network-addressable connection (e.g., using a web browser).

[0095] Now for reference Figure 11 It describes the 1050 cloud computing environment ( Figure 10 This provides a set of functional abstractions. It should be understood beforehand that... Figure 11 The components, layers, and functions shown are for illustrative purposes only, and embodiments of the invention are not limited thereto. As depicted, the following layers and corresponding functions are provided:

[0096] The hardware and software layer 1160 includes hardware and software components. Examples of hardware components include: a host 1161; a server 1162 based on a RISC (Reduced Instruction Set Computer) architecture; a server 1163; a blade server 1164; a storage device 1165; and network and networking components 1166. In some embodiments, software components include network application server software 1167 and database software 1168.

[0097] The virtualization layer 1170 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers 1171; virtual storage 1172; virtual networks 1173, including virtual private networks; virtual applications and operating systems 1174; and virtual clients 1175. In one example, the management layer 1180 can provide the functionality described below. Resource provisioning 1181 provides dynamic procurement of computing resources and other resources utilized to perform tasks within the cloud computing environment. Metering and pricing 1182 provides cost tracking for the utilization of resources within the cloud computing environment, and billing or pricing for the consumption of these resources.

[0098] In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User Portal 1183 provides consumers and system administrators with access to the cloud computing environment. Service Level Management 1184 provides the allocation and management of cloud computing resources to ensure that required service levels are met. Service Level Agreement (SLA) Planning and Fulfillment 1185 provides the pre-scheduling and procurement of cloud computing resources, whereby future demand for cloud computing resources is anticipated according to the SLA.

[0099] Workload layer 1190 provides examples of functionalities that can be utilized in a cloud computing environment. Examples of workloads and functionalities that can be provided from this layer include: mapping and navigation 1191; software development and lifecycle management 1192; virtual classroom education delivery 1193; data analysis and processing 1194; transaction processing 1195; and artificial intelligence algorithm processing 1196 according to one or more embodiments of the present invention.

[0100] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of another feature, step, operation, element, component, and / or combination thereof.

[0101] At least one embodiment of the present invention can provide beneficial effects, such as replacing complex, manually (e.g., custom) developed frameworks (e.g., a collection of one or more framework configurations) for model recovery logic. As illustratively described herein, the framework is configured and instantiated using a set of fault detection components and an associated model recovery pipeline. Once instantiated, the framework uses logs as input to insert into a given lifecycle and delivers new model artifacts for new model versions into the existing lifecycle pipeline. In one or more illustrative embodiments, the framework is a cloud-based framework and platform for end-to-end development and lifecycle management of AI applications.

[0102] Various embodiments of the invention have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or improvements to existing technologies in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A neural network device, comprising: A plurality of synaptic network capacitors, wherein the synaptic network capacitors in the plurality of synaptic network capacitors share a first output terminal; A second plurality of synaptic network capacitors, wherein the synaptic network capacitors in the second plurality of synaptic network capacitors share a second output terminal; as well as A metal shield is disposed between the first output terminal and the second output terminal.

2. The neural network device according to claim 1, wherein the metal shield disposed between the first output terminal and the second output terminal is connected to a ground terminal.

3. The neural network device according to claim 1, wherein the metal shield is disposed between the first output terminal and the second output terminal, such that the metal shield capacitively decouples the first output terminal from the second output terminal.

4. The neural network device of claim 1, wherein each of the first plurality of synaptic network capacitors includes an input terminal independent of the input terminals of each of the other synaptic network capacitors in the first plurality of synaptic network capacitors.

5. The neural network device of claim 4, wherein each of the second plurality of synaptic network capacitors includes an input terminal independent of the input terminals of each of the other synaptic network capacitors in the second plurality of synaptic network capacitors.

6. The neural network device of claim 1, wherein the input terminal of each synaptic network capacitor is addressable via a given word line of the neural network device, and each of the first plurality of synaptic network capacitors and the second plurality of synaptic network capacitors is further addressable via a given word line of the neural network device.

7. The neural network device of claim 1, wherein the neural network device is part of an artificial intelligence system.

8. A method comprising: Forming a first plurality of synaptic network capacitors, wherein the synaptic network capacitors in the first plurality of synaptic network capacitors share a first output terminal; A second plurality of synaptic network capacitors are formed, wherein the synaptic network capacitors in the second plurality of synaptic network capacitors share a second output terminal; as well as A metal shield is formed between the first output terminal and the second output terminal.

9. The method of claim 8, further comprising connecting the metal shield formed between the first output terminal and the second output terminal to a ground terminal.

10. The method of claim 8, wherein the metal shield formed between the first output terminal and the second output terminal capacitively decouples the first output terminal from the second output terminal.

11. The method of claim 8, wherein each of the first plurality of synaptic network capacitors includes an input terminal independent of the input terminals of each of the other synaptic network capacitors in the first plurality of synaptic network capacitors.

12. The method of claim 11, wherein each of the second plurality of synaptic network capacitors includes an input terminal independent of the input terminals of each of the other synaptic network capacitors in the second plurality of synaptic network capacitors.

13. The method of claim 8, further comprising addressing the input terminal of each synaptic network capacitor via a given word line of the neural network device, and further addressing each of the first plurality of synaptic network capacitors and the second plurality of synaptic network capacitors via a given positioning line of the neural network device.

14. The method of claim 13, wherein the neural network device is part of an artificial intelligence system.

15. An artificial intelligence system, comprising: At least one processing device includes a neural network structure having multiple layers of neuronal nodes operatively coupled by synaptic elements, wherein the synaptic elements are configured in at least two groups of multiple synaptic elements, and wherein the at least two groups of multiple synaptic elements include: A plurality of synaptic elements, wherein the synaptic elements in the plurality of synaptic elements share a first output terminal; A second plurality of synaptic elements, wherein the synaptic elements in the second plurality of synaptic elements share a second output terminal; and A metal shield is disposed between the first output terminal and the second output terminal.

16. The artificial intelligence system of claim 15, wherein the metal shield disposed between the first output terminal and the second output terminal is connected to a ground terminal.

17. The artificial intelligence system of claim 15, wherein the metal shield is disposed between the first output terminal and the second output terminal to decouple the first output terminal from the second output terminal.

18. The artificial intelligence system of claim 15, wherein each of the first plurality of synaptic elements includes an input terminal independent of the input terminals of each of the other synaptic elements in the first plurality of synaptic elements.

19. The artificial intelligence system of claim 18, wherein each of the second plurality of synaptic elements includes an input terminal independent of the input terminals of each of the other synaptic elements in the second plurality of synaptic elements.

20. The artificial intelligence system of claim 15, wherein the input terminal of each synaptic element is addressable via a given word line of the at least one processing device, and each of the first plurality of synaptic elements and the second plurality of synaptic elements is also addressable via a given positioning line of the at least one processing device.

Images