Integrated de-noising neural network for high density memory

Abstract

Methods and systems related to computer memory are disclosed herein. The memory according to the present disclosure may be a multi-valued memory in which each storage element of the memory may store a plurality of values compared to a standard binary storage element. The memory may include a decoder neural network and an encoder neural network to de-noise values in the memory. Various methods disclosed herein overcome design constraints that would otherwise limit the density of such memories.

Description

Cross-reference to related applications

[0001] This application claims priority to U.S. Patent Application No. 18 / 775,368, filed July 17, 2024, which claims the benefit of U.S. Provisional Patent Application No. 63 / 527,825, filed July 20, 2023, and U.S. Provisional Patent Application No. 63 / 546,922, filed November 1, 2023, all of which are incorporated herein by reference in their entirety for all purposes. background

[0002] Neural networks, particularly deep learning models, have proven remarkably effective in a wide range of applications, from image recognition to natural language processing. However, one of the defining characteristics of neural networks is their enormous parameter requirements. These parameters, essentially the numerical weights the network uses to make predictions or decisions, are crucial to the network's ability to learn complex patterns from data. As neural networks become more complex and handle increasingly intricate tasks, the demand for parameters continues to grow. This trend is expected to intensify in the future as researchers and engineers develop even more complex architectures and pursue higher levels of accuracy and generalization. This increasing parameter count presents challenges in terms of computational resources, energy consumption, and model interpretability, underscoring the need for continuous research and innovation to achieve a balance between model complexity and efficiency.

[0003] In addition to the issues mentioned above, the rapid increase in parameter requirements of modern neural networks has led to a significant increase in the memory resources needed to perform the computations necessary to execute these networks. Consequently, AI accelerators designed to enhance the performance of neural networks and other machine learning tasks require massive amounts of memory resources to operate efficiently. These accelerators process large amounts of data and require rapid access to model weights and intermediate results used to execute neural networks. However, communication between memory and the processor within the AI ​​accelerator presents a critical bottleneck. While the processor can perform computations at significantly higher speeds, fetching data from memory can be a time-consuming operation, resulting in idle processor cycles and reduced overall performance. Overcoming this memory-processor communication bottleneck is a key challenge in the field of AI hardware design. Innovations such as on-chip memory, high-bandwidth memory interfaces, and memory hierarchy optimizations are being sought to alleviate this limitation, thereby allowing AI accelerators to fully realize their potential for complex AI workloads. Invention Overview

[0004] This document discloses methods and systems relating to computer memory. Specifically, it discloses a high-density memory with an integrated denoising neural network. The high-density memory can be integrated with a processor and formed on the same substrate as the processor. The high-density memory can be integrated with artificial intelligence accelerators or any computing system workload that requires large amounts of data to perform computational tasks. The high-density memory can be integrated with a denoising neural network and formed on the same substrate as the denoising neural network. The denoising neural network can be configured to reduce the effects of various noises in the high-density memory, thereby ensuring that the value to be stored in the memory can be read later and identified as the same value. Using such a denoising neural network, design constraints on the memory can be relaxed, so the memory can be designed to be denser, lower power, or faster while maintaining the same performance in terms of storage fidelity.

[0005] In a particular embodiment of the invention, the high-density memory may be a multi-value memory, wherein each storage element is a multi-value storage element capable of storing any one of a plurality of values. For example, these values ​​may be stored as one of the following: a plurality of conductive states of a circuit element, a connection state of a circuit element, the amount of charge stored by a circuit element, or an oscillation state of a circuit element. The storage element may store a multi-bit digital value as one of a plurality of values ​​in analog form for reading from the memory array and being converted back to a multi-bit digital value. The high-density memory may include noise sources in the form of defects in the individual storage elements, crosstalk between storage elements in the array, noise during a read operation of the memory ultimately read by the read circuitry, and noise during a write operation of the memory ultimately written to the storage element. While these noise sources are visible in many memory architectures, they are particularly severe in high-density memories and memories with multi-value storage elements.

[0006] In a specific embodiment of the invention, the denoising network includes a decoder neural network. In a specific embodiment of the invention, the denoising network includes an encoder neural network. In a specific embodiment of the invention, the denoising network includes both an encoder neural network and a decoder neural network. The encoder neural network and the decoder neural network can form an autoencoder. In a specific embodiment, the encoder neural network, the decoder neural network, and at least one of the noise sources mentioned above can form a variational autoencoder.

[0007] The encoder neural network according to this disclosure can be configured to format values ​​to be written to memory to reduce the impact of noise on the storage fidelity of the memory. This formatting can be referred to herein as encoding. It is worth noting that this formatting does not necessarily involve reducing the dimension of the encoder's input, and the term "encoding" is used herein to mean that the true value has been formatted to offset errors and variations in memory. In fact, as described below, contrary to standard practice, the embodiments disclosed herein exhibit beneficial results when the encoder increases the dimension of the encoder's input. The decoder neural network according to this disclosure can be configured to format values ​​when reading them from memory to reduce the impact of noise on the memory array. This formatting can be referred to herein as decoding. It is worth noting that this formatting does not necessarily involve increasing the dimension of the decoder's input, and the term "decoding" is used herein to mean that the system attempts to recover the true value from memory and offset errors in memory. In fact, as described below, contrary to standard practice, the embodiments disclosed herein exhibit beneficial results when the decoder reduces the dimension of the decoder's input.

[0008] Figure 1 A denoising neural network according to a specific embodiment of the invention disclosed herein is illustrated, comprising an encoder neural network 110 and a decoder neural network 120 forming an autoencoder to improve the performance of noisy memory 100. Noisy memory 100 may be a high-density, multi-valued memory used in conjunction with a processor performing computations for a machine learning application, in which the memory stores parameters and activations of a model used for the machine learning application. Thus, a write value 101 may be a model parameter or activation intended to be written to memory, stored as a stored value 102, and then retrieved as a read value 103 when the computation requires it. Noisy memory 100 may include noise sources 104 attributable to the structure or operation of the memory.

[0009] The model parameters or activations stored by write value 101 are provided to encoder neural network 110 to be encoded into stored value 102, wherein the set of stored values ​​has a higher dimension than write value 101. Stored value 102 is stored in noisy memory 100, wherein one stored value is stored in each storage element of the noisy memory. Stored value 102 can then be read from memory and provided to decoder neural network 120, which decodes the stored value into read value 103. Encoder neural network 110 and decoder neural network 120 can be trained to ensure that write value 101 and read value 103 are approximately equal despite the presence of noise source 104. Grid markings on write value 101 and read value 103 are used herein to indicate the number of values ​​to be written to and read from memory, and grid markings on stored value 102 are intended to indicate the number of storage units required to store the stored value.

[0010] In a particular embodiment of the invention, the dimension of the stored value is higher than the dimensions of the written and read values ​​because the encoding is redundant. The encoder neural network 110 learns a specific encoding to facilitate decoding by the decoder neural network 120 in the presence of noise sources (such as noise source 104) inherent in the structure or operation of the noisy memory 100. When the encoder neural network 110 and the decoder neural network 120 form an autoencoder, the stored value 102 can be described in the latent space of the autoencoder. The additional dimension encoded in the latent space can store derived aspects of the written value 101, including the mean, statistical moments, and relationships between values, making data decoding easier in the presence of noise source 104. The encoder neural network 110 and the decoder neural network 120 can also learn statistics about noise sources that corrupt the data stored in the noisy memory 100. In a particular embodiment, the encoder neural network 110, the decoder neural network 120, and the noise source 104 can form a variational autoencoder, wherein the encoder neural network 110 and the decoder neural network 120 are trained in a training routine, wherein the noise source 104 provides the desired fluctuations in the latent space data.

[0011] exist Figure 1In the example, the noisy memory 100 will need to store more values ​​and therefore will require more storage elements than the data would originally require. In the figure, stored value 102 represents all the values ​​that must be stored for the written value 101 in a single write operation. Those skilled in the art will recognize that the noisy memory 100 will have far more data entries than are used in a single write operation. However, since the noisy memory 100 can be a multi-value memory, where each storage element can store multiple bits of value, the replication of storage elements required for each value to be stored can be offset by the fact that each storage element can store multiple values. For example, if the encoder expands the dimension of each bit of the written data to 1.5 times in the potential data space, but each storage element of the memory can store 2 bits of potential data space data, the result is an overall increase in memory density on a per-storage-element basis. Furthermore, the method disclosed herein can relax the design constraints on the memory, allowing the individual storage elements to be more compact than alternative methods, thus enhancing this benefit.

[0012] Examples of the benefits described in the previous paragraphs are in Figure 1 As shown at the bottom, the dimension of two data bits is increased by 1.5 times, but only a single three-bit storage element is required, resulting in an improved net density compared to conventional memory arrays with a single bit per storage element. As illustrated, encoder neural network 110 increases the dimension of the write value by 1.5 times when encoding the write value into an encoded write value, and decoder neural network 120 reduces the dimension of the read value to the same 1 / 1.5 times when decoding the read value into a decoded read value 103. Therefore, the dimension of write value 101 is equal to the dimension of read value 103. Furthermore, the factor by which the encoder neural network increases the dimension of the write value (i.e., 1.5 in this case) is less than the number of bits that can be stored in each storage element (i.e., 3 in this case). In the illustrated case, due to the use of a three-bit storage element, there is a net reduction in the number of storage elements required despite the use of encoder neural network 110.

[0013] In certain embodiments of the invention, the memory array and any encoder neural network or decoder neural network in the system are designed such that the denoised memory value is a multi-valued analog signal. In these embodiments, the decoder neural network or encoder will perform well because the ground truth value will be closer to the noisy value compared to when the stored value is a basic binary signal. In the case of standard binary values, neural networks have more difficulty correcting for the effects of noise because the ground truth value may exceed half the reference range of noisy values ​​(i.e., when noise pushes the ground truth value just over half a threshold point). In contrast, in the case of multi-valued analog signals, the reference range is divided into smaller segments, giving the neural network a better chance to correct for the ground truth value.

[0014] In certain embodiments of the invention, the memory array disclosed herein is designed to reduce the impact of noise sources on the values ​​in the memory array, allowing the encoder or decoder neural network to remain relatively simple and operate with fewer parameters. In alternative examples or combinations, the memory array may be designed to emphasize the impact of gradient-based noise sources on the stored values, rather than random or popcorn noise sources. In these embodiments, the encoder or decoder neural network will be able to learn how to counteract noise sources with fewer parameters.

[0015] In a particular embodiment of the present invention, a memory is provided. The memory includes an array of storage elements. Each storage element in the array of storage elements is a multi-value storage element. The memory further includes: an encoder neural network configured to receive write values ​​to be stored in the storage elements of the array and to encode the write values ​​into encoded write values; a write circuit configured to write the encoded write values ​​into the storage elements in the array as stored values; a read circuit configured to read the stored values ​​from the storage elements in the array; and a decoder neural network configured to receive read values ​​from the read circuit and to decode the read values ​​into decoded read values.

[0016] In a particular embodiment of the invention, a memory is provided. The memory includes an array of storage elements storing stored values. Each storage element in the array is a multi-value read-only storage element. The memory further includes: a read circuit configured to read stored values ​​from the storage elements in the array as read values; and a decoder neural network configured to receive the read values ​​from the read circuit and decode the read values ​​into decoded read values. The decoder neural network reduces the dimensionality of the read values ​​when decoding them into decoded read values. As used herein, reducing or increasing the dimensionality of a dataset means reducing or increasing the number of bits or other values ​​used to represent the dataset (i.e., reducing or increasing the cardinality of the set).

[0017] In a specific embodiment of the present invention, a method is provided. The method includes: providing a write value to an encoder neural network; encoding the write value into an encoded write value using the encoder neural network; and writing the encoded write value into an array of storage elements using a write circuit. Each storage element in the array of storage elements is a multi-value storage element, and the write value is stored as a stored value in the array of storage elements. The method further includes: reading a stored value from the array of storage elements as a read value using a read circuit; and decoding the read value into a decoded read value using a decoder neural network. Brief description of the attached diagram

[0018] The accompanying drawings illustrate the systems, methods, and various other aspects of this disclosure. Those skilled in the art will understand that the element boundaries (e.g., boxes, groups of boxes, or other shapes) shown in the drawings represent one example of a boundary. It is possible that in some examples, one element may be designed as multiple elements, or multiple elements may be designed as one element. In some examples, an element shown as an internal component of one element may be implemented as an external component of another element, and vice versa. Furthermore, elements may not be drawn to scale. The description below is non-limiting and non-exhaustive, with reference to the accompanying drawings. Components in the drawings are not necessarily to scale but are emphasized to illustrate principles.

[0019] Figure 1 A denoising neural network according to a specific embodiment of the invention disclosed herein is shown, the denoising neural network including an encoder neural network and a decoder neural network that form an autoencoder to improve the performance of noisy memory.

[0020] Figure 2 A multi-valued memory according to a specific embodiment of the invention disclosed herein is shown, the multi-valued memory including a RAM array, an encoder neural network, and a decoder neural network.

[0021] Figure 3 A multi-valued memory according to a specific embodiment of the invention disclosed herein is shown, the multi-valued memory including a RAM array, an encoder neural network, a decoder neural network, and an integrated training circuit system.

[0022] Figure 4 A multi-valued memory according to a specific embodiment of the invention disclosed herein is shown, the multi-valued memory including a ROM array, an encoder neural network, a decoder neural network, and an integrated training circuit system.

[0023] Figure 5 A multi-valued memory according to a specific embodiment of the invention disclosed herein is shown, the multi-valued memory including a ROM array having a decoder neural network.

[0024] Figure 6A ROM array memory cell according to a specific embodiment of the invention disclosed herein is shown, wherein potential error sources have been incorporated for training.

[0025] Figure 7 A RAM array memory cell according to a specific embodiment of the invention disclosed herein is shown, wherein the memory cell includes an inverter circuit.

[0026] Figure 8 A RAM array memory cell according to a specific embodiment of the invention disclosed herein is shown, wherein the memory cell includes a single access transistor.

[0027] Figure 9 Flowcharts are shown of various methods for operating a memory according to specific embodiments of the invention disclosed herein. Detailed description

[0028] Detailed reference will now be made to the various aspects of the systems and methods described herein, as well as implementations and embodiments of variations thereof. Although several exemplary variations of the systems and methods are described herein, other variations of the systems and methods may include aspects of the systems and methods described herein combined in any suitable manner having all or some of the aspects described.

[0029] This document discloses in detail methods and systems relating to computer memory. The methods and systems disclosed in this section are non-limiting embodiments of the invention, provided for illustrative purposes only, and are not intended to limit the full scope of the invention. It should be understood that the disclosed embodiments may overlap with each other or may not overlap with each other. Thus, a portion of an embodiment or a particular embodiment thereof may or may not fall within the scope of another embodiment or a particular embodiment thereof, and vice versa. Different embodiments from different aspects may be combined or practiced separately. Many different combinations and sub-combinations of representative embodiments shown within the broad framework of the invention, which may be apparent to those skilled in the art but are not explicitly shown or described, should not be considered excluded.

[0030] In a particular embodiment, at least one neural network circuit can be trained to assist read or write circuitry in recovering noisy values ​​from a memory array. The neural network can be trained to identify appropriate control signals for writing desired values ​​to and from the memory array. The neural network circuit can be an integrated hardware unit of the read or write circuitry and is trained to learn the characteristics of the device in which it is integrated. The neural network circuit can be configured to increase or decrease the dimension of data entering or leaving a latent space with redundant values ​​for storing more noise-resistant values ​​in the memory. Noisy values ​​can be read from the memory array, and the neural network can be trained to recover the true values ​​intended to be stored at these memory locations. Additionally, a modified value can be written to the memory array using a neural network trained to counteract the effects of noise from writing and storing information in the array. An encoded neural network can form part of the write circuitry disclosed herein. A decoded neural network can form part of the read circuitry disclosed herein.

[0031] Figure 2 A multi-value memory 200 with an encoder neural network 202 and a decoder neural network 206 is shown. The figure illustrates the entire loop as follows: data in the form of a write value 201 is stored in a RAM memory array 204 as stored data in the form of a stored value 211, and then the data is read from the RAM memory array 204 as output data in the form of a decoded read value 207. As can be seen in the figure, the encoder neural network 202 can receive or provide a write value 201 and deliver an encoded write value 210 to be stored in the RAM memory array 204 by a write circuit 203, and the decoder neural network 206 can obtain a read value 212 from a read circuit 205 and modify the noisy output value into a denoised output in the form of a decoded read value 207.

[0032] In certain embodiments, the written value 201 and the decoded read value 207 will have a lower dimension than the stored value 211, even if all three sets of values ​​represent the same date. This is because, in some embodiments, the stored value 211 is stored in the latent data space of the autoencoder formed by the encoder neural network 202 and the decoder neural network 206. Figure 2 The method shown is well-suited for use with RAM arrays storing values ​​using analog oscillatory states, such as those involving patterns or pulse widths. The encoder neural network and decoder neural network can be implemented in hardware and integrated with the ROM array.

[0033] Figure 3 A memory array 300 is shown, which has a memory array with Figure 2It possesses similar characteristics, but features an integrated training circuitry system to assist in adjusting the parameters of the encoder and decoder neural networks. Figure 3 In this system, the encoder neural network and write circuitry are combined into encoder and write circuitry 303, and the decoder neural network and read circuitry are combined into decoder and read circuitry 304. The integrated training circuitry system includes a multiplexer to feed training input 301 from the training data input generator for the training phase of the neural network, or to feed standard input 302 when the device is in normal operation and no longer being trained. As shown, the integrated training circuitry system also includes a loss calculator circuitry 305 that knows the input provided by training input 301. This loss calculator circuitry can compare training input 301 with decoded read values ​​207 to determine the performance of the encoder and decoder neural networks. The loss calculator circuitry 305 can then calculate a loss based on this comparison, which can be used to adjust the weights or other parameters of the decoder neural network. The figure also illustrates how the loss can be fed back to the decoder and encoder neural networks during training. Specifically, the loss can be fed back to the encoder neural network along a gradient flow signal path that short-circuits the RAM memory array 204 from the training path. Once trained, the weights of the encoder and decoder neural networks can be fixed using ROM or any form of memory. Alternatively, the encoder and decoder neural networks can be periodically retrained between phases of RAM operation to store actual normal input data.

[0034] In the illustrated case, the multi-value memory includes a RAM array. However, in alternative embodiments, the illustrated RAM array may be replaced by a ROM array, flash memory array, or other memory array. An encoder neural network is configured to receive write values ​​to be stored in the memory array, encode the write values ​​into encoded write values, and store the encoded write values ​​in the multi-value memory array. The encoder neural network can be trained on the illustrated RAM array to learn how to adjust values ​​to provide the best opportunity to write the true value to memory, store the true value appropriately, and then retrieve the true value later. For example, the encoder neural network may determine that the true value intended to be stored in a specific sector of memory needs to be increased by 10% when written to memory to ensure it is retrieved correctly. As another example, the encoder neural network may encode the write values ​​in a higher-dimensional data space in a way that makes the memory more robust to noise, such as by encoding relations detected in the manner in which the write values ​​are stored in the memory array. A decoder neural network is configured to receive read values ​​from the memory array, decode the read values ​​into decoded read values, and provide the decoded read values ​​as a denoised output of the multi-value memory. The decoder neural network can be trained on the illustrated RAM array to learn how to adjust values ​​when reading from the RAM array, providing the best chance of the decoder neural network providing the original true value to be written to memory. For example, the decoder neural network can determine that a value read from a specific sector of memory needs to be reduced by 5% when reading from memory to ensure the true value is retrieved. As another example, the decoder neural network can decode stored values ​​from a higher-dimensional space and fully utilize encoded relationships in the data to retrieve values ​​more accurately, despite the presence of noise sources in the memory array. In general, the encoder neural network and decoder neural network disclosed herein can learn a certain relationship between the address of memory elements and the adjustment of the value to be read or stored to counteract noise and error sources in the memory array, and can also learn a certain relationship between the addresses of memory elements and encode this information into the stored value.

[0035] Figure 4 A memory 400 is shown, which includes a ROM array 404 and similar to Figure 3The integrated training circuit system includes a multiplexer capable of transmitting training input 401 or standard input 402. The ROM array can be written to using an encoder neural network and write circuitry 403 and read using a decoder neural network and read circuitry 405. The ROM array 404 also includes a loss calculator 406, which calculates the loss and determines how to adjust the parameters of the decoder and read circuitry 405 and the encoder neural network and write circuitry 403 by comparing the decoded read value 207 with the training input 401. Once the ROM is programmed, the encoder neural network and write circuitry 403 are no longer needed because the cells in the ROM array 404 are now read-only. However, the memory 400 can still be used for various applications. For example, the encoder can be a programming circuit for programming values ​​into the ROM array 404. In such embodiments, a large number of test chips can be programmed with values, and the encoder can be trained using data collected from reading the memory on those test chips. In the future, additional ROM arrays on different chips can then be programmed using the trained encoder. Alternatively, in such embodiments, a simulator may be included in the memory 400 in parallel with the ROM array 404, wherein the simulator simulates the noisy ROM behavior of the ROM array 404. The simulator can be used to train the encoder neural network and the write circuit 403, and then the parameters of the encoder neural network and the write circuit 403 can be frozen (e.g., by burning the parameters into the ROM), and only the decoder will be trained separately using the actual values ​​programmed into the ROM array 404.

[0036] Figure 5A multi-valued memory according to a specific embodiment of the invention disclosed herein is illustrated, having a read circuit of a ROM array 501 augmented with a decoder neural network and read circuitry 502. While the illustrated example shows a ROM array 501, the illustrated memory array can be replaced by a RAM array, a flash memory array, or any other type of memory array. In a specific embodiment, the decoder neural network and read circuitry 502 have been trained on the ROM array 501 to filter noise from it. The ROM array 501 can be a multi-valued ROM array. As can be seen in the figure, the decoder neural network and read circuitry 502 can modify a noisy output value 503 into a denoised output 504. In a specific embodiment, the decoder neural network and read circuitry 502 can reduce the dimensionality of the data stored in the ROM array 501 when reading data to reduce the impact of noise on the data. The illustrated method also demonstrates how a neural network can be trained by: supplying a reference true value to an automated testing environment, such as a training programming circuit 512 for applying test inputs to the ROM array 501; and then comparing the read value of the denoised output 504 corresponding to those stored values ​​with the reference true value. The term "reference true" refers to the true value that needs to be stored and retrieved from memory. The difference between the two can be used in a loss function to train the neural network, thereby denoising the output, such as by a loss calculator 505 that can adjust the parameters of the decoder neural network and the read circuit 502. The neural network can learn error sources in the memory array, which allows increasing the density of ROM cells by storing multiple bits per cell with less concern about the effects of noise on these cells. Noise sources can be attributed to varying routing distances, idiosyncrasies, conductivity differences in configurable connections (e.g., fuses) between storage transistors and bias sources, read circuit variations, and others.

[0037] The decoder neural network can be configured to receive read values ​​from the memory array, decode the read values ​​into decoded read values, and provide the decoded read values ​​as a denoised output of the multi-value memory. The decoder neural network can be trained on the illustrated ROM array to learn how to adjust values ​​when reading from the RAM array to provide the optimal opportunity for the decoder neural network and read circuitry 502 to provide the original true value that needs to be stored in memory when reading from the array. For example, the decoder neural network and read circuitry 502 can determine that a value read from a specific sector of memory needs to be reduced by 5% when reading from memory to ensure that the true value is retrieved. As another example, the decoder neural network and read circuitry 502 can reduce the dimensionality of the stored data when generating the decoded read values ​​and utilize additional information stored in the latent space of the stored data to compensate for noise sources in the ROM array 501 and the read circuitry system.

[0038] In a particular embodiment, the decoder neural network circuitry can be trained to assist the read circuitry in recovering noisy values ​​from the memory array. The neural network can be trained to identify appropriate control signals for reading the appropriate value from the memory array. Alternatively, the neural network can be trained to adjust the way the value is read. For example, the neural network can determine that when reading a value stored in a specific sector of the ROM array, the stored signal of that value (e.g., charge on a circuit element, voltage on a circuit element, or current through a circuit element) needs to be increased by 5% to ensure correct reading. The neural network circuitry can be an integrated hardware unit of the read circuitry and is trained to learn the characteristics of the device in which the circuitry is integrated. Noisy values ​​can be read from the memory array, and the neural network can be trained to recover the true value intended to be stored at these memory locations. The neural network can be trained to counteract the effects of noise from writing, storing, and reading information from the array. The decoder neural network can form part of the read circuitry disclosed herein.

[0039] In a particular embodiment, the encoder neural network circuitry can be trained to assist the write circuitry in writing values ​​to a noisy memory array so that the values ​​can be recovered later when reading true values ​​from the memory array. The neural network can be trained to identify appropriate control signals for writing appropriate values ​​from the memory array. Alternatively, the neural network can be trained to adjust the way stored values ​​are processed. For example, the neural network can determine that the stored signal (e.g., charge on a circuit element, voltage on a circuit element, or current through a circuit element) of the value stored in a specific sector of the ROM array needs to be increased by 5% to ensure correct reading later. The neural network circuitry can be an integrated hardware unit of the read circuitry and is trained to learn the characteristics of the device in which the circuitry is integrated. The array can be noisy, and the neural network can be trained to write values ​​into the array so that the true values ​​intended to be stored at those memory locations can be read later from those memory locations. The neural network can be trained to counteract the effects of noise from writing, storing, and reading information from the array. The encoded neural network can form part of the write circuitry disclosed herein.

[0040] The denoising neural network disclosed herein includes one or more of the encoder neural network circuit and decoder neural network circuit disclosed herein, which can be trained in various ways, including supervised and unsupervised learning routines. Regarding supervised learning routines, a set of labeled data in truth form can be provided to be stored in memory, and as part of the loss function of the computational learning routine, the values ​​read from memory can be compared with the truth values. The loss can then be used in any form of backpropagation to adjust the weights of the denoising neural network.

[0041] likeFigure 3 As shown, the multi-value memory may include a loss calculator circuit coupled to the output of the decoder neural network. The loss calculator circuit can compare the denoised output with the training output and use this comparison to calculate the loss of the encoder neural network. The training output may be the expected true values ​​from storing those values ​​in the memory array. True values ​​can be supplied to the encoder neural network using a training input generator circuit. These identical values ​​can be accessed by the loss calculator circuit and compared with the read values ​​provided by the decoder neural network. The decoder neural network can be configured to adjust a set of weights of the decoder neural network using the loss. The decoder neural network can be further configured to adjust a set of weights of the decoder neural network via a gradient flow connection between the encoder neural network and the decoder neural network. The gradient flow connection may be a wire or bus capable of transmitting backpropagation signals from the first layer of the decoder back to the encoder for calculating the gradient adjustments of the weights in the final layer of the encoder neural network. The decoder neural network can be configured to pass gradient flow inputs for backpropagation weight adjustments to the encoder neural network using the gradient flow connection.

[0042] The multi-valued memory may include a multiplexer to feed training input from the training data input generator for the training phase of the neural network. As shown in the figure, the system may also include a training output generator and a loss calculator circuit that are aware of the input provided by the training data input generator. Figure 3 It also shows how the loss can be fed back into the decoder neural network during training.

[0043] like Figure 5 As shown, the multi-value memory may include a loss calculator circuit coupled to the output of the decoder neural network. In this implementation, the loss calculator circuit is connected to an automated test environment programming block in the form of a training programming circuit 512, which provides truth values ​​to the ROM array for storage and to the loss calculator circuit for training. The test environment can ensure that the appropriate truth value is applied to the loss calculator circuit when a specific memory address is read, because it also controls which address the truth value is stored at. This training can be performed before or after its value has been provided to the ROM memory for storage. The automated test environment can overwrite the stored value, or it can provide a temporary stored value to the ROM array before programming the final value of the ROM array. The automated test environment can also utilize a portion of the ROM array 501 for training, which is then no longer used once the decoder neural network has been trained. Figure 2 In this circuit, the output of the loss calculator circuit can be used to train the decoder neural network because the loss is used to adjust the weights of the decoder neural network.

[0044] The encoder neural network and decoder neural network circuits according to this disclosure may include various elements. The circuit may include elements typically associated with read and write circuitry for a memory array, such as the ability to receive an address from which data should be read or to which data should be written. The circuit may include inputs for receiving truth values ​​to be written to the memory. The circuit may include outputs on which read values ​​may be supplied or write signals may be provided to the memory array.

[0045] Weights for the neural networks of the decoder and encoder used in the multi-bit memory can be stored in various ways. Once trained, the weights of the decoder neural network or encoder neural network can be set using ROM or any form of non-volatile memory for permanent use. Alternatively, the weights can be retrained periodically between phases of operation of the multi-bit memory. The weights of the neural network that set the state of the decoder and encoder can be stored in PROM memory or RAM memory, and can be set after the neural network has been trained on the multi-bit memory it is serving. The memory used to store the weights can be the same type of memory as the memory array of the bit memory that the decoder or encoder is serving or a different type of memory. In a particular embodiment, the memory storing the encoder and decoder weights can be a higher quality memory than the multi-bit memory and can have fewer noise sources. This memory may be large on a per-cell basis, but it can be significantly smaller than a memory array using the methods disclosed below. In a particular embodiment, the memory used for the decoder and encoder weights can be less than 10% of the size of the entire multi-bit memory. The memory used to store the weights for the encoder, decoder, or encoder and decoder can be referred to as a parameter memory array to distinguish it from the memory array that the decoder or encoder is serving.

[0046] The parameters of either the encoder neural network or the decoder network disclosed in this paper can be trained in multiple stages. For example, the parameters of a neural network can typically be trained once based on the characteristics of a specific memory design, and once a given chip has been manufactured, the parameters can then be fine-tuned for specific parts. Lightweight fine-tuning methods can be used to tune these parameters. Lightweight fine-tuning of trained neural networks, such as LORAN (Low-Rank Adaptive Network), can be used to modify only a small subset of parameters, thereby reducing computational costs and memory usage. Techniques may involve fine-tuning a subset of low-rank matrices or layers within the network, rather than adjusting all weights. This allows the encoder and decoder to adapt to the noise sources inherent in a given chip with minimal changes, thereby maintaining original performance while incorporating new information. Alternatively, methods such as Parameter Efficient Fine-Tuning (PEFT) and adapter modules can be used, where small modules are added to the original encoder or decoder and trained, leaving most of the pre-trained parameters of the original encoder or decoder unaffected. These methods achieve efficient resource utilization and fast training times, making them suitable for deployment in resource-constrained environments.

[0047] Encoder neural networks and decoder neural networks can also have logic or arithmetic circuitry systems that compute encoded or decoded values ​​based on the inputs of the encoder or decoder and the stored weights. Encoder neural networks and decoder neural networks can also have logic or arithmetic circuitry systems that implement encoded or decoded values ​​based on the inputs of the encoder or decoder and the stored weights. Encoder neural networks and decoder neural networks can use logic or arithmetic circuitry systems to execute the neural network, where values ​​used or from memory are used as inputs and the weights of the neural network are used as weights. The output of the neural network can then be a denoised value (in the case of a decoder) or an encoded true value stored in memory (in the case of an encoder). Alternatively, the output of the neural network can be a computational result indicating how much of the true value from memory or the encoded true value needs to be modified to produce the encoded value or the recovered true value for storage, respectively.

[0048] In a particular embodiment of the invention, at least one of the encoder neural network and the decoder neural network is integrated on the same integrated circuit as the memory array. In a particular embodiment, the memory array is also integrated with a processor. The parameters used for the encoder neural network and the encoder neural network can be stored in read-only memory or random access memory. The read-only memory and the multi-value memory can be integrated on a single substrate. The read-only memory can have single-value memory cells. The read-only memory can be less than 10 percent the size of the multi-value memory.

[0049] In a particular embodiment, the memory array may be RAM and integrated with the processor, wherein each memory cell in the RAM includes an inverter circuit. The processor may perform operations using a set of logic transistors. The inverter circuit may be formed by a set of inverter transistors. The set of logic transistors and the set of inverter transistors may be formed using a common process flow.

[0050] In certain embodiments, the memory array may be a ROM, wherein each memory cell includes an access transistor and may also include a storage transistor, wherein the connection or conduction state of the storage transistor represents the value stored by the memory cell. Each memory cell may be a multi-bit cell because the access transistor or storage transistor can be programmed to multiple connection or conduction states. The memory array may be integrated with a processor. The processor may perform operations using a set of logic transistors. The access transistors and storage transistors (if present) may be formed together with the set of logic transistors using a common process flow. In these embodiments, integration of the ROM with the processing circuitry can be facilitated because the noise cancellation effect of neural networks will make the bit lines, word lines, and power lines of the ROM more non-uniform than in standard ROM circuitry, making the ROM layout more compatible with the layout required by the processing circuitry.

[0051] In certain embodiments of the invention, a multi-valued memory may be provided, wherein error and noise sources have been incorporated or otherwise reduced in such a manner as to limit the number of parameters required to effectively reduce the influence of error and noise sources on the encoder neural network, decoder neural network, or decoder and encoder neural network. In certain embodiments, a multi-valued memory is provided comprising a multi-valued read-only memory array, wherein the read-only memory is configured such that each memory cell in the multi-valued read-only memory array can be read by one of: a charge-sharing operation; and a steady-state current measurement operation. The multi-valued memory may further comprise a decoder neural network configured to: receive read values ​​from the memory array; decode the read values ​​into encoded read values; and provide the encoded read values ​​as a denoised output of the multi-valued memory. In these embodiments, the charge-sharing operation may be performed between reference voltages on one side of an access transistor connected in a memory cell of the multi-valued memory, and the steady-state current measurement operation may be performed on one side of an access transistor in the memory cell connected to a reference current on the opposite side.

[0052] In certain embodiments of the invention, the multi-bit memory is designed such that noise sources follow a gradient across the entire array, such as in the case of program variations across the entire memory array, and that noise sources do not follow random locations or popcorn noise distributions. In these methods, variations in the characteristics of individual transistors or in their respective routing paths within the memory array do not affect the values ​​stored in the memory or read from the memory array.

[0053] Figure 6 A ROM array memory cell according to a specific embodiment of the invention disclosed herein is shown, wherein potential error sources have been incorporated for training. Figure 6 Memory cells in a system are programmed by connecting the drain of a transistor to different reference voltages and read by measuring the voltage on the bit line, which is generated after the word line voltage goes high to turn on the read transistor and the capacitance of the bit line charges. Therefore, in systems such as... Figure 6 Among these methods, charge-sharing circuits can be used to explicitly read the connection states of transistors and the associated values ​​stored therefrom, so that the neural network does not need to learn the specificity of individual storage transistors. Specifically, reading the on-resistance and threshold voltage of the transistor does not affect the voltage to which the capacitor, which serves as a bit line, is charged during charge-sharing operation. Therefore, the neural network does not need to learn the changes in those values ​​from one memory cell to another throughout the entire memory array.

[0054] In a particular embodiment of the invention, a multi-valued memory is provided, wherein the multi-valued memory array is a RAM array comprising an array of memory cells, and each memory cell in the array of memory cells comprises an inverter circuit. The RAM array may be integrated with a processor. The processor may perform operations using a set of logic transistors. The inverter circuit may be formed by a set of inverter transistors. The set of logic transistors and the set of inverter transistors are formed using a common process flow. The multi-valued memory may further include a decoder neural network configured to: receive read values ​​from the memory array; decode the read values ​​into encoded read values; and provide the encoded read values ​​as a denoised output of the multi-valued memory. The multi-valued memory may further include an encoder neural network configured to receive write values ​​to be stored in the memory array, encode the write values ​​into encoded write values, and store the encoded write values ​​in the multi-valued memory array.

[0055] Figure 7A RAM array memory cell according to a specific embodiment of the invention disclosed herein is shown, wherein the memory cell includes an inverter circuit. The inverter circuit stores the value of the memory cell in a pulse pattern or pulse width of pulses oscillating through the inverter loop. The inverter circuit can be programmed by forcing a value at node 700, which will generate a pulse pattern cyclically passing through node 701. The inverter loop can be formed from transistors formed using the same process as the processor transistors used in the processor that the RAM array is serving. Thus, the RAM array can be tightly integrated with the processor's processing circuitry. Furthermore, using encoder neural networks, decoder neural networks, or encoder neural networks and decoder neural networks according to this disclosure, the RAM can be even more tightly integrated because it will be less susceptible to noise that would otherwise result from the irregular layout of the RAM array. When used in combination with such neural networks, the apparatus for forming the inverter circuit can also be smaller and correspondingly reduced in the stringency of its layout design.

[0056] In specific embodiments of the invention, the noisy memory array disclosed herein can be any form of multi-valued memory array having storage elements capable of storing multi-bit values. For example, the storage elements can be multi-bit DRAM cells, such as RAM cell 800. Figure 8 As shown, RAM cell 800 includes a single access transistor, with its gate connected to a word line, its source connected to a bit line, and its drain connected to a storage capacitor. RAM cell 800 can be programmed with different values ​​by placing different amounts of charge on the storage capacitor. Reading a value from the multi-bit memory cell then involves sensing the amount of charge stored on the capacitor using a read circuit coupled to the bit line when the word line is driven high.

[0057] Figure 9 A flowchart 900 illustrates various methods for operating a memory according to specific embodiments of the invention disclosed herein. Flowchart 900 includes a step 901 of providing a write value to an encoder neural network. This value may be a value intended to be stored in memory or may be a value intended to aid in training the encoder neural network or a decoder neural network paired with the encoder neural network. In an encoding step 902, flowchart 900 further encodes the write value into an encoded write value using the encoder neural network. This step may include adjusting individual values ​​and may include increasing the dimension of the write value when generating the encoded write value. These steps are optional because not all embodiments disclosed herein include an encoder neural network. Therefore, these steps may be skipped, and the method may begin with a step of writing a value to memory or programming a value into read-only memory.

[0058] Flowchart 900 also includes a step 903 of writing an encoded write value into an array of storage elements using write circuitry. Each storage element in the array of storage elements is a multi-value storage element, and the write value is stored as a stored value in the array of storage elements. This step may involve applying different voltages, currents, or other signals to the storage elements to store a specific analog value from a set of potential values ​​in the storage elements. In a particular embodiment, this step may be replaced by a step of programming a value into a ROM memory.

[0059] Flowchart 900 continues to step 904: using read circuitry to read a stored value as a read value from the array of storage elements. This step may include applying certain control signals to the array of storage elements to sense the analog values ​​stored therein and translating the multi-valued analog signals into multi-bit digital signals. Flowchart 900 also includes step 905, using a decoder neural network to decode the read value into a decoded read value. This step may include changing individual values. This step may also involve reducing the dimension of the stored value when converting the stored value into a decoded read value. This step may be performed to reduce the influence of noise sources on the stored value. The dimension of the written value may be equal to the dimension of the decoded read value. The written value may be written to a set of addresses in the array and the read value may be read from the same addresses (e.g., after the original written value is encoded, written to memory, stored in memory, read from memory, and decoded, the decoded read value may be the same data as the original written value). Multi-valued storage elements may each store multiple bits because these storage elements may store multiple analog values ​​corresponding to two or more states. In a particular embodiment, the factor by which the encoder neural network increases the dimension of the written value can be less than the number of bits that can be stored in each storage element.

[0060] Flowchart 900 continues to comparison step 906: A loss calculator circuit coupled to the output of the decoder neural network is used to compare the decoded read values ​​with the training output. This step may involve a basic subtraction of one set of values ​​with another set to obtain the comparison result. Flowchart 900 continues to step 907: The loss of the encoder neural network is calculated using the loss calculator circuit and the comparison result. This loss may be proportional to the comparison result. The loss may be proportional to the absolute value of the comparison result. The loss can also be calculated differently for different parts of the memory. The loss may be a function of the address from which the value is read and the difference between the values. The loss function may be a digital array where the positions in the array are related to memory addresses and the values ​​in the array are proportional to the comparison result. The value at a given position in the array corresponds to the address from which the read value is obtained for calculating the comparison result.

[0061] Flowchart 900 also includes step 908: Adjusting a set of weights of the decoder neural network using the loss. This step can be performed in association with standard methods used in machine learning such as gradient descent. Gradient descent adjusts the weights by computing the gradient of the loss function with respect to each parameter, and then moving in the opposite direction of the gradient to minimize the loss. Various versions of gradient descent can be used, such as stochastic gradient descent (SGD), which updates parameters using a single or mini-batch of training instances; and mini-batch gradient descent, which strikes a balance between SGD and full-batch methods. Other methods that can be used include optimization algorithms such as AdaGrad, which adapts the learning rate of each parameter based on historical gradients; RMSprop, which addresses the decreasing learning rate of AdaGrad by using a moving average of the squared gradient; and Adam, which combines the advantages of AdaGrad and RMSprop by computing an adaptive learning rate for each parameter and merging momentum. Flowchart 900 also includes step 909: Passing the gradient stream input used for backpropagation weight adjustment from the decoder neural network to the encoder neural network using a gradient stream connection. This step may involve an extension of standard backpropagation. Alternatively, this step could involve skipping connections, where connections are added directly from the encoder to the decoder, allowing gradients to flow more easily and reducing the risk of vanishing gradients. Alternatively, methods used for variational autoencoders (VAEs) can be used to help update the parameters of a neural network (e.g., weight adjustments) by introducing a probabilistic framework, where the encoder produces parameters from a probability distribution and the decoder samples from that distribution, thereby facilitating gradient flow. The probability distribution can be injected into the training routine, or it can be part of a noise source in the memory array. Adding noise to the input, for example, or using differential regularization methods can also help maintain robust gradient flow, ensuring that the encoder and decoder efficiently learn complementary representations.

[0062] While this specification has been described in detail with respect to specific embodiments of the invention, it should be understood that modifications, variations, and equivalents of these embodiments will readily occur to those skilled in the art upon gaining an understanding of the foregoing. Any of the method steps discussed above can be executed by a processor operating on a computer-readable, non-transitory medium storing instructions for those method steps. The computer-readable medium may be memory within a personal user device or network-accessible memory. Although the examples in this disclosure are generally related to artificial intelligence accelerators, the same methods can be used for any computing architecture with large memory requirements, including those generally related to cryptographic processing, graphics processing, and high-performance computing. The memory array according to this disclosure may be read-only memory, random access memory, flash memory, phase-change memory, or any other memory technology. The methods disclosed herein can also be applied to the transmission of noisy multi-level values ​​via links in both on-chip and off-chip processing systems, where neural networks are used to ensure accurate value recovery at the destination. In these embodiments, links may replace the memory arrays disclosed herein, and encoders may be present on the transmitting side of the link and / or decoders on the receiving side of the link. These and other modifications and variations of the invention may be practiced by those skilled in the art without departing from the scope of the invention as more specifically set forth in the appended claims.

Claims

1. A memory (100; 200), including: An array of storage elements, wherein each storage element in the array of storage elements is a multi-value storage element; An encoder neural network (110; 202) is configured to receive write values ​​(101; 201) for storage in the storage elements of the array and to encode the write values ​​(101; 201) into an encoded write value (210). A write circuit (203) is configured to write the encoded write value (210) into the storage elements in the array as a stored value (102; 211). Read circuit (205), configured to read the stored value (102; 211) from the storage elements in the array; and A decoder neural network (120; 206) is configured to receive read values ​​(103; 212) from the read circuit (205) and decode the read values ​​(103; 212) into decoded read values ​​(207).

2. The memory (100; 200) according to claim 1, wherein: The encoder neural network (110; 202) increases the dimension of the write value when encoding the write value (101; 201) into the encoded write value (210); The decoder neural network (120; 206) reduces the dimensionality of the read value when decoding the read value (103; 212) into the decoded read value (207); and The dimension of the written value (101; 201) is equal to the dimension of the decoded read value (207).

3. The memory (100; 200) according to claim 2, wherein: The encoder neural network (110; 202) increases the dimension of the written value (101; 201) by a factor smaller than the number of bits that can be stored in each of the storage elements.

4. The memory (100; 200) according to claim 1 further comprises: A loss calculator circuit (305; 406) coupled to the output of the decoder neural network (120; 206); The loss calculator circuit (305; 406) compares the decoded read value (207) with the training output and uses the comparison to calculate the loss of the decoder neural network (120; 206).

5. The memory (100; 200) according to claim 4, wherein: The decoder neural network (120; 206) is configured to adjust a set of weights of the decoder neural network (120; 206) using the loss.

6. The memory (100) according to claim 1; 200), also includes: Gradient flow connection between the encoder neural network (110; 202) and the decoder neural network (120; 206); The decoder neural network (120; 206) is configured to use the gradient flow connection to pass the gradient flow input for backpropagation weight adjustment to the encoder neural network (110; 202).

7. The memory (100; 200) according to claim 1, wherein: A set of parameters defining the encoder neural network (110; 202) and the encoder neural network (110; 202) is stored in a read-only memory (404); The read-only memory (404) and the memory (100; 200) are integrated on a single substrate; The read-only memory (404) has a single-value memory cell; and The size of the read-only memory (404) is less than 10 percent of the size of the memory (100; 200).

8. The memory (100; 200) according to claim 1, wherein: The encoder neural network (110; 202) and the decoder neural network (120; 206) form an autoencoder.

9. The memory (100; 200) according to claim 1, wherein: The memory (100; 200) includes a noise source (104); and The encoder neural network (110; 202), the noise source (104), and the decoder neural network (120; 206) form a variational autoencoder.

10. The memory (100; 200) according to claim 1, wherein: The array of storage elements is a random access memory (204) array (204); and Each storage element in the array of storage elements includes an inverter circuit.

11. The memory (100; 200) according to claim 10, wherein: The memory (100; 200) is integrated with the processor; The processor uses a set of logic transistors to perform operations; Each of the storage elements is formed by a set of inverter transistors; and The set of logic transistors and the set of inverter transistors are formed using a common process flow.

12. A memory (100; 200) comprising: An array of storage elements, which stores storage values ​​(102; 211), wherein each storage element in the array of storage elements is a multi-value read-only storage element; Read circuit (205), configured to read the stored value (102; 211) from the storage element in the array as a read value (103; 212); and A decoder neural network (120; 206) is configured to receive the read value (103; 212) from the read circuit (205) and decode the read value (103; 212) into a decoded read value (207). The decoder neural network (120; 206) reduces the dimension of the read value when decoding the read value (103; 212) into the decoded read value (207).

13. The memory (100) according to claim 12; 200), also includes: An encoder neural network (110; 202) is configured to receive write values ​​(101; 201) for storage in the storage elements of the array and to encode the write values ​​(101; 201) into an encoded write value (210); and A programming circuit configured to program the encoded write value (210) into the storage elements in the array as the stored value (102; 211). Wherein: (i) the encoder neural network (110; 202) increases the dimension of the write value when encoding the write value (101; 201) into the encoded write value (210); and (ii) the dimension of the write value (101; 201) is equal to the dimension of the decoded read value (207).

14. A method (900) comprising: To the encoder neural network (110; 202) provides (901) write value (101; 201）； The written value (101; 201) is encoded (902) into an encoded written value (210) using the encoder neural network (110; 202). The encoded write value (210) is written into an array of storage elements (903) using a write circuit (203), wherein each storage element in the array of storage elements is a multi-value storage element, and the write value (101; 201) is stored in the array of storage elements as a storage value (102; 211). The stored value (102; 211) is read (904) from the array of storage elements using the read circuit (205) as the read value (103; 212); and The read value (103; 212) is decoded (905) into a decoded read value (207) using a decoder neural network (120; 206).

15. The method (900) according to claim 14, wherein: The encoder neural network (110; 202) increases the dimension of the write value when encoding the write value (101; 201) into the encoded write value (210); The decoder neural network (120; 206) reduces the dimensionality of the read value when decoding the read value (103; 212) into the decoded read value (207); and The dimension of the written value (101; 201) is equal to the dimension of the decoded read value (207).

16. The method (900) according to claim 15, wherein: The encoder neural network (110; 202) increases the dimension of the written value (101; 201) by a factor smaller than the number of bits that can be stored in each of the storage elements.

17. The method (900) according to claim 14, further comprising: A loss calculator circuit (305; 406) coupled to the output of the decoder neural network (120; 206) is used for comparison (906): the decoded read value (207) is compared with the training output; as well as The loss of the decoder neural network (120; 206) is calculated (907) using the loss calculator circuit (305; 406) and the comparison.

18. The method (900) according to claim 17, further comprising: The loss is used to adjust a set of weights of the decoder neural network (120; 206).

19. The method (900) according to claim 14, further comprising: Gradient flow connections are used to input gradient flow for backpropagation weight adjustment from the decoder neural network (120); 206) is passed (909) to the encoder neural network (110; 202).

20. The method (900) according to claim 14, wherein: A set of parameters for the encoder neural network (110; 202) and the encoder neural network (110; 202) are stored in a read-only memory (404); The read-only memory (404) and the array of storage elements are integrated on a single substrate; The read-only memory (404) has single-value memory (100; 200) units; and The size of the read-only memory (404) is less than 10 percent of the size of the array of storage elements.

21. The method (900) according to claim 14, wherein: The encoder neural network (110; 202) and the decoder neural network (120; 206) form an autoencoder.

22. The method (900) according to claim 14, wherein: The array of storage elements includes a noise source (104); and The encoder neural network (110; 202), the noise source (104), and the decoder neural network (120; 206) form a variational autoencoder.

23. The method (900) according to claim 14, wherein: The array of storage elements is a random access memory (204); and Each storage element in the array of storage elements includes an inverter circuit.

24. The method (900) according to claim 23, wherein: The random access memory (204) is integrated with the processor; The processor uses a set of logic transistors to perform operations; The storage element is formed by a set of inverter transistors; and The set of logic transistors and the set of inverter transistors are formed using a common process flow.

Images