Computing system for data intensive tasks and method of operating the same

Abstract

A computing system includes a processor; data bus; and anon-volatile memory (NVM) circuit having an array of NVM cells and an embedded compute circuit coupled to the array of NVM cells on a single semiconductor substrate and communicating with the processor as a slave device on the data bus. A method for operating the computing system including producing a clock signal that controls operations of the synchronous digital circuit and a memory circuit of the computing system. A power supply regulator circuit is coupled to the synchronous digital circuit and the memory circuit. A frequency of the clock signal and a voltage of the power supply regulator are dynamically adjusted at run time based on a signal corresponding to an operating characteristic of the memory circuit.

Description

[0001] COMPUTING SYSTEM FOR DATA INTENSIVE TASKS AND METHOD OF OPERATING THE SAME

[0002] BACKGROUND

[0003] Technical Field

[0004] The present disclosure relates to a computing system for data intensive tasks and methods of operating such a computing system, more specifically a computing system where a monolithic non-volatile memory (NVM) with embedded compute circuit communicate to a host processor as a slave device on a data bus, a method of dynamically adjusting voltage and frequency to optimize memory bound computation tasks, and a method of reducing performance impact of error of memory access.

[0005] Description of Related Art

[0006] Modem neural networks include many vector-matrix multiplications where the vector is the mathematical representation of neurons in a neural network, and the matrix is the link between different neurons and contains the learnable parameters. State of the art processors can execute vector math efficiently only when all required data, including the much larger matrix data, is available to the processor.

[0007] Such a scenario often occurs for other data intensive applications, and is generally caused by a widening gap between processor performance and memory bandwidth. While processor development prioritizes performance, memory technology prioritizes density. To bridge this gap for data intensive applications, large on-die static-random-access memory (SRAM) is required to act as a cache or local scratchpad memory. And there is a tradeoff between density and performance for SRAM macro. For smaller computing systems, there is a similar dilemma with regard to nonvolatile memory (NVM) macros and associated memory subsystems. On-die NVM can achieve higher density than SRAM, but is generally much slower for data access. This necessitates copying stored content to SRAM.

[0008] For memory bound computations, computation speed can only be improved by improving memory performance and improving data bandwidth from memory to compute circuits. There is a need to boost memory’ performance for computers and to integrate compute functions in high density NVM.

[0009] SUMMARY

[0010] The objective of the present disclosure is to provide a computing system for data intensive tasks and methods of operating such a computing system.

[0011] Accordingly, a method for operating a computing system comprising a memory circuit and a synchronous digital circuit is provided, which comprises the step of producing a clock signal that controls operations of the synchronous digital circuit and the memory' circuit; wherein a power supply regulator circuit is coupled to the synchronous digital circuit and the memory circuit; wherein a frequency of the clock signal and a voltage of the power supply regulator are dynamically adjusted at run time based on a signal corresponding to an operating characteristic of the memory circuit.

[0012] In some preferred embodiments, the clock signal is synchronized to an input clock signal by a phase-locked-loop (PLL) circuit, or the clock signal is generated by a delay circuit not synchronized to an input clock.

[0013] In some preferred embodiments, the operating characteristic of the memory circuit is an error rate of data access operations, and the error rate is estimated based on a parity check method or an error-correction-coding (ECC) method or a method of comparing an output data to a predetermined value.

[0014] In some preferred embodiments, the operating characteristic of the memory circuit is inferred by an error rate of data access operations performed on another memory circuit whose operating characteristics are correlated to the memory circuit, and the error rate is estimated based on a parity check method or an error-correction-coding (ECC) method or a method of comparing to a predetermined data.

[0015] In some preferred embodiments, the operating characteristic of the memory circuit is inferred from one or a plurality of environmental conditions.

[0016] In another aspect, the present disclosure provides a method for operating a computing system comprising a memory circuit, an error-correction-coding (ECC) decoder, and a computation circuit, whereby an operation to detect and correct errors in an output data, which is a pre-ECC data, of the memory circuit by the ECC decoder is performed simultaneously with a computation of the pre-ECC data.

[0017] In some preferred embodiments, a significance of a detected error of the output data of the memory or a significance of an error in a computation result of the pre-ECC data is determined, and an error correction operation is only performed if the significance exceeds a certain threshold.

[0018] In some preferred embodiments, an error correction operation is performed by adding a correction term to a computation result based on the pre-ECC data. In some preferred embodiments, the computation includes a process of copying the pre-ECC data to a buffer memory or a cache memory circuit, which is performed simultaneously with an ECC decoding operation.

[0019] In still another aspect, the present disclosure provides a computing system comprising a processor; data bus; and a non-volatile memory (NVM) circuit comprising an array of NVM cells and an embedded compute circuit coupled to the array of NVM cells on a single semiconductor substrate and communicating with the processor as a slave device on the data bus.

[0020] In some preferred embodiments, the computing system further comprises a clock generator circuit and a power supply regulator circuit that are configured to dynamically adjust an output clock frequency and an output power supply voltage in response to an estimated or inferred error rate of the array of NVM cells.

[0021] In some preferred embodiments, the computing system further comprises a parity check circuit, an error-correction-coding (ECC) circuit, or a circuit to compare an output data from the NVM circuit to a predetermined value.

[0022] In some preferred embodiments, the embedded compute circuit in the NVM circuit is configured to initiate a computation simultaneously with an ECC decoding operation, and the NVM circuit further comprises a circuit to compute a correction term and add to a computation result.

[0023] In some preferred embodiments, the data bus is a serial peripheral interface (SPI), an extended variant of SPI including quad-SPI and xSPI, a double-data-rate (DDR) memory bus, or a variant of a DDR memory' bus. In some preferred embodiments, the NVM circuit has a normal mode and a compute mode that are entered and exited by modifying a state bit on a physical pin or a state register, performing a plurality of communications on the data bus, or inferring from a range of input address values.

[0024] In some preferred embodiments, basic memory operation commands are interpreted to perform computations in the compute mode, including interpreting a memory write command as sending input data and initiating computation, interpreting a memory' read command as retrieving computation output, and interpreting an address value as a pointer to an executing code stored in the array of NVM cells.

[0025] In some preferred embodiments, the embedded compute circuit is configured to perform a sequence of mathematical operations, including a vector-vector addition, a vector-vector dot product, and a vector-matrix multiplication, following a set of instructions stored in the array of NVM cells.

[0026] In some preferred embodiments, the NVM circuit further comprises a decompression circuit.

[0027] In some preferred embodiments, the array of NVM cells is a type of magnetic- random-access-memory (MRAM), resistive-random-access-memory (RRAM), phase- change-memory (PCM), ferroelectric-random-access-memory' (FRAM), or Flash memory that retains digital information without power.

[0028] In still another aspect, the present disclosure provides a memory' circuit comprising an array of memory' cells; and a delay circuit that adjusts a read operation of the array of memory cells based on a measured or inferred error rate of the read operation. In some preferred embodiments, the memory circuit further comprises a circuit to detect error in the read operation.

[0029] In some preferred embodiments, the memory- circuit further comprises a circuit to detect an environmental condition correlated to the error rate.

[0030] In some preferred embodiments, the memory circuit is configured to perform a burst read operation wherein an address is automatically incremented; and a next read operation commences automatically and is not synchronized to an input clock signal.

[0031] In some preferred embodiments, the memory circuit is configured to perform an error-correction-coding (ECC) decoding operation of an output data simultaneously with a next read operation.

[0032] In some preferred embodiments, the memory- circuit further comprises a buffer memory-.

[0033] In some preferred embodiments, the memory circuit is configured to perform a burst read operation to fill the buffer memory with data associated with a page; and to perform a column read operation to read data from the buffer memory-.

[0034] In some preferred embodiments, the memory circuit further comprises a pin that provides a signal to indicate when output data can be latched.

[0035] In still another aspect, the present disclosure provides a digital memory circuit comprising an array of memory cells and a plurality of error detection circuits; wherein the plurality of error detection circuits operate on an output data of the memory- circuit being requested by a host circuit. In some preferred embodiments, the error detection circuit is a parity checking circuit or an error-correction-coding (ECC) decoding circuit.

[0036] In some preferred embodiments, a subset of the error detection circuit is activated dynamically at run time based on a signal corresponding to an operating characteristic of the memory’ circuit and a system signal corresponding to a sensitivity to the characteristic of the memory circuit.

[0037] In some preferred embodiments, the operating characteristic of the memory circuit is an error rate of data access operations.

[0038] In some preferred embodiments, the error rate is estimated based on an error detection method.

[0039] In some preferred embodiments, the error detection circuit method is reading a plurality of memory cells and comparing to a predetermined value, or a parity checking method, or an ECC method.

[0040] In some preferred embodiments, the digital circuit further comprises another memory' circuit whose operating characteristics are correlated to the memory circuit; and the operating characteristic of the memory circuit is inferred from an operating characteristic of the second memory circuit.

[0041] In some preferred embodiments, the digital circuit further comprises one or more sensors that can detect environmental conditions correlated to the operating characteristic of the memory circuit, and the operating characteristic of the memory circuit is inferred from one or more environmental conditions. In some preferred embodiments, the digital circuit further comprises a plurality of the error detection circuits performing error detection simultaneously.

[0042] In some preferred embodiments, an output data is released based on error detection results of a subset of the error detection circuits operating simultaneously, without waiting for all of the error detection circuits to complete.

[0043] In some preferred embodiments, after an output data is released, an incomplete error detection operation is terminated.

[0044] In still another aspect, the present disclosure provides a digital circuit on a single semiconductor substrate comprising a first non-volatile memory circuit where each bit of information is stored in a single non-volatile circuit element, and a second non-volatile memory circuit where each bit of information is stored in a pair of non-volatile circuit elements.

[0045] In some preferred embodiments, the pair of non-volatile circuit elements are configured to always have different states.

[0046] In some preferred embodiments, the second memory' circuit uses a differential amplifier circuit to detect one of two possible configurations of the pair of non-volatile circuit elements, where each of two inputs of the differential amplifier is coupled to a non-volatile circuit elements of the pair of non-volatile circuit elements.

[0047] In some preferred embodiments, a method is used to divide static data stored in the first and second non-volatile memories based on a length of contiguous data access and on a requirement on random access latency; whereby data with short length of contiguous data access and a stringent requirement of random access latency is stored in the second non-volatile memory.

[0048] In still another aspect, the present disclosure provides a digital non-volatile memory circuit consisting of an array of non-volatile memory cells organized into a plurality of rows and columns, a first number of write circuits and a second number of read circuits, and the second number of read circuits is more than the first number of write circuits and is an integer multiple of the first number of write circuits.

[0049] In some preferred embodiments, the digital circuit further comprises a column multiplex circuit coupled to a plurality of the columns.

[0050] In some preferred embodiments, the digital circuit further comprises an ECC circuit.

[0051] In some preferred embodiments, a data unit associated with each memory' read command (read data unit) is used for ECC encoding and decoding and is associated with each memory' write command; and a plurality7of memory write operations are performed for each memory7write command.

[0052] In some preferred embodiments, a data unit associated with each read operation (read data unit) is used for ECC encoding and decoding and a memory' write command is associated with a data unit (write data unit) smaller than the read data unit; and in response to the write command, the non-volatile memory circuit read a read data unit containing a plurality of bit positions of a write data unit specified by the write command (target read data unit), perform ECC encoding of the target read data unit, and perform one or a plurality' of write operations to modify a value of the w rite data unit and a new' ECC parity' bits. In some preferred embodiments, a data unit associated with each write operation (write data unit) is used for ECC encoding and decoding and a memory read command is associated with a data unit (read data unit) larger than the write data unit; and in response to the read command, the non-volatile memory' circuit read the read data unit and perform a plurality' of ECC decoding operations on a plurality' of write data units inside the read data unit.

[0053] BRIEF DESCRIPTION OF THE DRAWINGS

[0054] For a better understanding of the aforementioned embodiments of the disclosure as yvell as additional embodiments thereof, reference should be made to the Description of Embodiments below, in conjunction with the folloyving drawings in which like reference numerals refer to corresponding parts throughout the figures.

[0055] FIG. 1 is a conventional computer architecture;

[0056] FIG. 2 is a common scenario for data intensive computation;

[0057] FIG. 3 is a conventional dynamic-voltage-frequency-scaling (DVFS) used for power management;

[0058] FIG. 4 is an embodiment of DVFS according to the present disclosure;

[0059] FIG. 5 is a simplified flow diagram of MD-DVFS control mechanism;

[0060] FIG. 6 is an exemplary delay circuit that controls a conventional NVM read process;

[0061] FIGS. 7(a) and (b) show a delay circuit in a feedback loop;

[0062] FIG. 8 is a method to initiate computation and ECC decode in parallel; FIG. 9 shows steps in a method of correcting a computation result from a memory error;

[0063] FIG. 10 is an example of parallel ECC decodes;

[0064] FIG. 11 shows a method to select optimal ECC decode;

[0065] FIG. 12 is an example of conventional NVM array;

[0066] FIG. 13 is an exemplary low-latency NVM array of the present disclosure;

[0067] FIG. 14 is an example high-bandwidth NVM array circuit of the present disclosure;

[0068] FIG. 15 is a NVM with embedded compute circuit; and

[0069] FIG. 16 is another NVM with embedded compute circuit.

[0070] DETAILED DESCRIPTION

[0071] The technical contents of this disclosure will become apparent with the detailed description of embodiments accompanied with the illustration of related drawings as follows. It is intended that the embodiments and drawings disclosed herein are to be considered illustrative rather than restrictive.

[0072] Referring to FIG. 1, a conventional computer architecture includes a processor 100, a memory 101, and a bus. The memory 101 acts as a slave device on the bus such that it can respond to requests on the bus but cannot initiate a request. The inclusion of a bus enables multiple processors of different kinds to communicate with multiple memory modules and also other devices such as peripheral or input / output devices effectively. However, to achieve design efficiency and system fl exibility, a significant amount of delay and energy are expended to ensure each processor and memory conforms to the bus protocol.

[0073] FIG. 2 illustrates a common scenario for data intensive computation such as training or inferencing of a deep neural network. This qualitative chart plots computation performance as a function of memory bandwidth, and breaks the computation regime into two distinct sections. On the left part of the chart, computation performance is proportional to memory bandwidth. This means that the processor is waiting for data from the memory and the only way to increase performance is by increasing memory' bandwidth. This part of the computation regime is referred to as memory-bound computation. When memory bandwidth is high enough, the computation is limited by the processor speed as in the right part of the graph. And this regime is referred to as a processor bound computation. The inception point of this chart is the boundary between memory bound and processor bound sections of the computation regime, and represents an optimal situation with respect to power consumption and computation performance.

[0074] FIG. 3 shows an example of conventional dynamic-voltage-frequency-scaling (DVFS). A software 304, and in most cases an operating system, determines a required execution speed of the hardware and sends a signal to a DVFS controller 300. The DVFS controller 300 adjusts a power supply regulator 301 to change an output voltage, and adjusts a clock generator 302 to change a clock frequency. The power supply and the clock signal are sent to a digital circuit 303. The adjustment of power supply voltages and clock frequencies modifies execution speed of the circuit, such as one or more processors, to optimize power consumption based on current operating conditions and current operating requirements. Note conventional DVFS is generally used to reduce power consumption of a computer hardware when peak performance is not needed, or when a component exceeds a temperature threshold.

[0075] FIG. 4 illustrates an embodiment of the present disclosure, where a DVFS controller 400 responds to input signals associated with a memory’ operating characteristic such as an error rate 407 and a sensitivity of current computation to memory errors 406, in addition to input signals associated with environmental conditions and current operating requirements 405. In response to the input signals, the DVFS controller 400 adjusts a voltage regulator 401 and a clock generator 402. And the voltage regulator 401 changes a power supply voltage of a digital circuit 403; and the clock generator 402 changes a clock frequency of the digital circuit 403. The digital circuit 403 comprises a memory 404. Such a novel DVFS method focuses on optimizing the memory performance by increasing memory' performance while maintaining memory error rate just below a threshold that is tolerable by the computation task, and is referred to as a memory-driven-DVFS or MD- DVFS. MD-DVFS allows a computing system to operate at a higher speed when computing a memory-bound task.

[0076] The memory error rate can be estimated by many known methods, including sampling some memory addresses and comparing to predetermined values, parity' checks, and various error-correction-coding (ECC) methods.

[0077] Furthermore, the memory error rate may be estimated on a secondary memory whose operating characteristics are closely related to a main memory. This has the advantage of not interfering with main computing tasks while the main memory’ is the primary’ bottleneck. The secondary memory may be used to store other information that is not directly related to the main computation task. Environmental factors such as temperature can be easily sensed and are highly- correlated with error rate of many ty pes of memory-. Furthermore, the error rate of most memory- technologies depends on an aging history- of constituent memory cells. These factors can be included in MD-DVFS control algorithms to improve the accuracy, response time, and flexibility- of the method.

[0078] FIG. 5 shows a simplified flow diagram of MD-DVFS control mechanism. The system starts at an initial setting for clock frequency and power supply voltage. As the computation task is being performed, memory error rate is estimated and used to make adjustments to both clock frequency and power supply- voltage.

[0079] An ECC circuit may be added to further improve overall memory performance, whereby the MD-DVFS controller uses an error signal from the ECC decoder to increase memory performance to a threshold related to an error correcting capability of the ECC decoder. Furthermore, MD-DVFS controls the computation circuit with a different clock frequency and a different power supply voltage, and adjusts the performance of the computation to essentially match memory bandwidth such that the computation system operates near the inception point of the graph in FIG. 3.

[0080] Similar to conventional DVFS method, the clock output of MD-DVFS may be generated by a phase-lock-loop (PLL) circuit synchronized to an input clock signal, where the clock frequency is adjusted by a multiplication factor linked to the control mechanism.

[0081] Alternatively, MD-DVFS may generate a clock using a delay circuit not synchronized with an input clock. An embodiment of the present disclosure uses a delay circuit common in many high density memory, especially non-volatile-memory (NVM), circuits to generate the output clock signal. Refer to FIG. 6, a memory- read operation of many high density NVM includes several sequential stages that are timed by a delay circuit 600. A first stage (tO) is where input command and address are decoded. And based on the decoded command and address, some memory' sensing circuits are placed into a balanced state where a small input signal can cause it to fall into one or two possible output levels and a particular row in an memory' array is activated; this process is often called a precharge stage. A second stage includes a time delay (tl) after the selected memory' cells are coupled to the sensing circuit so that the voltage levels on the bitlines connected to the selected memory cells have sufficient time to develop a difference based on the information stored in the selected memory' cells. A third stage (t2) activates a differential amplifier that amplifies a relatively small difference between the selected bitline voltage level and a reference signal to a level sufficient to drive a latch circuit. A fourth stage (t3) may7be included to perform ECC decoding.

[0082] In most cases, compute circuit can perform at a much faster speed as compared to a memory' circuit, especially a high density memory' circuit. So it is advantageous to use the minimum required time to read memory' cells as a clock to control the computation circuit using the output data of the memory'. As shown in FIG. 7(a), the delay circuit 700 controlling a read operation of a NVM 701 is used as a clock signal to a compute circuit 702 coupled to the NVM 701. The compute circuit 702 or the NVM 701 generates the next NVM command and address, and triggers a next read operation. As shown in FIG. 7(b), computation of an output data from the NVM is performed while the next data is being read from the NVM, thus maximizing overall computation throughput. As an example, a first NVM read 703 is performed during a first read cycle time; then a second NVM read 705 is performed while a first compute 704 is performed using data from the first NVM read 703; then a second compute 706 is performed using data from the second NVM read When the MD-DVFS clock is not synchronized with an external clock, a clockdomain-crossing mechanism is included in a computing system where some part of the system, such as a general purpose CPU, a memory controller, or a data bus. is not controlled by the clock signal of MD-DVFS.

[0083] The memory circuits are designed to cover the worst-case operating conditions in specification. So under most circumstances, a memory circuit can operate significantly faster than its rated clock speed. The MD-DVFS method of the present disclosure lets a memory' circuit operate at a speed allowed by the present operating condition and sensitivity' level of a current computation on memory' error. For example, many data intensive computations such as that related to signal processing or artificial intelligence can tolerate a high level of memory' error. Furthermore, by integrating an ECC method, the memory' can operate at a condition where a higher number of errors are generated, in addition to using ECC output as a feedback signal for clock and power supply voltage control.

[0084] Furthermore, when the delay circuit controller memory read operation is also used to control a local computation circuit tightly coupled to the memory, delay and energy dissipation due to data transfer from memory' to the compute circuit can be minimized. Such a combined memory' and computation circuit does not require a clock input and eliminates extra latency due to tolerances of a clock distribution network as well as extra energy requirement to drive such a clock distribution network.

[0085] The ECC decoding can be a significant overhead in memory read latency (t3 in FIG. 6), especially if the ECC method needs to correct more than a single error bit. The present disclosure provides a method to reduce the ECC decoding overhead. Referring to FIG. 8, an ECC decode 801 is performed simultaneously with an operation, such as computation or copying data to a buffer memory, on pre-ECC data 802 from the memory. When a delaycircuit 800 is used to generate MD-DVFS clock, the delay circuit 800 does not generate a timing signal related to the ECC decode 801.

[0086] Since the pre-ECC data may contain alow level of errors, computation results based on pre-ECC data may also be incorrect. The present disclosure provides methods to mitigate this risk; an example of the method is shown in FIG. 9.

[0087] When a processor requests data from a memory, a cache search 900 is performed to determine if the requested data is already in the cache. In a case where the requested data is in the cache (cache hit), the data is sent to the processor to perform computation 905. In another case where the requested data is not in the cache (cache miss), a memory read 901 is performed. Pre-ECC data output 902 from the memory- is sent to both an ECC decode 903 as well as the computation 905. After the ECC decode 903, a cache line fill 904 copies a post-ECC data into the cache. If at least one error is detected and determined to be significant, a corrective computation 906 is performed and a corrective data output is added to a computation result 907.

[0088] Firstly, the present disclosure provides a method to determine whether a correction to the compute results is needed by estimating the impact of an error. For example, in most memory bound computations, data from memory is numerical parameters used in mathematical calculations. In such cases, the significance of an error can be inferred from the significance of the error's bit location. A most-significant-bit or sign bit is more significant than a least-significant-bit for integer representations, and a bit error in the exponent portion of a floating point representation is more significant than a bit error in the mantissa portion. Secondly, if a correction to the compute results is needed, such a correction is calculated without interrupting the main computation process based on the pre-ECC data. For example, a computation task operates on a large amount of data from 300 memory read operations, and an error is detected in the 100th memory' read operation; the main computation process will continue with the 101st memory read operation and associated computation while a correction term based on the error in the 100th memory' read operation is computed. The correction term is added to a final result after all 300 memory' read operations and associated computation is completed. Since the majority7of pre-ECC data does not contain errors, the majority' of computation based on pre-ECC data is correct and only rare corrections are needed.

[0089] These two inventive methods can also be used in cases where the data output of the memory' is first stored in a temporary memory' such as a cache memory or a buffer memory'. In these cases, the pre-ECC data is copied into the temporary' memory simultaneously with the ECC decoding operation. If an error is detected and determined to be significant, corresponding data in the temporary' memory' can be corrected.

[0090] As an example, consider an embedded system consisting of a microcontroller CPU, a block of static-random-access-memory (SRAM), and a block of magnetic-randomaccess-memory (MRAM). As an example, the CPU is clocked up to 800 MHz, the SRAM can be clocked up to 400 MHz, and the MRAM can be clocked up to 80 MHz. By a conventional operating method, a large cache would be needed to prevent frequent stall of the CPU due to slower SRAM, and data stored in the MRAM needs to be copied to the SRAM at boot time. With MD-DVFS and performing computation on pre-ECC data, it may be possible to increase the SRAM speed to 600 MHz and increase the MRAM speed to 150-200MHz under most conditions. This will increase system performance for data intensive applications such as artificial intelligence Al inference, and eliminate the need to copy stored data from MRAM to SRAM at boot time.

[0091] The error correcting capability of an ECC method is generally correlated with the complexity of the decoding algorithm. An ECC that can correct two error bits will take longer to decode than an ECC that can only correct one error bit. To further reduce the ECC decoding time, the present disclosure provides a computing system comprising multiple ECC with different error correcting capabilities. In an exemplary computing system shown in FIG. 10, a memory 1000 is coupled to both a 1-bit ECC decoder 1001 and a 2-bit ECC decoder 1002. An output data from the memory 1000 is sent to both the 1-bit ECC decoder and the 2-bit ECC decoder. And both the 1-bit ECC decoder and the 2- bit ECC decoder start decoding operations simultaneously. Furthermore, if a 1-bit ECC decoding is completed successfully (zero or one bit error), a 2-bit ECC decoding operation is terminated.

[0092] As shown in FIG. 11. an example of a method to selectively activate one or a subset of a plurality of ECC circuits based on error signals of previous ECC decoding operations. As an example, a memory 1100 is coupled to both a l-bit ECC decoder 1101 and a 2-bit ECC decoder 1102. Both the 1-bit ECC decoder 1101 and the 2-bit ECC decoder 1102 are coupled to an ECC control circuit 1103. During operation, the ECC controller circuit 1103 selectively activates one or both of the 1-bit ECC decoder 1101 and the 2-bit ECC decoder 1102, based on a plurality of input signals that includes error signal 1104 from earlier decoding operations, an environmental signal 1105, and an error sensitivity signal 1106.

[0093] Most conventional NVM arrays are configured to store at least one bit of digital information in each of a non-volatile circuit element for maximizing memory density. As shown in FIG. 12, an array of non-volatile memory’ cells 1200, 1201. 1202, 1203 are configured into a plurality of rows and columns, and memory cells on different columns are coupled to different bitlines 1204, 1205. During read operation, a sensing circuit 1207 provides a stimulus and compares a response of a memory cell to a reference signal 1206.

[0094] In most computation applications, some data stored in NVM needs to be accessed at random. For example, execution codes, which almost always include branching instructions, require access in an order that changes at run-time. Such data is organized in smaller units, and random access latency is more important than average bandwidth of a large block of data. In conventional sy stems, such data needs to be loaded into an SRAM at boot time. The present disclosure provides a method of using a low-latency NVM circuit to store such data for fast access.

[0095] FIG. 13 shows an exemplary low-latency NVM array of the present disclosure based on a 2-cell array architecture and differential sensing. In such a memory array, two cells 1300 and 1302. 1301 and 1303 are paired to encode a single bit. The cell pairs are configured to always have different states: cell 1300 is in a different state of cell 1302, and cell 1301 is in a different state of cell 1303. Using magnetic-random-access-memory or MRAM as an example, two magnetic-tunnel -junction or MTJ devices are configured to have opposite states such that one device has low resistance and another device has high resistance. Each cell in the pair of cells is coupled to a bitline 1304 and a complementary bitline 1305. A differential sensing circuit 1307 is used to compare the resistance levels on these two devices to each other rather than to compare the resistance value of a single cell to a reference, thus increasing a sensing margin several times and removing the negative impact on performance by a lot of noises and variations. Such an array architecture, 2 memory cells with differential sensing, trades density for read performance. Typically, such a memory’ array can achieve a similar read speed as an SRAM array of similar size but also require a similar silicon area. For example, in advanced semiconductor fabrication technologies such as 22nm or 28nm, a latency of l-5ns can be achieved depending on other design parameters such as array size. The main advantage of such a low-latency NVM as compared to SRAM is reduction of standby power consumption and eliminating a need to copy stored data to a SRAM at boot time. The present invention discloses a computing system comprising a general purpose high density NVM macro, a computing circuit, and a low-latency NVM macro on a single semiconductor substrate. The computing system allows performance critical codes to be executed at CPU speed and other parameters such as Neural-Network model parameters to be quickly accessed, without keeping them in SRAM. This can significantly reduce the standby power consumption, with the added benefit of faster system startup time.

[0096] The present invention discloses a method to determine whether a stored data unit requires random access or will be part of a larger data unit before the execution of a computation. The computing system with a high density NVM based on a 1-cell array, a low-latency NVM based on a 2-cell array, and a processor integrated on a single semiconductor substrate, and stores data requiring low-latency random access in the low- latency NVM has the advantages of minimizing standby power consumption, reducing wake up time, and reducing cost.

[0097] The present disclosure further provides a computing system that integrates a low- latency NVM, a processor, and compute-in-memory (CIM) circuit on a single semiconductor substrate.

[0098] The peak bandwidth of a memory circuit is related to the product of the speed of each read or write circuit and the number of such read or write circuits that can be operated in parallel. For NVM, memory read performance has a much bigger impact on overall system performance. The present disclosure provides a high-bandwidth NVM array architecture w hereby more read circuits are included in a memory' array circuit as compared to the number of write circuits, thus increasing read bandwidth without an additional cost associated with a large number of write circuits.

[0099] FIG. 14 shows an example high-bandwidth NVM array circuit of the present disclosure. A plurality of memory cells 1400, 1401 are coupled to a bitline 1402, and a plurality of bitline 1402 is coupled to a bitline multiplexer (MUX) circuit 1403. One bitline is selected for a write operation and is connected to a write circuit 1404 during a write operation; and multiple (N) bitlines are connected to a same number N of read circuits 1405 1406 1407 1408 during a read operation. FIG. 14 shows four read circuits (N=4) as an illustrative example; but the present disclosure is not limited to any particular number of read circuits as long as it is more than a number of the write circuits. In this example w here N=4, a non-volatile-memory circuit of the present disclosure can achieve 4 times higher read data bandwidth than a conventional architecture without adding area and cost from a higher number of w rite circuits. For most non-volatile-memory circuits, the write circuit is much larger than the read circuit due to the need to support much higher voltage and current levels during the write operation.

[0100] Since the write operation and the read operation relate to a different number of data bits, all bits in a single write operation are referred to as a write-word, while all bits in a single read operation are referred to as a read-word.

[0101] The present disclosure provides a first ECC method of the high-bandwidth NVM array, wherein an ECC domain is the read- word. A host writes a read-word at a time, during which time the ECC parity bits are generated and written together with data in N write operations. The N write operations may be self timed without host input, or interrupted to allow the memory circuit to perform other tasks. During read operation, a read-word and its associated parity' bits are sensed, and ECC decoding is performed to detect and correct errors.

[0102] The present disclosure provides a second ECC method of the high-bandwidth NVM array, wherein an ECC domain is the read-word. A host writes a write-word at a time. Upon receiving an address associated with the write operation, the memory performs a read operation and stores all N write-word in the corresponding read-word. After a new writeword is received from the host, it is combined with the stored read- word to form an updated read-word. ECC parity bits are generated by an encoder circuit. During the write operation, both the new write-word and the new parity bits are written to the memory. During read operation, a read-word and its associated parity bits are sensed, and ECC decoding is performed to detect and correct errors.

[0103] The present disclosure provides a third ECC method of the high-bandwidth NVM array, wherein an ECC domain is the write-word. A host writes a write- word at a time. Upon receiving the write-yvord, parity bits are generated by an ECC encoder circuit. The write-word and parity’ bits are yvritten to the memory. During a read operation, a read-word consisting of N write-words and N parity bits are sensed. The ECC decoding operations are performed. The ECC decoding operations may be performed in parallel using N decoder circuit, or performed in sequence using a smaller number of decoder circuits, or performed in a pipelined sequence, or a combination of the decoding methods.

[0104] As shown in FIG. 15. the present disclosure provides a computing system that includes a processor 1500, a data bus, and a NVM 1501. The NVM 1501 comprises an array of NVM cells 1502 and an embedded compute circuit 1503 coupled to the array of NVM cells 1502 on a single semiconductor substrate; and the NVM 1501 communicates with the processor as a slave device on the data bus.

[0105] In an exemplary’ embodiment, the NVM circuit comprises a clock generator circuit, a power supply regulator circuit, or both. The clock generator circuit and power supply regulator circuit are used to control both memory operations and compute operations; and output of the clock generator circuit and the power supply regulator circuit is dynamically adjusted based on an estimated memory error rate.

[0106] To estimate an error rate, the NVM circuit further comprises a parity check circuit, an ECC decoding circuit, or a circuit to compare an output data from the NVM circuit to a predetermined value. The NVM circuit may further comprise an array of secondary’ NVM cells that track the operating conditions and usage history of the NVM cells.

[0107] In another embodiment of the present disclosure, the embedded compute circuit is configured to start a computation task simultaneously as an ECC decoding operation, using an output data of the NVM before the ECC decoding result (pre-ECC data). Furthermore, a correction term calculated from a detected error is added to a final computation result without repeating the calculation on pre-ECC data.

[0108] In another embodiment of the present disclosure, the data bus is a serial peripheral interface (SPI), extended variants of SPI including quad-SPI and xSPI, a double-data-rate (DDR) memory bus and variants of a DDR memory bus, an I2C also known as IIC and a variant of an I2C bus, an MMC and eMMC and a variant of an MMC bus, a NVMe and a variant of an NVMe bus, a USB and a variant of a USB bus, a UFS and a variant of UFS bus, and any data bus commonly used by memory products. In another embodiment of the present disclosure, the NVM circuit has a normal mode and a compute mode that are entered and exited by changing a digital input on a physical pin, or modifying a state bit in a state register, or performing one or a plurality of commands, or inferring from an input address values. By using an address outside of a physical range corresponding to the array of NVM cells or using a state register bit to indicate an intended operating mode, information related to computation tasks may be transmitted using the same commands as used for conventional memory operations.

[0109] When the NVM is in the compute mode, normal memory commands such as read and write, and address inputs are interpreted differently as in the normal mode. For example, a memory' write command is interpreted as transmitting an input data for computation; and a memory7read command is interpreted as requesting for computation results; and an address is interpreted as a pointer to different execution codes stored in the array of NVM cells or other execution parameters.

[0110] The NVM circuit with embedded compute circuit may also be integrated with the processor and other components on a single semiconductor substrate.

[0111] The array of NVM cells is a type of magnetic-random-access-memory (MRAM), resistive-random-access-memory' (RRAM). phase-change-memory (PCM), ferroelectric- random-access-memory (FRAM). or FLASH memory that retains digital information without power.

[0112] FIG. 16 shows another exemplary embodiment of the present disclosure. A computing system comprises a processor 1600, a data bus, and a NVM 1601. The NVM 1601 comprises an array of NVM cells 1602, a decompression circuit 1604 coupled to the array of NVM cells 1602, and an embedded compute circuit 1603 coupled to the decompression circuit 1604 on a single semiconductor substrate; and the NVM 1601 communicates with the processor as a slave device on the data bus.

[0113] The decompression circuit is configured to perform data decompression during a computation such that data bandwidth of the computation is much higher than data bandwidth of the array of NVM cells and the cost of the computing system is reduced. The decompression circuit may be a lossless algorithm such as Huffman coding, or a lossy algorithm such as transform coding.

[0114] While this disclosure has been described by means of specific embodiments, numerous modifications and variations could be made thereto by those skilled in the art without departing from the scope and spirit of this disclosure set forth in the claims.

Claims

WHAT IS CLAIMED IS:

1. A method for operating a computing system comprising a memory circuit and a synchronous digital circuit, the method comprising: producing a clock signal that controls operations of said synchronous digital circuit and said memory circuit; wherein a power supply regulator circuit is coupled to said synchronous digital circuit and said memory circuit; wherein a frequency of said clock signal and a voltage of said power supplyregulator are dynamically adjusted at run time based on a signal corresponding to an operating characteristic of said memory circuit.

2. The method of claim 1, wherein said clock signal is synchronized to an input clock signal by a phase-locked-loop (PLL) circuit, or said clock signal is generated by a delay circuit not synchronized to an input clock.

3. The method of claim 1, wherein the operating characteristic of said memory circuit is an error rate of data access operations, and said error rate is estimated based on a parity check method or an error-correction-coding (ECC) method or a method of comparing an output data to a predetermined value.

4. The method of claim 1, wherein the operating characteristic of said memory circuit is inferred by an error rate of data access operations performed on another memorycircuit whose operating characteristics are correlated to said memory circuit, and said error rate is estimated based on a parity- check method or an error-correction-coding (ECC) method or a method of comparing to a predetermined data.

5. The method of claim 1, wherein the operating characteristic of said memory circuit is inferred from one or a plurality of environmental conditions.

6. A method for operating a computing system comprising a memory circuit, an error-correction-coding (ECC) decoder, and a computation circuit, whereby an operation to detect and correct errors in an output data, which is a pre-ECC data, of said memory' circuit by said ECC decoder is performed simultaneously with a computation of said pre- ECC data.

7. A method of claim 6, wherein a significance of a detected error of the output data of said memory or a computation result of the pre-ECC data is determined, and an error correction operation is only performed if said significance exceeds a certain threshold.

8. A method of claim 6, wherein an error correction operation is performed by adding a correction term to a computation result based on the pre-ECC data.

9. A method of claim 6, wherein said computation includes a process of copying said pre-ECC data to a buffer memory or a cache memory circuit, which is performed simultaneously with an ECC decoding operation.

10. A computing system comprising: a processor;a non-volatile memory (NVM) circuit comprising an array of NVM cells and an embedded compute circuit coupled to said array of NVM cells on a single semiconductor substrate and communicating with said processor as a slave device on said data bus.

11. The computing system of claim 10, further comprising a clock generator circuit and a power supply regulator circuit that are configured to dynamically adjust an output clock frequency and an output power supply voltage in response to an estimated or inferred error rate of said array of NVM cells.

12. The computing system of claim 10, further comprising a parity check circuit, an error-correction-coding (ECC) circuit, or a circuit to compare an output data from said NVM circuit to a predetermined value.

13. The computing system of claim 12, wherein said embedded compute circuit in said NVM circuit is configured to initiate a computation simultaneously with an ECC decoding operation, and said NVM circuit further comprises a circuit to compute a correction term and add to a computation result.

14. The computing system of claim 10, wherein said data bus is a serial peripheral interface (SPI), an extended variant of SPI including quad-SPI and xSPI, a double-data-rate (DDR) memory us? or a variant of a DDR memory bus.

15. The computing system of claim 10, wherein said NVM circuit has a normal mode and a compute mode that are entered and exited by modifying a state bit on aphysical pin or a state register, performing a plurality of communications on said data bus, or inferring from a range of input address values.

16. The computing system of claim 15, whereby basic memory operation commands are interpreted to perform computations in said compute mode, including interpreting a memory write command as sending input data and initiating computation, interpreting a memory read command as retrieving computation output, and interpreting an address value as a pointer to an executing code stored in said array of NVM cells.

17. The computing system of claim 10, wherein said embedded compute circuit is configured to perform a sequence of mathematical operations, including a vectorvector addition, a vector-vector dot product, and a vector-matrix multiplication, following a set of instructions stored in said array of NVM cells.

18. The computing system of claim 10, wherein said NVM circuit further comprises a decompression circuit.

19. The computing system of claim 10, wherein said array of NVM cells is a type of magnetic-random-access-memory (MRAM), resistive-random-access-memory (RRAM), phase-change-memory (PCM), ferroelectric-random-access-memory (FRAM), or Flash memory that retains digital information without power.

Images