Memory device, memory system and data output method

Abstract

The invention provides a memory device, a memory system and a data output method. The memory device includes a plurality of memory cells and a controller. The controller is coupled to the plurality of memory units and is used for executing operation instructions on input data in the plurality of memory units. The plurality of memory units are used for storing data, receiving input data from the controller, and sequentially executing a plurality of operation operations on the input data and the stored data according to an operation instruction from the controller so as to generate a plurality of operation results to the controller. The controller is configured to: control the plurality of memory cells to sequentially perform a first arithmetic operation and a second arithmetic operation of the plurality of arithmetic operations; storing a first operation result corresponding to the first operation; outputting a first operation result; and storing a second operation result corresponding to the second operation when the first operation result is output.

Description

Technical Field

[0001] This invention relates to a memory device, memory system, and data output method, and particularly to a memory device, memory system, and data output method for in-memory computing (IMC). Background Technology

[0002] With the rapid growth of data volume and the rise of technologies such as cloud computing and big data, traditional computing models are facing performance bottlenecks, leading to the emergence of In-Memory Computing (IMC). IMC is a computing architecture that combines data storage and computation processes within memory to reduce communication latency between the processor and memory. Summary of the Invention

[0003] This invention describes methods, apparatus, systems, and techniques for managing in-memory arithmetic devices or in-memory arithmetic devices (e.g., digital memory arithmetic devices), which can be used to perform one or more operations in memory, such as multiply-accumulate (MAC) operations.

[0004] One embodiment of the present invention provides a memory device comprising a plurality of memory cells and a controller. The controller is coupled to the plurality of memory cells and is used to execute arithmetic instructions on input data in the plurality of memory cells. The plurality of memory cells are used to store data, receive input data from the controller, and sequentially perform a plurality of arithmetic operations on the input data and stored data according to the arithmetic instructions from the controller to generate a plurality of arithmetic results to the controller. The controller is used to: control the plurality of memory cells to sequentially execute a first arithmetic operation and a second arithmetic operation among the plurality of arithmetic operations; store a first arithmetic result corresponding to the first arithmetic operation; output the first arithmetic result; and when outputting the first arithmetic result, store a second arithmetic result corresponding to the second arithmetic operation.

[0005] In some embodiments of this type of memory device, each of the multiple memory cells includes a memory cell array and peripheral circuitry. The memory cell array is used to store corresponding portions of multiple weighted data as stored data. The peripheral circuitry is coupled to the memory cell array and is used to: receive corresponding portions of input data from the controller; read stored data from the memory cell array; and perform multiple computational operations on the corresponding portions of the input data and the stored data to generate multiple computational results.

[0006] In some embodiments of this type of memory device, the peripheral circuitry includes a sense amplifier circuit, an input latch, a multiplier circuit, and an adder circuit. The sense amplifier circuit is coupled to the memory cell array and is used to read corresponding portions of multiple weight data from the memory cell array. The input latch is used to store corresponding portions of input data from the controller. The multiplier circuit multiplies the corresponding portions of the multiple weight data with the corresponding portions of the input data to output multiple multiplication results. The adder circuit adds the multiple multiplication results to output a cell operation result. The controller adds the cell operation results of multiple memory cells to obtain one of multiple operation results.

[0007] In some embodiments of this type of memory device, when the controller receives a multiply-accumulate (MAC) instruction, the memory device enters MAC mode. In MAC mode: multiple memory units are used to read stored data corresponding to a first operation; when the multiple memory units have finished reading the stored data corresponding to the first operation, the multiple memory units are used to execute the first operation and read stored data corresponding to a second operation; when the multiple memory units have finished the first operation, the controller is used to store the first operation result; and when the multiple memory units have finished reading the stored data corresponding to the second operation, the multiple memory units are used to execute the second operation and read stored data corresponding to a third operation among the multiple operations.

[0008] In some embodiments of this type of memory device, when the controller receives a result read instruction, the memory device enters a result read mode. In the result read mode: multiple memory cells are used to perform a third operation; and the controller is used to: output a first operation result; while outputting the first operation result, store a second operation result corresponding to the second operation; after completing the output of the first operation result, output the second operation result; while outputting the second operation result, store a third operation result corresponding to the third operation; and after completing the output of the second operation result, output the third operation result.

[0009] In some embodiments of this type of memory device, the controller includes an output buffer. The output buffer stores multiple operation results. The output buffer includes a first buffer and a second buffer. The first buffer stores and transmits the multiple operation results. The second buffer receives multiple operation results from the first buffer, stores the multiple operation results, and outputs the multiple operation results. The second operation result corresponding to the second operation is stored in the first buffer in MAC mode and moved to the second buffer in result read mode.

[0010] In some embodiments of this type of memory device, the result reading mode is divided into multiple sub-modes by multiple delay intervals, multiple memory cells are used to perform one of multiple arithmetic operations at the start point of each of the multiple sub-modes, and the controller is used to stop outputting multiple arithmetic results at multiple delay intervals.

[0011] In some embodiments of this type of memory device, the controller is configured to receive a result read instruction after a wait time following the receipt of a MAC instruction. The wait time is determined by a digital signal containing at least one bit.

[0012] In some embodiments of this type of memory device, the controller is configured to receive a result read instruction a specified number of system clock cycles after receiving a MAC instruction. The specified number of clock cycles is determined by a digital signal containing at least one bit.

[0013] Another aspect of the present invention provides a memory system comprising a host device and a memory device. The memory device includes a plurality of memory cells and a controller. The controller is coupled to the plurality of memory cells and is used to receive arithmetic instructions containing input data from the host device and execute the arithmetic instructions on the plurality of memory cells. The plurality of memory cells are used to store data, receive input data from the controller, and sequentially perform a plurality of arithmetic operations on the input data and stored data according to the arithmetic instructions from the controller to generate a plurality of arithmetic results to the controller. The controller is used to: control the plurality of memory cells to sequentially execute a first arithmetic operation and a second arithmetic operation among the plurality of arithmetic operations; store a first arithmetic result corresponding to the first arithmetic operation; output the first arithmetic result to the host device; and, when outputting the first arithmetic result, store a second arithmetic result corresponding to the second arithmetic operation.

[0014] In some embodiments of this alternative memory system, each of the multiple memory cells includes a memory cell array and peripheral circuitry. The memory cell array stores corresponding portions of multiple weighted data as stored data. The peripheral circuitry is coupled to the memory cell array and is used to: receive corresponding portions of input data from the controller; read stored data from the memory cell array; and perform multiple computational operations on the corresponding portions of the input data and the stored data to generate multiple computational results.

[0015] In some embodiments of this alternative memory system, the peripheral circuitry includes a sense amplifier circuit, an input latch, a multiplier circuit, and an adder circuit. The sense amplifier circuit is coupled to the memory cell array and is used to read corresponding portions of multiple weight data from the memory cell array. The input latch is used to store corresponding portions of input data from the controller. The multiplier circuit multiplies the corresponding portions of the multiple weight data with the corresponding portions of the input data to output multiple multiplication results. The adder circuit adds the multiple multiplication results to output a unit operation result. The controller adds the unit operation results of multiple memory cells to obtain one of multiple operation results.

[0016] In some embodiments of this alternative memory system, the memory device enters MAC mode when the host device transmits a multiply-accumulate (MAC) instruction to the controller. In MAC mode: multiple memory units are used to read stored data corresponding to a first arithmetic operation; when the multiple memory units have finished reading the stored data corresponding to the first arithmetic operation, the multiple memory units are used to execute the first arithmetic operation and read stored data corresponding to a second arithmetic operation; when the multiple memory units have finished completing the first arithmetic operation, the controller is used to store the first arithmetic result and transmit a ready signal to the host device; and when the multiple memory units have finished reading the stored data corresponding to the second arithmetic operation, the multiple memory units are used to execute the second arithmetic operation and read stored data corresponding to a third arithmetic operation among the multiple arithmetic operations.

[0017] In some embodiments of this alternative memory system, when the host device receives a ready signal and transmits a result read instruction to the controller, the memory device enters a result read mode. In the result read mode: multiple memory cells are used to perform a third operation; and the controller is used to: output a first operation result to the host device; while outputting the first operation result, store a second operation result corresponding to the second operation; after completing the output of the first operation result, output the second operation result to the host device; while outputting the second operation result, store a third operation result corresponding to the third operation; and after completing the output of the second operation result, output the third operation result to the host device.

[0018] In some embodiments of this alternative memory system, the controller includes an output buffer. The output buffer stores multiple operation results. The output buffer includes a first buffer and a second buffer. The first buffer stores and transmits the multiple operation results. The second buffer receives multiple operation results from the first buffer, stores the multiple operation results, and outputs the multiple operation results to the host device. The second operation result corresponding to the second operation is stored in the first buffer in MAC mode and moved to the second buffer in result read mode.

[0019] In some embodiments of this alternative memory system, the host device is used to transmit a read status buffer command to the controller after transmitting a MAC instruction to determine the polling state of the memory device, and to transmit a result read instruction to the controller when the polling state of the memory device changes to a ready state.

[0020] In some embodiments of this alternative memory system, the result reading mode is divided into multiple sub-modes by multiple delay intervals, multiple memory cells are used to perform one of multiple computational operations at the start point of each of the multiple sub-modes, and the controller is used to stop outputting multiple computational results to the host device at multiple delay intervals.

[0021] In some embodiments of this alternative memory system, the controller is configured to receive a result read instruction after a wait period following the receipt of a MAC instruction. This wait period is determined by a digital signal containing at least one bit.

[0022] In some embodiments of this alternative memory system, the controller receives a result read instruction a specified number of system clock cycles after receiving the MAC instruction. The specified number of clock cycles is determined by a digital signal containing at least one bit.

[0023] Another aspect of the present invention provides a data output method applicable to a memory device. The data output method includes the following steps: (a) receiving an arithmetic instruction containing input data from a host device via the memory device; (b) transmitting the input data to a plurality of memory cells of the memory device via a controller of the memory device; (c) performing a plurality of arithmetic operations on the input data and stored data according to the arithmetic instruction via the plurality of memory cells to generate a plurality of arithmetic results to the controller, including the following operations: performing a first arithmetic operation among the plurality of arithmetic operations; and performing a second arithmetic operation among the plurality of arithmetic operations; (d) storing a first arithmetic result corresponding to the first arithmetic operation via the controller; (e) outputting the first arithmetic result to the host device via the controller; and (f) storing a second arithmetic result corresponding to the second arithmetic operation via the controller when the controller outputs the first arithmetic result.

[0024] In some embodiments of this alternative data output method, the data output method further includes the following steps: entering MAC mode in response to receiving a multiply-accumulate (MAC) instruction from the host device, wherein steps (a), (b), (c), and (d) are performed in MAC mode; transmitting a ready signal to the host device via a controller in response to storing a first operation result corresponding to the first operation; and entering a result reading mode in response to receiving a result reading instruction from the host device, wherein steps (e) and (f) are performed in the result reading mode.

[0025] One or more details disclosing the implementation are set forth in the accompanying drawings and the following description. Other features, manners, and advantages will become apparent from the specification, drawings, and claims. Attached Figure Description

[0026] The same numbers and names in the various figures of this invention are used to denote the same elements. It should also be understood that the various embodiments shown in the figures are merely illustrative representations and are not necessarily drawn to scale.

[0027] Figure 1 A schematic diagram of an example system containing a memory device that can be used as an in-memory computing (IMC) device is shown.

[0028] Figure 2 A schematic diagram of an instance memory device that can be used as an IMC device is shown;

[0029] Figure 3A A schematic diagram of an example memory device with a global adder for multiple memory cells is shown.

[0030] Figure 3B A schematic diagram of another example memory device is shown, having a global adder for multiple memory cells and one or more second-level adders;

[0031] Figure 4A A schematic diagram illustrating the instance data input path from the host device to the memory device is shown.

[0032] Figure 4B A schematic diagram illustrating another example of a data input path from a host device to a memory device is shown.

[0033] Figure 4C A schematic diagram illustrating the instance data output path of the self-memory device is shown;

[0034] Figure 4D A schematic diagram illustrating another example of a data output path from a self-memory device is shown.

[0035] Figure 5A Example instructions for MAC operations are illustrated under interface protocols such as SPI, QPI, or OPI.

[0036] Figures 5B to 5D It is illustrated for use in Figure 5A A flowchart of the instance process of executing instructions under the interface protocol;

[0037] Figure 6A Example instructions for performing MAC operations under interface protocols such as LPDDR are illustrated;

[0038] Figures 6B to 6D It is illustrated for use in Figure 6A A flowchart of the instance process of executing instructions under the interface protocol;

[0039] Figure 7 A timing diagram illustrating an example of performing MAC operations under the interface protocol is shown.

[0040] Figure 8A The example MAC operation is illustrated;

[0041] Figures 8B to 8E It is illustrated for Figure 8A Configure a buffer for instances of MAC operations;

[0042] Figure 9 A flowchart illustrating an example of a method for managing computing devices within memory is provided.

[0043] Figure 10 The flowchart illustrates the execution of instructions under interface protocols such as SPI, QPI, or OPI;

[0044] Figure 11 A schematic diagram illustrating the instance data output path from another instance memory device is shown.

[0045] Figure 12 A flowchart illustrating the execution of instructions under interface protocols such as LPDDR is provided; and

[0046] Figures 13A to 13B It is illustrated for Figure 12 Configuring a buffer for MAC operations.

[0047] Explanation of reference numerals in the attached figures:

[0048] 100: System

[0049] 110: Memory device

[0050] 112: Controller

[0051] 114: Interface

[0052] 116: Control Circuit

[0053] 118: Configuration buffer

[0054] 120: Main unit

[0055] 132: Memory Group

[0056] 200: Memory device

[0057] 210: Memory Group

[0058] 220: Controller

[0059] 230: Interface

[0060] 232: Data Cache

[0061] 234: SRAM buffer

[0062] 236: Address Generator

[0063] 238: X Decoder

[0064] 240: SCLK input

[0065] 241: Clock Generator

[0066] 242: Pattern Logic

[0067] 244: State Machine

[0068] 246: HV Generator

[0069] 248: Y decoder

[0070] 250: Sensing Amplifier

[0071] 252: Data cable

[0072] 254: Output buffer

[0073] 260: Timing control circuit

[0074] 262: Repair the control circuit

[0075] 264: Global Adder

[0076] 300: Memory device

[0077] 301: Controller

[0078] 302: Interface

[0079] 304: Control Circuit

[0080] 308: Memory Group

[0081] 312: Input buffer

[0082] 314: Output buffer

[0083] 314a: FIFO buffer

[0084] 314b: First buffer

[0085] 316: Timing control circuit

[0086] 318: Repair the control circuit

[0087] 320: Global Adder

[0088] 322: Clock Generator

[0089] 330: Memory unit

[0090] 331: Memory Cell Array

[0091] 332: Internal SA circuit

[0092] 333: Input latch

[0093] 334: Multiplier

[0094] 335: Adder

[0095] 350: Memory device

[0096] 352: Two-stage adder

[0097] 400, 410: Data input paths

[0098] 420, 430: Data output paths

[0099] 500: Table

[0100] 510, 520, 530: Process

[0101] 511~516, 521~525: Steps

[0102] 531~533: Steps

[0103] 600: Table

[0104] 610, 620, 630: Process

[0105] 611~613, 621~625: Steps

[0106] 631~634: Steps

[0107] 900: Process

[0108] 902, 904, 906, 908, 910: Steps

[0109] 700: First timing diagram

[0110] 710: Second timing diagram

[0111] 800: MAC Operations

[0112] CS#, RESET#: Pins

[0113] SIO[7:0]: Data pins

[0114] tSLCH, tCLSH: Duration

[0115] D0~D3,D254,D255: Character Units

[0116] A[31:24], A[23:16]: bytes

[0117] A[15:8], A[7:0]: bytes

[0118] EEh,11h: Command code

[0119] OP[0], OP[1], OP[2]: Option codes

[0120] OP[1:0], OP[2:0]: Option codes

[0121] T1~T5: Time

[0122] tMAC: Waiting time

[0123] tRNR: Delay interval Detailed Implementation

[0124] The embodiments of the present invention will be described below with reference to the relevant drawings. In the drawings, the same reference numerals denote the same or similar elements or method flows.

[0125] Embodiments of the present invention provide methods, apparatus, systems, and techniques for managing in-memory computing (IMC) devices or computing in memory (CIM) devices (e.g., digital computing in memory (dCIM) devices), which can be used to perform one or more operations in memory, such as multiply-accumulate (MAC) operations. Note that the terms "in-memory computing (IMC)" and "computing in memory (CIM)" are used interchangeably in this invention.

[0126] These technologies provide protocols, instructions, and configurations for IMC devices that can perform one or more computational operations or functions. For ease of explanation, MAC operations are described as example computational operations in this invention. However, it should be noted that the technologies implemented in this invention can also be used to implement other computational operations or other functions.

[0127] The present invention provides a scheme for performing MAC operations in an IMC device. The IMC device can be implemented as a global adder and / or one or more secondary adders to sum the multiplication results of MAC operations to obtain the MAC operation result. These techniques can provide configurable MAC operations in the IMC device, for example, by managing configuration buffers and / or command inputs. The configuration buffer may contain information on the startup dimension, weight dimension, weight / startup format, MAC operation parallel settings, interface switching, and / or read content selection. These techniques can support different types of protocols, including but not limited to Serial Peripheral Interface (SPI), Queued Serial Peripheral Interface (QPI), Octal Peripheral Interface (OPI), and Low-Power Double Data Rate (LPDDR) protocols.

[0128] The IMC device implemented in this invention can achieve: (1) high performance, wherein the IMC device can significantly improve the data processing speed because the access speed to memory is much faster than the access speed to disk storage; (2) low latency, wherein in-memory operations reduce the data transfer time between the host device and one or more memory devices; (3) real-time data processing, which can analyze and process large amounts of data in real time, making it very suitable for applications that require fast response, such as real-time inference processing for prediction; (4) efficiency improvement, which reduces input / output (I / O) operations, reduces energy consumption and hardware requirements, and makes the system run more efficiently.

[0129] These technologies can be applied to various types of non-volatile memory devices, such as NOR flash memory and NAND flash memory, or volatile memory devices, such as random access memory (RAM), such as dynamic random-access memory (DRAM) or static random-access memory (SRAM). These technologies can also be applied to various types of memory, such as single-level cell (SLC) devices, multi-level cell (MLC) devices (such as two-level cell devices), triple-level cell (TLC) devices, quad-level cell (QLC) devices, or penta-level cell (PLC) devices. Alternatively or concurrently, these technologies can be applied to various types of devices and systems, such as secure digital (SD) cards, embedded multimedia cards (eMMC) or solid-state drives (SSDs), embedded systems, computing network devices (such as network routers or network processors), cache memory controllers and transformation backup buffers, lookup tables, database engines, data compression hardware, artificial neural networks, intrusion prevention systems, custom computers, etc.

[0130] Figure 1 To illustrate the schematic diagram of example system 100, system 100 includes a memory device 110, which may be an in-memory computing (IMC) device or a CIM device. System 100 may include a host device 120 coupled to memory device 110 and used to control operations in memory device 110, such as in-memory computing (e.g., MAC) operations.

[0131] The host device 120 may include a host controller, which may include at least one processor and at least one memory coupled to the at least one processor and storing programming instructions executed by the at least one processor to perform one or more corresponding operations. For example, the at least one processor may include: a central processing unit (CPU), a graphics processing unit (GPU), a multi-core processor, a data processing unit (DPU), a tensor processing unit (TPU), a quantum processing unit (QPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a microprocessor, or any other processing device or combination thereof.

[0132] The memory device 110 includes a controller 112 and one or more memory banks 132. The controller 112 may include at least one interface 114 and control circuitry 116. The at least one interface 114 is coupled to the host device 120 and the control circuitry 116. The control circuitry 116 is coupled between the at least one interface 114 and the one or more memory banks 132.

[0133] At least one interface 114 is used to receive input data (e.g., data vector or data matrix) and / or commands / or operation instructions from the host device 120, and to output the received data / commands / instructions to the control circuit 116. At least one interface 114 is also used to output data (e.g., operation results) from the control circuit 116 to the host device 120.

[0134] Memory bank 132 may include a two-dimensional (2D) memory device or a three-dimensional (3D) memory device. In some embodiments, memory bank 132 is a non-volatile memory for long-term storage of instructions and / or data, such as a NOR flash memory device, a NAND flash memory device, or other suitable non-volatile memory device. Figure 3A and Figure 3B As described further in detail, each memory bank 132 may include one or more memory cells (e.g., a column of memory cells). Memory cells may include memory arrays, such as NOR memory arrays. Each memory cell may be used as an arithmetic unit (e.g., a MAC unit) to perform one or more arithmetic operations (e.g., MAC operations).

[0135] The memory cells can be used to store weight data or embedding data of one or more models (e.g., machine learning (ML) models or artificial intelligence (AI) models) corresponding to a specific function. The weight data or embedding data of each model can be stored in a corresponding region (e.g., a word line) of the memory cell or memory group 132. Each model can correspond to the starting address of the weight data or embedding data stored in the memory cell or memory group 132. The host device 120 can send operation instructions or commands to execute a model-specific function by including information with corresponding starting addresses in one or more memory cells of one or more memory groups 132, so that the controller 112 can read the stored weight data or embedding data from one or more memory cells based on the information with corresponding starting addresses and execute the operation of the model-specific function.

[0136] In some embodiments, system 100 includes a plurality of memory devices 110. Each memory device 110 may include one or more memory groups 132 and be used to perform corresponding functions different from each other. Each memory device 110 may be coupled to a host device 120. These memory devices 110 may be integrated into a die. In some embodiments, two or more memory devices 110 may be stacked together, for example, to provide a greater storage density.

[0137] In some implementations, controller 112 includes one or more configuration buffers 118. The one or more configuration buffers 118 may be included in or outside control circuitry 116. Operations in one or more memory banks 132 can be configured via command input and / or configuration of one or more configuration buffers 118. Each configuration buffer 118 corresponds to a feature and stores option codes for setting that feature, and controller 112 is used to set the option codes for each of the one or more configuration buffers 118. The one or more configuration buffers 118 can be pre-configured by host device 120 before sending a command to memory device 110 to perform an operation. Configuration buffers 118 can be implemented using one or more logic units, such as ADD, OR, NAND, NOR, SRAM, flip-flops (FF), such as D-type FFs, and / or latches, such as set-reset (SR) latches. Figures 8B to 8E Further details are provided below, stating that computational operations such as MAC operations can be configured by one or more configuration buffers.

[0138] In some instances, one or more configuration buffers 118 include at least one of the following: a configuration buffer for initiating a dimension, representing the length of the input data, wherein the option code for initiating the dimension represents an integer N, for example... Figure 8B As shown; a configuration buffer is used for the startup format, where the option codes for the startup format represent the symbolic information of the integer N, for example... Figure 8C As shown; the configuration buffer is used for the weight dimension, representing the size of the stored data, where the option code for the weight dimension represents an integer M, for example... Figure 8B As shown; the configuration buffer is used for the weight format and number of bits. The weight format represents the symbolic information of the integer M, and the number of bits represents the range of the weight dimension, for example... Figure 8C As shown; or a configuration buffer, used to select the number of one or more memory units or memory groups 132 for parallel execution of arithmetic instructions, wherein the option code specifies the number of one or more memory units or memory groups 132, for example Figure 8D As shown.

[0139] In some implementations, the interface of the memory device 110 may be switchable. For example, one or more configuration buffers 118 may include at least one configuration buffer for switching the protocol of at least one interface 114 between a first interface protocol and a second interface protocol, such as... Figure 8E As shown. The first interface protocol may include a Low-Power Double Data Rate (LPDDR) protocol, and the second interface protocol may include one of a Serial Peripheral Interface (SPI), a Queued Serial Peripheral Interface (QPI), or an Octal Peripheral Interface (OPI). For example, a configuration buffer can be written via the second interface protocol to switch from the second interface protocol to the first interface protocol, or a configuration buffer can be written via the first interface protocol to switch from the first interface protocol to the second interface protocol.

[0140] In some implementations, reading content from memory device 110 is optional. For example, one or more configuration buffers 118 may include configuration buffers for reading commands to switch between reading content and stored data, for example... Figure 8E As shown.

[0141] In some implementations, at least one interface 114 includes an input / output (I / O) interface, configured according to an interface protocol, which may include one of the following protocols: Serial Peripheral Interface (SPI), Queued Serial Peripheral Interface (QPI), or Octal Peripheral Interface (OPI). As an example, Figures 5A to 5D The corresponding instructions for MAC operations under the SPI / QPI / OPI protocol are explained in further detail.

[0142] In some implementations, at least one interface 114 includes: a first interface configured according to the LPDDR protocol; and a second interface configured according to one of the SPI protocol, QPI protocol, or OPI protocol, for example... Figures 6A to 6D Further details are provided. The second interface can be used to program corresponding stored data in one or more memory groups 132. The first interface can be used to perform at least one of the following: setting one or more corresponding configuration buffers 118; transferring input data to these memory groups 132; executing operation instructions on the input data and the corresponding stored data in one or more memory groups 132; or outputting the operation result to the first interface.

[0143] Figure 2 To illustrate the schematic diagram of the example memory device 200, the memory device 200 may be, for example... Figure 1 The memory device 110 is included. The memory device 200 may be implemented as an IMC or CIM device (e.g., a dCIM device) and used to perform computational operations. The memory device 200 may include one or more NOR flash memory devices. In some implementations, two or more memory devices 200 may be stacked together, for example to increase storage density.

[0144] like Figure 2 As shown, the memory device 200 includes multiple components that can be integrated onto a circuit board (e.g., a Si-based substrate) and packaged. The memory device 200 may have one or more memory groups 210 and a controller 220 (e.g., ...). Figure 1 The controller 112 (in the memory bank 210) may include other components besides the memory bank 210. Each memory bank may include multiple memory cells. For example... Figure 3A or Figure 3BFurther described in detail, a memory cell may include: a memory cell array having multiple memory cells; and peripheral circuitry coupled to the memory cell array. The memory cells may be connected in series to multiple column word lines and multiple row bit lines. Each memory cell may include at least one memory transistor used as a storage element to store data. The memory transistor may include a silicon-oxide-nitride-oxide-silicon (SONOS) transistor, a floating-gate transistor, a nitride read-only memory (NROM) transistor, or any suitable non-volatile memory MOS device capable of storing charge.

[0145] The memory device 200 (e.g., controller 220) may include an X decoder (or column decoder) 238 and an optional Y decoder (or row decoder) 248. Each memory cell may be coupled to the X decoder 238 via a corresponding word line and to the Y decoder 248 via a corresponding bit line. Thus, each memory cell may be selected for read or write operations via the X decoder 238 and the Y decoder 248 via the corresponding word line and the corresponding bit line.

[0146] Memory device 200 (e.g., controller 220) may include a memory interface (input / output, I / O) 230 having multiple pins for coupling to an external device (e.g., Figure 1 The host device 120. The memory interface 230 can be used to support one or more types of interface protocols (e.g., communication protocols with the controller) and interface commands. The memory interface 230 can be a Serial Peripheral Interface (SPI) or any other suitable interface.

[0147] In some embodiments, the pins in the memory interface 230 may include: SI / SIO0 for serial data input / serial data input and output; SO / SIO1 for serial data output / serial data input and output; SIO2 for serial data input or output; SIO3 for serial data input or output; the RESET# pin for hardware reset (active low); and the CS# pin for chip selection. The memory interface 230 may also include one or more other pins, such as WP# for write protection (active low) and / or Hold# for holding signal input.

[0148] The memory device 200 (e.g., controller 220) may include a data buffer 232, an SRAM buffer 234, an address generator 236, a synchronous clock (SCLK) input 240, a clock generator 241, mode logic 242, a state machine 244, and a high voltage (HV) generator 246. The SCLK input 240 can be used to receive the synchronous clock input, and the clock generator 241 can be used to generate a clock signal for the memory device 200 based on the synchronous clock input. The mode logic 242 can be used to determine whether a read or write operation exists and provide the determination result to the state machine 244.

[0149] The memory device 200 (e.g., controller 220) may also include: a sense amplifier 250, optionally connected to the Y decoder 248 via a data line 252; and an output buffer 254 for buffering the output signal from the sense amplifier 250 to the memory interface 230. The sense amplifier 250 may be part of read circuitry for reading data from the memory device 200. The sense amplifier 250 may be used to sense low-power signals from bit lines representing data bits (1 or 0) stored in memory cells and amplify small voltage swings to a recognizable logic level for correct data interpretation. The sense amplifier 250 may also communicate bidirectionally with the state machine 244, for example. The sense amplifier 250 may be coupled to a row of memory cells associated with a bit line.

[0150] host device (e.g., Figure 1 The host device 120 can generate commands, such as read commands and / or write commands, which can be executed to read data from memory device 200 and / or write data to memory device 200, respectively. Data written to or read from one or more memory groups 210 can be communicated or transferred between memory device 200 and controller and / or other components via a data bus (e.g., a system bus), which can be a multi-bit bus.

[0151] In some instances, during a read operation, memory device 200 receives a read command from a host device via memory interface 230. State machine 244 can provide control signals to HV generator 246 and sense amplifier 250. Sense amplifier 250 can also send information (e.g., sensed data logic level) back to state machine 244. HV generator 246 can provide voltages to X decoder 238 and Y decoder 248 to select a memory cell. Sense amplifier 250 can sense a low-power (voltage or current) signal representing a data bit (1 or 0) stored in the selected memory cell from the bit line and amplify the low-power signal to a recognizable logic level so that logic circuitry outside memory device 200 can correctly interpret the data bit. Output buffer 254 can receive the amplified voltage from sense amplifier 250 and output the amplified power signal to logic circuitry outside memory device 200 via memory interface 230.

[0152] In some instances, during a write operation, memory device 200 receives a write command from a host device. Data buffer 232 can temporarily store input data from memory interface 230, and address generator 236 can generate a corresponding physical address to store the input data in a designated memory cell of memory bank 210. Address generator 236 can be connected to X decoder 238 and Y decoder 248, which are controlled to select the designated memory cell via corresponding word lines and bit lines. SRAM buffer 234 can store input data from data buffer 232 in memory as long as power is on. State machine 244 can process write signals from SRAM buffer 234 and provide control signals to HV generator 246, which generates a write voltage and provides this write voltage to X decoder 238 and Y decoder 248. Y decoder 248 can be used to output a write voltage to the bit line to store input data in a specified memory cell.

[0153] Memory device 200 can be used as an IMC or CIM device to implement one or more computational operations or functions, such as MAC operations. Memory device 200 can store weighted data and / or embedded data in memory group 210 for computational operations or functions. Figures 3A to 4D Further details are provided, including that the memory device 200 (e.g., controller 220) may be supplemented with timing control circuitry 260, repair control circuitry 262, and global adder 264, and that each memory cell may include a memory cell array, input latches, multiplier circuitry, internal sense amplifier circuitry, and adder circuitry, for example, for performing MAC operations. Input latches may include one or more latches. Multiplier circuitry may include one or more multipliers. Internal sense amplifier circuitry may include one or more sense amplifiers.

[0154] For example, an input latch can be used to store input data. An internal sense amplifier circuit can read stored weights and / or embedded data from the memory cell array when the memory device 200 performs a MAC operation. The internal sense amplifier may be different from the sense amplifier 250 external to the memory cell or memory group 210. The size of the internal sense amplifier may be smaller than that of the sense amplifier 250. A multiplier circuit can be used to multiply the corresponding weights by the input data to obtain a multiplication result. An adder circuit can be used to add the multiplication results to obtain a sum.

[0155] Timing control circuitry 260 can be used to set the operation timing during arithmetic operations in each memory cell or memory bank. Timing control circuitry 260 can be coupled to clock generator 241 and state machine 244. Repair control circuitry 262 can receive input data from the host device and, in response to determining that a designated area in the input latch used to store a corresponding portion of the input data is damaged and / or a designated area in the corresponding memory bank used to store weight data is damaged, can remap the corresponding portion of the input data to a redundant area in the input latch of memory bank 210. Repair control circuitry 262 can be included in or externally coupled to state machine 244. Global adder 264 can be used to generate an arithmetic result or the result of a MAC operation based on the corresponding sums obtained by adder circuitry of one or more memory cells or memory bank 210 performing a MAC operation. Global adder 264 can be coupled to memory bank 210 and output buffer 254.

[0156] Figure 3A The schematic diagram illustrating an example of a memory device 300 shows a global adder 320 for a plurality of memory cells 330. The memory device 300 may be, for example... Figure 1 The memory device 110 or Figure 2 The memory device 200 is included. The memory device 300 may include a group of memory banks 308. Each memory bank 308 may include a group of memory cells 330, such as a row of memory cells 330. The memory bank 308 may be, for example... Figure 1 Memory group 132 or Figure 2 The memory bank 210 is included. Memory cells 330 can be arranged in an array. Memory device 300 can be used to perform MAC operations in memory cells 330. Memory cells 330 can be referred to as MAC arrays. Memory cells 330 can be memory arrays, such as NOR memory arrays.

[0157] In some implementation methods, for example, such as Figure 3AAs shown, memory cell 330 includes memory cell array 331, one or more internal sense amplifier (SA) circuits 332 (each circuit may include one or more internal sense amplifiers), one or more input latches 333 (each circuit may include one or more input latches), one or more multipliers 334 (each circuit may include one or more multipliers), and adder tree (or adder) 335. Memory cell array 331 includes memory cells coupled to word lines and bit lines. Weights of a model (e.g., an ML or AI model) can be stored in memory cells coupled to one or more corresponding word lines. The internal SAs in the internal SA circuits 332 can be connected to... Figure 2 The sensing amplifier 250 differs from the internal one. An internal SA can be coupled to a set (e.g., a row) of memory cells, coupled to corresponding word lines, and used to read data (e.g., weight data) stored in that row of memory cells. Input latches 333 can be used to store input data (e.g., input vector information). Each input latch can store bits. Each multiplier 334 is coupled to a corresponding input latch 333 and used to multiply the corresponding weight by the input data to obtain a multiplication result. Adders 335 can be used to add the multiplication results to obtain a sum.

[0158] like Figure 3A As shown, the memory device 300 includes a controller 301 coupled to the memory cells 330 or the memory groups 308. The controller 301 can be used to execute arithmetic instructions on input data in one or more memory groups 308, such as single-memory-group operations, multi-memory-group operations, or all-memory-group operations, as... Figure 8D Further details will be provided in the text.

[0159] Controller 301 can be with Figure 1 Controller 112 or Figure 2 The controller 220 is similar to or the same as the controller 301. Controller 301 may include at least one interface 302 (e.g., Figure 1 Interface 114 or Figure 2 Interface 230) and control circuit 304 (e.g., Figure 1 The control circuit 304 is coupled between at least one interface 302 and these memory cells 330 or memory groups 308. At least one interface 302 can be used to receive input data (e.g., from a host device, such as...). Figure 1The host device 120 is used to perform the calculation, and the result is output to, for example, a host device. As described above, at least one interface 302 may include one of the Serial Peripheral Interface (SPI) protocol, the Queued Serial Peripheral Interface (QPI) protocol, or the Octal Peripheral Interface (OPI) protocol. In some implementations, at least one interface 302 may include: a first interface configured according to the LPDDR protocol; and a second interface configured according to one of the SPI protocol, the QPI protocol, or the OPI protocol.

[0160] The control circuit 304 can be used to perform at least one of the following: programming corresponding stored data in one or more memory groups 308 or memory units 330; transferring input data to one or more memory groups 308 or memory units 330; executing arithmetic instructions on the input data and the data to be stored in one or more memory groups 308 or memory units 330; or outputting the arithmetic result to at least one interface 302.

[0161] In some implementation methods, for example, such as Figure 3A As shown, the control circuit 304 includes an input buffer 312 and an output buffer 314 (e.g., Figure 2 The output buffer 254. The input buffer 312 stores input data (e.g., input vector or matrix data) before transferring the input data to one or more memory banks 308 or memory cells 330. The input buffer 312 may include SRAM, a register, or any other volatile memory. The output buffer 314 stores output data (e.g., the result of a MAC operation) before outputting the result of an operation to at least one interface 302. The output buffer 314 may include SRAM, a register, or any other volatile memory. Figure 2 Similar to controller 220, controller 301 may include one or more other components, such as an address generator (such as...). Figure 2 Address generator 236) and / or state machine (such as Figure 2 (State machine 244).

[0162] In some implementation methods, for example, such as Figure 3A As shown, control circuit 304 includes repair control circuit 318 (e.g., Figure 2The repair control circuit 262 is configured to: in response to determining that a designated area in the input latch 333 of the memory unit 330 used for storing input data is damaged or defective, remap the input data to a redundant area in the input latch 333 of the memory unit 330. In some cases, in response to determining that a designated area in the memory unit 330 used for storing weights or embedded data is damaged or defective, the repair control circuit 318 may remap the weights or embedded data from the host device to a redundant area in the memory unit 330.

[0163] In some implementation methods, for example, such as Figure 3A As shown, the control circuit 304 includes a clock generator 322 (e.g., Figure 2 The clock generator 241 is used to generate clock signals for internal MAC operations, for example, to generate clock signals for each of one or more memory groups 308 or memory cells 330 that perform arithmetic operations. The control circuit 304 may also include timing control circuitry 316 (e.g., Figure 2 The timing control circuit 260 is used to set the operation timing during each operation performed on one or more memory groups 308 or memory cells 330. These operations may include two or more of the following: operation of one or more internal SA circuits 332, operation of one or more input latches 333, operation of one or more multipliers 334, and operation of adders 335.

[0164] In some implementation methods, for example, such as Figure 3A As shown, the control circuit 304 includes a global adder 320 (e.g., Figure 2 A global adder 264) is used to produce a result of an operation or a result of a MAC operation based on the corresponding sum obtained by the adder 335 of one or more memory units 330 that perform the MAC operation.

[0165] For example, such as Figure 8AAs shown, the MAC operation includes the following steps: multiplying a row weight with N weight values ​​in a matrix (M×N) with a vector with N vector values ​​using a multiplier 334 in the corresponding memory unit 330 to obtain N multiplication results; and adding the N multiplication results using an adder 335 and a global adder 320 in the corresponding memory unit 330 to obtain a single MAC result in the global adder 320. A row weight with N weight values ​​can be stored separately in the corresponding memory unit 330, where each memory unit 330 stores the corresponding weight value from the N weight values. The control circuit 304 can transfer the corresponding portion of the input vector to one or more input latches 333 in each memory unit 330. For example, each memory unit 330 stores P weight values ​​in the memory unit array 331, where P can be an integer, such as 5, 10, or any suitable number. The number of corresponding memory units can be an integer closest to and not less than N / P. For example, if N is 100 and P is 10, then the number of corresponding memory devices can be 10. If N is 100 and P is 12, then the number of corresponding memory devices can be 9 > 100 / 12.

[0166] If P weight values ​​are stored in memory cells coupled to word lines, one or more input latches 333 can store P vector values ​​of the input vector. One or more internal SA circuits 332 can read the stored P weight values ​​from the memory cell array 331, and one or more multiplication circuits 334 can multiply the P weight values ​​and P vector values ​​to obtain P multiplication results, and adders 335 can sum the P multiplication results to obtain a sum for the memory cells 330. Then, global adders 320 can sum the sums from the adders 335 of the corresponding memory cells 330 to obtain a single MAC result, that is, the result of multiplying N weight values ​​and N vector values.

[0167] If P weight values ​​are stored in memory cells coupled to wires (e.g., bit lines), one or more wires are coupled to the corresponding internal sense amplifier, the corresponding input latch 333, and the corresponding multiplier 334. Multiplying the P weight values ​​by the P vector values ​​can be achieved through two or more corresponding internal SA circuits 332, two or more input latches 333, and two or more corresponding multipliers 334. Adder 335 sums all multiplication results from two or more corresponding multipliers 334 to obtain the sum of the multiplication results of P weight values ​​multiplied by P vector values. Then, global adder 320 sums the sums from adders 335 in the corresponding memory cells 330 to obtain a single MAC result, i.e., the result of multiplying N weight values ​​by N vector values.

[0168] Figure 3BThe schematic diagram illustrating another example of the memory device 350 shows a global adder 320 and a two-level adder 352 for a plurality of memory cells 330. The memory device 350 can be used with... Figure 3A The memory device 350 is similar to the memory device 300, except that the memory device 350 includes two or more secondary adders 352. The memory device 350 can be used as an IMC or CIM device.

[0169] Each second-level adder 352 may be coupled to a corresponding memory cell 330 and used to add corresponding sums from the adder circuits of the corresponding memory cell 330 to obtain a corresponding sum. For example, each second-level adder 352 may be coupled to a memory bank 308, which may include a column of memory cells 330. A global adder 320 is used to produce a MAC result based on the corresponding sums from two or more second-level adders 352. For example, a MAC operation may be performed on 100 memory cells 330 or 5 memory banks 308. There may be 5 second-level adders 352, each adder 352 coupled to a memory bank 308 or 20 memory cells 330. Each second-level adder 352 may obtain a sum from the memory bank 308 or the corresponding 20 memory cells 330, and the global adder 320 may obtain a sum from the sums of the 5 second-level adders 352.

[0170] In some implementations, memory device 350 may include multi-stage adders. For example, memory device 350 may include multiple second-level adders 352 and one or more third-level adders (not shown). Each third-level adder may be coupled to two or more second-level adders 352 and used to obtain a third-level sum from the two or more second-level adders 352. The global adder 320 then produces a sum by adding the third-level sums from one or more third-level adders. In one example, a MAC operation may be performed on 100 memory cells 330. There are 10 second-level adders 352, each coupled to 10 memory cells 330. There are 2 third-level adders, each coupled to 5 second-level adders 352, and the global adder 320 may be coupled to these 2 third-level adders.

[0171] In some implementation methods, such as Figure 8A As shown, the data matrix includes M×N weights. As described above, the MAC operation calculates the sum of the products of one column of N weights and an input vector with N vector values ​​to obtain a single MAC result. The MAC operation can be repeated on other columns of weights to obtain other MAC results. The final MAC result can be a vector containing M MAC results, for example, such as... Figure 8AAs shown. The final MAC result (e.g., M individual results) of the MAC used to operate on the data matrix M×N and the input vector 1×N can be stored in the output buffer 314, which can output the final MAC result to the host device via the interface 302.

[0172] Similarly, the memory device can perform a MAC operation on a first data matrix M×N and a second data matrix N×M by repeating the MAC operation described above (e.g., a vector-to-matrix MAC operation) in the memory device. The second data matrix N×M can be considered as M sets of 1×N vectors. The final MAC result can be an M×M matrix. The final MAC result can be stored in an output buffer 314, which can output the final MAC result to the host device via interface 302.

[0173] Figures 4A to 4D This diagram illustrates the data input and output paths between the host device and the memory device. The host device may be, for example... Figure 1 The host device 120, and the memory device may be, for example, Figure 1 Memory device 110, Figure 2 Memory device 200, Figure 3A memory device 300 or Figure 3B The memory device 350.

[0174] Figure 4A This is a schematic diagram illustrating an example data input path 400 from a host device to a memory device. The data input path 400 may include a communication path between the host device and the memory device, as well as a communication bus or line within the memory device. Along the data input path 400, the interface 302 of the memory device receives input data (e.g., vector data) from the host device according to a protocol (e.g., SPI, QPI, or LPDDR) and transfers the input data to an input buffer 312 that stores the input data. The memory device may load the input data stored in the input buffer 312 into the input latch 333 of the corresponding memory cell 330. For example, the memory device may load a portion of the input data from the input buffer into the input latch 333 of each corresponding memory cell 330 via a communication bus within the memory device along the data input path 400.

[0175] Figure 4B This is a schematic diagram illustrating another example of a data input path 410 from a host device to a memory device. Data input path 410 is similar to... Figure 4AThe data input path 400 differs in that it does not directly transmit the input data to the corresponding memory unit 330, but instead first transmits the input data to the repair control circuit 318 of the memory device. As described above, in response to determining that a designated area in the input latch 333 used to store a corresponding portion of the input data is damaged, the repair control circuit 318 can remap the corresponding portion of the input data to a redundant area in the input latch 333 of the memory unit 330.

[0176] Figure 4C This is a schematic diagram illustrating an example data output path 420 from the memory device to the host device. If the memory device includes a global adder 320 but not a second-level adder, for example... Figure 3A The sum obtained by adder 335 in the corresponding memory cell 330 of the memory device 300 can be transmitted along the data output path 420 to the global adder 320. The global adder 320 produces a single MAC result based on the sum obtained by adder 335 (e.g., by adding the sums). The single MAC result can be sent to output buffer 314, which can store the single MAC result. In some cases, as described above, multiple MAC results can be obtained by repeating the MAC operation in the memory device, for example, the result of multiplying a data matrix M×N with an input vector 1×N or an input matrix N×M. The output buffer 314 can provide the final MAC result (including one or more MAC results) to the host device via interface 302. Interface 302 can be configured according to a protocol (e.g., SPI, QPI, or LPDDR).

[0177] Figure 4D This is a schematic diagram illustrating another example of a data output path 430 from a memory device to a host device. The memory device and... Figure 3B The memory device 350 is similar to or the same as that of the memory device 350, including a global adder 320 and one or more secondary adders 352. Data output path 430 is... Figure 4CThe data output path 420 is similar, except that instead of directly transmitting the sum of the adders 335 of the corresponding memory units 330 to the global adder 320, the sum of the adders 335 of the corresponding memory units 330 is first output to one or more secondary adders 352, which generate one or more level sums based on the sum of the adders 335. Then, the one or more secondary adders 352 output one or more level sums to the global adder 320, which generates a MAC result based on the one or more level sums. The global adder 320 can output the MAC result to an output buffer 314, which stores the MAC result and one or more optional other MAC results to obtain the final MAC result. Then, the output buffer 314 can output the final MAC result to the host device via interface 302. Interface 302 can be configured according to a protocol (e.g., SPI, QPI, or LPDDR).

[0178] Figure 5A Table 500 illustrates example instructions for performing MAC operations under interface protocols such as SPI, QPI, or OPI. Figures 5B to 5D Explanation in Figure 5A The flowcharts 510, 520, and 530 illustrate instances of instruction execution under the protocol. MAC operations, instance instructions, and / or instance processes can be executed by a memory device, such as... Figure 1 Memory device 110, Figure 2 Memory device 200, Figure 3A memory device 300 or Figure 3B The memory device 350.

[0179] like Figure 5A , Figure 5B As shown, the instructions "programming", "read buffer", and "read buffer" (e.g., items 1, 2, and 3 in Table 500) can be used with memory banks (e.g., ... Figure 1 Memory group 132 in Figure 2 Memory group 210 or Figure 3A or Figure 3B Memory group 308) or memory cell (e.g., Figure 3A , Figure 3B or Figures 4A to 4D The memory unit 330 contains the programmed data (e.g., weights / embedded data). In some instances, the embedded data is a representation of values ​​or objects, such as text, images, and audio, which machine learning (ML) or artificial intelligence (AI) models or systems and / or computational algorithms (e.g., semantic search algorithms) can use to understand complex knowledge.

[0180] like Figure 5B As shown, process 510 includes several steps. In step 511, data is programmed into a memory device. The memory device can be accessed via an interface (e.g., Figure 1 Interface 114 in Figure 2 Interface 230 or Figure 3A , Figure 3B or Figures 4A to 4D Interface 302 in the middle) from the host device (e.g., Figure 1 The host device 120 receives a command. The command may include data to be programmed and address information in the memory device. The address information may include the starting address for storing the data in the memory device.

[0181] In step 512, the stored data is read into the buffer. The buffer may be a buffer in a memory device (e.g., Figure 3A , Figure 3B or Figures 4A to 4D The output buffer 314 in the memory device may be an external buffer.

[0182] In step 513, the data is stored, for example, via a memory device or control circuitry (e.g., Figure 1 Control circuit 116 or Figure 3A or Figure 3B The control circuit 304 reads from the buffer. In step 514, it is determined whether the data read from the buffer, for example, via the memory device or the control circuit, matches the data to be programmed in the memory device. The step of determining whether the read data matches the data to be programmed may include the following steps: determining whether the difference between the read data and the data to be programmed is less than a threshold. This difference may be the number of bits, or the percentage of the different number of bits to the total number of bits of the data. The threshold may be, for example, a threshold for correction by an error correction code (ECC) circuit or a predetermined threshold.

[0183] If the read data matches the data to be programmed, process 510 is completed in step 515, indicating that the data has been successfully and accurately stored in the memory device. If the read data does not match the data to be programmed, an error message or notification is generated in step 516. The error message or notification can be sent back to the host device via an interface to cause the host device to take action, such as resending the command to program the data into the memory device.

[0184] like Figure 5A and Figure 5C As shown, the commands "Write to configuration buffer" and "Read from configuration buffer" (e.g., items 5 and 4 in Table 500) can be associated with setting the mode buffer of the memory device. Figure 5C As shown, process 520 includes multiple steps.

[0185] In step 521, the configuration buffer is written. The configuration buffer can be, for example... Figure 1 The configuration buffer 118 in the middle may be... Figures 8B to 8E The configuration buffer is described in further detail below. The configuration buffer can be configured via option codes. In some embodiments, the option code includes multiple bits. The memory device can receive commands to write to the configuration buffer, such as commands from a host device. The commands may include information about the configuration buffer, such as bits of the configuration buffer option code. The configuration buffer may be included in the controller of the memory device, such as in control circuitry.

[0186] In step 522, the written configuration buffer is read out, for example, via a memory device. In step 523, the memory device (e.g., control circuitry) determines whether the configuration buffer has been written correctly, for example, by determining whether the read configuration buffer matches the information in the command. If the configuration buffer has been written correctly, process 520 completes in step 524. If the configuration buffer has not been written correctly, an error message or notification is generated in step 525. The error message or notification may be sent to the host device via an interface. The host device may take action, such as resending the command to write the configuration buffer to the memory device.

[0187] like Figure 5A and Figure 5D As shown, the instructions "MAC with vector" and "read MAC result" (e.g., items 6 and 7 in Table 500) can be associated with performing MAC operations in a memory device. Figure 5D As shown, process 530 includes multiple steps.

[0188] In step 531, the memory device performs a MAC operation on the input data (e.g., a data vector) using weight data stored in one or more memory cells or memory groups, according to an arithmetic instruction (e.g., from a host device). The arithmetic instruction may include a MAC operation command, the input data, and address information (e.g., a starting address) in one or more memory cells or memory groups corresponding to the weight data stored in the memory device. As described above, the memory device can use a global adder (e.g., Figure 3A , Figure 3B or Figures 4A to 4D The global adder 320 in the code produces one or more MAC results. These one or more MAC results can be stored in an output buffer (e.g., ...). Figure 2 Output buffer 254 or Figure 3A , Figure 3B or Figures 4A to 4D The output buffer (314) is used to generate the final MAC result based on one or more MAC results.

[0189] In step 532, after the MAC operation is completed, the memory device reads the final MAC result from the output buffer and outputs the final MAC result to the host device via the interface.

[0190] Figure 6A Table 600 illustrates example instructions for performing MAC operations under interface protocols (such as LPDDR). Figures 6B to 6D Instructions for use in Figure 6A The flowcharts for instance instructions executed under the protocol are 610, 620, and 630. MAC operations, instance instructions, and / or instance processes can be executed by a memory device, such as... Figure 1 The memory device 110 shown Figure 2 The memory device 200 shown Figure 3A The memory device 300 shown or Figure 3B The memory device 350 is shown. The instructions in Table 600 can be used with... Figure 5A The corresponding instructions in Table 500 are similar.

[0191] Because LPDDR is a volatile memory (such as DRAM), it cannot store weighted data. As mentioned above, in addition to the first interface configured according to the LPDDR protocol, the memory device may further include a second interface configured according to another protocol, which may be one of SPI, QPI, or OPI protocols. The second interface can be used, for example, according to... Figure 5B Step 511 of process 510 programs the weight data into one or more memory cells or memory banks. The first interface (e.g., the LPDDR protocol) can be used to execute other instructions, such as those listed in Table 600.

[0192] For example, such as Figure 6A and Figure 6B As shown, the instructions "read buffer" and "read buffer" (e.g., items 1 and 2 in Table 600) can be used with the verification memory cell (e.g., Figure 3A , Figure 3B or Figures 4A to 4D Memory cell 330) or memory group (e.g., in the memory cell 330) or memory group (e.g., Figure 1 Memory group 132 in Figure 2 Memory group 210 or Figure 3A or Figure 3B This relates to the data (e.g., weight / embedded data) in memory group 308. Figure 6B As shown, process 610 includes multiple steps performed using a first interface, for example, after programming the weight data using a second interface. Process 610 may be similar to... Figure 5B Process 510 in the middle.

[0193] In step 611, the stored data is read into the buffer. The buffer may be a buffer in a memory device (e.g., Figure 3A , Figure 3B or Figures 4A to 4D The output buffer 314 in the middle). In step 612, the data is stored, for example, through a memory device or control circuitry (e.g., Figure 1 Control circuit 116 or Figure 3A or Figure 3B The control circuit 304 in the middle reads from the buffer.

[0194] In some implementations, it is determined whether data read from a buffer, for example via a memory device or control circuitry, matches data to be programmed in the memory device. The step of determining whether the read data matches the data to be programmed may include the following steps: determining whether the difference between the read data and the data to be programmed is less than a threshold. The difference may be the number of bits, or it may be the percentage of different bits in the total number of bits of data. The threshold may be, for example, a threshold for error correction code (ECC) circuitry or a predetermined threshold. If the read data matches the data to be programmed, process 610 is completed in step 613, indicating that the data has been successfully and accurately stored in the memory device. If the read data does not match the data to be programmed, an error message or notification may be generated, for example... Figure 5B Step 516. Error messages or notifications may be sent back to the host device via an interface to cause the host device to take action, such as retransmitting commands using a second interface to program data in the memory device.

[0195] like Figure 6A and Figure 6C As shown, the instructions "Write to mode register" and "Read from mode register" (e.g., items 4 and 3 in Table 600) can be associated with setting the mode register of the memory device. Procedure 620 can be similar to... Figure 5C In process 520. For example... Figure 6B As shown, process 620 includes multiple steps.

[0196] In step 621, the configuration buffer is written. The configuration buffer can be, for example... Figure 1 The configuration buffer 118 in the middle may be... Figures 8B to 8E The configuration buffer is described in further detail below. The configuration buffer can be configured via option codes. In some embodiments, the option code includes multiple bits. The memory device can receive commands to write to the configuration buffer, such as commands from a host device. The commands may include information about the configuration buffer, such as bits of the configuration buffer option code. The configuration buffer may be included in the controller of the memory device, such as in control circuitry.

[0197] In step 622, the written configuration buffer is read out, for example, via a memory device. In step 623, the memory device (e.g., control circuitry) determines whether the configuration buffer has been written correctly, for example, by determining whether the read configuration buffer matches the information in the command. If the configuration buffer has been written correctly, process 620 completes in step 624. If the configuration buffer has not been written correctly, an error message or notification is generated in step 625. The error message or notification may be sent to the host device via an interface. The host device may take action, such as resending the command to write the configuration buffer to the memory device.

[0198] like Figure 6A and Figure 6D As shown, the instructions "Write Vector Data", "MAC", and "Read MAC Result" (e.g., items 5, 6, and 7 in Table 600) can be associated with performing a MAC operation in a memory device. Process 630 can be similar to... Figure 5D Process 530 in the middle. For example... Figure 6D As shown, process 630 includes multiple steps.

[0199] In step 631, the memory device writes input data (e.g., vector data) into one or more memory cells or memory groups according to an operation instruction (e.g., from the host device). The operation instruction may include a command for MAC operation, input data, and address information (e.g., a starting address) in one or more memory cells or memory groups corresponding to weight data stored in one or more memory cells or memory groups. A corresponding portion of the input data may be written to the input latch (e.g., ...) of each of the one or more memory cells or memory groups. Figure 3A , Figure 3B or Figures 4A to 4D Input latch 333), for example, such as Figure 4A or Figure 4B As shown.

[0200] In step 632, the memory device performs a MAC operation on the input data using weighted data stored in one or more memory cells or memory groups according to the arithmetic instructions. As described above, the memory device can use a global adder (e.g., Figure 3A , Figure 3B or Figures 4A to 4D The global adder 320 in the code produces one or more MAC results. These one or more MAC results can be stored in an output buffer (e.g., ...). Figure 3A , Figure 3B or Figures 4A to 4D The output buffer (314) is used to generate the final MAC result based on one or more MAC results.

[0201] In step 633, after the MAC operation is completed, the memory device reads the final MAC result from the output buffer and outputs the final MAC result to the host device via the interface. Process 630 ends in step 634.

[0202] Figure 7 The following timing diagrams illustrate an example of a MAC operation performed via an interface according to a protocol: a first timing diagram 700, showing the receipt of a MAC operation command with input data (a); and a second timing diagram 710, showing the reading of the MAC result (b). The protocol may be the OPI protocol. Table 1 shows the example MAC-related instructions and protocols.

[0203] Table 1. MAC-related commands and protocols.

[0204]

[0205] Instructions can be received from the host device (e.g., Figure 1 The host device 120 in the middle transmits the data to the memory device (e.g., Figure 1 Memory device 110 in Figure 2 Memory device 200 in Figure 3A The memory device 300 or Figure 3B (Memory device 350 in the middle). As shown in Table 1 and Figure 7 As shown, when a memory device is selected, the instruction "MAC with Vector" can be transmitted to the memory device via the interface, with pin CS# at a low level. The interface can also receive a serial clock signal (SCLK). Instructions can be received using the interface's data pins SIO[7:0].

[0206] like Figure 7 As shown in Figure (a), the instruction "MAC with Vector" includes a command, a start address, and input data (e.g., vector data). The command can be represented by command codes 12h and EDh. The start address (ADDR) is 4 bytes long, for example, represented by A[31:24], A[23:16], A[15:8], A[7:0]. The input data can be represented by multiple character units, such as character units D1, D0, ..., D255, D254. The length of the input data depends on the start dimension defined in the corresponding configuration buffer, for example, ... Figure 8A and Figure 8B Further details are provided below. For example, the length can be an integer N. When the CS signal changes from low to high, the CS signal triggers the internal MAC to execute.

[0207] The host device can use the read status register (RDSR) command to send a MAC instruction to the memory device to read the execution status of the MAC instruction. When the MAC instruction completes, the memory device responds to the RDSR command to notify the host device, and then the host device can send a read command to read the MAC result from the memory device. As mentioned above, the MAC result can be stored in the output buffer of the memory device.

[0208] like Figure 7 As shown in Figure (b), when selecting a memory device, the instruction "Read MAC Result" can be transmitted to the memory device via the interface, where pin CS# is low. The interface can also receive a serial clock signal (SCLK). Instructions can be received using the interface's data pins SIO[7:0]. Instructions include a command, a start address, virtual data, and output data (e.g., MAC result). Commands can be represented by command codes EEh and 11h. The start address (ADDR) is 4 bytes long, for example, represented by A[31:24], A[23:16], A[15:8], A[7:0], but this is irrelevant because the ADDR is read from the output buffer, not from the memory device. Output data can be represented by multiple word units, for example, word units D1, D0, D3, D2... after the virtual cycle. The total length of the output data depends on the weight dimension defined in the corresponding configuration buffer, for example, as... Figure 8A and Figure 8B Further details will be provided below. For example, the total length can be an integer M.

[0209] Figure 8A The MAC operation 800 in the example illustrates the multiplication of the weight matrix M×N with the data vector 1×N, resulting in a vector 1×M. The MAC operation is performed using the function ∑(weights×vector), which includes multiplication and addition operations. For example, combining... Figure 3A and Figure 3B As stated above. Figures 8B to 8E Instructions for Figure 8A MAC operation instance configuration buffer. MAC operations can be performed via... Figures 8B to 8E Configure the configuration buffer shown. The configuration buffer can be, for example... Figure 1 The configuration buffer 118 in the memory device may be included, for example, in the memory device. Figure 1 Memory device 110 in Figure 2 Memory device 200 in Figure 3A The memory device 300 or Figure 3BThe memory device 350 is configured to perform read or write operations and can be stored in volatile or non-volatile memory. The configuration buffer can be configured via option codes (OPs) that may include multiple bits.

[0210] For example, each column of a weight matrix M×N has N weight values, which correspond to the number of vector values ​​N in the data vector. Therefore, the length of the N weight values ​​in a column of the weight matrix, or the length N of the data vector, can be considered as the starting dimension. This starting dimension can be configured through the corresponding configuration buffer, for example, as... Figure 8B As shown.

[0211] The weight matrix M×N has M columns and N rows, and the MAC result can include the corresponding M results in the result vector. Therefore, the size of the M columns in the weight matrix M×N or the length of the result vector can be regarded as the weight dimension. This weight dimension can be configured through the corresponding configuration buffer, for example, as shown below. Figure 8B As shown.

[0212] like Figure 8C As shown, a configuration buffer can be configured for the startup format, where the option code OP[2] for the startup format represents the symbol information of the integer N. In some instances, for example, such as Figure 8C As shown, option code OP[2]=1 indicates the selection of a signed integer; option code OP[2]=0 indicates the selection of an unsigned integer. A configuration buffer can be configured for the weight format, where option codes OP[1:0] represent the sign information of the integer M and the number of bits used to represent the weight dimension range. In some instances, for example, such as... Figure 8C As shown, OP0=1 indicates the selection of signed integers; OP0=0 indicates the selection of unsigned integers; OP1=1 indicates the selection of INT8; OP1=0 indicates the selection of INT4.

[0213] In signed integers, the quantity can be positive or negative. In some implementations, the leftmost bit of a signed integer is the sign bit (0 for positive, 1 for negative). For example, with 8 bits, the range is -128 to 127. Negative numbers can be represented using two's complement. Unsigned integers can only represent non-negative numbers, i.e., 0 and positive numbers. With 8 bits, since all bits are used to represent the value and there is no sign bit, the range is 0 to 255. For example, INT8 represents 8 bits (1 byte), corresponding to a signed range of -128 to 127 and an unsigned range of 0 to 255. Similarly, INT4 represents 4 bits (half a byte), corresponding to a signed range of -8 to 7 and an unsigned range of 0 to 15.

[0214] In some implementations, a configuration buffer can be used to select multiple memory cells or memory groups in a memory device for parallel execution of arithmetic instructions (e.g., MAC operations). The option codes OP[2:0] of the configuration buffer can specify the number of memory groups. For example, as... Figure 8D As shown, option codes OP[2:0] represent a single group operation, OP[2:0]=1 represents a double group operation, OP[2:0]=2 represents a four-group operation, OP[2:0]=3 represents an eight-group operation, and so on, up to all group operations. To obtain the highest MAC throughput, the configuration buffer can be set to select all group operations. To reduce power consumption, the configuration buffer can be set to select a single group operation. When the configuration buffer is set to select multiple groups instead of all group operations, this can be achieved using the "MAC with Vector" command (e.g., ...). Figure 5A or Figure 5D (as shown) or the "Write Vector Data" command (e.g., as shown) Figure 6A or Figure 6D As shown, allocate the selected group or memory unit.

[0215] In some implementations, the interface of the memory device (e.g., Figure 1 Interface 114 in the middle, Figure 3A , Figure 3B or Figures 4A to 4D Interface 302 in the configuration can be switchable. For example, the configuration buffer can be used to switch the interface protocol between a first interface protocol and a second interface protocol, for example, as shown in the example. Figure 8E As shown. The first interface protocol may include a Low-Power Double Data Rate (LPDDR) protocol, and the second interface protocol may include one of a Serial Peripheral Interface (SPI), a Queued Serial Peripheral Interface (QPI), or an Octal Peripheral Interface (OPI). Option code OP[0] can be set to select the first interface protocol or the second interface protocol. In some instances, for example, such as Figure 8EAs shown, option code OP[0]=0 indicates selection of the second interface protocol or SPI / QPI / OPI mode; option code OP[0]=1 indicates selection of the first interface protocol or LPDDR mode. In some implementations, the configuration buffer can be written via the second interface protocol (e.g., SPI / QPI / OPI) to switch from the second interface protocol (e.g., SPI / QPI / OPI) to the first interface protocol (e.g., LPDDR), or the configuration buffer can be written via the first interface protocol (e.g., LPDDR) to switch from the first interface protocol (LPDDR) to the second interface protocol (e.g., SPI / QPI / OPI).

[0216] In some implementations, the content read from the self-memory device can be optional. For example, a configuration buffer can be used to read commands to switch between the read content and the stored data, such that the same read command can be used to select the read command using the configuration buffer. In some instances, for example, such as... Figure 8E As shown, option code OP[1] can be set to select the content to be read. For example, option code OP[1]=0 indicates that the MAC result is selected (e.g., from...). Figure 5A , Figure 5D or Figure 6A , Figure 6D The output buffer shown is read); option code OP[1]=1 indicates that the weight / embedded data is read (e.g., from the output buffer read); Figure 5A , Figure 5B or Figure 6A , Figure 6B (Read the memory device shown).

[0217] Figure 9 This is a flowchart of an example process 900 for managing memory devices, such as in-memory computing (IMC) devices. Process 900 can be executed by the memory device, for example, Figure 1 Memory device 110 in Figure 2 Memory device 200 in Figure 3A The memory device 300 or Figure 3B The memory device 350 in the memory may include one or more memory banks, for example... Figure 1 Memory group 132 in Figure 2 Memory group 210 or Figure 3A or Figure 3B The memory group 308 in the memory. Each memory group may include one or more memory cells, for example Figure 3A , Figure 3B or Figures 4A to 4DThe memory cell 330 may include a memory array, such as a NOR flash memory array. The memory device may include a controller coupled to a memory bank or memory cell (e.g., Figure 1 Controller 112 or Figure 2 The controller 220 in the figure. The controller may include at least one interface (e.g., Figure 1 Interface 114 or Figure 3A , Figure 3B or Figures 4A to 4D Interface 302) and control circuit 116 (e.g., Figure 1 Control circuit 116 or Figure 3A or Figure 3B (Control circuit 304 in the middle).

[0218] Process 900 may include multiple steps. In step 902, the host device (e.g., via a memory device) is accessed. Figure 1 The host device 120 receives arithmetic instructions. The arithmetic instructions may include commands for performing arithmetic operations (e.g., MAC operations), address information, and input data. The input data may be a data vector or a data matrix, for example, such as... Figure 8A As shown. The address information may include the starting address of the weight data in the corresponding IMC device used to execute the calculation instructions. The calculation instructions may be, for example... Figure 7 The "MAC with Vector" instruction is shown in Figure (a).

[0219] In step 904, the input data is transferred to one or more memory cells or memory groups in the memory device, for example, such as Figure 4A or Figure 4B As shown. The host device can, for example, select one or more memory groups for single-group operation, multi-group operation, or full-group operation by configuring a buffer, such as... Figure 8D As shown. Each of one or more memory cells may include one or more input latches (e.g., Figure 3A , Figure 3B or Figures 4A to 4D The input latch 333 is used to receive and store the corresponding part of the input data.

[0220] In some implementations, the controller includes a repair control circuit (e.g., Figure 2 Repair control circuit 262 or Figure 3A , Figure 3B Or the repair control circuit 318 in 4B). The repair control circuit can be used to: in response to determining that a designated area in the input latch of one or more memory cells used to store a corresponding portion of the input data is damaged, remap the corresponding portion of the input data to a redundant area in the input latch of the memory cell.

[0221] In step 906, for each of one or more memory cells, the self-memory cell (e.g., a self-memory cell array, etc.) Figure 3A , Figure 3B or Figures 4A to 4D The stored data (e.g., weighted data) is read from the memory cell array 331 in the memory. The stored data can be read from the corresponding internal sense amplifier circuit (e.g., Figure 3A , Figure 3B or Figures 4A to 4D The internal SA circuit 332 in the memory can be used to read the weights. The weights can be stored in one or more memory cells, and the number of the one or more memory cells can be based on the size of the weights and the type of memory cells (e.g., SLC, MLC, TLC, QLC or PLC).

[0222] According to the arithmetic instructions, the arithmetic operation is performed on the corresponding input data and stored data. The multiplier circuit of the memory circuit (e.g.) Figure 3A , Figure 3B or Figures 4A to 4D The multiplier 334 in the circuit can multiply the corresponding input data with the stored data to obtain multiple multiplication results. Adder circuits (e.g.) Figure 3A , Figure 3B or Figures 4A to 4D The adder 335 in the code can add these multiplication results together to obtain the corresponding sum.

[0223] In some implementations, the controller executes arithmetic instructions in one or more memory units based on input data. The input data corresponds to multiple weights stored in the one or more memory units. Each of the one or more memory units can perform an arithmetic operation on a corresponding portion of the input data and the corresponding weights of those weights. Each of the one or more memory units can execute the arithmetic operations in parallel with each other. In some instances, the input data comprises a data vector with multiple vector values, the number of which is the same as the number of weights. The multiplier can multiply each corresponding weight by the corresponding vector value of the corresponding portion of the input data to obtain the corresponding multiplication result.

[0224] In step 908, the result of the executed arithmetic instruction is determined based on the execution result of the arithmetic operations in each of the one or more memory cells. The memory device may include a global adder (e.g., Figure 3A , Figure 3B , Figure 4C or Figure 4D The global adder 320 in the memory is used to generate an operation result based on the corresponding sum obtained from the adder circuit of one or more memory units.

[0225] In some implementations, the memory device further includes one or more secondary adders (e.g., Figure 3B or Figure 4D The secondary adder 352 is coupled to the global adder. Each of the one or more secondary adders may be coupled to a corresponding memory cell (or a corresponding memory bank) and used to add the corresponding sums from the adder circuits of the corresponding memory cell to obtain the corresponding level sum, and the global adder is used to produce the operation result based on one or more corresponding level sums from the one or more secondary adders.

[0226] In step 910, the memory device outputs the calculation result to the host device. The controller may include an input buffer (e.g., Figure 3A , Figure 3B , Figure 4A or Figure 4B The input buffer 312 is used to store input data before transferring the input data to one or more memory units; and the output buffer (e.g., Figure 2 Output buffer 254 or Figure 3A , Figure 3B , Figures 4C to 4D The output buffer 314 is used to store the calculation result before outputting the calculation result to the host device via at least one interface.

[0227] In some implementations, the controller includes at least one of the following: a clock generator (e.g., Figure 2 Clock generator 241 or Figure 3A , Figure 3B The clock generator 322 in the middle is used to generate a clock signal for each of one or more IMC devices performing arithmetic operations; or a timing control circuit (e.g., Figure 2 The timing control circuit 260 or Figure 3A , Figure 3B , Figures 4A to 4D The timing control circuit 316 is used to set the operation timing during the execution of an arithmetic operation in each of one or more memory cells. The operation may include two or more of the following: operation of the internal sense amplifier circuit, operation of the input latch, and operation of the adder circuit.

[0228] In some implementations, at least one interface is used to receive input data from a host device and output the calculation result to the host device. The control circuit is used to perform at least one of the following: programming corresponding data stored in one or more memory units; transferring input data to one or more memory units; executing calculation instructions on the input data and the corresponding data stored in one or more memory units; or outputting the calculation result to at least one interface.

[0229] In some implementations, the controller includes one or more configuration buffers (e.g., Figure 1 (Configuration buffer 118 in the configuration buffer). Each configuration buffer corresponds to a feature and stores option codes used to set that feature, for example, such as... Figures 8B to 8E As shown. The controller can be used to set the option codes for each of one or more configuration buffers. In some cases, one or more configuration buffers are used to be pre-configured by the host device before sending commands to the memory device to execute operation instructions.

[0230] In some instances, one or more configuration buffers include at least one of the following: a configuration buffer for initiating a dimension representing the length of the input data (e.g., such as...). Figure 8B As shown), where the option code for the startup dimension represents the integer N; the configuration buffer is used for the startup format (e.g., as shown). Figure 8C As shown), the option code used for the startup format represents the symbol information of the integer N; the configuration buffer, used for the weight dimension, represents the size of the stored data (e.g., as shown). Figure 8B As shown), the option code for the weight dimension represents the integer M; the configuration buffer is used for the weight format and the number of bits, where the weight format represents the sign information of the integer M, and the number of bits is used to represent the range of the weight dimension (e.g., as shown). Figure 8C (as shown); or a configuration buffer, used to select the number of one or more IMC devices for parallel execution of arithmetic instructions, wherein the option code specifies the number of one or more memory units or memory groups, for example, as Figure 8D As shown.

[0231] In some implementations, one or more configuration buffers include at least one of the following: a configuration buffer for switching the protocol of at least one interface between a first interface protocol and a second interface protocol; or a configuration buffer for reading commands to switch the read content between the calculation result and the stored data, for example, such as Figure 8E As shown. In some instances, the first interface protocol includes the Low-Power Double Data Rate (LPDDR) protocol, and the second interface protocol includes one of the Serial Peripheral Interface (SPI), the Queued Serial Peripheral Interface (QPI), or the Octal Peripheral Interface (OPI).

[0232] In some implementations, at least one interface includes an input / output (I / O) interface configured according to an interface protocol, which includes one of the Serial Peripheral Interface (SPI), Queued Serial Peripheral Interface (QPI), or Octal Peripheral Interface (OPI) protocols.

[0233] In some implementations, at least one interface includes: a first interface configured according to the LPDDR protocol; and a second interface configured according to one of the SPI, QPI, or OPI protocols. The second interface is used to program the corresponding stored data in these memory cells or memory groups, and the first interface is used to perform at least one of the following: setting one or more corresponding configuration buffers; transferring input data to the memory cells; executing arithmetic instructions on the input data and the corresponding stored data in the memory cells; or outputting the arithmetic result to the first interface.

[0234] In some implementations, the controller receives arithmetic instructions from the host device. These instructions include commands, input data, and information corresponding to the starting address of the stored data in each memory cell. The starting address corresponds to a model associated with the arithmetic operation, and the memory cell can read the stored data based on the starting address.

[0235] In some implementations, the controller is used to receive read status commands from the host device (e.g., ... Figure 7 The controller can receive read commands from the host device and output the calculation results to the host device based on the read commands. (The RDSR command shown is an example of this.)

[0236] In some implementations, memory cells in the memory device are used to perform specific functions corresponding to computational operations. Memory cells can be used to store the weights of multiple models, with the weight of each model stored in a corresponding area of ​​each memory cell. The weight of each model can be updated in the corresponding area of ​​each memory cell.

[0237] Figure 10 A flowchart illustrating the execution of instructions under interface protocols such as SPI, QPI, or OPI is provided. Figure 10 The upper part of the flowchart represents the instructions received by the memory device and its operating mode, while the lower part represents the process of the memory device performing digital memory in-memory operations (dCIM).

[0238] First, at time T1, the host device (e.g.,Figure 1 The host device 120 in the memory device transmits a multiply-accumulate (MAC) instruction to the controller in the memory device (e.g., Figure 1 The controller 120 in the memory device puts the memory device into MAC mode. In some embodiments, as described above, the multiply-accumulate (MAC) instruction can be a vectored MAC instruction.

[0239] At this time, the controller receives arithmetic instructions containing input data from the host device, and the memory unit (e.g., Figure 3A , Figure 3B or Figures 4A to 4D The memory unit 330 in the middle reads the stored data (e.g., weights) corresponding to the first MAC operation and transmits the vector to the latch.

[0240] Next, at time T2, the controller controls the memory unit to perform the first MAC operation and read the stored data corresponding to the second MAC operation. In other words, while the memory unit is reading the stored data corresponding to the first MAC operation, it will then continue reading the stored data corresponding to the second MAC operation.

[0241] In some embodiments, after the memory unit completes the first MAC operation, the controller stores the result of this operation (i.e., the first operation result). Specifically, the controller retrieves the first operation result from the global adder (e.g., ...). Figure 3A , Figure 3B , Figure 4C or Figure 4D The global adder 320 in the middle is sent to the output buffer (e.g., Figure 3A , Figure 3B , Figure 4C or Figure 4D The output buffer 314 in the diagram is therefore labeled as "GA to OB".

[0242] Next, at time T3, similar to time T2, the controller controls the memory unit to perform the second MAC operation and reads the stored data corresponding to the third MAC operation. However, since the output buffer still stores the first operation result at this time, after the second MAC operation is completed and the second operation result is generated, the second operation result is not immediately transmitted to the output buffer, but is only stored in the output buffer when the memory device (in the subsequent result reading mode) outputs the first operation result.

[0243] Furthermore, since the result of the second operation has not yet been stored, the memory unit will not immediately execute the third MAC operation after reading the stored data corresponding to the third MAC operation. Instead, the third MAC operation will only be completed after the memory device (in the subsequent result reading mode) stores the result of the second operation into the output buffer.

[0244] In some embodiments, after the host device sends a MAC instruction to the memory device, it sends a read status buffer command to the controller to confirm whether the polling state of the memory device is busy or ready. When the memory device completes the storage of the first calculation result, the polling state switches to the ready state. In some embodiments, the controller sends a ready signal to the host device to indicate that the memory device has completed the storage of the first calculation result.

[0245] When the host device confirms that the memory device has entered the ready state, it sends a result read instruction to the memory device, causing the memory device to enter the result read mode. At time T4, the memory device enters the result read mode and begins to output the result of the MAC operation.

[0246] As described above, in the result read mode, the memory device outputs the first operation result to the host device. Furthermore, since the memory device outputs the first operation result, the output buffer can store it again; therefore, when the first operation result is output, the second operation result can be transferred to the output buffer for storage. Further, since the second operation result is transferred to the output buffer, the memory unit can begin executing the third MAC operation and read the stored data corresponding to the fourth MAC operation. Similar to the second operation result, the third operation result can be transferred to the output buffer for storage when the second operation result is output.

[0247] Next, at time T5, the controller controls the memory unit to perform the fourth MAC operation (and completes the operation when the result of the third operation is stored), and reads the stored data corresponding to the fifth MAC operation, and so on.

[0248] Figure 11 A schematic diagram of instance data output path 420 from another instance memory device is shown. Figure 11 The configuration is similar to Figure 4C The configuration differs in that, in some higher-speed applications (e.g., LPDDR protocol), the output buffer 314 may include a first buffer 314b and a first-in first-out (FIFO) buffer 314a to store multiple data entries simultaneously.

[0249] Figure 12 A flowchart illustrating the execution of instructions under an interface protocol such as LPDDR is shown. Figure 12 The upper part of the flowchart represents the instructions received by the memory device and its operating mode, while the lower part represents the process of the memory device performing digital memory in-memory operations (dCIM).

[0250] Figure 12 Flowcharts and Figure 10 The flowcharts differ in several ways. The first difference is that after each MAC operation is completed, the result is first stored in the first buffer (e.g., ...). Figure 11 The first buffer 314b (labeled "REG to FIFO" in the diagram) is then moved to the second buffer (e.g., Figure 11 The results are stored in the FIFO buffer 314a (labeled "GA to REG" in the diagram). Therefore, between time T3 and T4, although the memory device has not yet entered the result read mode, since the first operation result has been stored in the FIFO buffer 314a, the second operation result can be stored in the first buffer 314b first. Similarly, in the result read mode, the first operation result is output to the host device, so the second output result can be moved from the first buffer 314b to the FIFO buffer 314a, and the third MAC operation can be executed to produce the third operation result in the first buffer 314b, and so on.

[0251] Figure 12 and Figure 10 The second difference between the embodiments is that, in Figure 12 In some embodiments, the result reading mode can be divided into multiple sub-modes by multiple delay intervals tRNR, and the memory device will stop outputting the calculation result during the delay interval and perform data reading and MAC operation at the beginning of the next sub-mode. In some embodiments, the delay interval tRNR is determined by the delay within the memory device.

[0252] Figure 12 and Figure 10 The third difference in the embodiment is that the host device does not send a read status buffer command to the controller to determine the polling status of the memory device, and therefore the controller does not send a ready signal to the host device. Instead, after a certain period of time has elapsed since the host device sent the MAC instruction to the memory device (e.g., ...), Figure 12 During the waiting time tMAC, the host device will send a result read instruction to the memory device, causing the memory device to enter the result read mode.

[0253] Figures 13A to 13B It is illustrated for Figure 12 Configuring a buffer for MAC operations. Figure 13A In some embodiments, option codes OP[0:1] are used to determine the length of the wait time tMAC. In some embodiments, for example, such as Figure 13A As shown, option code OP[0:1]=00 indicates a waiting time tMAC equal to 100 milliseconds; option code OP[0:1]=01 indicates a waiting time tMAC equal to 1000 milliseconds; option code OP[0:1]=10 indicates a waiting time tMAC equal to 5000 milliseconds; option code OP[0:1]=11 indicates a waiting time tMAC equal to 10000 milliseconds.

[0254] exist Figure 13B In some embodiments, option codes OP[0:1] are used to determine the number of system clock cycles represented by the wait time tMAC. In some embodiments, such as... Figure 13B As shown, option code OP[0:1]=00 indicates that the waiting time tMAC is equal to the length of 100,000 system clock cycles; option code OP[0:1]=01 indicates that the waiting time tMAC is equal to the length of 1 million system clock cycles; option code OP[0:1]=10 indicates that the waiting time tMAC is equal to the length of 5 million system clock cycles; option code OP[0:1]=11 indicates that the waiting time tMAC is equal to the length of 10 million system clock cycles.

[0255] It should be noted that Figure 13A and Figure 13B The number of bits for the option code is merely an example; option codes with other bit counts are all within the scope of this invention. In some embodiments, the option code may be a digital signal containing more than two bits (e.g., OP[0:2]) to allow the wait time tMAC to switch between longer time lengths and / or more system clock cycles.

[0256] The processes and logic described in this document can be executed by one or more programmable processors that execute one or more computer programs to perform the functions described herein. These processes and logic can also be executed by special-purpose logic circuits (e.g., field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs)), and the devices can also be implemented as special-purpose logic circuits.

[0257] Processors suitable for performing computer programming include, for example, general-purpose and special-purpose microprocessors, and one or more processors of any type of digital computer. Typically, the processor receives instructions and data from read-only memory or random access memory, or both. Basic components of a computer may include: a processor for executing instructions; and one or more memory devices for storing instructions and data. Typically, a computer may also include one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data, or be operatively coupled to one or more mass storage devices to receive data from or transfer data to them, or both. However, a computer does not necessarily need to have such devices. Computer-readable media suitable for storing computer programming instructions and data may include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices); magnetic disks. The processor and memory may be supplemented by or incorporated into special-purpose logic circuitry.

[0258] While this document may describe numerous details, these details should not be construed as limiting the scope of the claimed or potentially claimed invention, but rather as descriptions of features of particular embodiments. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as acting in certain combinations, and even initially claimed so, in some cases, one or more features from the claimed combination may be removed from that combination, and the claimed combination may be for sub-combinations or variations thereof. Similarly, although operations are depicted in a specific order in the figures, this should not be construed as requiring these operations to be performed in the specific or sequential order shown, or that all shown operations must be performed to achieve the desired result. The above are merely preferred embodiments of the invention, and various modifications and equivalent changes can be made to the invention without departing from the scope or spirit of the invention. In summary, all modifications and equivalent changes made to the invention within the scope of the appended claims are within the scope of the invention.

Claims

1. A memory device comprising: Multiple memory units; and A controller, coupled to the plurality of memory cells, is used to execute an arithmetic instruction on input data in the plurality of memory cells. The plurality of memory units are used to store data, receive input data from the controller, and sequentially perform multiple operations on the input data and the stored data according to the operation instructions from the controller, so as to generate multiple operation results and send them to the controller. The controller is used for: Control the plurality of memory units to execute a first operation and a second operation in sequence from the plurality of arithmetic operations; Store a first operation result corresponding to the first operation; Output the result of the first operation; and When outputting the first operation result, a second operation result corresponding to the second operation is stored.

2. The memory device of claim 1, wherein each of the plurality of memory cells comprises: An array of memory cells is used to store a corresponding portion of multiple weight data as the stored data; and An external circuit is coupled to the memory cell array and is used to: The controller receives a corresponding portion of the input data. The stored data is read from the memory cell array; as well as The multiple operations are performed on the corresponding portion of the input data and the stored data to produce the multiple operation results.

3. The memory device of claim 2, wherein the peripheral circuitry comprises: A sensing amplifier circuit is coupled to the memory cell array and is used to read the corresponding portion of the plurality of weight data from the memory cell array; An input latch for storing the corresponding portion of the input data from the controller; A multiplier circuit is used to multiply the corresponding parts of the multiple weight data and the corresponding parts of the input data to output multiple multiplication results; as well as An adder circuit is used to add multiple multiplication results to output a unit operation result, wherein the controller is used to add the unit operation results of multiple memory units to obtain one of the multiple operation results.

4. The memory device of claim 3, wherein when the controller receives a multiply-accumulate (MAC) instruction, the memory device is configured to enter a MAC mode, wherein in the MAC mode: The plurality of memory units are used to read the stored data corresponding to the first arithmetic operation; When the plurality of memory units complete reading the stored data corresponding to the first operation, the plurality of memory units are used to execute the first operation and read the stored data corresponding to the second operation; When the plurality of memory units complete the first operation, the controller is used to store the result of the first operation; as well as When the plurality of memory units complete reading the stored data corresponding to the second operation, the plurality of memory units are used to execute the second operation and read the stored data corresponding to a third operation among the plurality of operations.

5. The memory device of claim 4, wherein when the controller receives a result read instruction, the memory device is configured to enter a result read mode, wherein in the result read mode: These multiple memory units are used to perform the third arithmetic operation; and This controller is used for: Output the result of the first operation; When outputting the first operation result, the second operation result corresponding to the second operation is stored; After outputting the result of the first operation, output the result of the second operation. When outputting the second operation result, store a third operation result corresponding to the third operation; and After outputting the result of the second operation, output the result of the third operation.

6. The memory device of claim 5, wherein the controller includes an output buffer for storing the plurality of operation results, wherein the output buffer includes: A first buffer, used to store and transmit the multiple operation results; and A second buffer is used to receive the multiple calculation results from the first buffer, store the multiple calculation results, and output the multiple calculation results. The result of the second operation corresponding to the second operation is stored in the first buffer in the MAC mode and moved to the second buffer in the result reading mode.

7. The memory device of claim 5, wherein the result reading mode is divided into multiple sub-modes by multiple delay intervals, the multiple memory cells are used to execute one of the multiple arithmetic operations at a starting point in each of the multiple sub-modes, and the controller is used to stop outputting the multiple arithmetic results at the multiple delay intervals.

8. The memory device of claim 5, wherein the controller is configured to receive the result read instruction after a waiting time following the receipt of the MAC instruction, wherein the waiting time is determined by a digital signal comprising at least one bit.

9. The memory device of claim 5, wherein the controller is configured to receive the result read instruction a specified number of system clock cycles after receiving the MAC instruction, wherein the specified number of clock cycles is determined by a digital signal comprising at least one bit.

10. A memory system comprising: A main unit; and A memory device comprising: Multiple memory units; and A controller, coupled to the plurality of memory units, is configured to receive an arithmetic instruction containing input data from the host device and execute the arithmetic instruction on the plurality of memory units. The plurality of memory units are used to store data, receive input data from the controller, and sequentially perform multiple operations on the input data and the stored data according to the operation instructions from the controller, so as to generate multiple operation results and send them to the controller. The controller is used for: Control the plurality of memory units to execute a first operation and a second operation in sequence from the plurality of arithmetic operations; Store a first operation result corresponding to the first operation; Output the first calculation result to the host device; and When outputting the first operation result, a second operation result corresponding to the second operation is stored.

11. The memory system of claim 10, wherein each of the plurality of memory cells comprises: An array of memory cells is used to store a corresponding portion of multiple weight data as the stored data; and An external circuit is coupled to the memory cell array and is used to: The controller receives a corresponding portion of the input data. The stored data is read from the memory cell array; as well as The multiple operations are performed on the corresponding portion of the input data and the stored data to produce the multiple operation results.

12. The memory system of claim 11, wherein the peripheral circuitry comprises: A sensing amplifier circuit is coupled to the memory cell array and is used to read the corresponding portion of the plurality of weight data from the memory cell array; An input latch for storing the corresponding portion of the input data from the controller; A multiplier circuit is used to multiply the corresponding parts of the multiple weight data and the corresponding parts of the input data to output multiple multiplication results; as well as An adder circuit is used to add multiple multiplication results to output a unit operation result, wherein the controller is used to add the unit operation results of multiple memory units to obtain one of the multiple operation results.

13. The memory system of claim 12, wherein when the host device transmits a multiply-accumulate (MAC) instruction to the controller, the memory device is configured to enter a MAC mode, wherein in the MAC mode: The plurality of memory units are used to read the stored data corresponding to the first arithmetic operation; When the plurality of memory units complete reading the stored data corresponding to the first operation, the plurality of memory units are used to execute the first operation and read the stored data corresponding to the second operation; When the plurality of memory units complete the first operation, the controller stores the first operation result and transmits a ready signal to the host device. as well as When the plurality of memory units complete reading the stored data corresponding to the second operation, the plurality of memory units are used to execute the second operation and read the stored data corresponding to a third operation among the plurality of operations.

14. The memory system of claim 13, wherein when the host device receives the ready signal and transmits a result read instruction to the controller, the memory device is configured to enter a result read mode, wherein in the result read mode: These multiple memory units are used to perform the third arithmetic operation; and This controller is used for: The first calculation result is output to the host device; When outputting the first operation result, the second operation result corresponding to the second operation is stored; After completing the first calculation result, the second calculation result is output to the host device; When outputting the second operation result, store a third operation result corresponding to the third operation; as well as After completing the output of the second calculation result, the third calculation result is output to the host device.

15. The memory system of claim 14, wherein the controller includes an output buffer for storing the plurality of operation results, wherein the output buffer includes: A first buffer, used to store and transmit the multiple operation results; and A second buffer is used to receive the multiple calculation results from the first buffer, store the multiple calculation results, and output the multiple calculation results to the host device. The result of the second operation corresponding to the second operation is stored in the first buffer in the MAC mode and moved to the second buffer in the result reading mode.

16. The memory system of claim 14, wherein the host device is configured to transmit a read status buffer command to the controller after transmitting the MAC instruction to determine a polling state of the memory device, and to transmit the result read instruction to the controller when the polling state of the memory device changes to a ready state.

17. The memory system of claim 14, wherein the result reading mode is divided into multiple sub-modes by multiple delay intervals, the multiple memory cells are used to execute one of the multiple arithmetic operations at a starting point in each of the multiple sub-modes, and the controller is used to stop outputting the multiple arithmetic results to the host device during the multiple delay intervals.

18. The memory system of claim 14, wherein the host device is configured to transmit the result read instruction to the controller after a waiting time following the transmission of the MAC instruction, wherein the waiting time is determined by a digital signal comprising at least one bit.

19. The memory system of claim 14, wherein the host device is configured to transmit the result read instruction to the controller after a specified number of system clock cycles following the transmission of the MAC instruction, wherein the specified number of clock cycles is determined by a digital signal comprising at least one bit.

20. A data output method, applicable to a memory device, wherein the data output method comprises the following steps: (a) Receives an arithmetic instruction containing input data from a host device via the memory device; (b) The input data is transmitted to a plurality of memory cells of the memory device via a controller of the memory device; (c) Performing multiple operations on the input data and a stored data according to the operation instructions through the multiple memory units to generate multiple operation results to the controller, including the following operations: Perform a first operation among the plurality of operations; and Execute the second operation among the multiple operations; (d) The controller stores a first operation result corresponding to the first operation; (e) The controller outputs the first calculation result to the host device; and (f) When the controller outputs the first calculation result, the controller stores a second calculation result corresponding to the second calculation operation.

21. The data output method according to claim 20, further comprising the following steps: In response to receiving a multiply-accumulate (MAC) instruction from the host device, the device enters a MAC mode, wherein steps (a), (b), (c), and (d) are performed in the MAC mode; In response to storing the first operation result corresponding to the first operation, a ready signal is transmitted to the host device via the controller; and In response to receiving a result reading instruction from the host device, a result reading mode is entered, wherein steps (e) and (f) are executed in the result reading mode.

Images