Memory device and method of operation thereof

Abstract

A memory device has a memory array including a memory section for storing weight data, a weight buffer coupled to the memory section and configured to hold new weight data to be updated in the memory section, a logic circuit, and a compute circuit coupled to an output of the logic circuit. The logic circuit also has a first input coupled to the memory section by a bit line, and a second input configured to receive input data. The logic circuit is configured to generate intermediate data at the output, the intermediate data corresponding to the input data and the weight data read from the memory section by the bit line. The compute circuit is configured to generate output data based on the intermediate data, the output data corresponding to a computation performed on the input data and the weight data read from at least one memory section. Embodiments of the invention also disclose a method of operating a memory device.

Description

Technical Field

[0001] The embodiments of the present invention relate to memory devices and methods of operating the same. Background Technology

[0002] Recent developments in the field of artificial intelligence have spawned a variety of products and / or applications, including but not limited to speech recognition, image processing, machine learning, and natural language processing. Such products and / or applications typically utilize neural networks to process large amounts of data for learning, training, cognitive computing, and other tasks. Summary of the Invention

[0003] According to one aspect of the present invention, a memory device is provided, comprising: a memory array including at least one memory segment configured to store weight data; at least one weight buffer coupled to the at least one memory segment and configured to hold new weight data to be updated in the at least one memory segment; and a computing circuit coupled to the output of a logic circuit; the at least one logic circuit having: a first input coupled to the at least one memory segment via at least one bit line, a second input configured to receive input data, and an output; wherein the at least one logic circuit is configured to generate intermediate data at the output, the intermediate data corresponding to the input data and the weight data read from the at least one memory segment via the at least one bit line, and the computing circuit is configured to generate output data based on the intermediate data, the output data corresponding to a calculation performed on the input data and the weight data read from the at least one memory segment.

[0004] According to another aspect of the present invention, a method for operating a memory device is provided, the method comprising simultaneously performing: performing a computation in memory (CIM) operation using a first segment of weight data read from a first memory cell among a plurality of memory cells in a memory segment of the memory device, and updating a second segment of weight data in a second memory cell among a plurality of memory cells in the memory segment, the second memory cell being different from the first memory cell.

[0005] According to another aspect of the present invention, a memory device is provided, comprising: a memory array; an input driver coupled to a plurality of input data lines and configured to provide input data through the plurality of input data lines; and a multiply-accumulate (MAC) circuit coupled to a plurality of read bit lines and the plurality of input data lines and configured to perform a CIM operation based on weight data read from a plurality of memory segments and input data provided from the input driver; the memory array comprising: a plurality of memory segments configured to store weight data for CIM operation; a plurality of write bit lines coupled to the plurality of memory segments to update the weight data stored in the plurality of memory segments; and a plurality of read bit lines coupled to the plurality of memory segments to read the weight data stored in the plurality of memory segments for CIM operation. Attached Figure Description

[0006] The various aspects of the invention will be best understood from the following detailed description when read in conjunction with the accompanying drawings. It should be emphasized that, in accordance with standard industrial practice, the various components are not drawn to scale and are for illustrative purposes only. In fact, for clarity of discussion, the dimensions of the various components may be arbitrarily increased or decreased.

[0007] Figure 1A This is a schematic diagram of a memory device according to some embodiments.

[0008] Figure 1B This is a schematic diagram of a memory device according to some embodiments.

[0009] Figure 2A This is a schematic diagram of a memory device according to some embodiments.

[0010] Figure 2B This is a schematic diagram of a neural network according to some embodiments.

[0011] Figure 2C This is a schematic diagram of an integrated circuit (IC) device according to some embodiments.

[0012] Figure 3 This is a schematic diagram of a portion of a memory device according to some embodiments.

[0013] Figure 4A This is a schematic diagram of a portion of a memory segment corresponding to a memory device, according to some embodiments.

[0014] Figure 4B It is a timing diagram of example operations in a portion of a memory segment corresponding to a memory device according to some embodiments.

[0015] Figure 4C This is a schematic diagram of a portion of a memory segment corresponding to a memory device, according to some embodiments.

[0016] Figure 4D This is a schematic diagram of a portion of a memory segment corresponding to a memory device, according to some embodiments.

[0017] Figure 4E This is a schematic circuit diagram of an example single-port memory cell according to some embodiments.

[0018] Figure 5A This is a schematic diagram of a portion of a memory segment corresponding to a memory device, according to some embodiments.

[0019] Figure 5B This is a schematic diagram of a portion of a memory segment corresponding to a memory device, according to some embodiments.

[0020] Figure 5C This is a schematic diagram of a portion of a memory segment corresponding to a memory device, according to some embodiments.

[0021] Figure 6A This is a schematic diagram of a memory device according to some embodiments.

[0022] Figure 6B These are schematic circuit diagrams of input drivers and corresponding logic circuits in memory devices according to some embodiments.

[0023] Figure 7A This is a schematic diagram of a memory device according to some embodiments.

[0024] Figure 7B This is a schematic circuit diagram of a portion of an adder tree in a memory device according to some embodiments.

[0025] Figure 8 This is a flowchart of a method according to some embodiments. Detailed Implementation

[0026] The following disclosure provides numerous different embodiments or examples for implementing various features of the invention. Specific embodiments of components, values, operations, materials, arrangements, etc., are described below to simplify the invention. Of course, these are merely examples and not intended to be limiting. Other components, values, operations, materials, arrangements, etc., are conceivable. For example, in the following description, forming a first component above or on a second component can include embodiments where the first and second components are in direct contact, and can also include embodiments where an additional component can be formed between the first and second components, thereby allowing the first and second components to not be in direct contact. Furthermore, reference numerals and / or letters may be repeated in various examples. This repetition is for simplicity and clarity and does not in itself indicate a relationship between the various embodiments and / or configurations discussed.

[0027] Memory devices configured to perform computation in memory (CIM) operations (also referred to herein as CIM memory devices) are available for neural network applications and other applications. A CIM memory device includes a memory array configured to store weight data to be used with input data in one or more CIM operations. After one or more CIM operations, the weight data in the memory array is updated for further CIM operations.

[0028] In some embodiments, one or more weight buffers are included in the same memory macro, which contains a memory array storing weight data. The one or more weight buffers are coupled to corresponding memory segments within the memory array. In at least one embodiment, weight data in one or more memory segments is updated from the corresponding weight buffers while other memory segments are accessed to obtain weight data for CIM operations. In at least one embodiment, weight data in one or more memory cells of a memory segment is updated from the corresponding weight buffers, while weight data in other memory cells of the same memory segment is used for CIM operations. As a result, in one or more embodiments, weight data updates and CIM operations can be performed simultaneously. This differs from other methods, where the entire memory array is accessed for CIM operations, and therefore, CIM operations are stopped whenever weight data updates are performed. Because weight data updates and CIM operations are not performed simultaneously according to other methods, such methods may encounter one or more problems, including but not limited to performance degradation, increased processing time, and increased power consumption. According to some embodiments that allow simultaneous weight data updates and CIM operations, CIM memory devices can avoid such problems. In at least one embodiment, because one or more weight buffers are included in the same memory macro as the memory array storing the weight data, it is possible to eliminate weight buffers outside the memory macro or at least reduce their size. This is another difference from other methods that require external weight buffers. Compared to other methods, CIM memory devices without external weight buffers or with reduced-size external weight buffers, in at least one embodiment, offer one or more advantages, including but not limited to reduced chip area, lower manufacturing costs, and improved performance.

[0029] Figure 1A This is a schematic diagram of a memory device 100A according to some embodiments. A memory device is an integrated circuit (IC) device. In at least one embodiment, the memory device is a separate IC device. In some embodiments, the memory device is included as part of a larger IC device that includes circuitry for other functions besides the memory device itself.

[0030] Memory device 100A includes a memory macro 110 and a memory controller 120. Memory macro 110 includes a memory array 112, one or more weighted buffers 114, one or more registers 115, one or more logic circuits 116, and computing circuitry 117. Memory controller 120 includes a word line driver 122, a bit line driver 124, control circuitry 126, and an input buffer 128. In some embodiments, one or more elements of memory controller 120 are included in memory macro 110, and / or, one or more elements of memory macro 110 (other than memory array 112) are included in memory controller 120.

[0031] Macros have reusable configurations and can be used with a variety of IC devices of different types or designs.

[0032] In some embodiments, macros are understood in the context of an architecture analogous to a modular program, where subroutines / processes are called by a main program (or by other subroutines) to perform a given computational function. In this context, IC devices use macros to perform one or more given functions. Thus, in this context, and in terms of architecture, IC devices are analogous to a main program, and macros are analogous to subroutines / processes. In some embodiments, macros are software macros. In some embodiments, macros are hardware macros. In some embodiments, macros are software macros described numerically in register translation level (RTL) code. In some embodiments, macros are not composed, placed, and routed to allow software macros to be composed, placed, and routed for multiple process nodes. In some embodiments, macros are hardware macros described numerically in a binary file format (e.g., Graphical Database System II (GDSII) stream format), where the binary file format represents planar geometry, text labels, other information, and a macro layout diagram in one or more hierarchical forms. In some embodiments, macros are composed, placed, and routed to make the hardware macro specific to a particular process node.

[0033] A memory macro is a macro that includes memory cells, which are addressable to allow data to be written to or read from the memory cells. In some embodiments, a memory macro further includes circuitry configured to provide access to the memory cells and / or perform further functions associated with the memory cells. For example, memory macro 110 includes a memory cell MC as described herein, and weighted buffer 114, register 115, logic circuitry 116, and computational circuitry 117 form circuitry configured to provide CIM functionality associated with the memory cell MC. In at least one embodiment, a memory macro configured to provide CIM functionality is referred to as a CIM macro. The described macro configurations are examples. Other configurations are within the scope of various embodiments.

[0034] The memory cells MC of the memory macro 110 are arranged in multiple columns and rows of the memory array 112. The memory controller 120 is electrically coupled to the memory cells MC and configured to control the operation of the memory cells MC, including but not limited to read operations, write operations, etc.

[0035] The memory array 112 also includes multiple word lines (also referred to as “address lines”) WL1 to WLr extending along the rows of memory cells MC, and multiple bit lines (also referred to as “data lines”) BL1 to BLt extending along the columns, where r and t are natural numbers. Each of the memory cells MC is electrically connected to the memory controller 120 via at least one word line and at least one bit line. In some example operations, word lines are configured to transmit the address of a memory cell MC to be read from, or to transmit the address of a memory cell MC to be written to, etc. In at least one embodiment, a group of word lines is configured to be used as both read word lines and write word lines. Examples of bit lines include read bit lines for transmitting data read from a memory cell MC represented by a corresponding word line, write bit lines for transmitting data to be written to a memory cell MC represented by a corresponding word line, etc. In at least one embodiment, a group of bit lines is configured to be used as both read bit lines and write bit lines. Word lines are generally referred to herein as WL, and bit lines are generally referred to herein as BL. The number of word lines and / or bit lines in the memory array 112 is within the scope of various embodiments. Examples of memory types for the memory cell MC include, but are not limited to, static random access memory (SRAM), resistive RAM (RRAM), magnetoresistive RAM (MRAM), phase-change RAM (PCRAM), spin-transfer torque RAM (STTRAM), floating-gate metal-oxide-semiconductor field-effect transistor (FGMOS), spintronics, etc. In one or more example embodiments described herein, the memory cell MC includes an SRAM memory cell.

[0036] exist Figure 1A In the example configuration, the memory cell MC is a single-port memory cell. In some embodiments, the port of the memory cell is represented by a group of word lines WL and bit lines BL (referred to herein as a WL / BL group), which is configured to provide access to the memory cell in read operations (i.e., read access) and / or write operations (i.e., write access). A single-port memory cell has one WL / BL group, which is configured for both read and write access, but not simultaneously. A multi-port memory cell has several WL / BL groups, each configured for read-only access, write-only access, or both read and write access. Example reference for a single-port memory cell. Figure 1A , Figure 4E Described. Example reference for multi-port memory units. Figure 1B, Figure 5A , Figure 5B , Figure 5C Describe it.

[0037] Memory array 112 includes multiple memory segments. In some embodiments, a memory segment includes memory rows, memory columns, memory banks, etc. A memory row includes multiple memory cells coupled to the same word line WL. A memory column (also referred to as a "memory string") includes multiple memory cells coupled to the same bit line BL. A memory bank includes more than one memory row and / or more than one memory column. In at least one embodiment, a memory bank includes a portion of memory array 112 having multiple memory rows and multiple memory columns. In some embodiments, a memory segment includes multiple memory banks. In an example, a first memory segment 105 includes a memory column of memory cells MC coupled to bit line BL1, a second memory segment 107 includes a memory column of memory cells MC coupled to bit line BL2, etc. Other ways of dividing memory array 112 into multiple memory segments are within the scope of various embodiments.

[0038] Each of the memory cells MC is configured to store a segment of weight data to be used in CIM operations. In one or more example embodiments described herein, the memory cell MC is a single bit memory cell, i.e., each memory cell is configured to store one bit of weight data. This is an example, and multi-bit memory cells, where each memory cell is configured to store more than one bit of weight data, are found in a variety of embodiments. In some embodiments, a single bit memory cell is also referred to as a bit cell. For example, memory cell 113 coupled to word line WL1 and bit line BLt is configured to store a segment W1,t of weight data. The combination of multiple segments of weight data stored in multiple memory cells constitutes a weight value to be used in CIM operations. For simplicity, a segment of weight data stored in a memory cell MC, multiple segments of weight data stored in multiple memory cells MC, or all segments of weight data stored in all memory cells MC of the memory array 112 are referred to herein as weight data.

[0039] Weight buffer 114 is coupled to memory array 112 and configured to temporarily store new weight data to be updated in memory array 112. In some embodiments described herein, each memory segment is coupled to a corresponding weight buffer. In one or more embodiments described herein, a common weight buffer is coupled to several memory segments. Weight buffer 114 is coupled to memory cells MC in memory array 112 via bit lines BL. In a weight data update operation, new weight data is derived from weight buffer 114 and written to one or more memory cells MC via the corresponding bit lines BL. Figure 1AAs schematically illustrated, weight buffer 114 is coupled to memory controller 120 to receive new weight data and / or control signals, the control signals specifying when and / or in which memory cells (MCs) the new weight data is updated. In at least one embodiment, new weight data is received from external circuitry outside the memory device 100A (e.g., a processor as described herein). The new weight data is received via one or more input / output (I / O) circuitry (not shown) of memory controller 120 and forwarded to weight buffer 114. Example weight buffers include, but are not limited to, registers, memory cells, or other circuitry configured for data storage.

[0040] Register 115 has an input coupled to bit line BL to receive weight data read from one or more memory cells MC. Register 115 is configured to latch the weight data received from bit line BL and provide the latched weight data to logic circuitry 116 via the output of register 115. As a result, when the latched weight data is used in CIM operations at logic circuitry 116 and / or computation circuitry 117 as described herein, bit line BL can be used for write operations to update one or more memory cells MC with new weight data from weight buffer 114. As described herein, the simultaneous execution of weight data updates and CIM operations provides one or more advantages. Examples of register 115 include flip-flops, latches, etc. In some embodiments, each register in register 115 is coupled to a bit line in bit line BL of memory array 112. In one or more embodiments, registers (e.g., multi-bit registers) in register 115 are coupled to multiple bit lines in bit line BL of memory array 112.

[0041] According to some embodiments, in addition to the simultaneous execution of weight data updates and CIM operations described for different memory cells in a memory segment, weight data updates and CIM operations can also be performed simultaneously in different memory segments. For example, weight data in the first memory segment 105 is updated with new weight data provided by a corresponding weight buffer in the weight buffer 114 above bit line BL1, while weight data read from the second memory segment 107 above a different bit line BL2 is being used in a CIM operation. In at least one embodiment, the presence of different data on different bit lines BL does not affect or interfere with the simultaneous execution of weight data updates and CIM operations.

[0042] The inputs of logic circuit 116 are coupled to the output of register 115. An additional input of logic circuit 116 is coupled to receive input data D_IN for use with weighting data in CIM operations. Figure 1AIn the example configuration, input data D_IN is provided by input buffer 128 in memory controller 120. In one or more embodiments, input data D_IN is output data provided from another memory macro (not shown) of memory device 100A. In some embodiments, as described herein, input data D_IN is provided serially to logic circuit 116 as a stream of bits. Logic circuit 116 is configured to generate intermediate data 119 at its output corresponding to input data D_IN and weighted data read from one or more memory cells MC. Examples of logic circuit 116 include, but are not limited to, NOR gates, AND gates, any other logic gates, combinations of logic gates, etc.

[0043] Computational circuitry 117 is coupled to the output of logic circuitry 116 and configured to generate output data D_OUT based on intermediate data 119 output from logic circuitry 116. Output data D_OUT corresponds to a CIM operation performed on input data D_IN and weight data read from one or more memory cells MC. Examples of CIM operations include, but are not limited to, mathematical operations, logical operations, and combinations thereof. In some embodiments, computational circuitry 117 is configured to combine multiple intermediate data 119 output from multiple logic circuits 116 into output data D_OUT. In at least one embodiment, computational circuitry 117 includes a multiply-accumulate (MAC) circuit, and the CIM operation includes multiplication of one or more multi-bit weight values ​​with one or more multi-bit input data values. Additional computational circuitry configured to perform CIM operations other than multiplication is available in various embodiments. In some embodiments, output data D_OUT is provided as input data to another memory macro (not shown) of memory device 100A. In one or more embodiments, the output data D_OUT is output to external circuitry outside the memory device 100A, such as the processor described herein, via one or more I / O circuits (not shown) of the memory controller 120.

[0044] exist Figure 1A In an example configuration, controller 120 includes word line driver 122, bit line driver 124, control circuitry 126, and input buffer 128. In at least one embodiment, controller 120 also includes one or more clock generators that provide clock signals to various components of memory device 100A, one or more input / output (I / O) circuits for data exchange with external devices, and / or one or more controllers that control various operations in memory device 100A.

[0045] Word line driver 122 is coupled to memory array 112 via word line WL. Word line driver 122 is configured to decode the row address of the memory cell MC selected for access in a read or write operation. Word line driver 122 is configured to provide voltage to the selected word line WL corresponding to the decoded row address, and to provide different voltages to the other unselected word lines WL.

[0046] Bit line driver 124 is coupled to memory array 112 via bit line BL. Bit line driver 124 is configured to decode the column address of the memory cell MC selected for access in a read or write operation. Bit line driver 124 is configured to provide a voltage to the selected bit line BL corresponding to the decoded column address, and to provide different voltages to other unselected bit lines BL.

[0047] Control circuitry 126 is coupled to one or more of the following: weighted buffer 114, register 115, logic circuitry 116, computation circuitry 117, word line driver 122, bit line driver 124, and input buffer 128, to coordinate the operation of these circuits, drivers, and / or buffers in the overall operation of the memory device 100A. For example, control circuitry 126 is configured to generate various control signals for controlling the operation of one or more of the following: weighted buffer 114, register 115, logic circuitry 116, computation circuitry 117, word line driver 122, bit line driver 124, and input buffer 128.

[0048] Input buffer 128 is configured to receive input data from external circuitry (e.g., a processor as described herein) outside the memory device 100A. The input data is received via one or more I / O circuits (not shown) of the memory controller 120 and forwarded to logic circuitry 116 via input buffer 128. Example input buffers include, but are not limited to, registers, memory cells, or other circuitry configured for data storage.

[0049] In at least one embodiment, a CIM memory device (such as memory device 100A) is superior to other methods in which data moves back and forth between memory and processor because such back-and-forth data movement avoids performance and energy efficiency bottlenecks. Example CIM applications include, but are not limited to, artificial intelligence, image recognition, neural networks for machine learning, etc. In some embodiments, memory device 100A in one or more embodiments enables simultaneous weight data updates and CIM operations. Furthermore, a weight buffer 114 is included in memory macro 110, enabling the elimination of external weight buffers outside memory macro 110, or at least a reduction in the size of external weight buffers. As a result, in at least one embodiment, it is possible to achieve one or more advantages, including but not limited to reduced processing time, reduced power consumption, reduced chip area, reduced manufacturing costs, and improved performance.

[0050] Figure 1B This is a schematic diagram of a memory device 100B according to some embodiments. Figure 1B The components in have been made by Figure 1A The same label in Figure 1A The corresponding component in.

[0051] The difference between memory device 100A and memory device 100B is that memory device 100A includes single-port memory cells, while memory device 100B includes multi-port memory cells. Specifically, memory device 100B includes a memory macro 130 having a memory array 132 in which multi-port memory cells MC are arranged in multiple rows and multiple columns. Multiple read word lines RWL1 to RWLr (commonly referred to as "RWL") and multiple write word lines WWL1 to WWLr (commonly referred to as "WWL") extend along the rows. Multiple read bit lines RBL1 to RBLt (commonly referred to as "RBL") and multiple write bit lines WBL1 to WBLt (commonly referred to as "WBL") extend along the columns. Each memory cell MC is coupled to a pair of read word lines RWL and read bit lines RBL, and is coupled to another pair of write word lines WWL and write bit lines WBL. For example, memory cell 133 is coupled to a pair of read word lines RWL1 and read bit lines RBLt, and to another pair of write word lines WWL1 and write bit lines WBLt. For each memory cell MC, the RWL / RBL pair represents a read port, while the WWL / WBL pair represents a write port. In some embodiments, groups of word lines WL are configured as both write and read word lines. Weighted buffer 114 is coupled to memory array 132 via write bit lines WBL. Figure 1BIn the example configuration, register 115 is omitted and logic circuitry 116 is coupled to the read bit line RBL. In at least one embodiment, register 115 is included in memory device 100B and coupled between read bit line RBL and logic circuitry 116 in a manner similar to memory device 100A. In one or more embodiments, when register 115 is included in memory device 100B, it makes it possible to hold latched weighted data at the input of logic circuitry 116 for an extended period of time, which might be difficult to achieve without register 115.

[0052] In at least one embodiment, the multi-port memory cells of memory array 132 enable simultaneous execution of weight data updates and CIM operations. For example, when memory cell 133 is accessed in a read operation to read a segment W1,t of weight data for the corresponding CIM operation, the read segment W1,t of weight data is provided to logic circuitry 116 via the corresponding read bit line RBLt. Simultaneously, weight data updates are performed on any other memory cell in the same memory column or memory string. For example, concurrent with the CIM operation performed on memory cell 133, a segment of weight data in memory cell 135 is updated via a segment of new weight data provided from weight buffer 114 via the corresponding write bit line WBLt. The described CIM operations and weight data updates are performed on two different bit lines (i.e., read bit line RBLt and write bit line WBLt) and do not affect or otherwise interfere with each other. As a result, in one or more embodiments, weight data updates and CIM operations can be performed simultaneously. In at least one embodiment, one or more advantages of memory device 100A described herein may be implemented by memory device 100B.

[0053] Figure 2A This is a schematic diagram of a memory device 200A according to some embodiments.

[0054] Memory device 200A includes memory macros 202, 204, 206, and 208 and a memory controller 220. In some embodiments, one or more of memory macros 202, 204, 206, and 208 correspond to one or more of memory macros 110 and 130, and / or, memory controller 220 corresponds to memory controller 120. Figure 2A As shown, memory controller 220 is a common memory controller for memory macros 202, 204, 206, and 208. In at least one embodiment, at least one of memory macros 202, 204, 206, and 208 has its own memory controller. The number of four memory macros in memory device 200A is illustrative. Other configurations are within the scope of various embodiments.

[0055] Memory macros 202, 204, 206, and 208 are sequentially coupled, with the output data of the preceding memory macro serving as the input data for the subsequent memory macro. For example, input data DIN is input into memory macro 202. Memory macro 202 performs one or more CIM operations based on the input data DIN and the weight data stored in memory macro 202, generating output data DOUT2 as the result of the CIM operations. Output data DOUT2 is provided as input data DIN4 for memory macro 204. Memory macro 204 performs one or more CIM operations based on the input data DIN4 and the weight data stored in memory macro 204, generating output data DOUT4 as the result of the CIM operations. Output data DOUT4 is provided as input data DIN6 for memory macro 206. Memory macro 206 performs one or more CIM operations based on the input data DIN6 and the weight data stored in memory macro 206, generating output data DOUT6 as the result of the CIM operations. Output data DOUT6 is provided as input data DIN8 for memory macro 208. Memory macro 208 performs one or more CIM operations based on input data DIN8 and weight data stored in memory macro 208, and generates output data DOUT as the result of the CIM operations. One or more of the input data DIN, DIN4, DIN6, and DIN8 correspond to the weight data stored in memory macro 208. Figures 1A-1B The described input data D_IN, and / or one or more of the output data DOUT2, DOUT4, DOUT6, and DOUT, correspond to information about... Figures 1A-1B The described output data is D_OUT. In at least one embodiment, the configuration of the described memory macros 202, 204, 206, and 208 implements a neural network. In at least one embodiment, one or more advantages described herein can be achieved by memory device 200A.

[0056] Figure 2B This is a schematic diagram of a neural network 200B according to some embodiments.

[0057] Neural network 200B comprises multiple layers A through E, each layer containing multiple nodes (or neurons). Nodes in consecutive layers of neural network 200B are interconnected via connection matrices or arrays. For example, nodes in layers A and B are interconnected via connections in matrix 212, nodes in layers B and C are interconnected via connections in matrix 214, nodes in layers C and D are interconnected via connections in matrix 216, and nodes in layers D and E are interconnected via connections in matrix 218. Layer A is the input layer configured to receive input data 211. Input data 211 propagates through neural network 200B from one layer to the next via corresponding connection matrices between layers. As the data propagates through neural network 200B, it undergoes one or more computations and is output as output data 219 from layer E, which is the output layer of neural network 200B. Layers B, C, and D between input layer A and output layer E are sometimes referred to as latent layers or intermediate layers. Figure 2B The number of layers, the number of connection matrices, and the number of nodes in each layer are examples. Other configurations are within the scope of various embodiments. For example, in at least one embodiment, the neural network 200B does not include a hidden layer and has an input layer connected to the output layer via a connection matrix. In one or more embodiments, the neural network 200B has one, two, or more hidden layers.

[0058] In some embodiments, matrices 212, 214, 216, and 218 are correspondingly implemented by memory macros 202, 204, 206, and 208, with input data 211 corresponding to input data DIN and output data 219 corresponding to output data DOUT. Specifically, in matrix 212, the connection between a node in layer A and another node in layer B has a corresponding weight. For example, the connection between node A1 and node B1 has a weight W(A1, B1), which corresponds to a weight value stored in the memory array of memory macro 202. Memory macros 204, 206, and 208 are configured in a similar manner. When machine learning is performed using the neural network 200B, the weight data in one or more of the memory macros 202, 204, 206, and 208 is updated, for example, by a processor and via a memory controller 220. According to some embodiments, one or more advantages described herein can be achieved in the neural network 200B implemented wholly or partially by one or more memory macros and / or memory devices.

[0059] Figure 2C This is a schematic diagram of an integrated circuit (IC) device 200C according to some embodiments.

[0060] IC device 200C includes one or more hardware processors 232 and one or more memory devices 234 coupled to the processors 232 via one or more buses 236. In some embodiments, IC device 200C includes one or more additional circuits, including but not limited to cellular transceivers, GPS receivers, and network interface circuits for one or more of Wi-Fi, USB, Bluetooth, etc. Examples of processor 232 include, but are not limited to, central processing units (CPUs), multi-core CPUs, neural processing units (NPUs), graphics processing units (GPUs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), other programmable logic devices, multimedia processors, image signal processors (ISPs), etc. Examples of memory devices 234 include one or more memory devices and / or memory macros described herein. In at least one embodiment, each processor 232 is coupled to a corresponding memory device within the memory device 234.

[0061] Because one or more memory devices 234 are CIM memory devices, multiple calculations can be performed within the memory devices, which reduces the computational workload of the corresponding processor, reduces memory access time, and improves performance. In at least one embodiment, the IC device 200C is a system-on-a-chip (SoC). In at least one embodiment, one or more of the advantages described herein can be achieved by the IC device 200C.

[0062] Figure 3 This is a schematic diagram of a portion of a memory device 300 according to some embodiments. In at least one embodiment, memory device 300 corresponds to at least one of memory devices 100A and 100B. Figure 3 As shown, the memory device 300 includes at least a memory macro 330 and an input buffer 328. For simplicity, other components of the memory device 300 are omitted.

[0063] In some embodiments, memory macro 330 corresponds to at least one of memory macros 110 and 130. Figure 3 In the example configuration, memory macro 330 is configured for CIM operations and is referred to as the CIM macro. Memory macro 330 includes M memory segments 341, 342, and 343, M weighted buffers 351, 352, and 353, and M registers and logic circuitry (in... Figure 3The memory segments 341, 342, and 343 are designed with the label "Reg+LOC" and are 361, 362, and 363, where M is a natural number. Each of the M memory segments 341, 342, and 343 includes a memory row, memory column, or at least one memory bank in the memory array of the memory macro 330. In some embodiments, the memory array of the memory macro 330 corresponds to at least one of the memory arrays 112 and 132. Each of the memory segments 341, 342, and 343 is configured to store corresponding weight data W[1], ..., W[M-1], W[M]. Each of the memory segments 341, 342, and 343 is coupled to a corresponding weight buffer in the weight buffers 351, 352, and 353, and a corresponding Reg+LOC circuit in the Reg+LOC circuits 361, 362, and 363. In some embodiments, the weight buffers 351, 352, and 353 correspond to one or more weight buffers 114. Weight buffers 351, 352, and 353 are configured to provide new weight data to update the weight data stored in the corresponding memory segments 341, 342, and 343. In some embodiments, each of the Reg+LOC circuits 361, 362, and 363 includes a register corresponding to one or more registers 115 and logic circuitry corresponding to one or more logic circuits 116. The memory macro 330 further includes a MAC circuit 370 coupled to the output of the Reg+LOC circuits 361, 362, and 363. In some embodiments, the MAC circuit 370 corresponds to the calculation circuit 117.

[0064] Input buffer 328 is located outside memory macro 330. In some embodiments, input buffer 328 corresponds to input buffer 128. Input buffer 328 is configured to provide input data D_IN as multiple input data segments IN[1], ...IN[M-1], IN[M] to the corresponding Reg+LOC circuits 361, 362, 363. Reg+LOC circuits 361, 362, 363 are configured to generate intermediate data Y[1], ...Y[M-1], Y[M] corresponding to the input data segments IN[1], ...IN[M-1], IN[M] and the weighted data W[1], ...W[M-1], W[M] read from memory segments 341, 342, 343. Intermediate data Y[1], ...Y[M-1], Y[M] are provided to MAC circuit 370, which is configured to perform further mathematical and / or logical operations to combine the intermediate data Y[1], ...Y[M-1], Y[M] into output data D_OUT. In some embodiments, MAC circuit 370, together with the logic circuits in Reg+LOC circuits 361, 362, 363, is configured to perform multiplication of a multi-bit weight value represented by one or more weight data W[1], ...W[M-1], W[M] with a multi-bit input data value represented by one or more input data segments IN[1], ...IN[M-1], IN[M]. In at least one embodiment, registers are omitted from Reg+LOC circuits 361, 362, 363.

[0065] In some embodiments, weight data in one or more memory segments 341, 342, 343 is updated by providing new weight data from the corresponding weight buffer, while weight data read from another memory segment is used for CIM operations. For example, in a manner similar to Figure 1A In this manner, new weight data is provided via the first bit line to update memory segment 343, while weight data W[M-1] read from memory segment 342 is used for CIM operations via different second bit lines. In at least one embodiment, the presence of different data on different bit lines does not affect or interfere with the simultaneous weight data update and CIM operations. In at least one embodiment, one or more advantages described herein may be achieved by memory macro 330.

[0066] Figure 4A This is a schematic diagram of a portion corresponding to a memory segment of memory device 400A according to some embodiments. In some embodiments, memory device 400A corresponds to at least one of memory devices 100A and 300.

[0067] exist Figure 4AIn the portion shown, memory device 400A includes memory segment 412, weight buffer 414, register 415, logic circuit 416, and MAC circuit 417. In some embodiments, memory segment 412 is a portion of the memory array of memory device 400A. In some embodiments, memory segment 412 corresponds to at least one of memory segments 341, 342, and 343, weight buffer 414 corresponds to at least one of weight buffers 351, 352, and 353, register 415 and logic circuit 416 correspond to at least one of Reg+LOC circuits 361, 362, and 363, and MAC circuit 417 corresponds to MAC circuit 370.

[0068] exist Figure 4A In the example configuration, memory segment 412 is a memory column. The description herein regarding the configuration and operation of memory segment 412 as a memory column applies to other types of memory segments. Memory segment 412 includes multiple memory cells MC[0]...MC[N] correspondingly storing weight data W[0]...W[N]. Memory cells MC[0]...MC[N] are coupled to pairs of bit lines BL and complementary bit lines BLB. For simplicity, the bit line BL described herein and the description of the bit line BL apply to the complementary bit line BLB. Memory cells MC[0]...MC[N] are single-port memory cells configured to use bit lines BL in both read and write operations. Memory cells MC[0]...MC[N] are coupled to corresponding word lines WL[0]...WL[N] for access via the corresponding word lines in either read or write operations. Bit line BL is coupled to weight buffer 414 to receive new weight data to be updated in one of the memory cells MC[0]...MC[N] in a write operation. Bit line BL is further coupled to input Reg_In of register 415 to output weighted data from one of memory cells MC[0]...MC[N] to register 415 during a read operation. Register 415 is further configured to receive control signal LCK and has output Reg_Out. Output Reg_Out of register 415 is coupled to first input LOC_1 of logic circuit 416. Logic circuit 416 also includes a second input LOC_2, which is configured to receive input data IN, for example, from the input buffer described herein. Output LOC_Out of logic circuit 416 is coupled to MAC circuit 417.

[0069] Figure 4B This is a timing diagram of example operation 400B in a portion of a memory device 400A according to some embodiments. Figure 4B The example operation 400B is performed based on a clock signal CLK that has multiple clock pulses.

[0070] During the read period RD[0], memory cell MC[0] is accessed via pulse 422 on word line WL[0] during the read operation. The weight data W[0] currently stored in memory cell MC[0] is read from memory cell MC[0], appears on bit line BL, and is provided by bit line BL to the input Reg_In of register 415. Figure 4B In the diagram, weight data W[0] is schematically shown to indicate weight data W[0], which may include logic "1" (high level) or logic "0" (low level). Figure 4B Other weight data and / or input data are illustrated in a similar manner.

[0071] During the read period RD[0], pulse 424 of the control signal LCK is provided to register 415. For example, the control signal LCK is generated or provided by a memory controller corresponding to memory controller 120. In at least one embodiment, register 415 (e.g., a flip-flop) is configured to latch data or logic state at input Reg_In in response to the rising edge of pulse 424 of the control signal LCK. The latched data or logic state is maintained at output Reg_Out of register 415 until the rising edge of the next pulse 426 of the control signal LCK. As a result, the weighted data W[0] read from memory cell MC[0] is latched at output Reg_Out of register 415 during the period between the rising edges of pulses 424 and 426, and is unaffected by the data on bit line BL.

[0072] In the next time period (in Figure 4B In the CIM[0] / update operation, a CIM operation is performed using latched weight data W[0], while simultaneously updating the weight data in one or more memory cells in memory segment 412. During the CIM operation, the weight data W[0] latched at the output Reg_Out of register 415 is provided to input LOC_1 of logic circuit 416. Input data IN is provided to another input LOC_2 of logic circuit 416. In some embodiments, input data IN comprises multiple bits, which are serially provided to logic circuit 416 over several clock cycles to be processed together with the weight data W[0]. In at least one embodiment, logic circuit 416 is configured to multiply the bit sequence in input data IN with the weight data W[0] and output the multiplication result to MAC circuit 417, which is configured to perform further processing, such as addition, shifting, etc., to obtain the final result of the CIM operation.

[0073] When CIM operations are performed using latched weight data W[0], bit line BL is isolated from logic circuit 416 and MAC circuit 417 via register 415, and can be used to update weight data in one or more memory cells without affecting CIM operations and is unaffected by CIM memory devices. For example, one of the memory cells MC[1] to MC[N] is accessed in a write operation by pulse 428 on the corresponding word line WL[1] to WL[N]. The corresponding segments Wn[1] to Wn[N] of the new weight data are provided from weight buffer 414 to bit line BL and written or updated to the accessed memory cell in memory cells MC[1] to MC[N].

[0074] In some embodiments, weight data updates can be performed on multiple memory cells based on the bit sequence length in the input data IN that defines the CIM[0] / update time period length, while CIM operations are performed using latched weight data W[0] read from memory cell MC[0]. For example, when CIM operations are performed using latched weight data W[0], memory cell MC[0] is accessed by pulse 430 on the corresponding word line WL[0] during a write operation. The corresponding segment Wn[0] of the new weight data is provided from the weight buffer 414 to the bit line BL and is written or updated to the accessed memory cell MC[0]. In at least one embodiment, when performing CIM operations, weight data of more than two memory cells can be updated using latched weight data W[0] read from one memory cell.

[0075] In the next read period RD[1], during the read operation, the corresponding pulse on the corresponding word line WL[1] ( Figure 4B (Not shown in the image) Accessing memory cell MC[1]. The weight data W[1] currently stored in memory cell MC[1] is read from memory cell MC[1] and placed on bit line BL, provided that memory cell MC[1] has not been updated, for example, by pulse 428. However, if memory cell MC[1] was updated earlier, the new weight data Wn[1] is read and placed on bit line BL. In response to the rising edge of pulse 426 in control signal LCK, the weight data W[1] or the new weight data Wn[1] is latched at the output Reg_Out of register 415 and provided to the input LOC_1 of logic circuit 416 for use in Figure 4BThe CIM operation, labeled CIM[1] / update, is performed in the next time period, and the described process is repeated. In some embodiments, the described operations and configurations are applicable even when the memory segment 412 includes a multi-port memory cell. In some embodiments, the described operations and configurations enable the simultaneous execution of weight data updates and CIM operations, thereby saving processing time, power consumption, etc. In at least one embodiment, one or more advantages described herein may be achieved by memory device 400A and / or operation 400B.

[0076] Figure 4C This is a schematic diagram of a portion of the memory segment corresponding to the memory device 400C according to some embodiments. Figure 4C The components in have a combination of and Figure 4A The same label indicates Figure 4A The corresponding component in.

[0077] The difference between memory device 400C and memory device 400A is that the logic circuit 416 in memory device 400A is implemented by the NOR gate 436 in memory device 400C. The operation of memory device 400C is similar to that of memory device 400A. Figures 4A-4B The operation described herein. In at least one embodiment, one or more advantages described herein may be achieved by memory device 400C.

[0078] Figure 4D This is a schematic diagram of a portion of the memory segment corresponding to the memory device 400D according to some embodiments. Figure 4D The components in have a combination of and Figure 4A The same label indicates Figure 4A The corresponding component in.

[0079] The difference between memory device 400D and memory device 400A is that the logic circuit 416 in memory device 400A is implemented by the AND gate 446 in memory device 400D. The operation of memory device 400D is similar to that of memory device 400A. Figures 4A-4B The described operation. In at least one embodiment, one or more advantages described herein can be achieved by the memory device 400D. The described configuration of logic circuitry including NOR gates or AND gates is exemplary. Other logic circuitry configurations are within the scope of various embodiments.

[0080] Figure 4E This is a schematic circuit diagram of an example single-port memory cell 400E according to some embodiments. In at least one embodiment, memory cell 400E corresponds to one or more memory cells of memory devices 100A, 300, 400A, 400C, and 400D.

[0081] Memory cell 400E includes transistors M1 and M2 and inverters INV1 and INV2. The input of inverter INV2 is coupled to the output of inverter INV1 at node Q. The output of inverter INV2 is coupled to the input of inverter INV1 at node QB. The gates of transistors M1 and M2 are coupled to word line WL. Transistor M1 is series coupled between node Q and bit line BL. Transistor M2 is series coupled between node QB and complementary bit line BLB. Inverters INV1 and INV2 form a storage circuit ST for storing data corresponding to the logic state of node Q or QB. Transistors M1 and M2 are access transistors configured to couple the storage circuit ST to bit lines BL / BLB for read or write access in response to an appropriate voltage applied to word line WL. In some embodiments, each of the inverters INV1 and INV2 includes two transistors, thereby creating a total of six transistors in the memory cell 400E, which is also referred to as a 6T (6-transistor) SRAM memory cell. Examples of transistors in the memory cell 400E include, but are not limited to, metal-oxide-semiconductor field-effect transistors (MOSFETs), complementary metal-oxide-semiconductor (CMOS) transistors, bipolar junction transistors (BJTs), high-voltage transistors, high-frequency transistors, p-channel and / or n-channel field-effect transistors (PFETs / NFETs), FinFETs, planar MOS transistors with convex source / drain terminals, etc. The described configuration of a single-port memory cell is one example. Other single-port memory cell configurations are available across a variety of embodiments.

[0082] Figure 5A This is a schematic diagram of a portion corresponding to a memory segment of memory device 500A according to some embodiments. In some embodiments, memory device 500A corresponds to at least one of memory devices 100B and 300.

[0083] exist Figure 5A In the portion shown, memory device 500A includes a memory segment 512, a weighted buffer 514, and logic circuitry 516. Memory device 500A also includes a MAC circuit (not shown) coupled to the output of logic circuitry 516. In some embodiments, memory segment 512, weighted buffer 514, and logic circuitry 516 correspond to memory segment 412, weighted buffer 414, and logic circuitry 416 in memory device 400A.

[0084] The difference between memory device 400A and memory device 500A is that in memory device 400A, memory segment 412 includes a single-port memory cell, while in memory device 500A, memory segment 512 includes a multi-port memory cell. Figure 5AIn the example configuration, groups of word lines WL[0]...WL[N] are configured as write word lines and read word lines. Other configurations with groups of write word lines and separate groups of read word lines are within the scope of various embodiments. Multiport memory cells in memory segment 512 are coupled to weighted buffer 514 via write bit line WBLB, which is a different bit line from read bit line RBLB, through which multiport memory cells in memory segment 512 are coupled to logic circuitry 516. Figure 5A In the example configuration, the register corresponding to register 415 is omitted. In at least one embodiment, the register corresponding to register 415 is included in the memory device 500A, located between the read bit line RBLB and the logic circuit 516. In some embodiments, the logic circuit 516 includes a NOR gate or an AND gate. Other logic circuit configurations are available in a variety of embodiments.

[0085] Figure 5A A schematic circuit diagram of an example multi-port memory unit 510 according to some embodiments is shown. Figure 5A In the example configuration, the multi-port memory unit 510 includes inverters INV1, INV2, and INV3 and transmission gates TG1, TG2, and TG3. Inverters INV1 and INV2 are coupled to form as shown in the figure. Figure 4E The described storage circuit ST. Each transmission gate TG1, TG2, TG3 includes an NMOS transistor and an NMOS transistor. The gates of the PMOS and NMOS transistors in transmission gate TG1 are correspondingly coupled to a pair of write word lines WWL / WWLB. The gates of the NMOS and PMOS transistors in transmission gate TG2 are correspondingly coupled to the write word lines WWL / WWLB. The gates of the NMOS and PMOS transistors in transmission gate TG3 are correspondingly coupled to a pair of read word lines RWL / RWLB. Transmission gate TG1 is coupled between node Q and the output of inverter INV1. Transmission gate TG2 is coupled between node Q and the complementary write bit line WBLB. Transmission gate TG3 is coupled between the output of inverter INV3 and the complementary read bit line RBLB. The input of inverter INV3 is coupled to node QB. Transmission gates TG1 and TG2 form a write port WP. Inverter INV3 and transmission gate TG3 form an output buffer or read port RP. The described multi-port memory cell 510 has a total of 12 transistors, which is also referred to as a 12TSRAM memory cell. The configuration of the described multiport memory unit 510 is exemplary. Other multiport memory unit configurations are available in a variety of embodiments.

[0086] When transmission gate TG3 is turned on by the appropriate voltage on read word lines RWL / RWLB, the segment of weight data stored corresponding to the logic state of node QB is read through inverter INV3 and the turned-on transmission gate TG3, and provided to read bit line RBLB for CIM operation. Simultaneously, the new weight data is written to bit line WBLB to update another memory cell in memory segment 512 without affecting the storage and / or reading of weight data in memory cell MC[N], because transmission gates TG1 and TG2 are not turned on. As a result, as per the previous discussion... Figure 1B As described, CIM operations and weight data updates can be performed simultaneously on different memory cells in memory segment 512. In at least one embodiment, a register corresponding to register 415 is included in memory device 500A and coupled between read bit line RBLB and logic circuit 516 in a manner similar to memory device 400A. In one or more embodiments, when the register is included in memory device 500A, latched weight data can be held at the input of logic circuit 516 for an extended period of time, which might be difficult to achieve without the register. In at least one embodiment, one or more advantages described herein can be achieved by memory device 500A.

[0087] Figure 5B This is a schematic diagram of a portion corresponding to a memory segment of memory device 500B according to some embodiments. In some embodiments, memory device 500B corresponds to at least one of memory devices 100B, 300, and 500A.

[0088] exist Figure 5B In the portion shown, memory device 500B includes memory segment 522, weighted buffer 514, and logic circuits 526 and 536. Memory device 500B also includes a MAC circuit (not shown) coupled to the output of logic circuits 526 and 536. In some embodiments, memory segment 522 and logic circuits 526 and 536 correspond to memory segment 512 and logic circuit 516 in memory device 500A.

[0089] This document describes the differences between memory device 500A and memory device 500B. In memory device 500A, memory segment 512 includes multi-port memory cells, each with one read port RP, while in memory device 500B, memory segment 522 includes multi-port memory cells, each with two read ports RP1 and RP2. The memory cells in memory segment 522 are referred to herein as multi-read port memory cells. Figure 5BIn the example configuration, for simplicity, groups of word lines WL[0]...WL[N] are illustrated. In some embodiments, groups of write word lines WWL, separate groups of first read word lines RWL1 and second read word lines RWL2 are provided to individually access the write port WP and two read ports RP1, RP2 of each memory cell in memory segment 522. Other configurations are within the scope of various embodiments. The multi-read-port memory cells in memory segment 522 are coupled to a weighted buffer 514 via a write bit line WBLB, which is a different bit line from the first and second read bit lines RBLB1, RBLB2. Each multi-read-port memory cell in memory segment 522 is coupled to logic circuits 526, 536 via the first and second read bit lines RBLB1, RBLB2. In some embodiments, each of the logic circuits 526, 536 includes a NOR gate or an AND gate. Other logic circuit configurations are within the scope of various embodiments.

[0090] Figure 5B A schematic circuit diagram of an example multi-read port memory unit 520 according to some embodiments is shown. Figure 5B In the example configuration, the multi-read port memory unit 520 is similar to that described above. Figure 5A The described multi-port memory unit 510, in addition to Figure 5B The multi-read-port memory cell 520 additionally includes an inverter INV4 and a transmission gate TG4. Similar to the multi-port memory cell 510, the multi-read-port memory cell 520 includes an inverter INV3 and a transmission gate TG3, forming a first read port RP1 coupled to the first read word line RWL1 / RWLB1 and the first read bit line RBLB1. Similarly, the inverter INV4 and the transmission gate TG4 form a second read port RP2 coupled to the second read word line RWL2 / RWLB2 and the second read bit line RBLB2. The described configuration of the multi-read-port memory cell 520 is exemplary. Other multi-read-port memory cell configurations are available across a variety of embodiments.

[0091] In one or more embodiments, since each memory cell in memory segment 522 has two read ports, weight data can be read from two different memory cells via read bit lines RBLB1 and RBLB2 for CIM operations, and simultaneously, weight data in one or more other memory cells in memory segment 522 can be updated via write bit line WBLB. For example, first weight data on the first read bit line RBLB1 is read from memory cell MC[0] and combined (e.g., multiplied) with a first set of input data IN0 via logic circuitry 526.

[0092] The second weighted data on the second read bit line RBLB2 is read from memory cell MC[1] and combined (e.g., multiplied) with the second set of input data IN1 via logic circuit 536. The result or intermediate data output by logic circuits 526 and 536 is provided to the MAC circuit for further processing to complete the CIM operation. At the same time, one or more remaining memory cells (i.e., MC[2] to MC[N]) are updated by writing to bit line WBLB using new weighted data from weight buffer 514. As a result, as per the information about Figure 1B , Figure 5A One or more of the features described herein can simultaneously perform CIM operations and weight data updates for different memory cells in memory segment 522. In at least one embodiment, one or more of the advantages described herein can be achieved by memory device 500B.

[0093] Figure 5C This is a schematic diagram of a portion corresponding to a memory segment of memory device 500C according to some embodiments. In some embodiments, memory device 500C corresponds to at least one of memory devices 100B, 300, 500A, and 500B.

[0094] Compared to memory device 500B, memory device 500C additionally includes registers 525 and 535 correspondingly coupled to read bit lines RBLB1 and RBLB2 to latch weighted data on read bit lines RBLB1 and RBLB2 in response to the same control signal LCK, and in a manner similar to that regarding Figure 4A The described register 415 provides latched weight data to the corresponding logic circuits 526, 536. In at least one embodiment, registers 525, 535 enable the latched weight data to be held at the inputs of logic circuits 526, 536 for an extended period of time, which would be difficult to achieve without registers 525, 535. In at least one embodiment, one or more of the advantages described herein can be achieved by memory device 500C.

[0095] Figure 6A This is a schematic diagram of a memory device 600 according to some embodiments. In some embodiments, the memory device 600 corresponds to one or more of memory devices 100B, 300, and 500A.

[0096] Memory device 600 includes 64 memory macros 630, 630_1, 630_2, ..., 630_63. In at least one embodiment, the 64 memory macros are identical or similarly configured. Memory macro 630 is described in detail herein. For simplicity, detailed descriptions of other memory macros are omitted. In some embodiments, memory device 600 also includes a memory controller (not shown) coupled to one or more of the 64 memory macros and configured to control one or more of the 64 memory macros. In some embodiments, some of the 64 memory macros are sequentially coupled (as described with respect to FIG. 1) to implement a neural network. The number of 64 memory macros described is an example. Other numbers of memory macros included in memory device 600 are within the range of various embodiments.

[0097] The memory macro 630 includes a memory array 602 divided into 64 memory segments, each memory segment having 16 memory cells in 4 rows and 4 columns. Figure 6A The text indicates a representative memory segment 612. In at least one embodiment, the memory segment 612 corresponds to... Figure 5A The described memory segment 512. 64 memory segments are stacked side-by-side along the Y direction. In some embodiments, the memory cells in the memory array 602 are multi-port memory cells. In at least one embodiment, the multi-port memory cell is as described above. Figure 5A The described 12T SRAM memory cell. The number of memory segments and / or the size of each memory segment in memory array 602 are examples. Other configurations are within the scope of various embodiments.

[0098] Memory macro 630 also includes a weight buffer 614. In at least one embodiment, the weight buffer 614 corresponds to the weights about... Figure 5A The weight buffer 514 is described. The weight buffer 614 is coupled to the memory array 602 via a weight demultiplexer 642 and 256 write bit lines WBLB extending along the X direction. Four write bit lines WBLB are coupled to corresponding four memory columns in the memory segment 612, and are typically... Figure 6A The label "4×WBLB" indicates this. In at least one embodiment, each write bit line WBLB and its corresponding memory column corresponds to a reference. Figure 5AThe write bit lines WBLB and memory segment 512 (which is a memory column) are described. The four write bit lines WBLB coupled to memory segment 612 are configured to update four memory cells in a memory row within memory segment 612 using a 4-bit word D[3:0] output from weight buffer 614. Memory segment 612 and the corresponding four write bit lines WBLB are selected by six bits (i.e., bits [7:2]) of an 8-bit write access signal WA applied to weight demultiplexer 642. The remaining two bits (i.e., bits [1:0]) of the write access signal WA are applied to write address demultiplexer 641 to select one write word line WWL from the four write word lines WWL of memory array 602. The selected write word line WWL corresponds to the memory row in memory segment 612 where the word D[3:0] of weight data will be written. Figure 6A In the example configuration, four bits of weight data are updated simultaneously on a memory row in memory segment 612. In at least one embodiment, the four write word lines WWL correspond to about Figure 5A The write word lines are described. By using the weight demultiplexer 642, the weight buffer 614 can be used as a common weight buffer for multiple memory segments. The number of weight data bits, write bit lines WBLB, and write word lines WWL described are examples. Other configurations are within the scope of various embodiments.

[0099] A two-bit read access signal RA (i.e., [1:0]) is applied to the read address demultiplexer 643 to select one of the four read word lines RWL in the memory array 602. The selected read word line RWL corresponds to a memory row in the memory segment 612 from which a 4-bit word of stored weight data is read for CIM operations. Figure 6A In the example configuration, four bits of weight data are read simultaneously from the storage row in memory segment 612. In at least one embodiment, the four read word lines RWL correspond to about Figure 5A The description is written to the word line.

[0100] Memory macro 630 includes 256 read bit lines RBLBs, which are divided into 64 groups of 4 read bit lines RBLBs each. These 64 groups correspond to 64 memory segments. The four read bit lines RBLBs in each group are coupled to the corresponding four memory columns in the corresponding memory segment. For example, the four read bit lines RBLBs coupled to the four memory columns in memory segment 612 are typically composed of… Figure 6A The label "4×RBLB" indicates this. In at least one embodiment, each of the four read bit lines RBLB corresponds to about Figure 5AThe described read bit lines RBLB. When one of the read word lines RWL is selected, four bits of weight data are simultaneously read from the corresponding memory row in memory segment 612, and the four bits of weight data are output to the corresponding four read bit lines RBLB. When one of the read word lines RWL is selected, 256 bits of weight data are output to the corresponding 256 read bit lines RBLB across all 64 memory segments. The number of weight data bits, read bit lines RBLB, and read word lines RWL described are illustrative. Other configurations are within the scope of various embodiments.

[0101] Input driver 644 is configured to receive input data XIN, which comprises 64 groups [63:0] divided into 256 bits, with 4 bits in each group, i.e., [3:0]. Input driver 644 includes 64 input driver blocks, each configured to convert the 4 bits of the input data in the corresponding group into an identical 4-bit stream, sequentially outputting it to one of the 64 data lines XINLB coupled to input driver 644. The 4 parallel bits are converted into a 4 serial bit stream according to the selection signal XINSEL [3:0]. In some embodiments, the stream of four serial bits on data line 645 corresponds to about Figure 5A The input data IN is described. For a detailed description of the input driver 644, please refer to [link / reference]. Figure 6B Provided.

[0102] The memory macro 630 also includes 64 logic circuits, schematically shown as NOR gates in FIG. 6. Each of the 64 logic circuits is coupled to a corresponding data line and a corresponding group of four read bit lines RBLB among the 64 data lines XINLB. For example, logic circuit 646 is coupled to data line 645 and the four read bit lines RBLB corresponding to memory segment 612. In at least one embodiment, logic circuit 646 corresponds to about Figure 5A The described logic circuit 516. In some embodiments, the 64 logic circuits are 4-bit multipliers by 1-bit multipliers, as described herein. Figure 6B As described.

[0103] The memory macro 630 also includes an adder tree 650, which comprises multiple adders arranged in multiple stages to accumulate multiplication results from the outputs of 64 logic circuits or multipliers. Figure 6A In the example configuration, the adder tree 650 includes six levels, including adder 647 in the first level, adder 648 in the second level, and adder 649 in the sixth level. In some embodiments, the adder tree 650 and 64 logic circuits together form a MAC circuit 651. In one or more embodiments, the result output from the final adder 649 includes a 10-bit word. Figure 6AIn the example configuration, memory macro 630 further includes a shifter and an accumulator 652, which are configured to further perform shifting and / or accumulating in response to control signals SIGN and ACM_EN. In some embodiments, the shifter and accumulator 652 are simplified or omitted, or have different configurations. The result of the CIM operation based on the weighted data stored in memory array 602 and the input data XIN is output from memory macro 630 as NOUT. Figure 6A In the example configuration, weight buffer 614 and weight demultiplexer 642 are included in memory macro 630, while write address demultiplexer 641, read address demultiplexer 643 and input driver 644 are outside memory macro 630 and are included in memory controller as described herein.

[0104] Figure 6B This is a schematic circuit diagram of an input driver 644 and logic circuit 646 in a memory device 600 according to some embodiments.

[0105] Input driver 644 comprises 64 similarly configured input driver blocks. A representative input driver block 660 is shown in... Figure 6B The diagram is shown in more detail below. The input driver block 660 includes four flip-flops (one of which is indicated at 661) and a MUX 662. The four bits of the input data XIN[3:0] are correspondingly input in parallel to the four flip-flops, and the outputs of the four flip-flops are coupled to the inputs of the MUX 662. The MUX 662 sequentially outputs the four bits of the input data to the data line 645 according to the signal XINSEL[3:0].

[0106] Logic circuit 646 includes four NOR gates 670-673. The first inputs of NOR gates 670-673 are commonly coupled to data line 645 to receive a sequence of four bits of input data. The second inputs of NOR gates 670-673 are correspondingly coupled to read bit lines RBLB[0]-[3], which are coupled to four corresponding memory columns in memory segment 612. The four bits of weighted data read from the selected row of memory segment 612 are applied to the second inputs of NOR gates 670-673.

[0107] In the first clock cycle, NOR gates 670-673 multiply each of the four bits of the weight data with the first bit of the four-bit sequence of the input data, and output the corresponding first result to the adder in the first stage of the corresponding adder tree 650. In the second clock cycle, NOR gates 670-673 multiply each of the four bits of the weight data with the second bit of the four-bit sequence of the input data, and output the corresponding second result to the adder in the first stage of the corresponding adder tree 650. In the third clock cycle, NOR gates 670-673 multiply each of the four bits of the weight data with the third bit of the four-bit sequence of the input data, and output the corresponding third result to the adder in the first stage of the corresponding adder tree 650. In the fourth clock cycle, NOR gates 670-673 multiply each of the four bits of the weight data with the fourth bit of the four-bit sequence of the input data, and output the corresponding fourth result to the adder in the first stage of the corresponding adder tree 650. In at least one embodiment, one or more of the advantages described herein may be achieved by the memory device 600.

[0108] Figure 7A This is a schematic diagram of a memory device 700 according to some embodiments. In some embodiments, the memory device 700 corresponds to one or more of memory devices 100B, 300, 500A, and 600.

[0109] This document describes the differences between memory device 600 and memory device 700. In memory device 600, each memory segment includes multiple memory columns that are physically adjacent or contiguous. In memory device 700, each memory segment includes multiple memory columns that are physically non-adjacent or interleaved with memory columns of another memory segment.

[0110] For example, portion 702 of memory device 700 is in Figure 7AThe details are shown below. Section 702 includes four memory banks, such as banks 0-3, of the 64 memory banks of memory device 700. Each memory bank of memory device 700 corresponds to a memory segment described herein. Each memory bank of memory device 700 includes four memory columns, each memory column including four memory cells arranged along four corresponding rows. The memory columns extend along the X direction, and the rows extend along the Y direction perpendicular to the X direction. Section 702 includes an adder region 704, which includes first-stage and second-stage adders corresponding to banks 0-3. Banks 0 and 1 are physically arranged on one side of adder region 704 in the X direction. Banks 2 and 3 are physically arranged on the other side of adder region 704 in the X direction. Section 702 includes a logic circuit region LOC1 physically arranged along the X direction between banks 0, 1 and adder region 704. Part 702 also includes a logic circuit region LOC2 physically arranged along the X direction between memory banks 2 and 3 and adder region 704. Logic circuit regions LOC1 and LOC2 include various logic circuits as described herein.

[0111] The memory columns of memory banks 0 and 1 are arranged alternately along the Y direction. For clarity, the memory columns of memory bank 0 are typically... Figure 7A The memory banks 2 and 3 are indicated by 708. The memory columns of memory banks 2 and 3 are arranged alternately along the Y direction. Memory banks 0 to 3 are accessed in read operations via the corresponding read word lines RWL0_1[3:0], RWLB0_1[3:0], RWL1_2[3:0], RWLB1_2[3:0] and the corresponding read bit lines. The read bit lines are schematically shown by arrows extending from the memory columns of memory banks 0 to 3 to the corresponding logic circuit regions LOC1 and LOC2. For example, arrow 710 schematically represents the read bit line coupled between the memory column 711 of memory bank 1 and the corresponding logic circuit in logic circuit region LOC1. Memory banks 0 to 3 are accessed in write operations via the corresponding write word lines WWL[7:0], WWLB[7:0], WWL[15:8], WWLB[15:8] and the corresponding write bit line WBLB[3:0]. The write bit line WBLB is coupled to a weight buffer (not shown) to receive new weight data for updating one or more memory cells in section 702.

[0112] Input data on data line XINLB[0] is input to logic circuit region LOC1 to be multiplied with weighted data read from memory bank 0. Input data on data line XINLB[1] is input to logic circuit region LOC1 to be multiplied with weighted data read from memory bank 1. Input data on data line XINLB[2] is input to logic circuit region LOC2 to be multiplied with weighted data read from memory bank 2. Input data on data line XINLB[3] is input to logic circuit region LOC2 to be multiplied with weighted data read from memory bank 3. In some embodiments, each input data on data lines XINLB[0] to [3] includes a stream or string of input data bits, for example, as per the information provided. Figure 6B As described. The result of multiplying the weighted data read from memory cells in memory banks 0-3 with the input data stream is schematically shown by arrows extending from logic circuit regions LOC1, LOC2 to adder region 704. Adder region 704 includes multiple full adder (FA) units, each FA unit including a first-stage and a second-stage adder for accumulating the output of the multiplication. For details of FA unit 750, refer to [reference needed]. Figure 7B Describe it.

[0113] like Figure 7A As shown on the right, the memory device 700 also includes portions 712, 714, 716, 718, 720, 722, and 724, configured similarly to portion 702, and includes memory banks 4-31 of the memory device 700 and corresponding first and second stage adders. The output of the second stage adder in portions 702 and 712 is coupled to the input of the third stage adder 732. The output of the second stage adder in portions 714 and 716 is coupled to the input of the third stage adder 734. The output of the second stage adder in portions 718 and 720 is coupled to the input of the third stage adder 736. The output of the second stage adder in portions 722 and 724 is coupled to the input of the third stage adder 738. The outputs of the third stage adders 732 and 734 are coupled to the input of the fourth stage adder 740. The outputs of the third stage adders 736 and 738 are coupled to the input of the fourth stage adder 742. The outputs of the fourth-stage adders 740 and 742 are coupled to the input of the fifth-stage adder 744.

[0114] Parts 702, 712, 714, 716, 718, 720, 722, 724, third-stage adders 732, 734, 736, 738, fourth-stage adders 740, 742, and fifth-stage adder 744 together form part 745 of memory device 700. Memory device 700 also includes part 746, which is configured similarly to part 745. Part 746 of memory device 700 includes memory banks 32-63 with corresponding logic circuitry and first through fifth-stage adders. Memory device 700 also includes part 747, which includes a sixth-stage adder, a shifter, an accumulator circuit, and a write circuit. In some embodiments, memory device 700 operates in a manner similar to one or more of memory devices 500A, 600. In at least one embodiment, one or more advantages described herein may be achieved by memory device 700.

[0115] Figure 7B This is a schematic circuit diagram of the FA unit 750 in a memory device 700 according to some embodiments.

[0116] The FA unit 750 includes first-stage adders 781 and 782 and second-stage adders 783 and 784. The first-stage adders 781 and 782 are similarly configured, except that the second-stage adders 783 and 784 correspond to the polarity reversal circuits of the first-stage adders 781 and 782. The circuit diagram of the first-stage adder 781 is described in detail here.

[0117] The first-stage adder 781 includes PMOS transistors 751-760, NMOS transistors 761-770, and inverters 771 and 772. PMOS transistors 751-753 are series-coupled between the power supply voltage VDD and node S. PMOS transistors 754-756 are parallel-coupled between VDD and the enable terminal of inverter 771. PMOS transistors 757-758 are series-coupled between VDD and node CO. PMOS transistors 759-760 are parallel-coupled between VDD and the enable terminal of inverter 772. The gates of PMOS transistors 751 and 754 are coupled together and connected to node CI. The gates of PMOS transistors 752, 755, 757, and 760 are coupled together and connected to node B. The gates of PMOS transistors 753, 756, 758, and 759 are coupled together and connected to node A. NMOS transistors 761-763 are series-coupled between voltage VSS and node S. NMOS transistors 764-766 are connected in parallel between VSS and the enable terminal of inverter 771. NMOS transistors 767-768 are connected in series between VSS and node CO. NMOS transistors 769-770 are connected in parallel between VSS and the enable terminal of inverter 772. The gates of NMOS transistors 761 and 764 are coupled together and coupled to node CI. The gates of NMOS transistors 762, 765, 767, and 770 are coupled together and coupled to node B. The gates of NMOS transistors 763, 766, 768, and 769 are coupled together and coupled to node A. The input of inverter 771 is coupled to node S. The output of inverter 771 is coupled to node CO and the output of inverter 771. The input of inverter 772 is coupled to node CI.

[0118] Figure 8 This is a flowchart of method 800 according to some embodiments. In at least one embodiment, in reference to... Figures 1A-7B One or more ICs, memory devices, memory macros, or references described herein Figures 1A-7B The described method 800 includes one or more ICs, memory devices, or memory macros. Method 800 includes operations 805, 815, and 825, wherein operations 815 and 825 are executed simultaneously.

[0119] At operation 805, a first segment of weight data is read from the first memory cell among a plurality of memory cells in the memory segment of the memory device. For example, as per... Figure 4A , Figure 4B As described in one or more of the above, pulse 422 is applied to the word line WL[0] corresponding to the first memory cell MC[0] from which to be read, and the segment W[0] of the weighted data stored in the memory cell MC[0] is read out and placed on the bit line BL. Similarly, in Figure 5A In the memory device, word line WL[0] is selected, and segment W[0] of weighted data stored in memory cell MC[0] is read out and placed on read bit line RBLB.

[0120] At operation 815, the first segment of weight data read from the first memory cell is used to perform a computation in memory (CIM) operation. For example, as regarding Figure 4A , Figure 4B As described in one or more of the above, register 415, coupled to bit line BL, is controlled to latch a first segment W[0] of weighted data read from memory cell MC[0] at a first input of logic circuit 416. Input data IN is provided to a second input of logic circuit 416. Logic circuit 416 and MAC circuit 417 perform CIM operations on the latched segment W[0] of weighted data and input data IN, such as multiplication. For another example, as described in [the following text is incomplete and requires further context]. Figure 5A As described, the memory cells in the accessed memory segment are multi-port memory cells, which allow the segment W[0] of weight data read from memory cell MC[0] to be maintained on the read bit line RBLB without being affected by other activities, such as updating the weight data via another write bit line WBLB. The segment W[0] of the weight data read from memory cell MC[0] and the input data IN are in a manner similar to that described above. Figure 4A , Figure 4B The described method is used for CIM operations.

[0121] Operation 825 is performed at least partially concurrently with operation 815 to update the second segment of weight data in a second memory cell that is different from the first memory cell. For example, as regarding Figure 4A , Figure 4B As described in one or more of the embodiments, when a CIM operation is performed using segment W[0] of weight data read from memory cell MC[0] and input data IN, one of the memory cells MC[1] to MC[N] is accessed in a write operation via pulse 428 on the corresponding word line WL[1] to WL[N], and the corresponding new weight data segment Wn[1] to Wn[N] is provided from weight buffer 414 to bit line BL and written or updated in the accessed memory cell among memory cells MC[1] to MC[N]. In at least one embodiment, during a CIM operation, the described weight data update is performed on more than one memory cell using weight data read from memory cell MC[0]. For another example, as per the description... Figure 5AAs described, when a CIM operation is performed using weight data W[0] read from memory cell MC[0] via another bit line (i.e., read bit line RBLB), a weight data update is performed on another memory cell (e.g., memory cell MC[N]) to write the new weight data to memory cell MC[N] via read bit line WBLB. As a result, CIM operations and weight data updates can be performed simultaneously in a memory segment of a multi-port memory cell without requiring registers coupled to the bit lines of the memory segment. In some embodiments, registers are still coupled to the read bit line RBLB of the memory segment of the multi-port memory cell to maintain the read weight data over an extended period of time to ensure the correct execution of the CIM operation. In at least one embodiment, one or more of the advantages described herein may be achieved by method 800.

[0122] The described methods and algorithms include example operations, but they do not necessarily need to be performed in the order shown. Operations may be appropriately added, substituted, rearranged, and / or eliminated according to the spirit and scope of embodiments of this disclosure. Combinations of different features and / or different embodiments will be apparent to those skilled in the art upon reading this disclosure.

[0123] In some embodiments, a memory device includes a memory array comprising at least one memory segment configured to store weight data, at least one weight buffer coupled to the at least one memory segment and configured to hold new weight data to be updated in the at least one memory segment, at least one logic circuit, and computational circuitry coupled to an output of the logic circuitry. The logic circuitry further has a first input coupled to the at least one memory segment via at least one bit line, a second input configured to receive input data, and an output. The logic circuitry is configured to generate intermediate data at the output, the intermediate data corresponding to the input data and the weight data read from the at least one memory segment via the at least one bit line. The computational circuitry is configured to generate output data based on the intermediate data, the output data corresponding to a computation performed on the input data and the weight data read from the at least one memory segment.

[0124] In some embodiments, the memory array, at least one weight buffer, at least one logic circuit, and computing circuit are included in the same memory macro.

[0125] In some embodiments, at least one logic circuit includes a NOR gate or an AND gate.

[0126] In some embodiments, the computing circuitry includes a multiply-accumulate (MAC) circuit.

[0127] In some embodiments, the MAC circuitry includes an adder tree having multiple adder stages.

[0128] In some embodiments, the memory device further includes: at least one register coupled to at least one bit line between at least one memory segment and a first input of at least one logic circuit.

[0129] In some embodiments, at least one memory segment includes a single-port memory cell coupled to at least one bit line.

[0130] In some embodiments, at least one memory segment includes a plurality of memory cells, each memory cell having: a storage circuit, a segment configured to store corresponding weight data, and an output buffer coupled between the storage circuit and at least one bit line.

[0131] In some embodiments, the output buffer includes an inverter and a transmission gate, the inverter being coupled between the storage circuit and the transmission gate, and the transmission gate being coupled between the inverter and at least one bit line.

[0132] In some embodiments, at least one bit line includes a first bit line and a second bit line, each of the plurality of memory cells is a multi-port memory cell, the multi-port memory cell having: a first read port coupled to the first bit line, and a second read port coupled to the second bit line, and at least one logic circuit including: a first logic circuit having a first input coupled to the first bit line and a second input configured to receive a first portion of input data, and a second logic circuit having a first input coupled to the second bit line and a second input configured to receive a second portion of input data.

[0133] In some embodiments, the memory device further includes: a first register coupled to a first bit line and located between at least one memory segment and a first logic circuit; and a second register coupled to a second bit line and located between at least one memory segment and the second logic circuit.

[0134] In some embodiments, a method includes simultaneously performing: performing a computation in memory (CIM) operation using a first segment of weight data read from a first memory cell among a plurality of memory cells in a memory segment of a memory device, and updating a second segment of weight data in a second memory cell among a plurality of memory cells in the memory segment, the second memory cell being different from the first memory cell.

[0135] In some embodiments, the method further includes: simultaneously performing a CIM operation using first segment weight data read from a first memory cell: updating the first segment weight data in the first memory cell with the new first segment weight data.

[0136] In some embodiments, updating the first segment of weight data in the first memory cell and updating the second segment of weight data in the second memory cell includes: obtaining new first segment of weight data and new second segment of weight data correspondingly from the same weight cache coupled to the memory segment.

[0137] In some embodiments, the method further includes: simultaneously performing a CIM operation using first segment weight data read from a first memory cell: updating at least one third segment weight data in at least one third memory cell among a plurality of memory cells in the memory segment, the at least one third memory cell being different from the first memory cell and the second memory cell.

[0138] In some embodiments, the method further includes: latching a first segment of weight data read from a first memory cell at the output of a register, wherein performing a CIM operation using the first segment of weight data includes performing a CIM operation using the first segment of weight data latched at the output of the register.

[0139] In some embodiments, the plurality of memory cells in the memory segment include single-port memory cells coupled to the same bit line, and the bit line is coupled to the input of the register.

[0140] In some embodiments, performing a CIM operation using the first segment of weight data includes: performing a multiplication operation between the first segment of weight data and the input data at a logic circuit, and outputting the result of the multiplication operation to an adder tree.

[0141] In some embodiments, the memory device includes a memory array, an input driver, and a multiply-accumulate (MAC) circuit. The memory array includes multiple memory segments configured to store weight data for computation in memory (CIM) operations, multiple write bit lines coupled to the multiple memory segments to update the weight data stored in the multiple memory segments, and multiple read bit lines coupled to the multiple memory segments to read the weight data stored in the multiple memory segments for CIM operations. The input driver is coupled to multiple input data lines and configured to provide input data through the multiple input data lines. The MAC circuit is coupled to the multiple read bit lines and the multiple input data lines and configured to perform CIM operations based on the weight data read from the multiple memory segments and the input data provided from the input driver.

[0142] In some embodiments, the memory device further includes a weight buffer shared by multiple memory segments and coupled to multiple write bit lines, wherein the weight buffer is configured to provide new weight data to one of the multiple memory segments while reading another memory segment from the multiple memory segments to provide corresponding weight data stored in the other memory segment to MAC circuitry for CIM operation.

[0143] The foregoing has outlined features of several embodiments to enable those skilled in the art to better understand various aspects of this disclosure. Those skilled in the art will understand that they can readily use this disclosure as a basis for designing or modifying other processes and structures to achieve the same purposes and / or advantages of the embodiments described herein. Those skilled in the art will also recognize that such equivalent structures do not depart from the spirit and scope of the invention, and that they can be modified, substituted, and altered in various ways within the invention without departing from its spirit and scope.

Claims

1. A memory device, comprising: A memory array, including at least one memory segment configured to store weight data; At least one weight buffer is coupled to the at least one memory segment and is configured to hold new weight data to be updated in the at least one memory segment; At least one logic circuit having: The first input is coupled to the at least one memory segment via at least one bit line. The second input is configured to receive input data, and Output; A computing circuit coupled to the output of the at least one logic circuit; as well as At least one register, coupled to the at least one bit line, between the at least one memory segment and the first input of the at least one logic circuit. in The at least one logic circuit is configured to generate intermediate data at the output, the intermediate data corresponding to the input data and the weighted data read from the at least one memory segment via the at least one bit line, and The computing circuit is configured to generate output data based on the intermediate data, the output data corresponding to a calculation performed on the input data and the weight data read from the at least one memory segment. The at least one register is configured to latch weight data received from the at least one bit line, and to provide the latched weight data to the logic circuit through the output of the at least one register. When calculations are performed using the latched weight data, the at least one bit line is isolated from the logic circuit and the calculation circuit via the at least one register.

2. The memory device according to claim 1, wherein The memory array, the at least one weight buffer, the at least one logic circuit, and the computing circuit are included in the same memory macro.

3. The memory device according to claim 1, wherein... The at least one logic circuit includes a NOR gate or an AND gate.

4. The memory device according to claim 1, wherein The computing circuit includes a multiplication and accumulation circuit.

5. The memory device according to claim 4, wherein The multiply-accumulate circuit includes an adder tree with multiple adder stages.

6. The memory device according to claim 1, wherein The at least one memory segment includes a plurality of memory cells, wherein the memory cells are static random access memory.

7. The memory device according to claim 1, wherein The at least one memory segment includes a single-port memory cell coupled to the at least one bit line.

8. The memory device according to claim 1, wherein The at least one memory segment includes a plurality of memory cells, each memory cell having: Storage circuitry, configured to store segments corresponding to the weight data, and An output buffer is coupled between the storage circuit and the at least one bit line.

9. The memory device according to claim 8, wherein The output buffer includes an inverter and a transmission gate. The inverter is coupled between the storage circuit and the transmission gate, and The transmission gate is coupled between the inverter and the at least one bit line.

10. The memory device according to claim 1, wherein The at least one bit line includes a first bit line and a second bit line. Each of the plurality of memory cells in the at least one memory segment is a multi-port memory cell, the multi-port memory cell having: The first read port is coupled to the first bit line, and The second read port is coupled to the second bit line. The at least one logic circuit includes: A first logic circuit has a first input coupled to the first bit line and a second input configured to receive a first portion of the input data, and The second logic circuit has a first input coupled to the second bit line and a second input configured to receive a second portion of the input data.

11. The memory device of claim 10, further comprising: A first register, coupled to the first bit line, is located between the at least one memory segment and the first logic circuit; and The second register, coupled to the second bit line, is located between the at least one memory segment and the second logic circuit.

12. A method of operating a memory device, the method comprising: Simultaneously: Using a first segment of weight data read from a first memory cell among multiple memory cells in a memory segment of a memory device, memory computation operations are performed at the logic and computation circuits, and Update the second segment weight data in the second memory cell among the plurality of memory cells in the memory segment, wherein the second memory cell is different from the first memory cell; The method further includes: The first segment of weight data read from the first memory cell via the bit line is latched at the output of the register. Specifically, performing the memory calculation operation using the first segment of weight data includes performing the memory calculation operation using the first segment of weight data latched at the output of the register. Specifically, when performing calculation operations in the memory using the latched first segment of weight data, the bit line is isolated from the logic circuit and the calculation circuit through the register.

13. The method of claim 12, further comprising: This is performed simultaneously with the computational operation in the memory using the first segment of weight data read from the first memory unit: The first segment of weight data in the first memory unit is updated using the new first segment of weight data.

14. The method of claim 13, wherein Updating the first segment of weight data in the first memory unit and updating the second segment of weight data in the second memory unit includes: The new first segment of weight data and the new second segment of weight data are obtained from the same weight cache coupled to the memory segment.

15. The method of claim 12, further comprising: This is performed simultaneously with the computational operation in the memory using the first segment of weight data read from the first memory unit: Update at least one third segment weight data in at least one third memory cell among the plurality of memory cells in the memory segment, wherein the at least one third memory cell is different from the first memory cell and the second memory cell.

16. The method of claim 12, wherein The computing circuit includes a multiplication and accumulation circuit.

17. The method of claim 16, wherein The plurality of memory cells in the memory segment include single-port memory cells coupled to the same bit line, and The bit line is coupled to the input of the register.

18. The method of claim 12, wherein Performing the computation operation in the memory using the first segment of weight data includes: The first segment of weighted data is multiplied by the input data at the logic circuit. The result of the multiplication operation is output to the adder tree.

19. A memory device, comprising: Memory array, the memory array comprising: Multiple memory segments are configured to store weight data for computational operations in memory. Multiple write bit lines, coupled to the multiple memory segments, to update the weight data stored in the multiple memory segments, and Multiple read bit lines are coupled to the multiple memory segments to read the weight data stored in the multiple memory segments for computational operations in memory. An input driver, coupled to a plurality of input data lines, and configured to provide input data through the plurality of input data lines; Multiple logic circuits have a first input coupled to the multiple memory segments via the multiple read bit lines, a second input configured to receive the input data, and an output; Multiple registers, coupled to the multiple read bit lines, wherein the multiple registers are configured to latch weighted data received from the multiple read bit lines and to provide the latched weighted data to the multiple logic circuits through the outputs of the multiple registers; and A multiply-accumulate circuit, coupled to the plurality of read bit lines and the plurality of input data lines, is configured to perform memory computation operations based on the weighted data read from the plurality of memory segments and the input data provided from the input driver. Specifically, when performing calculation operations in the memory using the latched weight data, the register isolates the plurality of read bit lines from the logic circuit and the multiplication-accumulation circuit.

20. The memory device of claim 19, further comprising: A weighted buffer, shared by the plurality of memory segments and coupled to the plurality of write bit lines. The weight buffer is configured to provide new weight data to one of the plurality of memory segments, and simultaneously read another memory segment from the plurality of memory segments to provide the corresponding weight data stored in the other memory segment to the multiplication-accumulation circuit for computational operations in the memory.

Images