Processing unit controller in memory

Abstract

The invention relates to a processing unit (PU) controller in a memory. A memory device including a bank controller and the PU controller may also include a plurality of banks of memory cells. The PU controller may be coupled to the plurality of memory banks of a memory cell. The PU controller can further comprise a PU. The bank controller may provide data from the plurality of banks to the PU controller. The PU controller may receive the data from any of the plurality of memory banks. The PU controller may also provide the data to the PU. The PU may utilize the data to perform a plurality of operations.

Description

Technical Field

[0001] This disclosure generally relates to memory, and more specifically, to implementing a processing unit controller in memory. Background Technology

[0002] Memory devices are typically provided as internal semiconductor integrated circuits in computers or other electronic devices. Many different types of memory exist, including volatile and non-volatile memory. Volatile memory requires power to maintain its data and includes random access memory (RAM), dynamic random access memory (DRAM), and synchronous dynamic random access memory (SDRAM). Non-volatile memory provides persistent data by retaining the stored data when no power is supplied and includes NAND flash memory, NOR flash memory, read-only memory (ROM), electrically erasable programmable ROM (EEPROM), erasable programmable ROM (EPROM), and resistive variable memory, such as phase-change random access memory (PCRAM), resistive random access memory (RRAM), and magnetoresistive random access memory (MRAM).

[0003] Memory also serves as a volatile and non-volatile data storage device for various electronic applications. Non-volatile memory can be used in, for example, personal computers, Memory Sticks, digital cameras, cellular phones, portable music players such as MP3 players, movie players, and other electronic devices. Memory cells can be arranged in arrays, where arrays are used in memory devices. Summary of the Invention

[0004] In this example, an apparatus for implementing a processing unit controller in a memory is described. The apparatus includes: a plurality of memory banks (130, 230-1, 230-2, 230-3, 230-4, 230-5, 230-6, 230-7, 230-8, 230-9, 230-10, 230-11, 230-12, 230-13, 230-14, 230-15, 230-16) of the memory cells; and a memory bank controller (140) coupled to the plurality of memory banks of the memory cells. Storage; Processing Unit (PU) controllers (105, 205) coupled to the plurality of storage banks of the memory cell and including PUs (102, 202-1, 202-2, 202-3, 202-4, 202-5, 202-6, 202-7, 202-8, 202-9, 202-10, 202-11, 202-12, 202-13, 202-14, 202-15, 202-16). The storage bank controllers are configured to provide data from the plurality of storage banks to the PU controllers. The PU controllers are configured to: receive data from any of the plurality of storage banks; and provide the data to the PUs; and wherein the PUs are configured to perform a plurality of operations using the data.

[0005] In this example, a method for implementing a processing unit controller in memory is described. The method includes: providing data from storage banks (130, 230-1, 230-2, 230-3, 230-4, 230-5, 230-6, 230-7, 230-8, 230-9, 230-10, 230-11, 230-12, 230-13, 230-14, 230-15, 230-16) of memory cells in a memory device (120) to a processing unit (PU) controller (105, 205) by a storage bank controller (140); receiving the data from the storage banks of the memory cells by the PU controller, wherein the PU controller is coupled to the storage banks; determining an available PU among a plurality of PUs by the PU controller; providing the data to the available PU by the PU controller; and having the available PU perform a plurality of operations using the data.

[0006] In this example, an apparatus for implementing a processing unit controller in a memory is described. The apparatus includes: a plurality of memory banks (130, 230-1, 230-2, 230-3, 230-4, 230-5, 230-6, 230-7, 230-8, 230-9, 230-10, 230-11, 230-12, 230-13, 230-14, 230-15, 230-16) of memory cells; a memory bank controller (140); and a processing unit (PU). A controller (105, 205) is coupled to the plurality of memory banks of the memory cells and includes a plurality of PUs (102, 202-1, 202-2, 202-3, 202-4, 202-5, 202-6, 202-7, 202-8, 202-9, 202-10, 202-11, 202-12, 202-13, 202-14, 202-15, 202-16). The memory bank controller is configured to provide data from the plurality of memory banks to the PU controller. The PU controller is configured to: receive the data from any of the plurality of memory banks; determine an available PU among the plurality of PUs; and provide the data to the available PU; wherein the available PU is configured to use the data to perform a plurality of operations. Attached Figure Description

[0007] Figure 1 This is a block diagram of a device in the form of a computing system including a memory device, according to several embodiments of the present disclosure.

[0008] Figure 2 This is a block diagram of a processing unit controller according to several embodiments of the present disclosure.

[0009] Figure 3 Example flowcharts illustrating methods for implementing a processing unit controller in a memory according to several embodiments of the present disclosure.

[0010] Figure 4 This describes an example machine of a computer system, within which a set of instructions can be executed to cause the machine to perform any or more of the methods discussed herein. Detailed Implementation

[0011] This disclosure implements a processing unit (PU) controller in memory. The memory device may include multiple banks of memory cells. The PU controller may include the PU and be coupled to the multiple banks of memory cells. The PU controller may receive data from any of the multiple banks of memory. The PU controller may provide data to the PU. The PU may use the data to perform multiple operations.

[0012] In the previous method, the PU can receive data from the memory bank of the memory device. Each memory bank can be coupled to a single PU but not to other PUs in the memory device. Each memory bank can provide data to and receive data from a single PU, but cannot provide data to or receive data from other PUs in the memory device. In the previous method, data to be provided to multiple PUs in the memory device is stored in each of the memory banks coupled to the PUs. Storing data in each of the memory banks coupled to the PUs includes copying data and storing the copied data in each of the memory banks. If there are 16 memory banks in the memory device, then the data is copied 16 times and each instance of the data is stored in a different memory bank. Storing copies of the data in memory banks for providing to the PUs reduces the size of the memory banks available for storing different data.

[0013] To address these and other shortcomings of prior art methods, embodiments of this disclosure implement a controller called a PU controller to provide data from any of the storage units of a memory device to the PU and from any of the PUs of the memory device to the storage unit. Implementing a PU controller to route data from the storage unit to the PU and from the PU to the storage unit reduces the need to store data (e.g., copies of data) in each of the storage units. A single instance of data can be provided to each of the PUs because the PU controller can route data stored in a single storage unit to each of the PUs, thereby making more storage units available for storing different data.

[0014] As used herein, a PU may include hardware and / or firmware for performing multiple operations. A PU may include a MAC unit containing hardware and / or firmware for performing multiple multiplication operations and multiple accumulation operations (referred to as MAC operations).

[0015] For example, a PU can be used to implement an artificial neural network (ANN) using a MAC unit. As used in this paper, an ANN provides learning by forming probabilistic weight relationships between inputs and outputs. These probabilistic weight relationships can be provided by multiple nodes that make up the ANN. Nodes, along with weights, biases, and activation functions, can be used to produce the ANN's output based on its inputs. Multiple nodes of an ANN can be grouped to form layers of the ANN.

[0016] As used herein, Artificial Intelligence (AI) refers to the ability of a device to improve itself through “learning” (e.g., by storing patterns and / or instances that can be used to take action at a later time). Deep Learning refers to the ability of a device to learn from data provided as instances. Deep Learning can be a subset of AI. Neural networks and other types of networks can be categorized as deep learning. Improving the efficiency of implementing an ANN can improve the functionality of the memory device implementing the ANN and the device in which the memory device is implemented. For example, improving the latency, power consumption, and / or processing power of the memory device implementing the ANN can lead to improvements in the latency, power consumption, and / or processing power of the memory system.

[0017] As used herein, “several things” refers to one or more such things. For example, “several memory devices” may refer to one or more memory devices. “Multiple things” means two or more. Additionally, identifiers such as “N” as used herein (especially with respect to reference numerals in the figures) indicate that several specific features may be included within several embodiments of this disclosure.

[0018] The figures in this document follow a numbering convention, where the first few digits correspond to the figure number and the remaining digits identify the elements or components in the figure. Similar elements or components between different figures can be identified by using similar digits. It should be understood that the elements shown in the various embodiments herein may be added, interchanged, and / or eliminated to provide several additional embodiments of this disclosure. Furthermore, the scale and relative dimensions of the elements provided in the figures are intended to illustrate various embodiments of this disclosure and are not intended to be limiting.

[0019] Figure 1 This is a block diagram of a device in the form of a computing system 100 including a memory device 120, according to several embodiments of the present disclosure. As used herein, the memory device 120, the storage bank 130 of the memory cells (also referred to as the memory array 130), the host 110, the PU controller 105, and / or the PU 102 may also be individually considered as a “device”.

[0020] In this example, system 100 includes a host 110 coupled to memory device 120 via interface 156. Computing system 100 may be a personal laptop, desktop computer, digital camera, mobile phone, memory card reader, or Internet of Things (IoT) enabled device, as well as various other types of systems. Host 110 may include several processing resources (e.g., one or more processors, microprocessors, or some other type of control circuitry) capable of accessing memory 120. System 100 may include a separate integrated circuit, or both host 110 and memory device 120 may be on the same integrated circuit. For example, host 110 may be a system controller for a memory system including multiple memory devices 120, wherein system controller 110 provides access to the respective memory devices 120 via another processing resource, such as a central processing unit (CPU).

[0021] exist Figure 1 In the example shown, host 110 is responsible for executing the operating system (OS) and / or various applications that can be loaded into the OS (e.g., from memory device 120 via controller 140). Host 110 can provide access commands and / or safe mode initialization commands to the memory device via interface 156.

[0022] For clarity, system 100 has been simplified to focus on features particularly relevant to this disclosure. For example, memory array 130 may be a DRAM array, SRAM array, STT RAM array, PCRAM array, TRAM array, RRAM array, NAND flash array, and / or NOR flash array. Array 130 may include memory cells arranged in rows coupled by access lines (which may be referred to herein as word lines or select lines) and columns coupled by sense lines (which may be referred to herein as digital lines or data lines).

[0023] In various instances, memory device 120 may include volatile memory and / or non-volatile memory. For example, memory device 120 may be a DRAM memory device including DRAM array 130. Memory device 120 may include other types of memory arrays. Memory device 120 includes an address circuitry to latch address signals provided via interface 156. For example, interface 156 may include a physical interface employing a suitable protocol (e.g., a data bus, address bus, and command bus, or a combined data / address / command bus). This protocol may be custom or proprietary, or interface 156 may employ a standardized protocol, such as Peripheral Component Interconnect Fast (PCIe), Gen-Z, CCIX, or similar. Address signals are received and decoded by row decoder 146 and column decoder 152 to access memory array 130. Data can be read from memory array 130 by using a sensing circuitry to sense changes in voltage and / or current on a sensing line. For example, the sensing circuitry may include a sensing amplifier that can read and latch a page (e.g., a row) of data from memory array 130. The I / O circuitry can be used for bidirectional data communication with the host 110 via interface 156. The read / write circuitry is used to write data to or read data from the memory array 130.

[0024] Controller 140 decodes signals provided by host 110. These signals may include chip enable signals, write enable signals, and address latch signals for controlling operations performed on memory array 130 (including data read, data write, and data erase operations). In various embodiments, controller 140 is responsible for executing instructions from host 110. Controller 140 may include a state machine, sequencer, and / or some other type of control circuitry system, which may be implemented in hardware, firmware, or software, or any combination of the three.

[0025] In various examples, controller 140 may receive signals provided by host 110, including signals requesting operation to be performed by PU 102. As used herein, PU 102 may include hardware, firmware, and / or software that perform operations (e.g., multiplication, for example) using data provided by memory array 130 and / or host 110.

[0026] In various instances, an error correction code (ECC) circuitry 103 may be coupled to a column decoder 152. The ECC circuitry 103 may receive data from a memory array 130. The ECC circuitry 103 may perform error correction operations to correct errors in the data sensed from the memory array 130. A power supply unit (PU) 102 may be coupled to the ECC circuitry 103 via a PU controller 105. The PU 102 may perform multiple operations on the data received from the ECC circuitry 103. The PU 102 may provide an output to a data path 104. The data path 104 may provide data to an interface 156. In various instances, the data path 104 may include input / output (I / O lines) and / or receivers and / or drivers. As used herein, a receiver may include circuitry configured to receive signals. A driver may describe circuitry used to drive signals across one or more lines.

[0027] The memory controller 140 can cause data to be read from memory 130 and can cause data to be provided to PU controller 105 via sensing circuitry and ECC 103. For example, the memory controller 140 can control memory logic to cause row decoder 146 to activate a row of memory 130. The memory controller 140 can also control memory logic to cause column decoder 146 to select certain columns of memory 130. Data from the sense amplifier coupled to the selected column can be provided to ECC 103 via a global data line. Corrected data can be provided to PU controller 105 via a global data line. In various examples, the memory controller 140 can cause data generated by PU controller 105 to be stored in memory 130. For example, PU controller 105 can place output data on a global data line and signal to memory controller 140. The memory controller 140 can cause data to be provided to the sensing circuitry system of memory 130 to cause output data to be stored in memory 130. The memory controller 140 can be a different circuitry system than PU controller 105.

[0028] In various instances, the PU controller 105 may include hardware and / or firmware for distributing data provided by the storage bank 130 to the PU 102. For example, the PU controller 105 may provide any combination of data from the storage bank 130 to the PU 102.

[0029] Although PU 102 is shown as being inside PU controller 105, PU 102 may be implemented outside of PU controller 105. For example, PU controller 105 may be coupled to each of PU 102 such that PU controller 105 can provide data to each of PU 102. As shown, PU 102 may also be implemented inside PU controller 105. PU controller 105 may include conductive paths to each of PU 102 to allow data to be provided from PU controller 105 to each of PU 102.

[0030] In various instances, PU controller 105 may determine which of PUs 102 will receive data received by PU controller 105. For example, not all PUs 102 may be available to receive data from PU controller 105. In various instances, the service agreement may specify that only a subset of PUs 102 may receive data from PU controller 105, and other considerations may be taken into account to limit which PUs 102 receive data from PU controller 105 at any given time. PU controller 105 may schedule which of the available PUs 102 will receive data. As used herein, a PU 102 is available if it is not performing an operation and / or has not yet been scheduled to perform an operation at a determined future time.

[0031] Figure 2 This is a block diagram of a PU controller 205 according to several embodiments of the present disclosure. The PU controller 205 is shown as being integrated in a memory device 220. The memory device 220 is similar to... Figure 1 The memory device 120.

[0032] The memory device 220 includes a plurality of memory banks 230-1, 230-2, 230-3, 230-4, 230-5, 230-6, 230-7, 230-8, 230-9, 230-10, 230-11, 230-12, 230-13, 230-14, 230-15, and 230-16, collectively referred to as memory banks 230. Memory banks 230 are similar to... Figure 1 Storage bank 130. Storage bank 230 is coupled to ECCs 203-1, 203-2, 203-3, 203-4, 203-5, 203-6, 203-7, and 203-8, referred to as ECC 203, which is similar to... Figure 1ECC 103. The PU controller 205 is shown as including control circuitry systems 222-1, 222-2, register 221, and PUs 202-1, 202-2, 202-3, 202-4, 202-5, 202-6, 202-7, 202-8, 202-9, 202-10, 202-11, 202-12, 202-13, 202-14, 202-15, and 202-16, referred to as PU 202. PU 202 is similar to... Figure 1 PU 102.

[0033] The PU controller 205 can receive data from the storage unit 230 and can route the data to any one or more of the PUs 202. Data can be routed without requiring different instances of data to be stored in two or more of the storage units 230.

[0034] Each of PU 202 can typically be associated with one of memory banks 230. For example, PU 202-1 can correspond to memory bank 230-1. PU 202-2 can correspond to memory bank 230-2. PU 202-3 can correspond to memory bank 230-3. PU 202-4 can correspond to memory bank 230-4. PU 202-5 can correspond to memory bank 230-5. PU 202-6 can correspond to memory bank 230-6. PU 202-7 can correspond to memory bank 230-7. PU 202-8 can correspond to memory bank 230-8. PU 202-9 can correspond to memory bank 230-9. PU 202-10 can correspond to memory bank 230-10. PU 202-11 can correspond to memory bank 230-11. PU 202-12 can correspond to memory bank 230-12. PU202-13 corresponds to memory module 230-13. PU 202-14 corresponds to memory module 230-14. PU 202-15 corresponds to memory module 230-15. PU 202-16 corresponds to memory module 230-16.

[0035] In various instances, the mode of memory device 220 can be used to determine whether data is provided from the memory bank to its corresponding PU or whether data is provided from any of the memory banks to any of the available PUs 202.

[0036] For example, based on the memory device 220, the PU controller 205 can provide data to any or more of the PUs 202. Data can be read from any of the storage units 230. Alternatively, data can be routed from the storage unit to its corresponding PU.

[0037] Storage units can be organized into storage units (e.g., BG). For example, Figure 2The example shown contains four blocks (e.g., BG0, BG1, BG2, and BG3), where memory banks 230-7, 230-8, 230-15, and 230-16 constitute BG0, memory banks 230-5, 230-6, 230-13, and 230-14 constitute BG1, memory banks 230-1, 230-2, 230-9, and 230-10 constitute BG2, and memory banks 230-3, 230-4, 230-11, and 230-12 constitute BG3.

[0038] Storage bank 230 can provide data to PU controller 205 via ECC 203. For example, storage banks 230-1 and 230-2 can provide data to PU controller 205 via ECC 203-1. Storage banks 230-3 and 230-4 can provide data to PU controller 205 via ECC 203-2. Storage banks 230-5 and 230-6 can provide data to PU controller 205 via ECC 203-3. Storage banks 230-7 and 230-8 can provide data to PU controller 205 via ECC 203-4. Storage banks 230-9 and 230-10 can provide data to PU controller 205 via ECC 203-5. Storage banks 230-11 and 230-12 can provide data to PU controller 205 via ECC 203-6. Storage units 230-13 and 230-14 can provide data to PU controller 205 via ECC 203-7. Storage units 230-15 and 230-16 can provide data to PU controller 205 via ECC 203-8.

[0039] The PU controller 205 may include control logic 222-1 and 222-2, referred to as control logic 222. Control logic 222 may be oriented towards the memory bank. For example, control logic 222-1 can be configured to receive data from storage banks 230-1, 230-2, 230-3, 230-4, 230-5, 230-6, 230-7, 230-8 instead of storage banks 230-9, 230-10, 230-11, 230-12, 230-13, 230-14, 230-15, 230-16, because control logic 222-1 is physically coupled to storage banks 230-1, 230-2, 230-3, 230-4, 230-5, 230-6, 230-7, 230-8 instead of storage banks 230-9, 230-10, 230-11, 230-12, 230-13, 230-14, 230-15, 230-16. Control logic 222-2 can be configured to receive data from storage banks 230-9, 230-10, 230-11, 230-12, 230-13, 230-14, 230-15, 230-16 instead of storage banks 230-1, 230-2, 230-3, 230-4, 230-5, 230-6, 230-7, 230-8, because control logic 222-2 is physically coupled to storage banks 230-9, 230-10, 230-11, 230-12, 230-13, 230-14, 230-15, 230-16 instead of storage banks 230-1, 230-2, 230-3, 230-4, 230-5, 230-6, 230-7, 230-8.

[0040] Control logic 222 can be configured to receive data from memory 230 and store the data in register 221. Register 221 can store data and can provide data to PU 202. Data can be copied as it is provided to PU 202. For example, if data is provided to PU 202-1 and PU 202-2, then a first copy of the data can be provided from register 221 to PU 202-1 and a second copy of the data can be provided from register 221 to PU 202-2. In various examples, copies of data can be provided concurrently from register 221 to PU 202. Each of PU 202 can be coupled to register 221. For example, each of PU 202 can be coupled to register 221 via multiple lines and / or PU 202 can be coupled to register 221 via one or more buses.

[0041] PU 202 can perform multiple operations using data received from register 221. PU 202 can generate output data. The output data can be provided to and stored in register 221. Register 221 can provide the output data to the control logic 222 of PU controller 205. The control logic 222 can provide the output data to memory 230. In various examples, the same memory that provides the input data can receive the output data. In other examples, a different memory than the one that provides the input data can receive the output data.

[0042] Although not shown, output data generated by PU 202 can also be routed to memory 230 without first storing the output data in register 221. For example, the output path within PU controller 205 can differ from the input path within PU controller 205. In various examples, the timing of PU controller 205 can be synchronized with the timing of memory device 220. PU controller 205, control logic 222, and / or PU 202 can receive timing signals to allow PU controller 205 to synchronize with memory device 220.

[0043] In various instances, PU controller 205 may select PUs 202 to receive data provided by storage bank 230. For example, PU controller 205 may select one of PUs 202, a subset of PUs 202, or all PUs 202 (e.g., available PUs 202). PU controller 205 may rotate the use of PUs 202 to allow a constant data stream to be provided to PUs 202. For example, at a first time, storage bank 230-1 may provide first data. PU controller 205 may select a first number of PUs 202 and provide the first data to the first number of PUs 202. At a second time, storage bank 230-2 may provide second data. PU controller 205 may select a second number of PUs 202 and provide the second data to the second number of PUs 202. The first number of PUs 202 and the second number of PUs 202 may concurrently perform several operations on a portion thereof. At a third time, the first number of PUs 202 may terminate the execution of the operation and may have produced first output data. The PU controller 205 can receive third data from the storage unit 230-1. Given that a first number of PUs 202 are available and a second number of PUs 202 are unavailable, the PU controller 205 can select the first number of PUs 202 and can provide the third data to the first number of PUs 202.

[0044] The PU controller 205 can select the PU 202 based on several criteria. For example, the PU controller 205 can select the PU 202 based on availability, service contracts and / or energy consumption / availability, as well as other factors that the PU controller 205 can use to select the PU 202.

[0045] In various instances, PU controller 205 can facilitate the distribution of data read from a single memory bank 230 to multiple PUs 202. For example, data can be provided to PU controller 205 from memory banks 230-16. PU controller 205 can store the data in register 221. Data can be distributed from register 221 to PUs 202-5, 202-6, 202-13, and 202-14. Each of PUs 202-5, 202-6, 202-13, and 202-14 can receive a different copy of the data stored in register 221. Control logic 222 can signal register 221 to cause the charge stored in register 221 to be copied and provided to PUs 202-5, 202-6, 202-13, and 202-14. Control logic 222 can provide signals to PUs 202-5, 202-6, 202-13, and 202-14 to cause PUs 202-5, 202-6, 202-13, and 202-14 to store data and use the data to perform multiple operations.

[0046] Output data generated by PUs 202-5, 202-6, 202-13, and 202-14 can be provided to memory bank 230-16 or different memory banks, such as memory bank 230-4. The PUs selected by PU controller 205 for receiving data can be consecutive PUs and / or non-consecutive PUs. Consecutive PUs include PUs that are adjacent to each other. Non-consecutive PUs include non-adjacent PUs. Control logic 222 can provide signals to PUs 202-5, 202-6, 202-13, and 202-14 to cause output data to be provided from PUs 202-5, 202-6, 202-13, and 202-14 to register 221 for storage. Control logic 222 can also provide signals to register 221 to cause register 221 to provide output data to control logic 222. Control logic 222 can route the output data to memory bank 230-16 or different memory banks.

[0047] In various instances, PU controller 205 facilitates the distribution of data read from multiple storage banks 230 to a single PU from PU 202. For example, data may be provided to PU controller 205 from storage banks 230-16, 230-8. PU controller 205 may store the data in register 221. The data may be distributed from register 221 to PU 202-1. Control logic 222 may signal register 221 to cause the charge stored in register 221 to be provided to PU 202-1. Control logic 222 may signal PU 202-1 to cause PU 202-1 to store data and use the data to perform multiple operations.

[0048] Output data generated by PU 202-1 can be provided to memory banks 230-16, 230-8, or different memory banks, such as memory banks 230-2 and 230-3. Memory banks configured to receive output data can be contiguous and / or non-contiguous. Contiguous memory banks include those that are adjacent (e.g., memory banks 230-11 and 230-12). Non-contiguous memory banks include those that are not adjacent (e.g., memory banks 230-7 and 230-9). Control logic 222 can signal PU 202-1 to cause output data to be provided from PU 202-1 to register 221 for storage. Control logic 222 can signal register 221 to cause register 221 to provide output data to control logic 222. Control logic 222 can route output data to memory banks 230-16, 230-8, or different memory banks.

[0049] In various instances, PU controller 205 facilitates the distribution of data read from multiple storage banks 230 to multiple PUs 202. For example, data may be provided to PU controller 205 from storage banks 230-16 and 230-8. PU controller 205 may store the data in register 221. The data may be distributed from register 221 to PUs 202-11, 202-12, and 202-16. Control logic 222 may signal register 221 to cause the charge stored in register 221 to be provided to PUs 202-11, 202-12, and 202-16. Control logic 222 may signal PUs 202-11, 202-12, and 202-16 to cause PUs 202-11, 202-12, and 202-16 to use the data to perform multiple operations.

[0050] Output data generated by PUs 202-11, 202-12, and 202-16 can be provided to memory banks 230-16, 230-8, or different memory banks 230-2 and 230-3. Control logic 222 can signal PUs 202-11, 202-12, and 202-16 to cause output data to be provided from PUs 202-11, 202-12, and 202-16 to register 221 for storage. Control logic 222 can also signal register 221 to cause register 221 to provide output data to control logic 222. Control logic 222 can route output data to memory banks 230-16, 230-8, or different memory banks.

[0051] As described herein, the mapping from memory bank 230 to PU 202 can occur in two phases. In the first phase, data can be received from and / or provided to one or more blocks. In the second phase, data can be received from or stored in register 221. The mapping may include receiving data from and / or providing data to memory bank 230. PU controller 205 can be configured to provide data to or receive data from each of memory banks 230. PU controller 205 can be coupled to each of memory banks 230. For example, PU controller 205 can be coupled to memory bank 230 via one or more global data lines. The mapping may also include providing data to PU 202. Data can be provided to PU 202 from register 221. Each of PU 202 can be coupled to register 221. Data can be received via control logic 222 and provided via register 221, defining whether data is routed from memory 230 to PU 202 or from PU 202 to memory 230.

[0052] In various instances, the host's storage controller may provide in-memory processing (PIM) commands to the memory device. The memory device may provide PIM commands to the PU controller 205. PIM commands may be provided as matrix addresses and / or vector addresses. For example, the memory controller may provide a matrix address to the memory device. The memory device may interpret the matrix address into a PIM command. The memory device may provide a PIM command to the PU controller 205. Alternatively, the matrix address may be provided directly to the PU controller 205, and the PU controller may translate the matrix address into a PIM command. The matrix address may have 32 * read burst length (RDBL)16. The matrix address may have a length equal to 16RDBL multiplied by 32. The vector address may have a length equal to 1RDBL16.

[0053] Matrix addresses and vector addresses can be used to access data from memory bank 230. For example, a matrix address may contain memory bank addresses, row addresses, and / or column addresses. Although a single memory bank address, row address, and / or column address is described, a matrix address may contain multiple memory bank addresses, multiple row addresses, and / or multiple column addresses. A vector address may also contain memory bank addresses, row addresses, and / or column addresses.

[0054] Once the PU controller 205 receives a PIM command in the form of a matrix address and / or a vector address, the PU controller 205 can generate an access command to read matrix data and / or vector data from one or more of the memory banks 230. The access command can be executed by the memory device 220 to cause the memory device 220 to access the matrix data and / or vector data. The matrix data and / or vector data can be read from the memory bank 230 and made available to the PU controller 205. The PU controller 205 can provide the matrix data and / or vector data to the PU 202 as previously described. The PU controller 205 can also generate an access command (e.g., a write command) to cause the output data generated by the PU 202 to be stored back into the memory bank 230. In various examples, the output data generated by the PU 202 can also be provided to the host via the input / output circuitry (I / O) of the memory device 220.

[0055] Figure 3 This describes example flowcharts of a method 380 for implementing a processing unit controller in memory according to several embodiments of the present disclosure. The method can be executed by a memory device of a computing system. For example, the method can be executed by a PU controller or PU of the memory device. Method 380 can be executed by processing logic, which may include hardware (e.g., processing device, circuit system, dedicated logic, programmable logic, microcode, device hardware, integrated circuit, etc.), software (e.g., instructions that run or execute on the processing device), or a combination thereof. In some embodiments, method 380 is executed by… Figure 1 The memory controller 105 and Figure 2 The memory controller 205 performs the process. Although shown in a specific sequence or order, the order of the processes may be modified unless otherwise specified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes may be executed in different orders, and some processes may be executed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0056] At 381, the memory controller (e.g.) Figure 1 The memory controller 140 can provide data from the memory bank of the memory cell to the PU controller. For example, the memory controller can cause data to be sensed from the memory bank of the memory cell and transmitted to the global data line via the sensing circuitry of the memory bank. The PU controller can receive the data via the global data line.

[0057] At 382, ​​the PU controller (e.g.) Figure 1 PU controller 105 and Figure 2 The PU controller 205 can access the memory of the memory cell (e.g., Figure 1 storage 130 and Figure 2The memory cell 230 receives data. The PU controller may be coupled to the memory cell's memory. For example, the memory cell may transmit data to the ECC (e.g., via the memory cell's sensing circuitry system). Figure 1 ECC 103 and Figure 2 The ECC (Electronic Control Center 203) provides data. The ECC can provide data to the PU controller. The PU controller can be indirectly coupled to the memory via the ECC.

[0058] At 383, the PU controller can determine the available PUs among multiple PUs. PUs may be unavailable for various reasons. For example, some of the multiple PUs may be unavailable because parts of the multiple PUs are being executed to perform multiple operations. In various cases, unavailable PUs may be performing multiple operations concurrently.

[0059] However, unavailable PUs can execute independently of each other and available PUs. For example, the first part of an unavailable PU can begin execution at the first time, the second part at the second time, and the third part at the third time. Between the third and fourth times, the first, second, and third parts of the unavailable PU can execute concurrently. The first part of the unavailable PU can finish execution at the fourth time, the second part at the fifth time, and the third part at the sixth time. The execution of the first part does not depend on the execution of the second and third parts. The execution of the second part does not depend on the execution of the first and third parts. The execution of the third part does not depend on the execution of the first and second parts. Any of the first, second, and third parts can execute even if none of the other parts execute.

[0060] At position 384, the PU controller can provide data to available PUs. For example, the PU controller can store data in its registers. The PU controller's registers can provide copies of the data to available PUs.

[0061] At 385, a usable PU can use data to perform multiple operations. In various instances, a usable PU performing multiple operations may become unavailable after it begins performing multiple operations. Usable PUs performing multiple operations may not be independent of each other. Usable PUs can be dependent if they are performing multiple operations simultaneously, if the data used to perform the multiple operations is the same data, or if the data is associated with the same ANN.

[0062] Multiple PUs can be coupled to a PU controller. The PU controller can provide data to available PUs by providing data to the PU controller from an external source. In such implementations, the PU controller can control PUs even if the PU is not part of the PU controller and is implemented externally to the PU controller.

[0063] The PU controller may include registers. Data received from memory can be stored in registers before being supplied to multiple PUs. Data can be supplied from registers to multiple PUs, regardless of whether the multiple PUs are implemented inside or outside the PU controller. For example, if multiple PUs are implemented inside the PU controller, then the PU controller can internally supply data from registers to the available PUs.

[0064] Data can be provided to available PUs sequentially or concurrently. For example, data can be provided from registers to available PUs simultaneously. Data can be provided as signals via multiple lines. Available PUs can store signals relatively simultaneously. Data can be provided to registers sequentially. For example, data can be provided to a first available PU, followed by data to a second available PU. The second available PU cannot receive data until the first available PU has received data. The first available PU can begin execution after receiving data or can postpone execution until the second available PU is ready to perform multiple operations using the same data.

[0065] Available PUs can concurrently provide output data generated by each of the available PUs to memory. For example, first output data generated by a first PU and second output data generated by a second PU can be stored in a register and concurrently provided from the register to memory. As described herein, concurrency describes actions occurring relatively simultaneously.

[0066] although Figure 2 Not shown, but the PU controller may include input registers (e.g., register 221) and output registers. Input registers can be used to store data received by the PU controller. Output registers can be used to store output data generated by the PU. Implementing input and output registers in the PU controller allows the PU controller to receive and output data simultaneously.

[0067] Output data stored in the output register can be sequentially provided to the memory. For example, before the second output data generated by the second PU is provided to the memory by the output register, the first output data generated by the first PU can be provided to the memory by the output register.

[0068] A first portion of the output data stored in the output register can be provided to the memory bank. A second portion of the output data stored in the output register can be provided to the system-on-a-chip (SoC) coupled to the memory device, which includes the memory bank of memory cells, a PU controller, and multiple PUs. For example, the PU controller can be coupled to the input / output circuitry of the memory device, allowing the PU controller to provide data to the memory device externally.

[0069] In various instances, the memory controller can provide the PU controller with data from multiple memory banks. The PU controller (e.g., Figure 1 PU controller 105 and Figure 2 The PU controller 205 can be obtained from any of the multiple storage banks (e.g., Figure 1 storage 130 and Figure 2 The storage bank 230 receives data. Multiple storage banks may include memory cells. The PU controller may be coupled to multiple storage banks of the memory cells. The PU controller may include a PU (e.g., Figure 2 (PU 202 in the original text). The PU controller can be configured to receive data from each of a plurality of memory banks in the memory cell. The data may include matrix data and / or vector data. The data can be used to implement an ANN. The data can also be used to execute an ANN. For example, data including matrix data and / or vector data may contain the input data and weights of the ANN. The PU controller's PU can use the matrix data and vectors to perform multiplication to process the input data through the ANN to produce the output of the ANN.

[0070] A PU controller can provide data to a PU. The PU controller can route data provided by a first memory bank out of multiple memory banks to the PU at a first time. At a second time, the PU controller can route data provided by a second memory bank out of multiple memory banks to the PU. Even if a PU does not correspond to a memory bank, the PU controller can still route data from a memory bank to the PU. For example, in a traditional architecture, each PU can be implemented to process data from an associated memory bank rather than other memory banks. Given that the PU routes data from a memory bank rather than other memory banks, the PU can be described as corresponding to a memory bank. The PU controller can be implemented to route data from other memory banks and memory banks to the PU, thereby allowing memory devices to utilize PU resources more efficiently than limiting the PU to processing data provided by a single memory bank.

[0071] The PU can use data to perform multiple operations. For example, first data provided by a first memory bank can be stored in a first register of the PU. Second data provided by a second memory bank can be provided to one or more MAC units of the PU along with the first data. The MAC units can use matrix data and vector data to perform multiple multiplication operations to execute an ANN. The outputs of the MAC units can be accumulated. The accumulated result can be the output of a layer of the ANN and / or the output of the ANN. In various instances, the outputs of the PU can be provided to different PUs, can be stored in multiple memory banks, and / or can be provided externally to a memory device.

[0072] The first memory bank of a memory cell can provide first data to the PU controller. The PU controller can provide the first data to the PU by routing the first data to the PU. The PU can be indirectly coupled to the multiple memory banks of the memory cell through the PU controller. The PU controller can route matrix data from the first memory bank and vector data from the second memory bank.

[0073] The PU controller can provide output data generated by the PU using the first data to the first memory for storage. For example, the PU can provide output data to the control logic of the PU controller. The control logic of the PU controller can then provide output data to the first memory for storage.

[0074] The second memory bank of a memory cell, one of multiple memory banks in the memory unit, can provide second data to the PU controller. In various instances, the second memory bank and the first memory bank of the memory unit can provide data to the PU controller concurrently. For example, the first control logic of the PU controller can receive first data from the first memory bank. While the first control logic receives the first data, the second control logic of the PU controller can receive second data from the second memory bank. The first and second control logics can store the first and second data concurrently in the registers of the PU controller and / or can store the first and second data sequentially in the registers of the PU controller.

[0075] The PU controller can provide second data to the PU after providing first data. For example, the PU can receive first data and second data from a register. The register can provide the first data to the PU, and then the register can provide the second data to the PU.

[0076] The PU controller can provide output data generated by the PU using second data to a first memory bank for storage. The PU controller can also provide output data generated by the PU using first data to a second memory bank for storage. The output generated by the PU can be provided to any of the multiple memory banks.

[0077] In various instances, a device may include multiple banks of memory in a memory cell, a PU controller, and a bank controller. The bank controller may provide data from the multiple banks of memory to the PU controller. The PU controller may be coupled to the multiple banks of memory in the memory cell. The PU controller may include multiple PUs. The controller may receive data from any of the multiple banks of memory. The PU controller may determine the available PU among the multiple PUs. The PU controller may provide data to the available PU. The available PU may use the data to perform multiple operations.

[0078] The PU controller can provide output data generated by multiple operations from available PUs to multiple memory banks. The PU controller can also provide first data from data received from a first memory bank among the multiple memory banks to a first available PU from the available PUs. The PU controller can also use the first data to perform a first plurality of operations to generate first output data. The PU controller can provide second data from data received from a second memory bank among the multiple memory banks to a second available PU from the available PUs.

[0079] The PU controller can utilize the second data to perform a second plurality of operations to generate second output data. The PU controller can provide first output data from a first available PU to a second storage bank. The PU controller can also provide second output data from a second available PU to a first output storage bank. The PU controller is not limited to providing output data to a storage bank that provides input data for generating output data. The PU controller can also provide output data to a storage bank that does not provide input data for generating output data.

[0080] Figure 4 An example machine illustrating computer system 490 is described, within which a set of instructions can be executed to cause the machine to perform any or more of the methods discussed herein. In some embodiments, computer system 490 may correspond to a host system (e.g., Figure 1 The host 110), which includes, is coupled to, or utilizes a memory system (e.g., Figure 1 The memory device 120) or the PU controller that can be used to execute (e.g., Figure 1 The operation of the PU controller 105. In alternative embodiments, the machine may be connected (e.g., networked) to other machines in a LAN, intranet, extranet, and / or the Internet. The machine may operate as a server or client machine in a client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or client machine in a cloud computing infrastructure or environment.

[0081] A machine can be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network device, a server, a network router, a switch, or a bridge, or any machine capable of executing a set of instructions (sequentially or otherwise) specifying actions to be taken by the machine. Furthermore, while a single machine is described, the term "machine" should also be considered as any collection of machines that individually or collectively execute a set (or more) of instructions to perform any or more of the methods discussed herein.

[0082] Example computer system 490 includes a processing device 491, a main memory 493 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) (e.g., synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM)), static memory 497 (e.g., flash memory, static random access memory (SRAM), etc.) and a data storage system 498, which can communicate with each other via a bus 496.

[0083] Processing device 491 represents one or more general-purpose processing devices, such as a microprocessor, central processing unit, or the like. More specifically, the processing device may be a Complex Instruction Set Computing (CISC) microprocessor, a Reduced Instruction Set Computing (RISC) microprocessor, a Very Long Instruction Word (VLIW) microprocessor, or a processor implementing other instruction sets, or several processors implementing combinations of instruction sets. Processing device 491 may also be one or more special-purpose processing devices, such as an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), a network processor, or the like. Processing device 491 is configured to execute instructions 492 for performing the operations and steps discussed herein. Computer system 490 may further include a network interface device 494 for communication via network 495.

[0084] Data storage system 498 may include machine-readable storage medium 499 (also referred to as computer-readable medium) storing one or more sets of instructions 492 or software embodying any or more of the methods or functions described herein. Instructions 492 may also reside wholly or at least partially in main memory 493 and / or processing device 491 during execution by computer system 490, which also constitute machine-readable storage medium.

[0085] In one embodiment, instruction 492 includes instructions for implementing the corresponding Figure 1 The PU controller 105 provides functional instructions. Although the machine-readable storage medium 499 is shown as a single medium in the exemplary embodiment, the term "machine-readable storage medium" should be considered as a single medium or multiple media containing one or more sets of instructions. The term "machine-readable storage medium" should also be considered as any medium capable of storing or encoding a set of instructions for machine execution and causing the machine to perform any or more of the methods of this disclosure. Therefore, the term "machine-readable storage medium" should be considered as including (but not limited to) solid-state memory, optical media, and magnetic media.

[0086] Although specific embodiments have been illustrated and described herein, those skilled in the art will understand that arrangements calculated to achieve the same results may be substituted for the specific embodiments shown. This disclosure is intended to cover adaptations or variations of various embodiments of this disclosure. It should be understood that the above description has been carried out in an illustrative rather than restrictive manner. Those skilled in the art will understand, upon review of the above description, combinations of the above embodiments and other embodiments not explicitly described herein. The scope of the various embodiments of this disclosure includes other applications in which the above structures and methods are used. Therefore, the scope of the various embodiments of this disclosure should be determined with reference to the appended claims, together with the full scope of the equivalents granted therein.

[0087] In the “Detailed Description”, various features are grouped together in a single embodiment for the purpose of simplifying this disclosure. The method of this disclosure should not be interpreted as reflecting an intention that the disclosed embodiments of this disclosure must use more features than those expressly recited in each claim. Rather, as reflected in the appended claims, the subject matter of the invention exhibits not all features of a single disclosed embodiment. Therefore, the appended claims are hereby incorporated into the “Detailed Description”, wherein each claim is considered an independent, separate embodiment.

Claims

1. An apparatus for implementing a processing unit controller in a memory, comprising: Multiple memory banks of a memory cell (130, 230-1, 230-2, 230-3, 230-4, 230-5, 230-6, 230-7, 230-8, 230-9, 230-10, 230-11, 230-12, 230-13, 230-14, 230-15, 230-16); A memory bank controller (140) coupled to the plurality of memory banks of the memory cells; Processing unit (PU) controllers (105, 205) are coupled to the plurality of memory banks of the memory unit and include PUs (102, 202-1, 202-2, 202-3, 202-4, 202-5, 202-6, 202-7, 202-8, 202-9, 202-10, 202-11, 202-12, 202-13, 202-14, 202-15, 202-16); The memory controller is configured to provide data from the plurality of memories to the PU controller; The PU controller is configured to: Receive data from any of the plurality of storage units; and The data is provided to the PU; and The PU is configured to use the data to perform multiple operations.

2. The device of claim 1, wherein the PU controller is further configured to: Receive first data from the first storage bank of the plurality of storage banks in the memory cell; and The first data is provided to the PU.

3. The device of claim 2, wherein the PU controller is further configured to provide output data to the first storage for storage, wherein the output data is generated by the PU using the first data.

4. The device of claim 2, wherein the memory controller is further configured to provide the PU controller with second data from a second memory bank of a memory bank among the plurality of memory banks of the memory unit.

5. The device of claim 4, wherein the PU controller is further configured to provide the second data to the PU after providing the first data to the PU.

6. The device of claim 4, wherein the PU controller is further configured to provide output data to the first storage for storage, the output data being generated by the PU using the second data.

7. The device of claim 6, wherein the PU controller is further configured to provide different output data to the second storage for storage, the different output data being generated by the PU using the first data.

8. A method for implementing a processing unit controller in a memory, comprising: The storage controller (140) provides data from the storage units (130, 230-1, 230-2, 230-3, 230-4, 230-5, 230-6, 230-7, 230-8, 230-9, 230-10, 230-11, 230-12, 230-13, 230-14, 230-15, 230-16) of the memory cells of the memory device (120) to the processing unit (PU) controller (105, 205); The PU controller receives the data from the memory bank of the memory unit, wherein the PU controller is coupled to the memory bank; The available PUs among the multiple PUs are determined by the PU controller; The data is provided by the PU controller to the available PU; and The available PU can use the data to perform multiple operations.

9. The method of claim 8, wherein the plurality of PUs are coupled to the PU controller, and wherein the method further comprises providing the data to the available PUs from an external source.

10. The method of claim 8, further comprising storing the data received from the storage in the register (221) of the PU controller.

11. The method of claim 10, wherein the PU controller includes the plurality of PUs, and wherein the method further includes internally providing the data from the register to the available PUs.

12. The method of claim 8, further comprising: The output data generated by each of the available PUs is stored in the output register of the PU controller; Provide the memory bank with a first portion of the output data stored in the output register; and A second portion of the output data stored in the output register is provided to the system-on-chip (SOC) coupled to the memory device, the memory device comprising the memory bank of the memory cell, the PU controller, and the plurality of PUs.

13. An apparatus for implementing a processing unit controller in a memory, comprising: Multiple memory banks of a memory cell (130, 230-1, 230-2, 230-3, 230-4, 230-5, 230-6, 230-7, 230-8, 230-9, 230-10, 230-11, 230-12, 230-13, 230-14, 230-15, 230-16); Storage controller (140); Processing unit (PU) controllers (105, 205) are coupled to the plurality of memory banks of the memory unit and include a plurality of PUs (102, 202-1, 202-2, 202-3, 202-4, 202-5, 202-6, 202-7, 202-8, 202-9, 202-10, 202-11, 202-12, 202-13, 202-14, 202-15, 202-16); The memory controller is configured to provide data from the plurality of memories to the PU controller; The PU controller is configured to: Receive the data from any of the plurality of storage units; and The available PUs among the plurality of PUs are determined by the PU controller; and The data is provided to the available PU; and The available PU is configured to use the data to perform multiple operations.

14. The device of claim 13, wherein the PU controller is further configured to provide the plurality of storage units with output data generated by the plurality of operations from the available PUs.

15. The device of claim 14, wherein the PU controller is further configured to: Provide the first available PU from the available PU with the first available PU receiving the first data from the first storage bank among the plurality of storage banks; The first data is used to perform a first plurality of operations to generate first output data; Provide a second available PU from the available PU with second data received from the data from a second storage bank among the plurality of storage banks; The second data is used to perform a second plurality of operations to produce a second output data; Provide the first output data from the first available PU to the second storage; and The second output data from the second available PU is provided to the first storage.

Images