Debugging operations for AI operations

Abstract

The present disclosure relates to debugging operations of artificial intelligence operations. An example device may include multiple memory arrays and a controller, wherein the controller is configured to perform artificial intelligence (AI) operations on data stored in the multiple memory arrays and to perform debugging operations on the AI ​​operations.

Description

[0001] Information about divisional applications

[0002] This case is a divisional application. The parent case of the divisional application is an invention patent application with an application date of August 27, 2020, application number 202010875460.4, and invention name “Debugging operation of artificial intelligence operation”. Technical Field

[0003] The present disclosure relates generally to memory devices, and more particularly to apparatus and methods for debugging operations of artificial intelligence operations. Background Art

[0004] Memory devices are typically provided as internal semiconductor integrated circuits in computers or other electronic devices. There are many different types of memory, including volatile memory and non-volatile memory. Volatile memory may require power to maintain its data and includes random access memory (RAM), dynamic random access memory (DRAM) and synchronous dynamic random access memory (SDRAM), etc. Non-volatile memory can provide persistent data by retaining the stored data when power is off and can include NAND flash memory, NOR flash memory, read-only memory (ROM), electrically erasable programmable ROM (EEPROM), erasable programmable ROM (EPROM) and resistance variable memory, such as phase change random access memory (PCRAM), resistive random access memory (RRAM) and magnetoresistive random access memory (MRAM), etc.

[0005] Memories are also used as volatile and nonvolatile data storage areas for a wide range of electronic applications. Nonvolatile memories can be used, for example, in personal computers, portable memory sticks, digital cameras, cellular phones, portable music players such as MP3 players, movie players, and other electronic devices. Memory cells can be arranged in an array, wherein the array is used in a memory device. Summary of the invention

[0006] In one aspect, the present disclosure relates to a device for debugging operations of artificial intelligence (AI) operations, the device comprising: multiple memory arrays; and a controller, wherein the controller is configured to cause the device to: perform artificial intelligence (AI) operations on data stored in the multiple memory arrays; and perform debugging operations on the artificial intelligence (AI) operations.

[0007] On the other hand, the present disclosure relates to a device for debugging operations of artificial intelligence (AI) operations, the device comprising: multiple memory arrays; and a controller, wherein the controller is configured to: receive a debug command from a host to perform a debug operation on the artificial intelligence (AI) operation executed on the device; and execute the debug command by causing the debug operation to be performed on the artificial intelligence (AI) operation.

[0008] In another aspect, the present disclosure relates to a device for debugging operations of artificial intelligence (AI) operations, the device comprising: multiple memory arrays; and a controller, wherein the controller is configured to cause the device to: perform debugging operations on artificial intelligence (AI) operations by writing an indicator in a register to a specific state in response to receiving a command from a host.

[0009] In another aspect, the present disclosure relates to a method for a debug operation of an artificial intelligence (AI) operation, the method comprising: performing an artificial intelligence (AI) operation on data stored in a plurality of memory arrays; and performing a debug operation on the artificial intelligence (AI) operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a block diagram of an apparatus in the form of a computing system including a memory device with an artificial intelligence (AI) accelerator according to multiple embodiments of the present disclosure.

[0011] Figure 2 is a block diagram of multiple registers on a memory device with an artificial intelligence (AI) accelerator according to multiple embodiments of the present disclosure.

[0012] Figure 3A and 3B is a block diagram of multiple bits in multiple registers on a memory device with an artificial intelligence (AI) accelerator according to multiple embodiments of the present disclosure.

[0013] Figure 4 is a block diagram of multiple blocks of a memory device with an artificial intelligence (AI) accelerator according to multiple embodiments of the present disclosure.

[0014] Figure 5 is a flow chart illustrating an example artificial intelligence (AI) process in a memory device with an artificial intelligence (AI) accelerator according to various embodiments of the present disclosure.

[0015] Fig. 6A is a block diagram of an apparatus in the form of a computing system including a memory device with an artificial intelligence (AI) accelerator according to multiple embodiments of the present disclosure.

[0016] Figure 6Bis a block diagram of an apparatus in the form of a computing system including a memory device with an artificial intelligence (AI) accelerator having shared input / output (I / O) lines in a data path local to a memory bank portion of an array according to multiple embodiments of the present disclosure.

[0017] Figure 7 is a schematic diagram illustrating a readout circuit system of a memory device according to various embodiments of the present disclosure, the readout circuit system including a computing component.

[0018] Figure 8 is a schematic diagram illustrating circuitry for multiple shared I / O lines in a data path of an array according to various embodiments of the present disclosure.

[0019] Fig. 9A is an example of a block diagram illustrating portions of an array coupled to a computing unit having multiple logic strips via multiple shared I / O lines in a data path local to the array.

[0020] Fig. 9B is a block diagram example showing multiple arrays coupled to multiple computing components in a computing unit via multiple shared I / O lines in data paths local to the array, wherein the spacing of the computing components is equal to the spacing of the data paths of the shared I / O lines and the spacing is a multiple of the spacing of the digit lines to the array. DETAILED DESCRIPTION

[0021] The present disclosure includes devices and methods related to performing debug operations on artificial intelligence operations. An example device may include multiple memory arrays and a controller configured to perform artificial intelligence (AI) operations on data stored in the multiple memory arrays and to perform debug operations on the AI ​​operations. The debug operations may include stopping the AI ​​operations and storing errors of the AI ​​operations and results of the debug operations in temporary memory blocks of the multiple memory arrays.

[0022] In some instances, registers of the controller can be used to perform debug operations. To start a debug operation, the first bit of the register can be programmed to a first state, and to stop and / or prevent a debug operation, the first bit of the register can be programmed to a second state. For example, the controller can program a bit and / or multiple bits of a register to a specific state in response to receiving a command from a host. Different registers of the controller can define the layers at which the debug operation will stop the AI ​​operation, change the contents of the neural network, change the bias value, change the activation function, observe the output of a layer, send the output of a layer, and / or send data.

[0023] Performing a debug operation on an artificial intelligence operation by stopping the AI ​​operation and storing errors of the AI ​​operation and results of the debug operation can improve the reliability of the memory device. For example, the memory device can detect and correct errors and update input data, bias values, activation functions, and / or neural networks based on the detected and corrected errors to prevent future errors.

[0024] In the following detailed description of the present disclosure, reference is made to the accompanying drawings, which form a part of the detailed description and in which are shown by way of illustration how various embodiments of the present disclosure may be practiced. The description of these embodiments is detailed enough to enable one of ordinary skill in the art to practice the embodiments of the present disclosure, and it should be understood that other embodiments may be utilized and that process changes, electrical changes, and / or structural changes may be made without departing from the scope of the present disclosure. As used herein, the designator "N" indicates that a plurality of the particular features so designated may be included in multiple embodiments of the present disclosure.

[0025] As used herein, "a plurality of" things may refer to one or more of such things. For example, a plurality of memory devices may refer to one or more of the memory devices. In addition, designators such as "N" as used herein, especially with respect to reference numerals in the accompanying drawings, indicate that a plurality of the particular feature so designated may be included in multiple embodiments of the present disclosure.

[0026] The accompanying drawings herein follow a numbering convention, wherein one or more first digits correspond to the figure number, and the remaining digits identify the elements or components in the figure. Similar elements or components between different figures can be identified by using similar numerals. As should be understood, the elements shown in the various embodiments herein can be added, exchanged and / or eliminated to provide multiple other embodiments of the present disclosure. In addition, the ratio and relative scale of the elements provided in the figures are intended to show various embodiments of the present disclosure, and are not intended to be used in a restrictive sense.

[0027] Figure 1 1 is a block diagram of an apparatus in the form of a computing system 100 including a memory device 120 according to various embodiments of the present disclosure. As used herein, the memory device 120, the memory arrays 125-1, . . . , 125-N, the memory controller 122, and / or the AI ​​accelerator 124 may also be respectively considered as a “device”.

[0028] like Figure 1As shown, host 102 can be coupled to memory device 120. Host 102 can be a laptop computer, a personal computer, a digital camera, a digital recording and playback device, a mobile phone, a PDA, a memory card reader, an interface hub, and other host systems and can include a memory access device, such as a processor. It should be understood by those of ordinary skill in the art that a "processor" can include one or more processors, such as a parallel processing system, multiple coprocessors, etc.

[0029] The host 102 includes a host controller 108 for communicating with the memory device 120. The host controller 108 can send commands to the memory device 120. The host controller 108 can communicate with the memory device 120, the memory controller 122 on the memory device 120, and / or the AI ​​accelerator 124 on the memory device 120 to perform AI operations, read data, write data, and / or erase data, among other operations. The AI ​​accelerator 124 may also include a processor that communicates with the memory device 120. Figures 6A-9B Components configured to perform AI operations described in association. AI operations may include machine learning or neural network operations, which may include training operations or reasoning operations or both. In some instances, each memory device 120 may represent a layer within a neural network or a deep neural network (e.g., a network with three or more hidden layers). Or each memory device 120 may be or include a node of a neural network, and the layers of the neural network may be composed of multiple memory devices or portions of several memory devices 120. Memory devices 120 may store weights (or models) for AI operations in memory array 125.

[0030] The physical host interface may provide an interface for passing control signals, address signals, data signals, and other signals between a memory device 120 having a compatible receiver for the physical host interface and a host 102. The signals may be transmitted between the host 102 and the memory device 120, for example, on multiple buses such as a data bus and / or an address bus.

[0031] The memory device 120 may include a controller 120, an AI accelerator 124, and a memory array 125-1, ..., 125-N. The memory device 120 may be a low power double data rate dynamic random access memory such as an LPDDR5 device and / or a graphics double data rate dynamic random access memory such as GDDR6, GDDR7, LPDDR6, DDR4, and other types of devices. The memory array 125-1, ..., 125-N may include a plurality of memory cells, such as volatile memory cells (e.g., DRAM memory cells and other types of volatile memory cells) and / or non-volatile memory cells (e.g., RRAM memory cells and other types of non-volatile memory cells). The memory device 120 may read data and / or write data to the memory array 125-1, ..., 125-N. The memory array 125-1, ..., 125-N may store data used during AI operations performed on the memory device 120. The memory arrays 125 - 1 , . . . , 125 -N may store inputs, outputs, weight matrices, and bias information of a neural network and / or activation function information used by an AI accelerator to perform AI operations on the memory device 120 .

[0032] The host controller 108, the memory controller 122, and / or the AI ​​accelerator 124 on the memory device 120 may include a control circuit system, for example, hardware, firmware, and / or software. In one or more embodiments, the host controller 108, the controller memory 122, and / or the AI ​​accelerator 124 may be an application-specific integrated circuit (ASIC) coupled to a printed circuit board including a physical interface. The register 130 may be located on the memory controller 122, on the AI ​​accelerator 124, and / or in the memory array 125-1, ..., 125-N and may be accessed by the controller 122. Moreover, the memory controller 122 on the memory device 120 may include a register 130. The register 130 may be programmed to provide information for the AI ​​accelerator to perform AI operations. The register 130 may include any number of registers. The register 130 may be written to and / or read by the host 102, the memory controller 122, and / or the AI ​​accelerator 124. The register 130 may provide input, output bias, neural network, and / or activation function information for the AI ​​accelerator 124. Registers 130 may include a mode register 131 for selecting an operating mode of memory device 120. For example, the AI ​​operating mode may be selected by writing a word such as 0xAA and / or 0x2AA to register 131, which disables access to registers associated with normal operation of memory device 120 and allows access to registers associated with AI operation. Furthermore, the AI ​​operating mode may be selected using a signature that uses an encryption algorithm authenticated by a key stored in memory device 120.

[0033] The AI ​​accelerator 124 may include hardware 126 and / or software / firmware 128 for performing AI operations. Figures 6A-9B Components configured to perform AI operations described in association. Hardware 126 may include adders / multipliers 126 for performing logical operations associated with AI operations. Memory controller 122 and / or AI accelerator 124 may receive commands from host 102 to perform AI operations. Memory device 120 may use the AI ​​accelerator 124, data in memory array 125-1, ..., 125-N, and information in register 130 to perform the AI ​​operation requested in the command of host 102. The memory device may report back information to host 120, such as, for example, the results and / or error information of the AI ​​operation. The AI ​​operation performed by AI accelerator 124 may be performed without using external processing resources.

[0034] The memory arrays 125-1, ..., 125-N may provide the main memory of the memory system or may be used as another memory or storage area of ​​the entire memory system. Each memory array 125-1, ..., 125-N may include multiple blocks of memory cells. The blocks of memory cells may be used to store data used during the AI ​​operation performed by the memory device 120. For example, the memory arrays 125-1, ..., 125-N may include DRAM memory cells. Embodiments are not limited to a specific type of memory device. For example, the memory device may include RAM, ROM, DRAM, SDRAM, PCRAM, RRAM, 3DXPoint, flash memory, etc.

[0035] For example, the memory device 120 may perform an AI operation that is or includes one or more reasoning steps. The memory array 125 may be a layer of a neural network or may each be a separate node, and the memory device 120 may be a layer; or the memory device 120 may be a node within a larger network. Additionally or alternatively, the memory array 125 may store data or weights or both to be used (e.g., summed) within the node. Each node (e.g., memory array 125) may combine inputs from data read from cells of the same or different memory arrays 125 with weights read from cells of the memory array 125. The combination of weights and data may be summed, for example, within the periphery of the memory array 125 or within the hardware 126 using an adder / multiplier 127. In such cases, the result of the summation may be passed to an activation function represented or instantiated within the periphery of the memory array 125 or within the hardware 126. The result may be passed to another memory device 120 or may be used within the AI ​​accelerator 124 (e.g., by software / firmware 128) to make decisions or train a network that includes the memory device 120.

[0036] The network using memory device 120 may be capable of or used for supervised or unsupervised learning. This may be combined with other learning or training regimes. In some cases, a trained network or model is imported into or used with memory device 120, and the operation of memory device 120 is primarily or only related to reasoning.

[0037] Figure 1 Embodiments of the present disclosure may include additional circuitry that is not shown to avoid obscuring the embodiments of the present disclosure. For example, memory device 120 may include address circuitry for latching address signals provided by I / O circuitry using I / O connections. The address signals may be received and decoded by row decoders and column decoders to access memory arrays 125-1, ..., 125-N. Those skilled in the art will appreciate that the number of address input connections may depend on the density and architecture of memory arrays 125-1, ..., 125-N.

[0038] Figure 2 2 is a block diagram of a plurality of registers on a memory device having an artificial intelligence (AI) accelerator according to various embodiments of the present disclosure. Register 230 may be an AI register and may contain input information, output information, neural network information, and / or activation function information, as well as other types of information, for use by an AI accelerator, a controller, and / or a memory array of a memory device (e.g., Figure 1 The registers may be based on the AI ​​accelerator 124, memory controller 122, and / or memory array 125-1, ..., 125-N in the host, the AI ​​accelerator, and / or the controller (e.g., Figure 1The AI ​​accelerator 124 and the memory controller 122 are read and / or written based on commands from the host 102, the AI ​​accelerator 124, and the memory controller 122.

[0039] Register 232-0 may define parameters associated with the AI ​​mode of the memory device. A bit in register 232-0 may start an AI operation, restart an AI operation, indicate that the contents of the register are valid, clear the contents of the register, and / or exit the AI ​​mode.

[0040] Registers 232-1, 232-2, 232-3, 232-4, and 232-5 may define the size of the inputs used in the AI ​​operation, the number of inputs used in the AI ​​operation, and the starting and ending addresses of the inputs used in the AI ​​operation. Registers 232-7, 232-8, 232-9, 232-10, and 232-11 may define the size of the outputs of the AI ​​operation, the number of outputs in the AI ​​operation, and the starting and ending addresses of the outputs of the AI ​​operation.

[0041] Register 232-12 may be used to implement input blocks, neuron blocks, output blocks, bias blocks, activation functions, and temporary blocks used during AI operations.

[0042] Registers 232-13, 232-14, 232-15, 232-16, 232-17, 232-18, 232-19, 232-20, 232-21, 232-22, 232-23, 232-24, and 232-25 may be used to define a neural network used during AI operations. Registers 232-13, 232-14, 232-15, 232-16, 232-17, 232-18, 232-19, 232-20, 232-21, 232-22, 232-23, 232-24, and 232-25 may define the size, number, and location of neurons and / or layers of a neural network used during AI operations.

[0043] Registers 232-26 may enable a debug / hold mode of the AI ​​accelerator and enable observation of output at the layer of the AI ​​operation. For example, the controller may be configured to perform an AI operation on data stored in a plurality of memory arrays, stop the AI ​​operation, and perform a debug operation on the AI ​​operation.

[0044] Registers 232-26 may indicate that an activation function should be applied during an AI operation, and the AI ​​operation may proceed forward in the AI ​​operation (e.g., execute the next step in the AI ​​operation). The controller may enable the AI ​​accelerator to perform a debug operation on the AI ​​operation by programming a bit and / or multiple bits of registers 232-26 to a specific state. In some instances, the first bit of registers 232-26 may be programmed to a first state to start a debug operation and may be programmed to a second state to stop and / or prevent a debug operation. For example, in response to receiving a command from a host, the controller may program a bit and / or multiple bits of registers 232-26 to a specific state.

[0045] Register 232-26 may indicate that the AI ​​operation may proceed forward in the AI ​​operation (e.g., perform the next step in the AI ​​operation). In some examples, a second bit of register 232-26 may be programmed to a first state to indicate that the AI ​​operation may proceed forward in the AI ​​operation and may be programmed to a second state to indicate that the AI ​​operation may not proceed forward.

[0046] Register 232-26 can indicate that the temporary block where the output of the layer is located is valid. In response to the temporary block being indicated as valid by register 232-26, the temporary block can be read by the host. The third bit of register 232-26 can be programmed to a first state to indicate that the temporary block is valid and can be programmed to a second state to indicate that the temporary block is invalid.

[0047] The debug operation may include storing data (e.g., errors and / or results from the AI ​​operation) in a temporary block of the plurality of memory arrays and verifying the data. For example, the fourth bit of register 232-26 may be programmed to a first state to store errors from the AI ​​operation for correction. In some instances, when the fourth bit of register 232-26 is programmed to a second state, no errors are stored. The data in the temporary block may be changed by a controller on a host and / or memory device (e.g., to correct errors in the data) so that the changed data may be used in the AI ​​operation as the AI ​​operation proceeds forward.

[0048] The results of the debugging operation can be stored and / or transmitted. For example, the results of the debugging operation can be stored in different blocks in the multiple memory arrays and / or sent to the host. The fifth bit of the register 232-26 can be programmed to a first state to send the results of the debugging operation to the host, and the sixth bit of the register 232-26 can be programmed to a first state to store the results of the debugging operation in different blocks in the multiple memory arrays.

[0049] Registers 232-27, 232-28, and 232-29 may define the layers at which the debug / hold mode will stop the AI ​​operation, change the contents of the neural network, change the input value, change the bias value, change the activation function, observe the output of the layer, and / or send the output of the layer. In some instances, registers 232-27, 232-28, and 232-29 may define the layers at which errors consistently occur and / or the layers at which the neural network is changed. Registers 232-30, 232-31, 232-32, and 232-33 may define the size of the temporary blocks used in the AI ​​operation and the start and end addresses of the temporary blocks used in the AI ​​operation. Register 232-30 may define the start and end addresses of the first temporary blocks used in the AI ​​operation, and register 232-33 may define the start and end addresses of the first temporary blocks used in the AI ​​operation. Registers 232-31 and 232-32 may define the size of the temporary blocks used in the AI ​​operation.

[0050] Registers 232-34, 232-35, 232-36, 232-37, 232-38, and 232-39 may be associated with activation functions used in AI operations. Register 232-34 may enable the use of an activation function block, enable the use of an activation function for each neuron, enable the use of an activation function for each layer, and enable the use of an external activation function. Register 232-35 may define the start address and end address of the location of the activation function. Registers 232-36, 232-37, 232-38, and 232-39 may define the resolution of the input (e.g., x-axis) and output (e.g., y-axis) of the activation function.

[0051] Registers 232-40, 232-41, 232-42, 232-43, and 232-44 may define the size of the offset value used in the AI ​​operation, the number of offset values ​​used in the AI ​​operation, and the start address and the end address of the offset value used in the AI ​​operation.

[0052] Registers 232-45 may provide status information for AI computations and information for debug / hold mode. Registers 232-45 may enable debug / hold mode, indicate that the AI ​​accelerator is performing an AI operation, indicate that the full capabilities of the AI ​​accelerator should be used, indicate that only matrix calculations for the AI ​​operation should be performed, and / or indicate that the AI ​​operation may continue to the next neuron and / or layer.

[0053] Registers 232-46 may provide error information about AI operations. Registers 232-46 may indicate errors in the AI ​​operation sequence, errors in the AI ​​operation algorithm, errors in the data page that cannot be corrected by ECC, and / or errors in the data page that can be corrected by ECC.

[0054] Registers 232-47 may indicate an activation function to be used in an AI operation. Registers 232-47 may indicate one of a plurality of predefined activation functions that may be used in an AI operation and / or a custom activation function located in a block that may be used in an AI operation.

[0055] Registers 232-48, 232-49, and 232-50 may indicate the neuron and / or layer on which the AI ​​operation is being performed. In the event that an error occurs during the AI ​​operation, registers 232-48, 232-49, and 232-50 may store the location of the neuron and / or layer on which the error occurred.

[0056] Figure 3A and 3B 334-5, 334-6, and 334-7, to indicate information associated with performing an AI operation. The plurality of registers may include 8 bits to store information associated with performing an AI operation; however, the plurality of registers may include any number of bits based on the size of the memory including the AI ​​accelerator.

[0057] Register 332-0 may define parameters associated with the AI ​​mode of the memory device. Bit 334-5 of register 332-0 may be a read / write bit and may indicate that, although other programming conventions may be used, for example, at the beginning of when programmed to 1b, the detailed description of the AI ​​operation may restart 360. Once the AI ​​operation restarts, bit 334-5 of register 332-0 may be reset to 0b. Bit 334-4 of register 332-0 may be a read / write bit and may indicate that, when programmed to 1b, the detailed description of the AI ​​operation may restart 361. Once the AI ​​operation starts, bit 334-4 of register 332-0 may be reset to 0b.

[0058] Bit 334-3 of register 332-0 may be a read / write bit and may indicate that when programmed to 1b, the contents of the AI ​​register are valid 362 and when programmed to 0b, the contents of the AI ​​register are invalid. Bit 334-2 of register 332-0 may be a read / write bit and may indicate that when programmed to 1b, the contents of the AI ​​register are to be cleared 363. Bit 334-1 of register 332-0 may be a read-only bit and may indicate that when programmed to 1b, the AI ​​accelerator is in use 364 and performing AI operations. Bit 334-0 of register 332-0 may be a write-only bit and may indicate that when programmed to 1b, the memory device is to exit 365 AI mode.

[0059] Registers 332-1, 332-2, 332-3, 332-4, and 332-5 may define the size of the input used in the AI ​​operation, the number of inputs used in the AI ​​operation, and the starting and ending addresses of the inputs used in the AI ​​operation. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-1 and 332-2 may define the size 366 of the input used in the AI ​​operation. The size of the input may indicate the width of the input in terms of the number of bits and / or may indicate the input type, such as floating point, integer, and / or double, among other types. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-3 and 332-4 may indicate the number 367 of the input used in the AI ​​operation. Bits 334-0, 334-1, 334-2, and 334-3 of register 332-5 may indicate a starting address 368 of a block in the memory array of inputs used in the AI ​​operation. Bits 334-0, 334-1, 334-2, and 334-3 of register 332-5 may indicate an ending address 369 of a block in the memory array of inputs used in the AI ​​operation. If the starting address 368 and the ending address 369 are the same address, only one input block is indicated for the AI ​​operation.

[0060] Registers 332-7, 332-8, 332-9, 332-10, and 332-11 may define the size of the output of the AI ​​operation, the number of outputs in the AI ​​operation, and the starting and ending addresses of the outputs of the AI ​​operation. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-7 and 332-8 may define the size 370 of the output used in the AI ​​operation. The size of the output may indicate the width of the output in terms of the number of bits and / or may indicate the type of output, such as floating point, integer, and / or double precision, among other types. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-9 and 332-10 may indicate the number 371 of the output used in the AI ​​operation. Bits 334-4, 334-5, 334-6, and 334-7 of register 332-11 may indicate a starting address 372 of a block in the memory array of outputs used in the AI ​​operation. Bits 334-0, 334-1, 334-2, and 334-3 of register 332-11 may indicate an ending address 373 of a block in the memory array of outputs used in the AI ​​operation. If the starting address 372 and the ending address 373 are the same address, only one output block is indicated for the AI ​​operation.

[0061] Register 332-12 can be used to implement input blocks, neuron blocks, output blocks, bias blocks, activation functions, and temporary blocks used during AI operations. Bit 334-0 of register 332-12 can enable input block 380, bit 334-1 of register 332-12 can enable neural network block 379, bit 334-2 of register 332-12 can enable output block 378, bit 334-3 of register 332-12 can enable bias block 377, bit 334-4 of register 332-12 can enable activation function block 376, and bits 334-5 and 334-6 of register 332-12 can enable first temporary block 375 and second temporary block 374.

[0062] Registers 332-13, 332-14, 332-15, 332-16, 332-17, 332-18, 332-19, 332-20, 332-21, 332-22, 332-23, 332-24, and 332-25 may be used to define a neural network used during an AI operation. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-13 and 332-14 may define the number of rows 381 in a matrix used in an AI operation. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-15 and 332-16 may define the number of columns 382 in the matrix used in the AI ​​operation.

[0063] Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-17 and 332-18 may define the size 383 of the neuron used in the AI ​​operation. The size of the neuron may indicate the width of the neuron in terms of the number of bits and / or may indicate the input type, such as floating point, integer, and / or double precision, among other types. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-19, 332-20, and 322-21 may indicate the number 384 of neurons of the neural network used in the AI ​​operation. Bits 334-4, 334-5, 334-6, and 334-7 of register 332-22 may indicate the starting address 385 of a block in the memory array of neurons used in the AI ​​operation. Bits 334-0, 334-1, 334-2, and 334-3 of register 332-5 may indicate an end address 386 of a block in the memory array of neurons used in the AI ​​operation. If the start address 385 and the end address 386 are the same address, only one block of neurons is indicated for the AI ​​operation. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-23, 332-24, and 322-25 may indicate a number of layers 387 of a neural network used in the AI ​​operation.

[0064] Registers 332-26 may enable a debug / hold mode of the AI ​​accelerator and enable observation of output at the layer of the AI ​​operation. For example, the controller may be configured to perform an AI operation on data stored in a plurality of memory arrays, stop the AI ​​operation, and perform a debug operation on the AI ​​operation.

[0065] The controller may indicate that the AI ​​accelerator is in debug / hold mode by programming a bit and / or multiple bits of register 332-26 to a specific state. Bit 334-0 of register 332-26 may indicate that the AI ​​accelerator is in debug / hold mode and that activation function 391 should be applied during AI operation. In some instances, bit 334-0 may be programmed to a first state to start a debug operation and may be programmed to a second state to stop and / or prevent a debug operation from occurring. For example, in response to receiving a command from a host, the controller may program bit 334-0 to a first state and / or a second state.

[0066] Bit 334-1 of register 332-26 may indicate that the AI ​​operation may proceed forward 390 in the AI ​​operation (e.g., execute the next step in the AI ​​operation). In some instances, bit 334-1 may be programmed to a first state to indicate that the AI ​​operation may proceed forward 390 in the AI ​​operation and may be programmed to a second state to indicate that the AI ​​operation may not proceed forward. The programming of bit 334-1 may be embedded in the execution of the AI ​​operation. The AI ​​operation may be stopped, the results of the AI ​​operation may be observed, errors may be corrected, and / or register 332-26 may be reprogrammed to include another step and / or remove a step in the AI ​​operation. In some instances, the previous contents at a particular step of the AI ​​operation may restore the temporary contents to view the effect of the modification on the AI ​​operation.

[0067] Bits 334-2 and 334-3 of register 232-26 may indicate that the temporary block at which the output of the layer is located is valid 388 and 389. For example, bits 334-2 and 334-3 of register 232-26 may be programmed to a first state to indicate that the temporary block is valid 388 and 389 and may be programmed to a second state to indicate that the temporary block is invalid. In some instances, a signal is sent to a host and / or controller from which data should not be read from a temporary block to indicate that the temporary block is invalid. The data in the temporary block may be changed by a controller on a host and / or memory device so that the changed data may be used in the AI ​​operation as the AI ​​operation proceeds forward.

[0068] The results of the debug operation can be stored and / or transmitted. For example, the results of the debug operation can be stored in different blocks in the multiple memory arrays and / or sent to the host. In some instances, one or more bits in bits 334-4, ..., 334-7 can be programmed to a first state to send the results of the debug operation to the host, and one or more bits in bits 334-4, ..., 334-7 can be programmed to a first state to store the results of the debug operation in different blocks in the multiple memory arrays.

[0069] Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6 and 334-7 of registers 332-27, 332-28 and 332-29 may define the layer at which the debug / hold mode will stop the 392AI operation and observe the output of the layer.

[0070] Registers 332-30, 332-31, 332-32, and 332-33 may define the size of a temporary block used in an AI operation and the start address and end address of a temporary block used in an AI operation. Bits 334-4, 334-5, 334-6, and 334-7 of register 332-30 may define a start address 393 of a first temporary block used in an AI operation. Bits 334-0, 334-1, 334-2, and 334-3 of register 332-30 may define an end address 394 of a first temporary block used in an AI operation. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-31 and 332-32 may define a size 395 of a temporary block used in an AI operation. The size of the temporary block may indicate the width of the temporary block in terms of the number of bits and / or may indicate an input type, such as floating point, integer, and / or double precision, among other types. Bits 334-4, 334-5, 334-6, and 334-7 of register 332-33 may define a starting address 396 of a second temporary block used in the AI ​​operation. Bits 334-0, 334-1, 334-2, and 334-3 of register 332-34 may define an ending address 397 of a second temporary block used in the AI ​​operation.

[0071] Registers 332-34, 332-35, 332-36, 332-37, 332-38, and 332-39 may be associated with an activation function used in an AI operation. Bit 334-0 of register 332-34 may enable use of activation function 3101. Bit 334-1 of register 332-34 may enable keeping the AI ​​at the neuron 3110 and using the activation function for each neuron. Bit 334-2 of register 332-34 may enable keeping the AI ​​at the layer 399 and using the activation function for each layer. Bit 334-3 of register 332-34 may enable use of an external activation function 398.

[0072] Bits 334-4, 334-5, 334-6, and 334-7 of register 332-35 may define the start address 3102 of the activation function block used in the AI ​​operation. Bits 334-0, 334-1, 334-2, and 334-3 of register 332-35 may define the end address 3103 of the activation function block used in the AI ​​operation. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-36 and 332-37 may define the resolution 3104 of the input (e.g., x-axis) of the activation function. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-38 and 332-39 may define a resolution 3105 of the output (e.g., y-axis) of the activation function for a given x-axis value of the custom activation function.

[0073] Registers 332-40, 332-41, 332-42, 332-43, and 332-44 may define the size of the bias value used in the AI ​​operation, the number of bias values ​​used in the AI ​​operation, and the starting and ending addresses of the bias values ​​used in the AI ​​operation. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-40 and 332-41 may define the size 3106 of the bias value used in the AI ​​operation. The size of the bias value may indicate the width of the bias value in terms of the number of bits and / or may indicate the bias value type, such as floating point, integer, and / or double precision, among other types. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-42 and 332-43 may indicate the number 3107 of offset values ​​used in the AI ​​operation. Bits 334-4, 334-5, 334-6, and 334-7 of register 332-44 may indicate the starting address 3108 of a block in the memory array of offset values ​​used in the AI ​​operation. Bits 334-0, 334-1, 334-2, and 334-3 of register 332-44 may indicate the ending address 3109 of a block in the memory array of offset values ​​used in the AI ​​operation. If the starting address 3108 and the ending address 3109 are the same address, only one offset value block is indicated for the AI ​​operation.

[0074] Register 332-45 may provide status information for AI computations and information for debug / hold mode. Bit 334-0 of register 332-45 may activate debug / hold mode 3114. Bit 334-1 of the register may indicate that the AI ​​accelerator is busy 3113 and is performing an AI operation. Bit 334-2 of register 332-45 may indicate that the AI ​​accelerator is turned on 3112 and / or that the full capabilities of the AI ​​accelerator should be used. Bit 334-3 of register 332-45 may indicate that only matrix calculations for AI operations should be performed 3111. Bit 334-4 of register 332-45 may indicate that the AI ​​operation may proceed forward 3110 and continue to the next neuron and / or layer.

[0075] Register 332-46 may provide error information about the AI ​​operation. The host may retrieve the AI ​​operation information while the AI ​​operation is being performed. Bit 334-3 of register 332-46 may indicate that there is an error 3115 in the sequence of the AI ​​operation. Bit 334-2 of register 332-46 may indicate that there is an error 3116 in the algorithm of the AI ​​operation. Bit 334-1 of register 332-46 may indicate that there is an error 3117 in the data page that cannot be corrected by the ECC. Bit 334-0 of register 332-46 may indicate that there is an error 3118 in the data page that can be corrected by the ECC.

[0076] Register 332-47 may indicate an activation function to be used in an AI operation. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, and 334-6 of register 332-47 may indicate that one of a plurality of predefined activation functions 3120 may be used in an AI operation. Bit 334-7 of register 332-47 may indicate that a custom activation function 3119 located in a block may be used in an AI operation.

[0077] Registers 332-48, 332-49, and 332-50 may indicate the neuron and / or layer on which the AI ​​operation is being performed. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-48, 332-49, and 332-50 may indicate the address of the neuron and / or layer on which the AI ​​operation is being performed. In the event that an error occurs during an AI operation, registers 332-48, 332-49, and 332-50 may indicate the neuron and / or layer on which the error occurred.

[0078] Figure 44 is a block diagram of multiple blocks of a memory device having an artificial intelligence (AI) accelerator according to multiple embodiments of the present disclosure. Input block 440 is a block in the memory array that stores input data. The data in input block 440 can be used as input for AI operations. The address of input block 440 can be in register 5 (e.g., Figure 2 Registers 232-5 and Figure 3A 332-5). Embodiments are not limited to one input block, as there may be multiple input blocks. Data input block 440 may be sent from the host to the memory device. The data may be accompanied by a command indicating that an AI operation should be performed on the memory device using the data.

[0079] Output block 442 is a block in the memory array that stores output data from AI operations. The data in output block 442 can be used to store the output from AI operations and send to the host. The address of output block 442 can be in register 11 (e.g., Figure 2 Registers 232-11 and Figure 3A The embodiment is not limited to one output block, as there may be multiple output blocks.

[0080] The data in output block 442 may be sent to the host when the AI ​​operation is completed and / or maintained. Temporary blocks 444-1 and 444-2 may be blocks in the memory array that temporarily store data while the AI ​​operation is being performed. Figure 4 The memory device includes two temporary blocks 444-1 and 444-2, but the memory device may include one or more temporary blocks. Data may be stored in temporary blocks 444-1 and 444-2 as the AI ​​operation iterates through the neurons and layers of the neural network used for the AI ​​operation. The address of temporary block 444 may be in registers 30 and 33 (e.g., Figure 2 Registers 232-30 and 232-33 in Figure 3B The embodiment is not limited to two temporary blocks, as there may be multiple temporary blocks.

[0081] The activation function block 446 may be a block stored in a memory array and / or in memory controller firmware that stores activation functions for AI operations. The activation function block 446 may store predefined activation functions and / or custom activation functions created by the host and / or the AI ​​accelerator. The address of the activation function block 446 may be in register 35 (e.g., Figure 2 Registers 232-35 and Figure 3B 332-35) in the embodiment. The embodiment is not limited to one activation function block, because there may be multiple activation function blocks.

[0082] The deviation value block 448 is a block in the memory array that stores deviation values ​​for AI operations. The address of the deviation value block 448 can be in the register 44 (e.g., Figure 2 Registers 232-44 and Figure 3B 332-44 in . The embodiment is not limited to one deviation value block, as there may be multiple deviation value blocks.

[0083] Neural network blocks 450-1, 450-2, 450-3, 450-4, 450-5, 450-6, 450-7, 450-8, 450-9, and 450-10 are blocks in the memory array that store neural networks for AI operations. Neural network blocks 450-1, 450-2, 450-3, 450-4, 450-5, 450-6, 450-7, 450-8, 450-9, and 450-10 may store information of neurons and layers used in AI operations. The addresses of neural network blocks 450-1, 450-2, 450-3, 450-4, 450-5, 450-6, 450-7, 450-8, 450-9, and 450-10 may be in registers 22 (e.g., Figure 2 Registers 232-22 and Figure 3A 332-22) as indicated in the instructions.

[0084] Figure 5 is a flow chart illustrating an example artificial intelligence process in a memory device with an artificial intelligence (AI) accelerator according to multiple embodiments of the present disclosure. In response to starting an AI operation, the AI ​​accelerator may write input data 540 and neural network data 550 into an input block and a neural network block, respectively. The AI ​​accelerator may perform an AI operation using the input data 540 and the neural network data 550. The results may be stored in temporary blocks 544-1 and 544-2. Temporary blocks 554-1 and 544-2 may be used to store data when performing matrix calculations, adding bias data, and / or applying activation functions during AI operations.

[0085] The AI ​​accelerator may receive partial results of the AI ​​operation stored in temporary blocks 544-1 and 544-2 and deviation value data 548 and perform the AI ​​operation using the partial results of the AI ​​operation and deviation value data 548. The results may be stored in temporary blocks 544-1 and 544-2.

[0086] The AI ​​accelerator may receive the partial results of the AI ​​operation stored in temporary blocks 544 - 1 and 544 - 2 and activation function data 546 and perform the AI ​​operation using the partial results of the AI ​​operation and activation function data 546. The result may be stored in output block 542.

[0087] The controller may be configured to perform AI operations on data stored in the plurality of memory arrays, stop AI operations, and perform debugging operations on AI operations. Debug operations 552, 554, 556 may occur after the AI ​​accelerator performs AI operations using input data 540 and neural network data 550, after performing AI operations using partial results of AI operations and bias value data 548, and / or after performing AI operations using partial results of AI operations and activation function data 546.

[0088] The controller may enable the device to perform a debug operation on the AI ​​operation by programming a bit and / or multiple bits of a register to a specific state. In some instances, the first bit of the register may be programmed to a first state to start a debug operation and may be programmed to a second state to stop and / or prevent the debug operation. For example, in response to receiving a command from a host, the controller may program a bit and / or multiple bits of a register to a specific state.

[0089] The register may indicate that the AI ​​operation may proceed forward in the AI ​​operation (e.g., perform the next step in the AI ​​operation). In some examples, a second bit may be programmed to a first state to indicate that the AI ​​operation may proceed forward in the AI ​​operation and may be programmed to a second state to indicate that the AI ​​operation may not proceed forward.

[0090] The register may indicate that the temporary block where the output of the layer is located is valid. The third bit and / or the fourth bit of the register may be programmed to a first state to indicate that the temporary block is valid and may be programmed to a second state to indicate that the temporary block is invalid.

[0091] The debugging operation may include storing data from the AI ​​operation in temporary blocks of the plurality of memory arrays and verifying the temporary blocks. The data in the temporary blocks (e.g., partial results from the AI ​​operation, errors and / or neural networks from the AI ​​operation, activation functions, input values, and / or bias values) may be changed by a controller on a host and / or memory device so that the changed data may be used in the AI ​​operation as the AI ​​operation proceeds.

[0092] The results of the debugging operation may also be stored in temporary blocks 544-1 and / or 544-2. The results of the debugging operation, partial results from the AI ​​operation, errors from the AI ​​operation and / or neural networks, activation functions, input values ​​and / or deviation values ​​may be stored and / or transmitted. For example, the results of the debugging operation may be stored in the output block 542 in the plurality of memory arrays and / or sent to the host. The fifth bit of the register may be programmed to a first state to send the results of the debugging operation to the host, and the sixth bit may be programmed to a first state to store the results of the debugging operation in the output block.

[0093] Fig. 6A 6 is a block diagram of an apparatus in the form of a computing system 600 including a memory device 620 having an artificial intelligence (AI) accelerator, the memory device including a memory array 625, according to various embodiments of the present disclosure. As used herein, the memory device 620, the controller 640, the memory array 625, the readout circuit system 6138, and / or the plurality of additional latches 6140 may also be individually considered to be a "device".

[0094] exist Fig. 6A In AI accelerators (e.g. Figure 1 The AI ​​accelerator 124 in the embodiment may include a readout circuit system 6138 configured to perform operations associated with AI operations such as logical operations and additional latches 6140 and a Figures 6A-9B Other components described in association with Figures 6A-9B As described in association, a memory device (eg, memory device 620 ) may be configured to perform operations associated with AI operations as part of an AI accelerator.

[0095] As used herein, additional latches are intended to refer to additional functionality (e.g., amplifiers, select logic) that senses, couples, and / or moves (e.g., reads, stores, caches) data values ​​of memory cells in an array and is distinct from Figure 6B , 7 , 8, 9A and 9B. The plurality of computing components 6148-1, ..., 6148-M and / or logic strips 6152-1, ..., 6152-N in the computing unit in the data path of the plurality of shared I / O lines 6144 shown in FIG. Fig. 6A and 6B As shown, logic strips 6152-1, ..., 6152-N in a data path of a plurality of shared input / output (I / O) lines 6144 local to the array may be associated with respective bank portions 6150-1, ..., 6150-N of memory cells in a bank 6146-1. Bank 6146-1 may be one of a plurality of banks on the memory device 620.

[0096] Fig. 6AThe system 600 in includes a host 602 coupled (e.g., connected) to a memory device 620. The host 602 can be a host system, such as a personal laptop computer, a desktop computer, a digital camera, a smart phone, or a memory card reader, as well as various other types of hosts. The host 602 can include a system motherboard and / or a backplane and can include multiple processing resources (e.g., one or more processors, microprocessors, or some other type of control circuit system). The system 600 can include separate integrated circuits, or both the host 602 and the memory device 620 can be on the same integrated circuit. The system 600 can be, for example, a server system and / or a high performance computing (HPC) system and / or a portion thereof. Although Fig. 6A The examples shown in the present disclosure illustrate a system having a Von Neumann architecture, but embodiments of the present disclosure may be implemented on a non-Von Neumann architecture, which may not include one or more components (e.g., CPU, ALU, etc.) typically associated with a Von Neumann architecture.

[0097] For clarity, system 600 has been simplified to focus on features that are particularly relevant to the present disclosure. Memory array 625 may be a DRAM array, an SRAM array, an STT RAM array, a PCRAM array, a TRAM array, an RRAM array, a NAND flash array, and / or a NOR flash array, among other types of arrays. Array 625 may include memory cells arranged in rows coupled by access lines (which may be referred to herein as word lines or select lines) and columns coupled by read lines (which may be referred to herein as data lines or digit lines). Although in Fig. 6A A single array 625 is shown in FIG. 6 , but embodiments are not limited thereto. For example, the memory device 620 may include multiple arrays 625 (eg, multiple banks of DRAM cells, NAND flash memory cells, etc.).

[0098] The memory device 620 may include an address circuit system 6525 to latch an address signal provided by the I / O circuit system 6134 via a data bus 656 (e.g., an I / O bus connected to the host 602) (e.g., provided to an external ALU circuit system and / or DRAM DQ via a local I / O line and a global I / O line). As used herein, a DRAM DQ may enable input of data to a memory bank and / or output of data from a memory bank (e.g., output from and / or input to a controller 640 and / or a host 602) via a bus (e.g., a data bus 656). During a write operation, a voltage (high = 1, low = 0) may be applied to a DQ (e.g., a pin). This voltage may be converted to an appropriate signal and stored in a selected memory cell. During a read operation, once the access is completed and the output is enabled (e.g., by enabling a low output signal), the data value read from the selected memory cell may appear at the DQ. At other times, the DQ may be in a high impedance state so that the DQ does not pull or sink current and does not present a signal to the system. As described herein, this can also reduce DQ contention when two or more devices (e.g., memory banks) share a combined data bus. Such DQs are separate and distinct from the multiple shared I / O lines 6144 ( Figure 6B middle).

[0099] For example, status and exception information may be provided from the controller 640 of the memory device 620 to the channel controller 604 via an out-of-band (OOB) bus 657, such as a high-speed interface (HSI), which in turn may be provided from the channel controller 604 to the host 602. The channel controller 604 may include logic components to allocate multiple locations in an array of each corresponding memory bank (e.g., a controller for a sub-array) to store memory bank commands, application instructions (e.g., for a sequence of operations), and arguments (PIM (processing in memory) commands) for the respective memory banks associated with the operation of each memory device in the plurality of memory devices 620. The channel controller 604 may dispatch commands (e.g., PIM commands) to the plurality of memory devices 620 to store those program instructions in a given memory bank 6146 ( Figure 6B )Inside.

[0100] The address signals are received by the address circuit system 6525 and decoded by the row decoder 6132 and the column decoder 6142 to access the memory array 625. Data can be read out (read) from the memory array 625 by sensing the voltage and / or current changes on the sense lines (digit lines) using a plurality of sense amplifiers of the sense circuit system 6138 as described herein. The sense amplifiers can read and latch a page (e.g., row) of data from the memory array 625. As described herein, additional computational circuit systems can be coupled to the sense circuit system 6138 and can be used in combination with the sense amplifiers to sense, store (e.g., cache and / or buffer), perform computational functions (e.g., operate), and / or move data. The I / O circuit system 6134 can be used to communicate bidirectional data with the host 602 via a data bus 656 (e.g., a 64-bit wide data bus). The write circuit system 6136 can be used to write data into the memory array 625.

[0101] Controller 640 (eg, Fig. 6A 625) can decode signals (e.g., commands) provided by the control bus 654 from the host 602. These signals can include chip enable signals, write enable signals, and / or address latch signals that can be used to control operations performed on the memory array 625, including data read, data storage, data movement (e.g., copying, transferring, and / or transmitting data values), data write, and / or data erase operations, as well as other operations. In various embodiments, the controller 640 can be responsible for executing instructions from the host 602 and accessing the memory array 625. The controller 640 can be a state machine, a sequencer, or some other type of controller. The controller 640 can control the shifting of data (e.g., right or left) in the rows of the array (e.g., the memory array 625) and execute microcode instructions to perform operations such as calculation operations, such as AND, OR, NOR, XOR, addition, subtraction, multiplication, division, etc.

[0102] Examples of readout circuitry 6138 are described further below (e.g., in Figures 6A-9B For example, in some embodiments, the readout circuit system 6138 can include multiple readout amplifiers and multiple computational components, which can be used as accumulators and can be used to perform operations in each sub-array (e.g., on data associated with complementary readout lines).

[0103] In some embodiments, the readout circuitry 6138 can be used to perform operations using data stored in the memory array 625 as input and participate in the movement of data to different locations in the memory array 625 for copy, transfer, transmission, write, logic and / or storage operations without transferring data through readout line address accesses (e.g., without activating column decode signals). In this way, various computational functions can use and be performed within the readout circuitry 6138 rather than being performed by (or associated with) processing resources external to the readout circuitry 6138 (e.g., by a processor associated with the host 602 and / or other processing circuitry such as ALU circuitry located on the device 620 such as on the controller 640 or elsewhere). In addition, however, computational functions are performed on data values ​​of multiple computational components 6148-1, ..., 6148-M and / or logic strips 6152-1, ..., 6152-N moved from rows of the array into computational cells in accordance with embodiments of the present disclosure. Furthermore, as an example, according to an embodiment, computational operations may be controlled in a computational cell at a speed of 2 nanoseconds (ns) without shifting data values ​​back into the rows, compared to the exemplary time of 60 nanoseconds (ns) required to fire a row in an array.

[0104] In various previous approaches, data associated with operands, for example, will be read from memory by a read circuit system and provided to an external ALU circuit system via an I / O line (e.g., via a local I / O line and / or a global I / O line). The external ALU circuit system may include multiple registers and will perform a computational function using the operands, and the results will be transmitted back to the array via the I / O circuit system 6134.

[0105] In contrast, embodiments according to the present disclosure perform computational functions on data values ​​in a plurality of computational components 6148-1, ..., 6148-M and / or logic strips 6152-1, ..., 6152-N in a computational cell in a data path local to the array that is moved from a row of the array via a plurality of shared I / O lines 6144. Additionally, the readout circuitry 6138 may be configured to perform operations on data stored in the memory array 625 and store results back to the memory array 625 without enabling I / O lines (e.g., local I / O lines) coupled to the readout circuitry 6138. However, once loaded, computational operations may be controlled faster in the computational cell, e.g., at a rate of 2 nanoseconds (ns), without moving data values ​​back into the row, compared to an example time, e.g., 60 nanoseconds (ns), required to activate a row in the array. The readout circuitry 6138 may be formed to be spaced apart from the memory cells of the array. The spacing of the plurality of computing components 6148-1, ..., 6148-M and / or logic strips 6152-1, ..., 6152-N associated with the data paths of the plurality of shared I / O lines 6144 is equal to the spacing of the data paths and is a function of the spacing of the digit lines to the memory cell array. For example, the spacing of the computing components is an integer multiple of the spacing of the digit lines to the memory cell array.

[0106] For example, the readout circuit system 6138 described herein can be formed with the same pitch as a pair of complementary readout lines (e.g., digit lines). For example, a pair of complementary memory cells can have 6F 2 The cell size of the pitch (e.g., 3F×2F), where F is the feature size. If the pitch of a pair of complementary sense lines of a complementary memory cell is 3F, then a sense circuit system with a certain pitch indicates that the sense circuit system (e.g., the sense amplifier and corresponding computing components of each corresponding pair of complementary sense lines) is formed to fit within the 3F pitch of the complementary sense lines. Similarly, the pitch of the computing components 6148-1, ..., 6148-M and / or the logic strips 6152-1, ..., 6152-N associated with the data paths of the plurality of shared I / O lines 6144 is a function of the 3F pitch of the complementary sense lines. For example, the pitch of the computing components 6148-1, ..., 6148-M and / or the logic 6152-1, ..., 6152-N will be an integer multiple of the 3F pitch of the digit line to the memory cell array.

[0107] In contrast, the circuitry of one or more processing resources (e.g., a computing engine such as an ALU) of various existing systems may not conform to the spacing rules associated with a memory array. For example, a memory cell of a memory array may have a 4F 2 or 6F 2Cell size. Thus, devices (e.g., logic gates) associated with the ALU circuitry of previous systems may not be formed to be spaced apart from the memory cells (e.g., the same spacing as the readout lines), which can affect, for example, chip size and / or memory density. As described herein, in the context of some computing systems and subsystems (e.g., central processing units (CPUs)), data may be processed at locations that are not spaced apart from the memory (e.g., memory cells in an array) and / or on the chip. For example, data may be processed by, for example, processing resources that are associated with a host rather than being spaced apart from the memory.

[0108] Thus, in various embodiments, circuitry external to the array 625 and readout circuitry 6138 is not required to perform computational functions because the readout circuitry 6138 can perform appropriate operations to perform such computational functions or can perform such operations in a data path of a plurality of shared I / O lines local to the array without the need to use external processing resources. Thus, the plurality of computational components 6148-1, ..., 6148-M and / or logic strips 6152-1, ..., 6152-N in the computational cells in the data path of the readout circuitry 6138 and / or the plurality of shared I / O lines 6144 can be used to supplement or replace such external processing resources (or at least the bandwidth consumption of such external processing resources) at least to some extent. In some embodiments, the plurality of computing components 6148-1, ..., 6148-M and / or logic strips 6152-1, ..., 6152-N in the computing unit in the data path of the plurality of shared I / O lines 6144 may be used to perform operations (e.g., to execute instructions) in addition to operations performed by an external processing resource (e.g., host 602). For example, the host 602 and / or the readout circuitry 6138 may be restricted to perform only certain operations and / or a certain number of operations.

[0109] The operations described herein may include operations associated with a device having process-in-memory (PIM) capabilities. Device operations having PIM capabilities may use bit-vector-based operations. As used herein, the term "bit vector" is intended to mean a physically contiguous number of bits on a bit vector memory device (e.g., a PIM device) that are physically contiguously stored in a row of a memory cell array. Therefore, as used herein, "bit vector operations" are intended to mean operations performed on bit vectors that are contiguous portions of a virtual address space (e.g., used by a PIM device). For example, a row of a virtual address space in a PIM device may have a bit length of 16K bits (e.g., corresponding to 16K complementary memory cell pairs in a DRAM configuration). The readout circuit system 6138 as described herein for such a 16K bit row may include corresponding 16K processing elements (e.g., computing components as described herein) formed to be spaced apart from the readout lines that are selectively coupled to the corresponding memory cells in the 16-bit row. The computational components in the PIM device may operate as a single bit processing element (PE) on a single bit of a bit vector of a row of memory cells read out by the readout circuit system 6138 (e.g., read out by a sense amplifier paired with the computational components as described herein and / or stored in the sense amplifier). Similarly, the plurality of computational components 6148-1, ..., 6148-M and / or logic strips 6152-1, ..., 6152-N in the computational units in the data path of the plurality of shared I / O lines 6144 may operate as a single bit processing element (PE) on a single bit of a bit vector of a row of memory cells read out in the array.

[0110] Enabling an I / O line may include enabling (e.g., turning on, activating) a transistor having a gate coupled to a decode signal (e.g., a column decode signal) and a source / drain coupled to the I / O line. However, embodiments are not limited to enabling I / O lines. For example, in various embodiments, readout circuitry (e.g., 6138) may be used to perform operations without enabling a column decode line of the array.

[0111] However, the plurality of shared I / O lines 6144 may be enabled to load data values ​​into the plurality of computing components 6148-1, ..., 6148-M and / or logic strips 6152-1, ..., 6152-N in a computing cell in a data path of the plurality of shared I / O lines 6144, where computing operations may be controlled more quickly. For example, computing operations may be performed at a speed of 2 nanoseconds (ns) in the plurality of computing components 6148-1, ..., 6148-M and / or logic strips 6152-1, ..., 6152-N in a computing cell. This increase in speed may be attributed to not having to utilize the associated time used to excite a row in an array, such as 60 nanoseconds (ns), to move data values ​​back into the row.

[0112] Figure 6B 6146-1 may represent an example memory bank of the memory device 620. Figure 6B As shown, the memory bank 6146-1 can include multiple main memory columns (horizontally shown as X) (e.g., 16,384 columns in the example DRAM memory bank). In addition, the memory bank 6146-1 can be divided into memory bank portions (e.g., quadrants of 32 sub-arrays) 6150-1, 6150-2, ..., 6150-N. Each memory bank portion can be associated with multiple computing components 6148-1, ..., 6148-M and logic strips 6152-1, ..., 6152-N in the computing unit in the data path of the multiple shared I / O lines 6144. Each memory bank portion in the memory bank portions 6150-1, ..., 6150-N can include multiple rows (vertically shown as Y) (e.g., each portion can be a quadrant containing 32 sub-arrays, and in the example DRAM memory bank, each sub-array can contain 512 rows). Example embodiments are not limited to the example horizontal and / or vertical orientation of the columns and rows described herein or their example numbering.

[0113] like Figure 6B As shown, the memory bank 6146-1 can be divided into a plurality of memory bank portions 6150-1, ..., 6150-N. Each memory bank portion can have associated therewith a plurality of computing components 6148-1, ..., 6148-M and logic strips 6152-1, ..., 6152-N in computing units in the data paths of the plurality of shared I / O lines 6144. The memory bank 6146-1 can include a controller 640 to direct operations on data values ​​of the plurality of computing components 6148-1, ..., 6148-M and logic strips 6152-1, ..., 6152-N in computing units in the data paths of the plurality of shared I / O lines 6144.

[0114] Figure 7 is a schematic diagram showing a readout circuit system 7138 according to various embodiments of the present disclosure. The readout circuit system 7138 may correspond to Fig. 6A Readout circuitry 6138 is shown.

[0115] like Figure 7As shown in the example embodiments of the present invention, the memory cells may include storage elements (e.g., capacitors) and access devices (e.g., transistors). For example, a first memory cell may include a transistor 7160-1 and a capacitor 7162-1, and a second memory cell may include a transistor 7160-2 and a capacitor 7162-2, etc. Although other embodiments of the configuration may be used (e.g., 2T2C with two transistors and two capacitors per memory cell), in the present embodiment, the memory array 725 is a DRAM array of 1T1B (one transistor one capacitor) memory cells. In various embodiments, the memory cells may be destructively read memory cells (e.g., reading the data stored in the cell destroys the data, causing the data originally stored in the cell to be refreshed after being read).

[0116] The cells of the memory array 725 may be arranged into rows coupled by access (word) lines 7164-X (row X), 7164-Y (row Y), etc., and columns coupled by pairs of complementary sense lines (e.g., Figure 7 The individual readout lines corresponding to each pair of complementary readout lines may also be referred to as digit lines 7168-1DIGIT(D) and 7168-2DIGIT(D)_ or digit lines 7168-2DIGIT(D)_. Figure 8 and 9A -9B. Although Figure 7 Only one pair of complementary digit lines is shown in FIG. 1 , but embodiments of the present disclosure are not limited thereto, and the memory cell array may include additional columns of memory cells and digit lines (eg, 4,096 columns, 8,192 columns, 16,384 columns, etc.).

[0117] Although the rows and columns are shown as being oriented orthogonally in planes, embodiments are not limited thereto. For example, the rows and columns may be oriented in any feasible three-dimensional configuration relative to each other. For example, the rows and columns may be oriented at any angle relative to each other, may be oriented in a substantially horizontal plane or a substantially vertical plane, and / or may be oriented in a folded topology as well as other possible three-dimensional configurations.

[0118] The memory cells can be coupled to different digit lines and word lines. For example, the first source / drain region of transistor 7160-1 can be coupled to digit line 7168-1(D), the second source / drain region of transistor 7160-1 can be coupled to capacitor 7162-1, and the gate of transistor 7160-1 can be coupled to word line 7164-Y. The first source / drain region of transistor 7160-2 can be coupled to digit line 7168-2(D)_, the second source / drain region of transistor 7160-2 can be coupled to capacitor 7162-2, and the gate of transistor 7160-2 can be coupled to word line 7164-X. Figure 7 As shown, a cell plate can be coupled to each of capacitors 7162-1 and 7162-2. The cell plate can be a common node to which a reference voltage (eg, ground) can be applied in various memory array configurations.

[0119] According to various embodiments of the present disclosure, the memory array 725 is configured to be coupled to a readout circuit system 7138. In the present embodiment, the readout circuit system 7138 includes a sense amplifier 7170 and a computational component 7148 corresponding to the memory cells of the corresponding column (e.g., coupled to the complementary bit lines of the corresponding pair). The sense amplifier 7170 can be coupled to a pair of complementary bit lines 7168-1 and 7168-2. The computational component 7148 can be coupled to the sense amplifier 7170 through transmission gates 7172-1 and 7172-2. The gates of the transmission gates 7172-1 and 7172-2 can be coupled to the operation selection logic 7178.

[0120] The operation selection logic 7178 may be configured to include a transmission gate logic for controlling a transmission gate of a pair of complementary digit lines that are not transposed between the coupling sense amplifier 7170 and the computation component 7148, and a swap gate logic for controlling a swap gate of a pair of complementary digit lines that are transposed between the coupling sense amplifier 7170 and the computation component 7148. The operation selection logic 7178 may also be coupled to the pair of complementary digit lines 7168-1 and 7168-2. The operation selection logic 7178 may be configured to control the continuity of the transmission gates 7172-1 and 7172-2 based on the selected operation.

[0121] The sense amplifier 7170 may be operated to determine the data value (eg, logic state) stored in the selected memory cell. The sense amplifier 7170 may include a cross-coupled latch, which may be referred to herein as a master latch. Figure 7 In the example shown, the circuit system corresponding to the sense amplifier 7170 includes a latch 7175, which includes four transistors coupled to a pair of complementary digital lines D 7168-1 and (D)_7168-2. However, the embodiments are not limited to this example. The latch 7175 can be a cross-coupled latch (for example, the gates of a pair of transistors such as n-channel transistors (for example, NMOS transistors) 7182-1 and 7182-2 are cross-coupled with another pair of transistors such as p-channel transistors (for example, PMOS transistors) 7184-1 and 7184-2). The cross-coupled latch 7175 including transistors 7182-1, 7182-2, 7184-1 and 7184-2 can be referred to as a master latch.

[0122] In operation, when a memory cell is being sensed (e.g., read), the voltage on one of the digit lines 7168-1(D) or 7168-2(D)_ will be slightly greater than the voltage on the other of the digit lines 7168-1(D) or 7168-2(D)_. The ACT signal and the RNL* signal may be driven low to enable (e.g., excite) the sense amplifier 7170. The digit line 7168-1(D) or 7168-2(D)_ having the lower voltage will cause one of the PMOS transistors 7184-1 or 7184-2 to conduct to a greater extent than the other of the PMOS transistors 7184-1 or 7184-2, thereby driving the digit line 7168-1(D) or 7168-2(D)_ having the higher voltage high to a greater extent than the other digit line 7168-1(D) or 7168-2(D)_ is driven high.

[0123] Similarly, the digit line 7168-1(D) or 7168-2(D)_ having the higher voltage will turn on one of the NMOS transistors 7182-1 or 7182-2 to a greater extent than the other of the NMOS transistors 7182-1 or 7182-2, thereby driving the digit line 7168-1(D) or 7168-2(D)_ having the lower voltage low to a greater extent than the other digit line 7168-1(D) or 7168-2(D)_ is driven low. As a result, after a short delay, the digit line 7168-1(D) or 7168-2(D)_ having the slightly greater voltage is driven to the supply voltage V by the pull-out transistor. CC The voltage of the other digit line 7168-1 (D) or 7168-2 (D) is driven to the voltage of the reference voltage (e.g., ground) through the sink transistor. Therefore, the cross-coupled NMOS transistors 7182-1 and 7182-2 and the PMOS transistors 7184-1 and 7184-2 are used as a sense amplifier pair that amplifies the differential voltage on the digit lines 7168-1 (D) and 7168-2 (D) and operates to latch the data value read from the selected memory cell on the node 7177-1 and / or 7177-2.

[0124] The embodiments are not limited to Figure 7 7170 configuration shown in FIG. As an example, the sense amplifier 7170 can be a current mode sense amplifier and a single-ended sense amplifier (e.g., a sense amplifier coupled to one digit line). Moreover, the embodiments of the present disclosure are not limited to the following. Figure 7 The folded digit line architecture shown is a folded digit line architecture.

[0125] The sense amplifier 7170 can be operated in conjunction with the computational component 7148 to perform various operations using data from the array as input. In various embodiments, the results of the operations can be stored back to the array without transferring the data via a digit line address access (e.g., without activating a column decode signal so that the data is transferred to the circuitry external to the array and the readout circuitry via a local I / O line). In this way, various embodiments of the present disclosure can achieve operations and computational functions associated therewith using less power than various previous methods. In addition, because various embodiments eliminate the need to transfer data over local and global I / O lines to perform computational functions (e.g., between memory and discrete processors), various embodiments can achieve increased (e.g., faster) processing capabilities compared to previous methods.

[0126] The sense amplifier 7170 can further include a balancing circuit system 7174, which can be configured to balance the digit lines 7168-1 (D) and 7168-2 (D). In this example, the balancing circuit system 7174 includes a transistor 7188 coupled between the digit lines 7168-1 (D) and 7168-2 (D). The balancing circuit system 7174 also includes transistors 7180-1 and 7180-2, each of which has a voltage coupled to a balancing voltage (e.g., V DD / 2) of the first source / drain region, where V DD is a supply voltage associated with the array. The second source / drain region of transistor 7180-1 can be coupled to digit line 7168-1(D), and the second source / drain region of transistor 7180-2 can be coupled to digit line 7168-2(D). The gates of transistors 7188, 7180-1, and 7180-2 can be coupled together and to a balance (EQ) control signal line 7186. Thus, activating EQ enables transistors 7188, 7180-1, and 7180-2, which effectively shorts digit lines 7168-1(D) and 7168-2(D) together and to a balance voltage (e.g., V CC / 2).

[0127] although Figure 7 The sense amplifier 7170 is shown to include a balancing circuit system 7174, but the embodiment is not limited thereto, and the balancing circuit system 7174 may be implemented separately from the sense amplifier 7170 to be used with the sense amplifier 7170. Figure 7 The configurations shown are implemented in different configurations or not at all.

[0128] As further described below, in various embodiments, the readout circuit system 7138 (e.g., the readout amplifier 7170 and the computational component 7148) can be operated to perform a selected operation and initially store the result in one of the readout amplifier 7170 or the computational component 7148 without transmitting data from the readout circuit system via local or global I / O lines (e.g., without performing a read line address access by activating, for example, a column decode signal).

[0129] However, further, for the embodiments described herein, a sense amplifier is provided and in some embodiments may also include a Fig. 9A and 9B The readout circuitry 7138 of the illustrated computing component may also couple memory cells from multiplexed columns of memory cells in the array to the memory cells as shown in conjunction with Fig. 9A and 9B The computing components 9148-1, ..., 9148-M and / or logic strips 9152-1, ..., 9152-N in the computing unit 9198 in the data path of the plurality of shared I / O lines 9144 of the array in question. In this way, the computing components 9148-1, ..., 9148-M and / or logic strips 9152-1, ..., 9152-N can be selected via the selection logic (in combination with Fig. 9A and 9B As discussed) is indirectly coupled to a column of memory cells through the multiple shared I / O lines 9144.

[0130] The execution of operations (e.g., Boolean logic operations involving data values) is basic and commonly used. Boolean logic operations are used in many higher-level operations. Therefore, the speed and / or power efficiency that can be achieved through improved operations can translate into the speed and / or power efficiency of higher-order functions.

[0131] like Figure 7 As shown, the computation component 7148 may also include a latch, which may be referred to herein as an auxiliary latch 7190. The auxiliary latch 7190 may be configured and operated in a manner similar to that described above with respect to the primary latch 7175, except that a pair of cross-coupled p-channel transistors (e.g., PMOS transistors) included in the auxiliary latch may have their respective sources coupled to a supply voltage 7176-2 (e.g., V DD ), and a pair of cross-coupled n-channel transistors (e.g., NMOS transistors) of the auxiliary latch can selectively couple their respective sources to a reference voltage 7176-1 (e.g., ground) so that the auxiliary latch is continuously enabled. The alignment of the computation component 748 is not limited to Figure 7 Configurations are shown, and various other embodiments are possible.

[0132] As described herein, a memory device (e.g., Fig. 6A The memory device 620 in the example may be configured to be coupled to a host (e.g., 602) via a data bus (e.g., 656) and a control bus (e.g., 654). The memory bank 6146 in the memory device 620 may include multiple memory bank portions ( Figure 6B 6150-1, ..., 6150-N in FIG. ...N). The memory bank 6146 may include multiple columns ( Figure 8 ) coupled to a plurality of array readout circuitry (e.g., Fig. 6A 6138 and Figure 7 and Figure 8 ). The readout circuitry may include a sense amplifier and a computational component (e.g., Figure 7 7170 and 7148 respectively).

[0133] Each memory bank portion 6150 can be connected to multiple shared I / O lines ( Figure 6B 6144) in a data path of a computing unit (e.g., Figure 6B 6152-0, 6152-1, ..., 6152-N-1) in the memory. A controller coupled to the memory bank (e.g., Figure 6A-6B 640) can be configured to direct data values ​​to be moved to array local shared I / O lines 8144 / 9144 ( Figure 8 and 9A ) in the data path of the computing unit 8198 / 9198 ( Figure 8 and 9A -9B) in logical strip 9152 ( Fig. 9A ) in the computing component 9148( Fig. 9A ).

[0134] The memory device may include logic strips (e.g., Figure 6B 6152 and Fig. 9A 9152), the logic strip having multiple columns ( Figure 6B ) in multiple computing components (e.g., Fig. 9A 9148-1, ..., 9148-Z). Figure 7 As further discussed, the plurality of sense amplifiers 7170 and / or computational components 7148 in the readout circuitry 7138 may be selectively coupled (e.g., via Figure 8 Column selection circuitry 8194-1 and 8194-2) to multiple shared I / O lines 8144 ( Figure 8). The column select circuitry may be configured to selectively sense data in a particular column of memory cells of the array by selectively coupling to a plurality (eg, four, eight, and sixteen, among other possibilities) of sense amplifiers and / or computational components.

[0135] In some embodiments, multiple logical stripes (e.g., Figure 6B 6152-1, ..., 6152-N) in can correspond to Figure 6B , 6150-N (e.g., a quadrant having multiple sub-arrays). A logical strip may include shared I / O lines 9144 local to array 725 ( Fig. 9A ) in the data path similar to Figure 7 The illustrated computing components 7148 include multiple computing components 9148-1, ..., 9148-Z ( Fig. 9A ). Figure 8 As shown, data values ​​read from a row of the array may be selected by column select logic via a plurality of shared I / O lines 8144 ( Figure 8 ) are moved in parallel to the plurality of shared I / O lines 8144 ( Figure 8 ) in the data path of the computing unit 8198 ( Figure 8 ) in multiple computing components 9148 ( Fig. 9A ). In some embodiments, the amount of data may correspond to at least one thousand bits of width of the plurality of shared I / O lines.

[0136] As described herein, the memory cell array may include an embodiment of a DRAM memory cell in which a controller is configured to move (e.g., copy, transfer, and / or transmit) data from a source location to a destination location via a shared I / O line in response to a command. In various embodiments, the source location may be in a first memory bank, and the destination location may be a shared I / O line 8144 ( Figure 8 ) in the data path of the computing unit 8198 ( Figure 8 )middle.

[0137] like Figure 8 As described, the device can be configured to transfer data from a source location (comprising a particular row associated with a first number of sense amplifiers and computational components (e.g., Figure 8 819) and column address) to a shared I / O line (e.g., Figure 8 In addition, the device can be configured to move data to a destination location that contains data associated with the shared I / O line 9144 ( Fig. 9A ) in the data path of the computing unit 9198 ( Fig. 9B) associated with a specific logic strip 952 ( Fig. 9A As the reader will appreciate, each shared I / O line 8144 ( Figure 8 ) may actually include complementary shared I / O line pairs (e.g., Figure 8 In some embodiments described herein, 2048 shared I / O lines (e.g., complementary pairs of shared I / O lines) may be configured as 2048-bit wide shared I / O lines. In some embodiments described herein, 1024 shared I / O lines (e.g., complementary pairs of shared I / O lines) may be configured as 1024-bit wide shared I / O lines.

[0138] Figure 8 is a schematic diagram showing a circuit system for data movement in a memory device according to various embodiments of the present disclosure. Figure 8 Eight sense amplifiers are shown (e.g., sense amplifiers 0, 1, ..., 7 at 8170-0, 8170-1, ..., 8170-7, respectively), each of which is coupled to a corresponding pair of complementary shared I / O lines 8144 (e.g., shared I / O line and shared I / O line*). Figure 8 Also shown are eight computational components (e.g., computational components 0, 1, ..., 7 at 8148-0, 8148-1, ..., 8148-7), each coupled to a corresponding sense amplifier (e.g., as shown for sense amplifier 0 at 8170-0) through corresponding transmission gates 8172-1 and 8172-2 and digit lines 8168-1 and 8168-2. For example, the transmission gates may be as shown. Figure 2 The connections shown are made and can be controlled by operating the selection signal Pass. For example, the output of the selection logic can be coupled to the gates of the transmission gates 8172-1 and 8172-2 and the digit lines 8168-1 and 8168-2. The corresponding sense amplifier and computing component pairs can help form the readout circuit system indicated at 8138-0, 8138-1, ..., 8138-7.

[0139] As combined Figure 7 As described, the data values ​​present on a pair of complementary digital bit lines 8168-1 and 8168-2 can be loaded into the computing component 8148-0. For example, when the transmission gates 8172-1 and 8172-2 are enabled, the data values ​​on the pair of complementary digital bit lines 8168-1 and 8168-2 can be transferred from the sense amplifier to the computing component (e.g., 8170-0 to 8148-0). The data values ​​on the pair of complementary digital bit lines 8168-1 and 8168-2 can be the data values ​​stored in the sense amplifier when the sense amplifier 8170-0 is activated.

[0140] Figure 8 The read amplifiers 8170-0, 8170-1, ..., 8170-7 in the Figure 7 The readout amplifier 7170 is shown in FIG. Figure 8 The computing components 8148-0, 8148-1, ..., 8148-7 shown in FIG. 8148 may each correspond to Figure 7 7148 shown in FIG. The combination of a sense amplifier and a computational component can facilitate a sense circuit system (e.g., 8138-0, 8138-1, ..., 8138-7) of a portion of a DRAM memory subarray 8145 coupled to a shared I / O line 8144 that is shared by multiple logic strips in a data path of the shared I / O line 8144.

[0141] Figure 8 The configurations of the illustrated embodiments are shown for clarity purposes, but are not limited to these configurations. For example, for sense amplifiers 8170-0, 8170-1, ..., 8170-7 in conjunction with computing components 8148-0, 8148-1, ..., 8148-7 and shared I / O lines 8144 Figure 8 The configuration shown in is not limited to half of the combination of sense amplifiers 8170-0, 8170-1, ..., 8170-7 and computing components 8148-0, 8148-1, ..., 8148-7 of the readout circuit system being formed above the column 8192 of memory cells (not shown) and half being formed below the column 8192 of memory cells. The number of such combinations of sense amplifiers and computing components that form the readout circuit system configured to be coupled to the shared I / O line is also not limited to eight. In addition, the configuration of the shared I / O line 8144 is not limited to being divided into two for coupling each of the two sets of complementary digit lines 8168-1 and 8168-2, respectively, nor is the positioning of the shared I / O line 8144 limited to being in the middle of the combination of sense amplifiers and computing components that form the readout circuit system (e.g., rather than at either end of the combination of sense amplifiers and computing components).

[0142] Figure 8 The illustrated circuitry also shows column selection circuitry 8194-1 and 8194-2 configured to implement data movement operations (e.g., such as, for example, selecting columns 8194-1 and 8194-2) with respect to a particular column 8192 of the subarray 8145, its associated complementary bit lines 8168-1 and 8168-2, and the shared I / O line 8144. Figure 6A-6BFor example, column selection circuitry 8194-1 has select lines 0, 2, 4, and 6 configured to couple to corresponding columns such as column 0 (332-0), column 2, column 4, and column 6. Column selection circuitry 8194-2 has select lines 1, 3, 5, and 7 configured to couple to corresponding columns such as column 1, column 3, column 5, and column 7. In various embodiments, in combination with Figure 8 The depicted column selection circuitry 8194 may represent at least a portion of the functionality implemented by and contained within a multiplexer, such as an eight (8) way multiplexer, a sixteen (16) way multiplexer, or the like.

[0143] The controller 840 may be coupled to the column select circuitry 8194 to control a select line (e.g., select line 0) to access data values ​​stored in a sense amplifier, a computational component, and / or present on a pair of complementary digit lines (e.g., 8168-1 and 8168-2 when select transistors 8196-1 and 8196-2 are activated by a signal from select line 0). Activation of select transistors 8196-1 and 8196-2 (e.g., as directed by the controller 540) enables coupling of the sense amplifier 8170-0, the computational component 8148-0, and / or the complementary digit lines 8168-1 and 8168-2 of column 0 (8192-0) to move the data values ​​on digit line 0 and digit line 0* to the shared I / O line 8144. For example, the data value moved may be a data value from a particular row 819 stored (cached) in the sense amplifier 8170-0 and / or the computational component 8148-0. The data value from each of columns 0 through 7 may be similarly selected by the controller activating the appropriate select transistors.

[0144] Additionally, enabling (e.g., activating) select transistors (e.g., select transistors 8196-1 and 8196-2) may enable particular sense amplifiers and / or computational components (e.g., 8170-0 and / or 8148-0, respectively) to be coupled to shared I / O line 8144 so that data values ​​stored by the amplifiers and / or computational components may be moved to (e.g., placed on and / or transferred to) shared I / O line 8144. In some embodiments, one column at a time (e.g., column 8192-0) is selected to be coupled to a particular shared I / O line 8144 to move (e.g., copy, transfer and / or transmit) stored data values. Figure 8In the example configuration of , shared I / O lines 8144 are shown as shared differential I / O line pairs (e.g., shared I / O lines and shared I / O lines *). Thus, selection of column 0 (8192-0) can result in two data values ​​(e.g., two bits of value 0 and / or 1) from a row (e.g., row 819) and / or as stored in sense amplifiers and / or computational components associated with complementary bit lines 8168-1 and 8168-2. These data values ​​can be input in parallel to each shared differential I / O pair (e.g., shared I / O and shared I / O *) of shared differential I / O lines 8144.

[0145] Fig. 9A 9152-N is a block diagram showing multiple portions 9150 of an array 925 coupled to a computing unit 9198 having multiple logic strips 9152-1, ..., 9152-N via multiple shared I / O lines 9144 in a data path local to the array 925. Fig. 9A In the example embodiment of FIG. 1 , a memory bank portion 9150 (e.g., a memory bank quadrant) having a plurality of sub-arrays 9145-1, ..., 9145-32 is shown. Fig. 9A , thirty-two (32) subarrays are shown in the memory bank quadrant 9150. However, the embodiments are not limited to this example. This example shows a memory bank portion having 16K columns multiplexed to shared I / O lines 9144 by sixteen (16) way multiplexers. Thus, the 16K columns are multiplexed to 1K shared I / O lines 9144 so that each 16 columns can provide data values ​​that can be moved in parallel as a group of 1024 (1K) bits to the computation unit 9198. Here, the shared I / O lines 9144 provide a 1K bit wide data path for the computation unit 9198.

[0146] exist Fig. 9A In the example of each logic strip 9152-1, ..., 9152-N has a plurality of computing components 9148-1, ..., 9148-Z, the plurality of computing components being combined with the present invention Figure 7The same as described in the readout circuit system 7138 of the embodiment. In some embodiments, each of the plurality of logic strips 9152-1, ..., 9152-N is configured to use the plurality of computing components 9148-1, ..., 9148-Z to perform computing functions. In some embodiments, each of the plurality of logic strips 9152-1, ..., 9152-Z can use the plurality of computing components 9148-1, ..., 9148-Z to perform different logical operations. For example, in some embodiments, at least one of the plurality of logic strips 9152-1, ..., 9152-Z can be configured to perform a long shift accelerator operation, such as an eight (8) or sixty-four (64) bit barrel shifter operation. This example can also provide partial reordering in eight (8) bit data chunks and can support gather / scatter operations in 256-bit data chunks with 8-bit crossbars. In another example, in some embodiments, at least one of the plurality of logic strips 9152-1, ..., 9152-Z may be configured to perform Kogge-Stone acceleration to generate partial carry-ahead to accelerate horizontal addition. In another example, in some embodiments, at least one of the plurality of logic strips 9152-1, ..., 9152-Z may be configured to perform "data block" math acceleration. This example may provide vertical mode acceleration in small bit groups (e.g., 4 or 8 bit data blocks). In another example, in some embodiments, the plurality of logic strips 9152-1, ..., 9152-Z may be configured to act as explicit mask registers to implement Boolean operations such as those that a compiler would use. As used herein, "data block" is intended to refer to a bit length that is smaller than the addressed data row, for example, a 256-bit data block (within a 128-byte addressable row) may be addressed to match the bit width to a particular interface. A 256-bit interface matching a 16K+ column memory array may be desirable.

[0147] According to an embodiment, the controller 540 ( Figure 5 ) can execute microcode instructions to direct the parallel movement of 1K-bit data values ​​from each multiplexed column associated with a specific access row in the multiple sub-arrays 9145-1, ..., 9145-32 to specific computing components 9148-1, ..., 9148-Z in a specific logic strip 9152-1, ..., 9152-N in a computing unit 9198.

[0148] According to some embodiments, the butterfly network 9202 may be used to connect a 1K-bit data value to corresponding computing components in the plurality of computing components 9148-1, ..., 9148-Z in corresponding logic strips in the plurality of logic strips 9152-1, ..., 9152-N. By way of example and not by way of limitation, the 1K-bit data value may be moved in parallel to logic strips associated with each of the 32 subarrays 9145-1, ..., 9145-32 in each of the 4 quadrants of the memory portion 9150. In this example, each of the 128 logic strips 9152-1, ..., 9152-N having 1K computing components 9148-1, ..., 9148-Z may be included in the computing unit 9198. The computing unit 9198 may be configured to receive the 1K-bit data value from the controller 640 ( Fig. 6A ) to operate the data values ​​of the plurality of computing components 9148-1, ..., 9148-Z loaded into the logic strips 9152-1, ..., 9152-N of the computing unit 9198, so as to be combined with the present invention Figure 7 The readout circuit system 7138 of the same as described above performs operations on the data values, such as AND, OR, NOR, XOR, addition, subtraction, multiplication, division, etc. As noted above, once the data values ​​are loaded into the calculation unit 9198, according to the controller 640 ( Fig. 6A ) can control computing operations in the computing unit faster, for example, at a speed of about 2 nanoseconds (ns), without having to move data values ​​back to array 525 ( Figure 5 ) in the row. For example, the computing unit 9198 can be used to compare the rows ( Fig. 6A ) and access it, for example, approximately 60 nanoseconds (ns) to perform computing operations much faster.

[0149] exist Fig. 9A In an example embodiment, the spacing of the plurality of computing components 9148-1, ..., 9148-Z and / or logic strips 9152-1, ..., 9152-N in the computing unit 9198 in the data path of the plurality of shared I / O lines 9144 is equal to the spacing of the data path of the shared I / O lines. According to an embodiment, the spacing of the data path is the spacing of the digit line to the array 525 of the memory cell ( Figure 5 ), such as a multiple (2x, 4x, etc.). For example, the spacing of the plurality of computing components 9148-1, ..., 9148-Z and / or logic 9152-1, ..., 9152-N is an integer multiple of the spacing from the digit line to the memory cell array.

[0150] Fig. 9Bis a block diagram example showing multiple arrays coupled to multiple computing components in a computing unit via multiple shared I / O lines in data paths local to the array, wherein the spacing of the computing components is equal to the spacing of the data paths of the shared I / O lines and the spacing is a multiple of the spacing of the digit lines to the array. Fig. 9B The example shows that multiple arrays can have memory cells accessed via digit lines 9168, such as memory bank quadrants 9150-1, 9150-2, portions of an array, etc.

[0151] exist Fig. 9B In the example of FIG. 1 , memory bank quadrants 9150-1 and 9150-2 are shown with a pitch of approximately sixteen thousand (16K) digit lines 9168 wide, based on a given feature size (design rule) of the digit line manufacturing process. Also shown are a plurality of shared I / O lines 9144-1, 9144-2, ..., 9144-Z, which may have different pitches that are a function of, for example, a multiple of, a given feature size (design rule) of the digit line manufacturing process. Fig. 9B In the example of FIG. 1 , the data paths of the plurality of shared I / O lines 9144-1, 9144-Z have a pitch that is approximately sixteen (16) times greater than the pitch of the digit lines 9168. Thus, in this example, approximately one thousand (1K) shared I / O lines 9144-1, ..., 9144-Z are shown multiplexed to 16K digit lines 9168 via a 16:1 multiplexer, such as 9204-1, ..., 9204-Z and 9206-1, ..., 9206-Z, respectively. However, embodiments are not limited to the numerical examples provided herein, and more or fewer digit lines 9168 may be multiplexed to the plurality of shared I / O lines 9144-1, ..., 9144-Z. For example, the pitch of shared I / O lines 9144-1, ..., 9144-Z may be a multiple other than 16 times (eg, 16x) the pitch of digit lines 9168 as set by a given feature size (design rules) of the digit line fabrication process.

[0152] like Fig. 9B As shown in the example of FIG. 1 , for example, a computing component in a plurality of computing components 9148-1, ..., 9148-Z and 9149-1, ..., 9149-Z can be associated with each shared I / O line 9144-1, ..., 9144-Z, respectively. The plurality of computing components 9148-1, ..., 9148-Z and 9149-1, ..., 9149-Z can be in a plurality of logical strips of computing units shown as 9198-1 and 9198-2, respectively, for example Fig. 9A As shown in 9152-1, 9152-2, ..., 9152-N. Fig. 9BAs shown in the example, for example, the spacing of the computing components in the multiple computing components 9148-1, ..., 9148-Z and 9149-1, ..., 9149-Z associated with each shared I / O line 9144-1, ..., 9144-Z can be equal to the spacing of the data paths of the shared I / O lines 9144-1, ..., 9144-Z, and therefore sixteen times (e.g., 16x) the spacing of the digital lines 9168 of the array, such as 9150-1 and 9150-2. According to various embodiments, because the computing components 9148-1, ..., 9148-Z and 9149-1, ..., 9149-Z in the data path of the shared I / O lines 9144-1, ..., 9144-Z are not restricted to a one-to-one (e.g., 1x) relationship with the pitch of the digit lines 8168, the computing components 9148-1, ..., 9148-Z and 9149-1, ..., 9149-Z are not restricted to the "vertical" alignment of the arrays 9150-1 and 9150-2, and in the present example can be sixteen times (16x) larger. In this way, the computing components 9148-1, ..., 9148-Z and 9149-1, ..., 9149-Z in the data path of the shared I / O lines 9144-1, ..., 9144-Z can be used to perform a more robust set of logical operations on the data values ​​stored therein (e.g., by having a larger footprint and space), such as the long shift acceleration mentioned above, while remaining close to the arrays 9150-1 and 9150-1 and not straying in peripheral areas of the array or memory die.

[0153] Although specific embodiments have been shown and described herein, it will be appreciated by those skilled in the art that arrangements calculated to achieve the same results may replace the specific embodiments shown. The present disclosure is intended to cover the adaptation or variation of the various embodiments of the present disclosure. It should be understood that the above description is carried out in an illustrative and non-restrictive manner. Reviewing the above description, the combination of the above embodiments and other embodiments not specifically described herein will be apparent to those skilled in the art. The scope of the various embodiments of the present disclosure includes other applications in which the above structures and methods are used. Therefore, the scope of the various embodiments of the present disclosure should be determined with reference to the attached claims together with the full scope of equivalents to which such claims are assigned.

[0154] In the foregoing detailed description, various features are grouped together in a single embodiment for the purpose of streamlining the present disclosure. This approach of the present disclosure should not be interpreted as reflecting an intention that the disclosed embodiments of the present disclosure must use more features than those explicitly stated in each claim. On the contrary, as reflected in the following claims, the inventive subject matter lies in less than all the features of a single disclosed embodiment. Therefore, the following claims are hereby incorporated into the detailed description, wherein each claim represents a separate embodiment on its own.

Claims

1. A device comprising: Artificial intelligence AI accelerator; a plurality of memory arrays coupled to the AI ​​accelerator; a plurality of registers coupled to the AI ​​accelerator; as well as a controller coupled to the AI ​​accelerator, wherein the controller is configured to: performing an AI operation via the AI ​​accelerator using at least one of the activation function data, a partial result of the AI ​​operation, or the deviation value data; In response to an error occurring, storing a location of a layer of the neural network where an error occurred during the AI ​​operation in a first register of the plurality of registers; stopping the AI ​​operation at the layer of the neural network where the error occurs in response to a particular bit of a second register of the plurality of registers being programmed to a particular state; and A debug operation is performed on the AI ​​operation in response to a specific bit of a third register of the plurality of registers being programmed to a specific state.

2. The apparatus of claim 1, wherein the controller is configured to perform the AI ​​operation on data stored in the plurality of memory arrays.

3. The apparatus of claim 1, wherein the controller is configured to perform the debug operation to identify an error of the AI ​​operation.

4. The apparatus of claim 1, wherein the controller is configured to store data in a temporary memory block of the plurality of memory arrays in response to a particular bit of a fourth register being programmed to a particular state.

5. The apparatus of claim 4, wherein the data comprises identified errors from the debug operation.

6. A device comprising: Artificial intelligence AI accelerator; a plurality of memory arrays coupled to the AI ​​accelerator; a plurality of registers coupled to the AI ​​accelerator; as well as a controller coupled to the AI ​​accelerator, wherein the controller is configured to: performing an AI operation via the AI ​​accelerator using at least one of the activation function data, a partial result of the AI ​​operation, or the deviation value data; In response to an error occurring, storing a location of a layer of the neural network where an error occurred during the AI ​​operation in a first register of the plurality of registers; stopping the AI ​​operation at the layer of the neural network where the error occurred; performing a debug operation on the AI ​​operation to identify an error of the AI ​​operation in response to a specific bit of a second register of the plurality of registers being programmed to a specific state; storing data in a temporary memory block of the plurality of memory arrays in response to a particular bit of a third register of the plurality of registers being programmed to a particular state; as well as The data stored in the temporary memory block is changed to correct the identified error.

7. The apparatus of claim 6, wherein the controller is configured to stop the AI ​​operation in response to a different bit of the second register of the plurality of registers being programmed to a particular state.

8. The apparatus of claim 7, wherein the controller is configured to continue the AI ​​operation in response to the different bit of the second register being programmed to a different state.

9. The apparatus of claim 8, wherein the controller is configured to continue the AI ​​operation using the changed data.

10. The apparatus of claim 6, wherein the controller is configured to verify the temporary memory block.

11. The apparatus of claim 6, wherein the controller is configured to send a result of the debug operation.

12. A method comprising: Receive debug commands from the host; programming a specific bit of a first register of a plurality of registers to a specific state in response to receiving the debug command from the host; performing an artificial intelligence (AI) operation via an AI accelerator coupled to the plurality of registers using at least one of the activation function data, the partial result of the AI ​​operation, or the deviation value data; In response to the error occurring, storing a location of a layer of the neural network where the error occurred during the AI ​​operation in a second register of the plurality of registers; stopping the AI ​​operation at the layer of the neural network where the error occurred; performing a debug operation on the AI ​​operation in response to the specific bit of the first register being programmed to the specific state; programming a specific bit of a third register of the plurality of registers to a specific state; and Data is stored in a temporary memory block in response to the particular bit of the third register being programmed to the particular state.

13. The method of claim 12, comprising performing the AI ​​operation on data stored in a plurality of memory arrays.

14. The method of claim 13, comprising using at least one of input data or neural network data to perform the AI ​​operation.

15. The method of claim 12, comprising programming different bits of the first register to different states.

16. The method of claim 15, comprising indicating that the AI ​​operation is to proceed in response to programming the different bit of the first register to the different state to continue the AI ​​operation.

17. The method of claim 15, comprising indicating that the temporary memory block is valid in response to programming the different bits of the first register to the different state.

18. The method of claim 15, comprising storing a result of the debug operation in response to programming the different bits of the first register to the different state.

19. The method of claim 15, comprising sending a result of the debug operation in response to programming the different bits of the first register to the different state.

Images