Activation function for artificial intelligence operations

Abstract

The present disclosure includes devices and methods related to defining activation functions for artificial intelligence (AI) operations. An example device can include a number of memory arrays and a controller, where the controller includes a number of activation function registers, where the number of activation function registers define activation functions for artificial intelligence (AI) operations performed by the device.

Description

Technical Field

[0001] This disclosure generally relates to memory devices, and more specifically, to apparatus and methods for defining activation functions for artificial intelligence (AI) operations. Background Technology

[0002] Memory devices are typically provided as internal semiconductor integrated circuits in computers or other electronic devices. Many different types of memory exist, including volatile and non-volatile memory. Volatile memory may require power to maintain its data and includes random access memory (RAM), dynamic random access memory (DRAM), and synchronous dynamic random access memory (SDRAM), among others. Non-volatile memory provides permanent data by retaining the stored data when no power is supplied and can include NAND flash memory, NOR flash memory, read-only memory (ROM), electrically erasable programmable ROM (EEPROM), erasable programmable ROM (EPROM), and resistive variable memory, such as phase-change random access memory (PCRAM), resistive random access memory (RRAM), and magnetoresistive random access memory (MRAM), among others.

[0003] Memory is also used as a volatile and non-volatile data storage device for a wide range of electronic applications. Non-volatile memory can be used in, for example, personal computers, portable memory sticks, digital cameras, cellular phones, portable music players such as MP3 players and video players, and other electronic devices. Memory cells can be arranged in arrays, wherein the array is used in the memory device.

[0004] The memory may be a portion of a memory module (e.g., a dual in-line memory module (DIMM)) used in a computing device. The memory module may include volatile memory such as DRAM and / or non-volatile memory such as flash memory or RRAM. The DIMM may be used as the main memory in a computing system. Attached Figure Description

[0005] Figure 1 This is a block diagram of a device in the form of a computing system including a memory device with an artificial intelligence (AI) accelerator, according to several embodiments of the present disclosure.

[0006] Figure 2 This is a block diagram of several registers on a memory device with an artificial intelligence (AI) accelerator according to several embodiments of the present disclosure.

[0007] Figure 3A and 3B This is a block diagram of several bits in several registers on a memory device with an artificial intelligence (AI) accelerator according to several embodiments of the present disclosure.

[0008] Figure 4 This is a block diagram of several blocks of a memory device having an artificial intelligence (AI) accelerator according to several embodiments of the present disclosure.

[0009] Figure 5 A flowchart illustrating an example artificial intelligence process in a memory device with an artificial intelligence (AI) accelerator according to several embodiments of the present disclosure.

[0010] Figure 6-1 and Figure 6-2 A table illustrating several activation functions for artificial intelligence (AI) operations according to several embodiments of the present disclosure.

[0011] Figure 7A This is a block diagram of a device in the form of a computing system including a memory device with an artificial intelligence (AI) accelerator, according to several embodiments of the present disclosure.

[0012] Figure 7B This is a block diagram of a computing system in the form of a memory device with an artificial intelligence (AI) accelerator, which includes shared input / output (I / O) lines in a local data path of a group segment of an array, according to several embodiments of the present disclosure.

[0013] Figure 8 The schematic diagram illustrates a sensing circuit system of a memory device according to several embodiments of the present disclosure, the sensing circuit system including a computing component.

[0014] Figure 9 This is a schematic diagram illustrating a circuit system for multiple shared I / O lines in a data path for an array according to several embodiments of the present disclosure.

[0015] Figure 10A This is a block diagram example illustrating multiple segments of an array that are coupled to computing units with multiple logic bars via multiple shared I / O lines in the array's local data path.

[0016] Figure 10B This is a block diagram example illustrating multiple arrays coupled to multiple computing components in a computing unit via multiple shared I / O lines in the array's local data path, wherein the computing components have a spacing equal to the spacing of the data path of the shared I / O lines and a multiple of the spacing between the digital lines and the array. Detailed Implementation

[0017] This disclosure includes devices and methods relating to defining and selecting activation functions for artificial intelligence (AI) operations. Example devices may include a plurality of memory arrays and a controller including a plurality of activation function registers, wherein the plurality of activation function registers define activation functions for AI operations performed by the device.

[0018] In several embodiments, the activation function may be a predefined activation function and / or a user-defined activation function. Predefined activation functions may include, but are not limited to, identity functions (e.g., identity relations, identity mappings, and / or identity transformations), binary step functions, logic functions, hyperbolic tangent functions (e.g., tanH), arctangent functions (e.g., ArcTan), soft sign functions, inverse square root unit (ISRU) functions, modified linear unit (ReLU) functions, leaked modified linear unit (Leaked ReLU) functions, parameterized modified linear unit (PReLU) functions, randomized leaked modified linear unit (RReLU) functions, exponential linear unit (ELU) functions, and / or scaled exponential linear unit (SELU) functions, as well as other functions.

[0019] AI operations may include updating, changing, and / or generating activation functions (e.g., custom activation functions). Custom activation functions can be generated based on the results of a previous AI operation. For example, a custom activation function could be based on the results of a debugging operation. Choosing a custom activation function can improve the functionality and efficiency of AI operations.

[0020] In some instances, the device may be a memory device of a computing system. An activation function register on the memory device reduces latency and power consumption when defining activation functions for AI operations, compared to a host defining activation functions for the memory device. Host-defined activation functions are exchanged between the memory device and the host, increasing latency and power consumption in the process of selecting and using activation functions during AI operations. While activation functions defined according to this disclosure can be executed on the memory device, data is not transferred from the memory device. For example, the memory device can process data and select the activation function to be executed based on the processed data, without sending data to the host or receiving activation functions from the host.

[0021] In the following detailed description of this disclosure, reference is made to the accompanying drawings, which form a part of this disclosure, and several embodiments of this disclosure are illustrated by way of illustration. These embodiments are described in sufficient detail to enable those skilled in the art to practice embodiments of this disclosure, and it should be understood that other embodiments may be utilized and process, electrical, and / or structural changes may be made without departing from the scope of this disclosure. As used herein, the designation “N” indicates that several specific features so designated may be included with several embodiments of this disclosure.

[0022] As used herein, “several things” can refer to one or more of such things. For example, “several memory devices” can refer to one or more memory devices. Additionally, the designation “N” as used herein, especially with respect to reference numerals in the figures, indicates that several specific features such a designation may be included with several embodiments of this disclosure.

[0023] The figures in this document follow a numbering convention, wherein the first one or more digits correspond to the figure number, and the remaining digits identify elements or components in the figure. Similar elements or components between different figures may be identified by using similar digits. It should be understood that elements shown in the various embodiments herein may be added, interchanged, and / or excluded to provide several additional embodiments of this disclosure. Furthermore, the scale and relative dimensions of the elements provided in the figures are intended to illustrate various embodiments of this disclosure and are not intended to be limiting.

[0024] Figure 1 This is a block diagram of a device in the form of a computing system 100 including a memory device 120, according to several embodiments of the present disclosure. As used herein, the memory device 120, memory arrays 125-1...125-N, memory controller 122, and / or AI accelerator 124 may also be individually considered as a “device”.

[0025] like Figure 1 As described herein, host 102 may be coupled to memory device 120. Host 102 may be a laptop computer, personal computer, digital camera, digital recording and playback device, mobile phone, PDA, memory card reader, interface hub, and other host system, and may include memory access device, such as a processor. Those skilled in the art will understand that "processor" may be one or more processors, such as a parallel processing system, several coprocessors, etc.

[0026] Host 102 includes host controller 104 for communication with memory device 120. Host controller 104 can send commands to memory device 120. Host controller 104 can communicate with memory device 120, memory controller 122 on memory device 120, and / or AI accelerator 124 on memory device 120 to perform AI operations, read data, write data and / or erase data, and other operations. AI accelerator 124 may also include... Figures 7A to 10BThe associated description refers to components configured to perform AI operations. AI operations may include machine learning or neural network operations, which may include training or inference operations, or both. In one instance, each memory device 120 may represent a layer within a neural network or deep neural network (e.g., a network with three or more hidden layers). Alternatively, each memory device 120 may be or include a node of a neural network, and layers of the neural network may be composed of multiple memory devices or portions of several memory devices 120. Memory devices 120 may store weights (or models) used for AI operations in memory array 125.

[0027] The physical host interface provides an interface for transmitting control, address, data, and other signals between the memory device 120 and the host 102, which has a compatible receiver for the physical host interface. For example, signals may be transmitted between the host 102 and the memory device 120 on several buses (e.g., a data bus and / or an address bus).

[0028] Memory device 120 may include controller 122, AI accelerator 124, and memory arrays 125-1…125-N. Memory device 120 may be a low-power double data rate dynamic random access memory (DRAM), such as an LPDDR5 device, and / or graphics double data rate DRAM, such as GDDR6, GDDR7, LPDDR6, DDR4, and other types of devices. Memory arrays 125-1…125-N may include several memory cells, such as volatile memory cells (e.g., DRAM memory cells, and other types of volatile memory cells) and / or non-volatile memory cells (e.g., RRAM memory cells, and other types of non-volatile memory cells). Memory device 120 may read data and / or write data to memory arrays 125-1…125-N. Memory arrays 125-1…125-N may store data used during AI operations performed on memory device 120. The memory arrays 125-1…125-N can store the inputs, outputs, weight matrices, and bias information of the neural network, and / or activation function information used by the AI ​​accelerator to perform AI operations on the memory device 120.

[0029] The host controller 104, the memory controller 122 on the memory device 120, and / or the AI ​​accelerator 124 may include control circuitry, such as hardware, firmware, and / or software. In one or more embodiments, the host controller 104, the memory controller 122, and / or the AI ​​accelerator 124 may be application-specific integrated circuits (ASICs) coupled to a printed circuit board including a physical interface register 130, which may be located on the memory controller 122, the AI ​​accelerator 124, and / or in memory arrays 125-1…125-N and accessible by the controller 122. Furthermore, the memory controller 122 on the memory device 120 may include register 130. Register 130 is programmable to provide information for the AI ​​accelerator to perform AI operations. Register 130 may include any number of registers. Register 130 can be written to and / or read from the host 102, the memory controller 122, and / or the AI ​​accelerator 124. Register 130 can provide input and output biases, neural network, and / or activation function information for the AI ​​accelerator 124. Register 130 may include a mode register 131 to select an operating mode for memory device 120. For example, an AI operating mode can be selected by writing a word to register 131 (e.g., 0xAA and / or 0x2AA), which disables access to registers associated with normal operation of memory device 120 and allows access to registers associated with AI operation. Furthermore, a signature can be used to select the AI ​​operating mode, the signature employing a cryptographic algorithm verified by a key stored in memory device 120.

[0030] AI accelerator 124 may include hardware 126 and / or software / firmware 128 to perform AI operations. Furthermore, AI accelerator 124 may also include... Figures 7A to 10B The associated components are configured to perform AI operations. Hardware 126 may include adder / multiplier 127 to perform logical operations associated with the AI ​​operation. Memory controller 122 and / or AI accelerator 124 may receive commands from host 102 to perform AI operations. Memory device 120 may use data in AI accelerator 124, memory arrays 125-1…125-N, and information in registers 130 to perform the AI ​​operation requested in the command from host 102. The memory device may report information such as results and / or error information of the AI ​​operation back to host 102. The AI ​​operation performed by AI accelerator 124 may be performed without using external processing resources.

[0031] Memory arrays 125-1…125-N may provide main memory for a memory system or may be used as additional memory or storage devices throughout the memory system. Each memory array 125-1…125-N may include several blocks of memory cells. These memory cell blocks may be used to store data used during AI operations performed by memory device 120. Memory arrays 125-1…125-N may include, for example, DRAM memory cells. Embodiments are not limited to a specific type of memory device. For example, memory devices may include RAM, ROM, DRAM, SDRAM, PCRAM, RRAM, 3D XPoint, and flash memory, etc.

[0032] With the aid of examples, memory device 120 can perform AI operations as or including one or more inference steps. Memory array 125 can be a layer of a neural network or each can be an individual node, and memory device 120 can be a layer; or memory device 120 can be a node within a larger network. Alternatively, memory array 125 can store data or weights, or both, to be used (e.g., summed) within a node. Each node (e.g., memory array 125) can combine inputs from data read from cells of the same or different memory arrays 125 with weights read from cells of memory array 125. For example, adder / multiplier 127 can be used to sum the combination of weights and data within the periphery of memory array 125 or within hardware 126. In these cases, the result of the summation can be passed to an activation function represented or instantiated within the periphery of memory array 125 or within hardware 126. The results can be passed to another memory device 120, or used within the AI ​​accelerator 124 (e.g., by software / firmware 128) to make decisions or train the network including the memory device 120.

[0033] The network employing memory device 120 may be capable of being used for supervised or unsupervised learning. This can be combined with other learning or training schemes. In some cases, a trained network or model is imported into or used with memory device 120, and the operation of memory device 120 is primarily or exclusively related to inference.

[0034] Figure 1 Embodiments may include additional circuitry not described to avoid obscuring the embodiments of this disclosure. For example, memory device 120 may include address circuitry to latch address signals provided on I / O connections via I / O circuitry. Address signals can be received and decoded by row decoders and column decoders to access memory arrays 125-1…125-N. Those skilled in the art will understand that the number of address input connections may depend on the density and architecture of the memory arrays 125-1…125-N.

[0035] Figure 2 This is a block diagram of several registers on a memory device having an artificial intelligence (AI) accelerator according to several embodiments of the present disclosure. Register 230 may be an AI register and includes input information, output information, neural network information and / or activation function information, and other types of information, for use by the AI ​​accelerator, controller, and / or memory array of the memory device (e.g., Figure 1 The AI ​​accelerator 124, memory controller 122, and / or memory array 125-1…125-N are used. Registers can be based on those from the host, AI accelerator, and / or controller (e.g., Figure 1 The host 102, AI accelerator 124, and memory controller 122 in the system read and / or write according to commands.

[0036] Register 232-0 can define parameters associated with the AI ​​mode of the memory device. Bits in register 232-0 can initiate AI operation, restart AI operation, indicate that the contents of the register are valid, clear the contents of the register, and / or exit AI mode.

[0037] Registers 232-1, 232-2, 232-3, 232-4, and 232-5 can limit the size of the inputs used for AI operations, the number of inputs used for AI operations, and the start and end addresses of the inputs used for AI operations. Registers 232-7, 232-8, 232-9, 232-10, and 232-11 can limit the size of the outputs of AI operations, the number of outputs in AI operations, and the start and end addresses of the outputs of AI operations.

[0038] Register 232-12 can be used to enable the use of input blocks, neuron blocks, output blocks, bias blocks, activation functions, and temporary blocks during AI operations.

[0039] Registers 232-13, 232-14, 232-15, 232-16, 232-17, 232-18, 232-19, 232-20, 232-21, 232-22, 232-23, 232-24, and 232-25 can be used to define the neural network used during AI operation. Registers 232-13, 232-14, 232-15, 232-16, 232-17, 232-18, 232-19, 232-20, 232-21, 232-22, 232-23, 232-24, and 232-25 can define the size, number, and location of neurons and / or layers of the neural network used during AI operation.

[0040] Registers 232-26 enable the debug / hold mode of the AI ​​accelerator and the output to be observed at the layer during AI operation. Registers 232-26 can indicate that activation should be applied during AI operation, and that the AI ​​operation can be progressively advanced within the AI ​​operation (e.g., performing the next step in the AI ​​operation). Registers 232-26 can indicate that a temporary block containing the layer's output is valid. Data in the temporary block can be changed by a controller on the host and / or memory device, such that the changed data can be used in the AI ​​operation as it is progressively advanced. Registers 232-27, 232-28, and 232-29 can define the layer, wherein the debug / hold mode will stop the AI ​​operation, change the contents of the neural network, and / or observe the output of the layer.

[0041] Registers 232-30, 232-31, 232-32, and 232-33 can define the size of the temporary block used for AI operations, as well as the start and end addresses of the temporary block used for AI operations. Register 232-30 can define the start and end addresses of the first temporary block used for AI operations, and register 232-33 can define the start and end addresses of the first temporary block used for AI operations. Registers 232-31 and 232-32 can define the size of the temporary block used for AI operations.

[0042] Registers 232-34, 232-35, 232-36, 232-37, 232-38, and 232-39 can be associated with activation functions used for AI operations. Register 232-34 enables the use of activation function blocks, activation functions for each neuron, activation functions for each layer, and external activation functions. Register 232-35 specifies the start and end addresses of the activation function's location. Registers 232-36, 232-37, 232-38, and 232-39 specify the resolution of the input (e.g., x-axis) and output (e.g., y-axis) of the activation function and / or custom activation functions.

[0043] Registers 232-40, 232-41, 232-42, 232-43, and 232-44 can limit the size of the bias value used for AI operations, the number of bias values ​​used for AI operations, and the start and end addresses of the bias values ​​used for AI operations.

[0044] Registers 232-45 provide status information for AI computation and information for debug / hold mode. Registers 232-45 can enable debug / hold mode, indicate that the AI ​​accelerator is performing an AI operation, indicate that the full capabilities of the AI ​​accelerator should be used, indicate that only matrix computations for the AI ​​operation should be performed, and / or indicate that the AI ​​operation can continue to the next neuron and / or layer.

[0045] Registers 232-46 can provide error information about AI operations. Registers 232-46 can indicate that there is an error in the sequence of AI operations, an error in the algorithm of AI operations, an error in a data page that ECC cannot correct, and / or an error in a data page that ECC can correct.

[0046] Registers 232-47 can indicate the activation function used in AI operations. Registers 232-47 can indicate one of several predefined activation functions that can be used for AI operations, and / or a custom activation function located in the block that can be used for AI operations.

[0047] Registers 232-48, 232-49, and 232-50 can indicate the neuron and / or layer performing the AI ​​operation. In the event of an error during the AI ​​operation, registers 232-48, 232-49, and 232-50 can store the location of the neuron and / or layer where the error occurred.

[0048] Figure 3A and 3B This is a block diagram of several bits in several registers on a memory device with an artificial intelligence (AI) accelerator according to several embodiments of the present disclosure. Each register 332-0…332-50 may include several bits, bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7, to indicate information associated with performing AI operations. The several registers may include 8 bits storing information associated with performing AI operations; however, the several registers may include any number of bits based on the size of the memory including the AI ​​accelerator.

[0049] Register 332-0 can define parameters associated with the AI ​​mode of the memory device. Bit 334-5 of register 332-0 can be a read / write bit and can indicate that the refinement of the AI ​​operation can be restarted 360 when programmed to, for example, 1b, but other programming conventions can be used. Once the AI ​​operation has been restarted, bit 334-5 of register 332-0 can be reset to 0b. Bit 334-4 of register 332-0 can be a read / write bit and can indicate that the refinement of the AI ​​operation can be started 361 when programmed to 1b. Once the AI ​​operation has been started, bit 334-4 of register 332-0 can be reset to 0b.

[0050] Bit 334-3 of register 332-0 can be a read / write bit, indicating that the contents of the AI ​​register are valid when programmed to 1b (362) and invalid when programmed to 0b. Bit 334-2 of register 332-0 can be a read / write bit, indicating that the contents of the AI ​​register will be cleared when programmed to 1b (363). Bit 334-1 of register 332-0 can be a read-only bit, indicating that the AI ​​accelerator is in use and performing AI operations when programmed to 1b (363). Bit 334-0 of register 332-0 can be a write-only bit, indicating that the memory device will exit AI mode when programmed to 1b (365).

[0051] Registers 332-1, 332-2, 332-3, 332-4, and 332-5 can define the size of the input used for AI operations, the number of inputs used for AI operations, and the start and end addresses of the inputs used for AI operations. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-1 and 332-2 can define the size 366 of the input used for AI operations. The size of the input can indicate the width of the input based on the number of bits and / or the type of the input (e.g., floating-point, integer, and / or double, and other types). Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-3 and 332-4 can indicate the number 367 of the inputs used for AI operations. Bits 334-4, 334-5, 334-6, and 334-7 of register 332-5 indicate the start address 368 of the block in the memory array used as input for AI operations. Bits 334-0, 334-1, 334-2, and 334-3 of register 332-5 indicate the end address 369 of the block in the memory array used as input for AI operations. If the start address 368 and the end address 369 are the same, then only one block of input is indicated for the AI ​​operation.

[0052] Registers 332-7, 332-8, 332-9, 332-10, and 332-11 can define the size of the output of the AI ​​operation, the number of outputs in the AI ​​operation, and the start and end addresses of the outputs of the AI ​​operation. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-7 and 332-8 can define the size 370 of the output used for the AI ​​operation. The size of the output can indicate the width of the output based on the number of bits and / or the type of the output (e.g., floating-point, integer, and / or double, and other types). Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-9 and 332-10 can indicate the number 371 of the outputs used for the AI ​​operation. Bits 334-4, 334-5, 334-6, and 334-7 of register 332-11 indicate the start address 372 of the block in the memory array for the output of the AI ​​operation. Bits 334-0, 334-1, 334-2, and 334-3 of register 332-11 indicate the end address 373 of the block in the memory array for the output of the AI ​​operation. If the start address 372 and the end address 373 are the same, then only one block of output is indicated for the AI ​​operation.

[0053] Registers 332-12 can be used to enable the use of input blocks, neuron blocks, output blocks, bias blocks, activation functions, and temporary blocks during AI operations. Bit 334-0 of register 332-12 enables input block 380, bit 334-1 enables neural network block 379, bit 334-2 enables output block 378, bit 334-3 enables bias block 377, bit 334-4 enables activation function block 376, and bits 334-5 and 334-6 enable first temporary block 375 and second temporary block 374.

[0054] Registers 332-13, 332-14, 332-15, 332-16, 332-17, 332-18, 332-19, 332-20, 332-21, 332-22, 332-23, 332-24, and 332-25 can be used to define the neural network used during AI operations. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-13 and 332-14 can define the number of rows (381) in the matrix used for AI operations. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-15 and 332-16 may limit the number of columns in the matrix used for AI operations to 382.

[0055] Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-17 and 332-18 define the size 383 of the neurons used for AI operations. The size of the neuron can indicate the width of the neuron based on the number of bits and / or the type of input (e.g., floating-point, integer, and / or double, and other types). Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-19, 332-20, and 322-21 indicate the number 384 of neurons in the neural network used for AI operations. Bits 334-4, 334-5, 334-6, and 334-7 of register 332-22 indicate the start address 385 of the block in the memory array of neurons used for AI operations. Bits 334-0, 334-1, 334-2, and 334-3 of register 332-5 indicate the end address 386 of the block in the memory array of the neurons used for AI operations. If the start address 385 and the end address 386 are the same, then only one block of neurons is indicated for AI operations. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-23, 332-24, and 322-25 indicate the number 387 of layers in the neural network used for AI operations.

[0056] Register 332-26 enables debug / hold mode for the AI ​​accelerator and the output to be observed at the layer of the AI ​​operation. Bit 334-0 of register 332-26 indicates that the AI ​​accelerator is in debug / hold mode and that the activation function should be applied during the AI ​​operation. Bit 334-1 of register 332-26 indicates that the AI ​​operation can be progressively advanced within the AI ​​operation (e.g., performing the next step in the AI ​​operation). Programming of bit 334-1 can be embedded in the execution of the AI ​​operation. The AI ​​operation can be stopped, the result of the AI ​​operation can be observed, errors can be corrected, and / or register 332-26 can be reprogrammed to include another step and / or remove a step from the AI ​​operation. In some instances, previous content at a specific step of the AI ​​operation can be restored to temporary content to see the effect of modifications on the AI ​​operation. Bits 334-2 and 334-3 of register 332-26 indicate that the temporary block containing the output of the layer is valid. The data in the temporary block can be changed by a controller on the host and / or memory device, so that the changed data can be used in the AI ​​operation as it progresses.

[0057] Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-27, 332-28, and 332-29 can limit debug / hold mode to stop the 392 AI operation and observe the output of the layer.

[0058] Registers 332-30, 332-31, 332-32, and 332-33 can define the size of the temporary block used for AI operations, as well as the start and end addresses of the temporary block used for AI operations. Bits 334-4, 334-5, 334-6, and 334-7 of register 332-30 can define the start address 393 of the first temporary block used for AI operations. Bits 334-0, 334-1, 334-2, and 334-3 of register 332-30 can define the end address 394 of the first temporary block used for AI operations. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-31 and 332-32 can define the size 395 of the temporary block used for AI operations. The size of the temporary block can be indicated by the number of bits and / or the type of input (e.g., floating-point, integer, and / or double, and other types). Bits 334-4, 334-5, 334-6, and 334-7 of registers 332-33 can define the start address 396 of the second temporary block used for AI operations. Bits 334-0, 334-1, 334-2, and 334-3 of registers 332-34 can define the end address 397 of the second temporary block used for AI operations.

[0059] Registers 332-34, 332-35, 332-36, 332-37, 332-38, and 332-39 can be associated with activation functions used for AI operations. Bit 334-0 of register 332-34 enables the use of activation function block 3101. Bit 334-1 of register 332-34 enables the use of activation functions to hold the AI ​​at neuron 3100 and for each neuron. Bit 334-2 of register 332-34 enables the use of activation functions to hold the AI ​​at layer 399 and for each layer. Bit 334-3 of register 332-34 enables the use of external activation function 398.

[0060] Bits 334-4, 334-5, 334-6, and 334-7 of registers 332-35 can define the start address 3102 of the activation function block used for AI operations. Bits 334-0, 334-1, 334-2, and 334-3 of registers 332-35 can define the end address 3103 of the activation function block used for AI operations. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-36 and 332-37 can define the resolution 3104 of the input (e.g., the x-axis) of the activation function. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-38 and 332-39 can limit the resolution and / or output (e.g., y-axis) of the activation function for a given x-axis value of a custom activation function 3105.

[0061] Registers 332-40, 332-41, 332-42, 332-43, and 332-44 can define the size of the bias value used for AI operations, the number of bias values ​​used for AI operations, and the start and end addresses of the bias values ​​used for AI operations. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-40 and 332-41 can define the size 3106 of the bias value used for AI operations. The size of the bias value can indicate the width of the bias value based on the number of bits and / or the type of the bias value (e.g., floating-point, integer, and / or double, and other types). Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-42 and 332-43 indicate the number 3107 of bias values ​​used for AI operations. Bits 334-4, 334-5, 334-6, and 334-7 of register 332-44 indicate the start address 3108 of the block in the memory array used for bias values ​​of AI operations. Bits 334-0, 334-1, 334-2, and 334-3 of register 332-44 indicate the end address 3109 of the block in the memory array used for bias values ​​of AI operations. If the start address 3108 and the end address 3109 are the same address, then only one block of bias values ​​is indicated for AI operations.

[0062] Registers 332-45 provide status information for AI computation and information for debug / hold mode. Bit 334-0 of register 332-45 activates debug / hold mode 3114. Bit 334-1 of register 332-45 indicates that the AI ​​accelerator is busy 3113 and performing AI operations. Bit 334-2 of register 332-45 indicates that the AI ​​accelerator is on 3112 and / or that the full capabilities of the AI ​​accelerator should be used. Bit 334-3 of register 332-45 indicates that only matrix computations for AI operations 3111 should be performed. Bit 334-4 of register 332-45 indicates that AI operations can proceed step-by-step 3110 and continue to the next neuron and / or layer.

[0063] Registers 332-46 provide error information regarding AI operations. The host can retrieve AI operation information during the execution of an AI operation. Bit 334-3 of register 332-46 indicates an error 3115 in the sequence of AI operations. Bit 334-2 of register 332-46 indicates an error 3116 in the algorithm of the AI ​​operation. Bit 334-1 of register 332-46 indicates an error 3117 in a data page that the ECC cannot correct. Bit 334-0 of register 332-46 indicates an error 3118 in a data page that the ECC can correct.

[0064] Registers 332-47 may indicate the activation function used in AI operation. In some instances, the controller may include several activation function registers, including registers 332-47, to define and use (e.g., initiate) the activation function in AI operation.

[0065] Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, and 334-6 of registers 332-47 can indicate one of several predefined activation functions 3120 available for the AI ​​operation. The predefined activation functions may include identity functions (e.g., identity relations, identity mappings, and / or identity transformations), binary step functions, logic functions (e.g., sigmoid functions and / or soft step functions), hyperbolic tangent functions (e.g., tanH), arctangent functions (e.g., ArcTan), soft sign functions, inverse square root unit (ISRU) functions, modified linear unit (ReLU) functions, leaked modified linear unit (Leaked ReLU) functions, parameterized modified linear unit (PReLU) functions, randomized leaked modified linear unit (RReLU) functions, exponential linear unit (ELU) functions, and / or scaled exponential linear unit (SELU) functions, as well as other functions. In some instances, the predefined activation function may be executed in response to the result of the AI ​​operation.

[0066] The controller can instruct and execute one of several predefined activation functions 3120 by programming one and / or more bits of registers 332-47 to a specific state. For example, bit 334-0 of registers 332-47 can be programmed to a first state of an identity function performing AI operations and / or a second state of preventing and / or stopping the execution of the identity function, and bit 334-1 of registers 332-47 can be programmed to a first state of a binary step function performing AI operations and / or a second state of preventing and / or stopping the execution of the binary step function.

[0067] Bits 334-7 of registers 332-47 can indicate that a custom activation function 3119 located in the block is available for AI operations. AI operations may include updating, changing, and / or generating custom activation functions. Custom activation function 3119 can be generated based on the result of a previous AI operation. For example, custom activation function 3119 may be based on the result of a debugging operation.

[0068] The controller can instruct the custom activation function 3119 to a specific state by programming one and / or more bits of registers 332-47. For example, bits 334-7 of registers 332-47 can be programmed to a first state that uses the custom activation function 3119 in AI operation and / or a second state that prevents and / or stops the use of the custom activation function.

[0069] Registers 332-48, 332-49, and 332-50 can indicate the neuron and / or layer performing the AI ​​operation. Bits 334-0, 334-1, 334-2, 334-3, 334-4, 334-5, 334-6, and 334-7 of registers 332-48, 332-49, and 332-50 can indicate the address 3121 of the neuron and / or layer performing the AI ​​operation. In the event of an error during AI operation, registers 332-48, 332-49, and 332-50 can indicate the neuron and / or layer where the error occurred.

[0070] Figure 4 This is a block diagram of several blocks of a memory device with an artificial intelligence (AI) accelerator according to several embodiments of the present disclosure. Input block 440 is a block in a memory array that stores input data. The data in input block 440 can be used as input for AI operations. The address of input block 440 can be stored in register 5 (e.g., ...). Figure 2 Register 232-5 and Figure 3A The data is indicated in registers 332-5. Since multiple input blocks can exist, the embodiment is not limited to a single input block. Data input block 440 can be sent from the host to the memory device. The data may be accompanied by a command indicating that an AI operation should be performed on the memory device using the data.

[0071] Output block 420 is a block in the memory array that stores output data from the AI ​​operation. Data in output block 442 can be used to store the output from the AI ​​operation and send it to the host. The address of output block 442 can be accessed in register 11 (e.g., ...). Figure 2 Register 232-11 and Figure 3A The output is indicated in registers 332-11. Since multiple output blocks can exist, the embodiment is not limited to a single output block.

[0072] After the AI ​​operation is completed and / or maintained, the data in output block 442 can be sent to the host. Temporary blocks 444-1 and 444-2 can be blocks in the memory array that temporarily store data while the AI ​​operation is being performed. Although Figure 4 It includes two temporary blocks 444-1 and 444-2, but the memory device may include one or more temporary blocks. Data can be stored in temporary blocks 444-1 and 444-2, while AI operations iterate through neurons and layers of the neural network used for AI operations. The addresses of temporary blocks 444-1 and 444-2 can be accessed in registers 30 and 33 (e.g., ...). Figure 2 Registers 232-30 and 232-33 in the and Figure 3B As indicated in 332-30 and 332-33 of the document. Since multiple temporary blocks can exist, the embodiment is not limited to two temporary blocks.

[0073] Activation function block 446 may be a block in the memory array storing activation functions for AI operations and / or in the memory controller firmware. Activation function block 446 may store predefined activation functions and / or custom activation functions generated by the host and / or AI accelerator. The address of activation function block 446 may be in register 35 (e.g., ...). Figure 2 Registers 232-35 and Figure 3B As indicated in 332-35 of the document. Since multiple activation function blocks can exist, the embodiment is not limited to one activation function block.

[0074] Bias value block 448 is a block in the memory array that stores the bias values ​​for AI operations. The address of bias value block 448 can be found in register 44 (e.g., ...). Figure 2 Registers 232-44 and Figure 3B As indicated in 332-44 of the document. Since multiple bias value blocks can exist, the embodiment is not limited to a single bias value block.

[0075] Neural network blocks 450-1, 450-2, 450-3, 450-4, 450-5, 450-6, 450-7, 450-8, 450-9, and 450-10 are blocks in the memory array that store the neural networks for AI operations. Neural network blocks 450-1, 450-2, 450-3, 450-4, 450-5, 450-6, 450-7, 450-8, 450-9, and 450-10 can store information about neurons and layers used for AI operations. The addresses of neural network blocks 450-1, 450-2, 450-3, 450-4, 450-5, 450-6, 450-7, 450-8, 450-9, and 450-10 can be accessed in register 22 (e.g., ...). Figure 2 Registers 232-22 and Figure 3A As indicated in 332-22).

[0076] Figure 5 This is a flowchart illustrating an example artificial intelligence process in a memory device with an artificial intelligence (AI) accelerator according to several embodiments of the present disclosure. In response to initiating an AI operation, the AI ​​accelerator may write input data 540 and neural network data 550 to input and neural network blocks, respectively. The AI ​​accelerator may use the input data 540 and neural network data 550 to perform AI operations. The results may be stored in temporary blocks 544-1 and 544-2. Temporary blocks 544-1 and 544-2 may be used to store data while performing matrix calculations, adding bias data, and / or applying activation functions during AI operations.

[0077] The AI ​​accelerator can receive partial results and bias value data 548 of AI operations stored in temporary blocks 544-1 and 544-2, and perform AI operations using the partial results and bias value data 548. The results can be stored in temporary blocks 544-1 and 544-2.

[0078] The AI ​​accelerator can receive partial results of AI operations stored in temporary blocks 544-1 and 544-2, and activation function data 546, and perform AI operations using these partial results and activation function data 546. The results can be stored in output block 542.

[0079] In several embodiments, activation function data 546 may be the result of one or more activation functions of an AI operation. The activation function may be a predefined activation function generated as a result of a previous AI operation or a custom activation function, as previously mentioned. Figure 3B As described. Several activation functions may include identity functions (e.g., identity relations, identity mappings, and / or identity transformations), binary step functions, logic functions, hyperbolic tangent functions (e.g., tanH), arctangent functions (e.g., ArcTan), soft sign functions, inverse square root unit (ISRU) functions, modified linear unit (ReLU) functions, leaked modified linear unit (Leaked ReLU) functions, parameterized modified linear unit (PReLU) functions, randomized leaked modified linear unit (RReLU) functions, exponential linear unit (ELU) functions, and / or scaled exponential linear unit (SELU) functions.

[0080] Figure 6-1 , Figure 6-2Table 6123 is provided to illustrate several activation functions 6122-1...6122-13 for artificial intelligence (AI) operations according to several embodiments of the present disclosure. Table 6123 includes a name 6124, a curve 6125, an equation 6126, a derived equation 6127, and an interval 6128 for each of the activation functions 6122-1...6122-13. Several activation functions 6122-1…6122-13 may include identity functions (e.g., identity relations, identity mappings, and / or identity transformations), binary step functions, logistic functions, hyperbolic tangent functions (e.g., tanH), arctangent functions (e.g., ArcTan), soft sign functions, inverse square root unit (ISRU) functions, modified linear unit (ReLU) functions, leaked modified linear unit (Leaked ReLU) functions, parameterized modified linear unit (PReLU) functions, randomized leaked modified linear unit (RReLU) functions, exponential linear unit (ELU) functions, and / or scaled exponential linear unit (SELU) functions. In some instances, the logistic function may include sigmoid functions and / or soft step functions. Activation functions for AI operations may include, but are not limited to, the several activation functions 6122-1…6122-13 described in Table 6123.

[0081] Several activation functions 6122-1...6122-13 can be predefined activation functions, and one or more bits in the register can indicate the predefined activation function to be used in the AI ​​operation, as previously combined. Figure 3B As discussed above. In some instances, several activation functions 6122-1...6122-13 can be modified (e.g., changed) to custom activation functions. For example, combining... Figure 3B The discussion outlines the possibility of generating a custom activation function based on the result of a previous AI operation. In some instances, the custom activation function can be based on the result of the previous AI operation and one or more of several activation functions 6122-1…6122-13. The activation function can be selected between predefined and custom activation functions based on the result of the previous AI operation, and different activation functions can be selected for each layer.

[0082] Figure 7A This is a block diagram of a device in the form of a computing system 700 including a memory device 720 with a memory array 7157 and an artificial intelligence (AI) accelerator, according to several embodiments of the present disclosure. As used herein, the memory device 720, controller 722, memory array 7157, sensing circuitry 7138, and / or several additional latches 7140 may also be considered individually as a “device”.

[0083] exist Figure 7A In China, AI accelerators (e.g., Figure 1The AI ​​accelerator 124 in the figure may include a sensing circuit system 7138 and an additional latch 7140, as well as... Figures 7A to 10B Other components described in the related section are configured to perform operations associated with AI operations, such as logical operations. See below for details. Figures 7A to 10B In connection with this description, the memory device (e.g., memory device 720) may be configured to perform operations associated with AI operations as part of an AI accelerator.

[0084] As used herein, an additional latch is intended to mean sensing, coupling, and / or moving (e.g., reading, storing, caching) the data value of a memory cell in the array and is different from... Figure 7B , 8 The additional functionality (e.g., amplifiers, selection logic) of multiple computing components 7148-1…7148-M and / or logic bars 7152-1…7152-N in the computing unit within the data path of the multiple shared I / O lines 7144 shown in 9, 10A, and 10B. Figure 7A and 7B As shown, logic bars 7152-1...7152-N in the data path of multiple shared input / output (I / O) lines 7144 local to the array can be associated with various group segments 7150-1...7150-N of the memory cells in group 7146-1. Group 7146-1 can be one of multiple groups on the memory device 720.

[0085] Figure 7A System 700 includes a host 702 coupled (e.g., connected) to memory device 720. Host 702 may be, for example, a host system of a personal laptop computer, desktop computer, digital camera, smartphone, or memory card reader, as well as various other types of host systems. Host 702 may include a system motherboard and / or backplane, and may include several processing resources (e.g., one or more processors, microprocessors, or some other type of control circuitry). System 700 may include a separate integrated circuit, or both host 702 and memory device 720 may be on the same integrated circuit. System 700 may be, for example, a server system and / or a high-performance computing (HPC) system and / or a portion thereof. Although Figure 7A The examples shown illustrate systems with a von Neumann architecture, but embodiments of this disclosure can be implemented in non-von Neumann architectures that may not include one or more components typically associated with a von Neumann architecture (e.g., CPU, ALU, etc.).

[0086] For clarity, system 700 has been simplified to focus on features particularly relevant to this disclosure. Memory array 7157 may be a DRAM array, SRAM array, STT RAM array, PCRAM array, TRAM array, RRAM array, NAND flash array, and / or NOR flash array, as well as other types of arrays. Array 7157 may include memory cells arranged in rows coupled by access lines (which may be referred to herein as word lines or select lines) and columns coupled by sense lines (which may be referred to herein as data lines or digital lines). Although Figure 7A The illustration shows a single array 7157, but the embodiments are not limited thereto. For example, the memory device 720 may include several arrays 7157 (e.g., several groups of DRAM cells, NAND flash cells, etc.).

[0087] Memory device 720 may include address circuitry 7525 to latch address signals provided on data bus 756 (e.g., an I / O bus connected to host 702) via I / O circuitry 7134 (e.g., provided to external ALU circuitry and / or DRAM DQ via local I / O lines and global I / O lines). As used herein, DRAM DQ enables data input to and / or output from a group (e.g., from and / or to controller 722 and / or host 702) via a bus (e.g., data bus 756). During a write operation, a voltage (high = 1, low = 0) may be applied to DQ (e.g., a pin). This voltage may be converted to an appropriate signal and stored in the selected memory cell. During a read operation, once the access is complete and the output is enabled (e.g., by outputting an enable signal low), the data value read from the selected memory cell may appear at DQ. At other times, DQ may be in a high-impedance state such that DQ neither supplies nor draws current and does not present signals to the system. This also reduces DQ contention when two or more devices (e.g., groups) share a combined data bus, as described herein. This DQ is related to multiple shared I / O lines 7144 in the local data path of array 7157 (in... Figure 7B They are separate and distinct from each other.

[0088] For example, status and exception information can be provided from the controller 722 of the memory device 720 to the channel controller 704 via an out-of-band (OOB) bus 757, such as a high-speed interface (HSI), and the status and exception information can then be provided from the channel controller 704 to the host 702. The channel controller 704 may include logic components to allocate multiple locations (e.g., controllers for subarrays) in the array for each corresponding group to store group commands, application instructions (e.g., sequences of operations), and independent variables (PIM commands) for each group associated with the operation of each of the multiple memory devices 720. The channel controller 704 may schedule commands (e.g., PIM commands) to the multiple memory devices 720 to be executed in a given group 7146 of the memory devices 720. Figure 7B The program instructions are stored inside.

[0089] Address signals are received via address circuitry 7525 and decoded by row decoder 7132 and column decoder 7142 to access memory array 7157. Data can be sensed (read) from memory array 7157 by sensing voltage and / or current changes on sense lines (digital lines) using several sense amplifiers as described herein in sense circuitry 7138. The sense amplifiers can read and latch data pages (e.g., rows) from memory array 7157. Additional computational circuitry can be coupled to sense circuitry 7138 as described herein and can be used in combination with sense amplifiers to sense, store (e.g., cache and / or buffer), perform computational functions (e.g., operations), and / or move data. I / O circuitry 7134 can be used for bidirectional data communication with host 702 via data bus 756 (e.g., a 64-bit wide data bus). Write circuitry 7136 can be used to write data to memory array 7157.

[0090] Controller 722 (e.g., group control logic, sequencer, and timing circuitry) can decode signals (e.g., commands) provided from host 702 via control bus 754. These signals may include chip enable signals, write enable signals, and / or address latch signals, which can be used to control operations performed on memory array 7157, including data sensing, data storage, data movement (e.g., copying, transferring, and / or delivering data values), data writing and / or data erasure operations, and other operations. In various embodiments, controller 722 may be responsible for executing instructions from host 702 and accessing memory array 7157. Controller 722 may be a state machine, sequencer, or some other type of controller. Controller 722 may control the shifting of data (e.g., right or left) in rows of the array (e.g., memory array 7157) and execute microcode instructions to perform operations such as computation operations, such as AND, OR, NOR, XOR, addition, subtraction, multiplication, division, etc.

[0091] In the following text (e.g., in Figures 7A to 8 (B) Further examples of the sensing circuit system 7138 are described. For example, in some embodiments, the sensing circuit system 7138 may include a plurality of sensing amplifiers and a plurality of computing components that can act as accumulators and can be used to perform operations in each subarray (e.g., on data associated with complementary sensing lines).

[0092] In some embodiments, the sensing circuitry 7138 can be used to perform operations using data stored in the memory array 7157 as input, and participate in the movement of data to different locations in the memory array 7157 for copying, transferring, conveying, writing, logical and / or storage operations, without transferring data via sensing line address access (e.g., without initiating column decoding signals). Thus, various computational functions can be performed using and within the sensing circuitry 7138, rather than by (or in combination with) processing resources external to the sensing circuitry 7138 (e.g., by a processor and / or other processing circuitry associated with host 702, such as an ALU circuitry located on device 720 (e.g., on controller 722 or elsewhere)). However, alternatively, according to embodiments of this disclosure, computational functions are performed on data values ​​moving from rows of the array to multiple computational components 7148-1…7148-M and / or logic bars 7152-1…7152-N in the computational unit. Furthermore, as an example, according to the embodiment, computational operations can be controlled in the computational unit at a rate of 2 nanoseconds (ns) compared to the instance time required to start a row in an array, for example, 60 nanoseconds (ns), without having to move data values ​​back to the row.

[0093] In various prior methods, for example, data associated with operands is read from memory via a sensing circuitry system and provided to an external ALU circuitry system via I / O lines (e.g., via local I / O lines and / or global I / O lines). The external ALU circuitry system may include several registers and will perform computational functions using the operands, and the results will be transmitted back to the array via the I / O circuitry system 7134.

[0094] In contrast, embodiments of this disclosure perform computational functions on data values ​​of multiple computational components 7148-1…7148-M and / or logic bars 7152-1…7152-N in computational cells that are moved from rows of the array via multiple shared I / O lines 7144 into a local data path within the array. Additionally, the sensing circuitry 7138 can be configured to perform operations on data stored in a memory array 7157 and store the results back into the memory array 7157 without activating I / O lines (e.g., local I / O lines) coupled to the sensing circuitry 7138. However, compared to the instance time required to start a row in the array, for example, 60 nanoseconds (ns), computational operations can be controlled much faster in the computational cells once loaded, for example, at a speed of 2 nanoseconds (ns) without having to move the data values ​​back to the rows. The sensing circuitry 7138 can be formed at intervals with the memory cells of the array. Multiple computing components 7148-1…7148-M and / or logic bars 7152-1…7152-N associated with data paths of multiple shared I / O lines 7144 have a spacing equal to the spacing of the data paths and a function of the spacing between the digital lines and the memory cell array. For example, the computing components have a spacing that is an integer multiple of the spacing between the digital lines and the memory cell array.

[0095] For example, the sensing circuit system 7138 described herein can be formed with a pair of complementary sensing lines (e.g., digital lines) at the same spacing. As an example, a pair of complementary memory cells can have a spacing of 6F. 2 The cell size is (e.g., 3F x 2F), where F is the feature size. If the spacing between a pair of complementary sensing lines for a complementary memory cell is 3F, then the spaced sensing circuitry instructs the sensing circuitry (e.g., the sensing amplifier and corresponding computing component for each corresponding pair of complementary sensing lines) to be formed within the 3F spacing of the complementary sensing lines. Similarly, the computing components 7148-1…7148-M and / or logic bars 7152-1…7152-N associated with the data paths of the multiple shared I / O lines 7144 have a spacing that is a function of the 3F spacing of the complementary sensing lines. For example, the computing components 7148-1…7148-M and / or logic bars 7152-1…7152-N will have a spacing that is an integer multiple of the 3F spacing between the digital lines and the memory cell array.

[0096] Conversely, the circuitry of processing resources (e.g., computing engines, such as the ALU) in various prior systems may not conform to the spacing rules associated with memory arrays. For example, the memory cells of a memory array may have 4F... 2 Or 6F 2Cell size. Therefore, devices (e.g., logic gates) associated with the ALU circuitry of a previous system may not be able to be spaced with memory cells (e.g., spaced the same as sensing lines), which could affect, for example, chip size and / or memory density. In the context of some computing systems and subsystems (e.g., central processing units (CPUs)), data can be processed in locations not spaced from memory (e.g., memory cells in an array) and / or not on the chip, as described herein. For example, data can be processed using processing resources associated with the host rather than spaced from memory.

[0097] Therefore, in several embodiments, the external circuitry of array 7157 and sensing circuitry 7138 are not required to perform computational functions, because sensing circuitry 7138 can perform these computational functions appropriately, or these operations can be performed locally on the data paths of multiple shared I / O lines of the array, without using external processing resources. Thus, the multiple computing components 7148-1…7148-M and / or logic bars 7152-1…7152-N in the computing units of sensing circuitry 7138 and / or the data paths of multiple shared I / O lines 7144 can be used to at least partially supplement or replace such external processing resources (or at least the bandwidth consumption of such external processing resources). In some embodiments, multiple computing components 7148-1…7148-M and / or logic bars 7152-1…7152-N in the computing units of the sensing circuit system 7138 and / or the multiple shared I / O lines 7144 can be used to perform operations other than those performed by external processing resources (e.g., host 702) (e.g., executing instructions). For example, host 702 and / or sensing circuit system 7138 may be limited to performing only certain operations and / or a certain number of operations.

[0098] The operations described herein may include operations associated with a device having processing in memory (PIM) capability. Device operations with PIM capability may use bit-vector-based operations. As used herein, the term "bit vector" refers to a physically contiguous number of bits stored in a row of a memory cell array on a bit-vector memory device (e.g., a PIM device). Thus, as used herein, "bit vector operation" means an operation performed on a bit vector that is a contiguous portion of a virtual address space (e.g., used by the PIM device). For example, a row of virtual address space in a PIM device may have a bit length of 16K bits (e.g., corresponding to 16K complementary pairs of memory cells in a DRAM configuration). As described herein, a sensing circuit system 7138 for such a 16K bit row may include corresponding 16K processing elements (e.g., computing components as described herein), the 16K processing elements being formed at intervals with sensing lines optionally coupled to corresponding memory cells in the 16-bit row. The computing components in the PIM device can operate as a single bit of the bit vector of a row of memory cells sensed by the sensing circuitry 7138 (e.g., sensed by a sense amplifier paired with the computing component and / or stored in the sense amplifier, as described herein). Similarly, logic bars 7152-1...7152-N in the computing cells of multiple computing components 7148-1...7148-M and / or multiple data paths sharing I / O lines 7144 can operate as a single bit of the bit vector of a row of memory cells sensed in the array.

[0099] Enabling I / O lines may include enabling (e.g., turning on, activating) a transistor having a gate coupled to a decoded signal (e.g., a column decoded signal) and a source / drain coupled to the I / O line. However, embodiments are not limited to disabling I / O lines. For example, in several embodiments, a sensing circuitry (e.g., 7138) may be used to perform operations without enabling the column decoded lines of the array.

[0100] However, multiple shared I / O lines 7144 can be enabled to load data values ​​into multiple computing components 7148-1…7148-M and / or logic bars 7152-1…7152-N in the computing unit within the data paths of the multiple shared I / O lines 7144, where computing operations can be controlled much faster. For example, in the multiple computing components 7148-1…7148-M and / or logic bars 7152-1…7152-N in the computing unit, computing operations can be performed at a speed of 2 nanoseconds (ns). This speed enhancement can be attributed to the fact that the data values ​​do not need to be moved back into the rows, for example, with an associated time of 60 nanoseconds (ns) for starting rows in the array.

[0101] Figure 7BThis is a block diagram of a device in the form of a computing system comprising a memory device with an artificial intelligence (AI) accelerator having shared input / output (I / O) lines in a local data path within a group segment of an array, according to several embodiments of the present disclosure. For example, group 7146-1 may represent an instance group of memory device 720. Figure 7B As shown, group 7146-1 may include multiple main memory columns (displayed horizontally as X) (e.g., 16,384 columns in an example DRAM group). Additionally, group 7146-1 may be divided into group segments (e.g., quadrants of 32 subarrays) 7150-1, 7150-2…7150-N. Each group segment may be associated with multiple computing components 7148-1…7148-M in logical bars 7152-1…7152-N within computing units in a data path sharing multiple I / O lines 7144. Each of the group segments 7150-1…7150-N may include multiple rows (displayed vertically as Y) (e.g., each segment may be a quadrant comprising 32 subarrays, each of which may comprise 512 rows in an example DRAM group). Example embodiments are not limited to the instance horizontal and / or vertical orientation of the columns and rows described herein, or their instance numbering.

[0102] like Figure 7B As shown, group 7146-1 can be divided into multiple group segments 7150-1…7150-N. Each group segment can have multiple computing components 7148-1…7148-M and logic bars 7152-1…7152-N in computing units within a data path of multiple shared I / O lines 7144 associated with it. Group 7146-1 may include a controller 722 to direct operations on data values ​​loaded onto the multiple computing components 7148-1…7148-M in the logic bars 7152-1…7152-N within the computing units of the multiple shared I / O lines 7144.

[0103] Figure 8 This is a schematic diagram illustrating a sensing circuit system 8138 according to several embodiments of the present disclosure. The sensing circuit system 8138 may correspond to... Figure 7A The sensing circuit system 7138 shown in the figure.

[0104] like Figure 8As shown in the example embodiments, memory cells may include storage elements (e.g., capacitors) and access means (e.g., transistors). For example, a first memory cell may include transistor 8160-1 and capacitor 8162-1, and a second memory cell may include transistor 8160-2 and capacitor 8162-2, etc. In this embodiment, memory array 825 is a DRAM array of one transistor-one capacitor (1T1B) memory cells, but other configuration embodiments may be used (e.g., a 2T2C array with two transistors and two capacitors per memory cell). In several embodiments, memory cells may be destructively read memory cells (e.g., reading data stored in the cell destroys the data, causing the data initially stored in the cell to be refreshed after being read).

[0105] The cells of the memory array 825 can be coupled in rows via access (word) lines 8164-X (row X), 8164-Y (row Y), etc., and via complementary sensing line pairs (e.g., Figure 8 The column arrangement of digital lines DIGIT(D) and DIGIT(D)_ is shown in the figure. The individual sensing lines corresponding to each pair of complementary sensing lines can also be referred to as digital line 8168-1 for DIGIT(D) and digital line 8168-2 for DIGIT(D)_, or... Figure 9 and 10A The corresponding reference numerals in Figure 10B. Although in Figure 8 Only one pair of complementary digital lines is shown in the illustration, but embodiments of this disclosure are not limited thereto, and the memory cell array may include additional columns of memory cells and digital lines (e.g., 4,096, 8,192, 16,384, etc.).

[0106] Although rows and columns are described as being vertically oriented in a plane, the embodiments are not limited thereto. For example, rows and columns may be oriented relative to each other in any feasible three-dimensional configuration. For example, rows and columns may be oriented relative to each other at any angle, may be oriented in a substantially horizontal or substantially vertical plane, and / or may be oriented in a folded topology and other possible three-dimensional configurations.

[0107] Memory cells can be coupled to different digital lines and word lines. For example, the first source / drain region of transistor 8160-1 can be coupled to digital line 8168-1 (D), the second source / drain region of transistor 8160-1 can be coupled to capacitor 8162-1, and the gate of transistor 8160-1 can be coupled to word line 8164-Y. The first source / drain region of transistor 8160-2 can be coupled to digital line 8168-2 (D), the second source / drain region of transistor 8160-2 can be coupled to capacitor 8162-2, and the gate of transistor 8160-2 can be coupled to word line 8164-X. For example... Figure 8The cell board shown can be coupled to each of capacitors 8162-1 and 8162-2. The cell board can be a common node to which a reference voltage (e.g., ground) can be applied in various memory array configurations.

[0108] According to several embodiments of this disclosure, memory array 825 is configured to be coupled to sensing circuitry system 8138. In this embodiment, sensing circuitry system 8138 includes a sensing amplifier 8170 corresponding to a respective column of memory cells (e.g., coupled to a pair of complementary digital lines) and a computing component 8148. Sensing amplifier 8170 may be coupled to the pair of complementary digital lines 8168-1 and 8168-2. Computing component 8148 may be coupled to sensing amplifier 8170 via turn-on gates 8172-1 and 8172-2. The gates of turn-on gates 8172-1 and 8172-2 may be coupled to operation selection logic 8178.

[0109] Operation selection logic 8178 can be configured to include pass-through logic for controlling pass-through gates and switch-through logic for controlling switch-through gates, wherein the pass-through gates are coupled to the pair of complementary digital lines between the sense amplifier 8170 and the computing component 8148 without transposition, and the switch-through gates are coupled to the pair of complementary digital lines between the sense amplifier 8170 and the computing component 8148 with transposition. Operation selection logic 8178 can also be coupled to the pair of complementary digital lines 8168-1 and 8168-2. Operation selection logic 8178 can be configured to control the continuity of pass-through gates 8172-1 and 8172-2 based on the selected operation.

[0110] An operable sense amplifier 8170 determines a data value (e.g., a logic state) stored in a selected memory cell. The sense amplifier 8170 may include a cross-coupled latch, which may be referred to herein as a master latch. Figure 8 In the example illustrated, the circuit system corresponding to the sense amplifier 8170 includes a latch 8175 comprising four transistors coupled to a pair of complementary digital lines 8168-1 (D) and 8168-2 (D). However, the embodiments are not limited to this example. The latch 8175 may be a cross-coupled latch (e.g., the gates of a pair of transistors), such as n-channel transistors (e.g., NMOS transistors) 8182-1 and 8182-2 cross-coupled to the gates of another pair of transistors (e.g., p-channel transistors (e.g., PMOS transistors) 8184-1 and 8184-2). The cross-coupled latch 8175 comprising transistors 8182-1, 8182-2, 8184-1, and 8184-2 may be referred to as the master latch.

[0111] In operation, when a memory cell is being sensed (e.g., read), the voltage on one of the digital lines 8168-1 (D) or 8168-2 (D) will be slightly greater than the voltage on the other of the digital lines 8168-1 (D) or 8168-2 (D)_. The ACT signal and the RNL* signal can be driven low to enable (e.g., start) the sense amplifier 8170. The digital line 8168-1 (D) or 8168-2 (D)_ with the lower voltage will turn on one of the PMOS transistors 8184-1 or 8184-2 to a greater extent than the other of the PMOS transistors 8184-1 or 8184-2, thereby driving the digital line 8168-1 (D) or 8168-2 (D)_ with the higher voltage to a greater extent than driving the other digital line 8168-1 (D) or 8168-2 (D)_ high.

[0112] Similarly, a digital line 8168-1 (D) or 8168-2 (D) with a higher voltage will turn on one of the NMOS transistors 8182-1 or 8182-2 to a greater extent than the other, thereby driving a digital line 8168-1 (D) or 8168-2 (D) with a lower voltage to a greater extent than driving the other digital line 8168-1 (D) or 8168-2 (D) with a lower voltage. Therefore, after a short delay, the digital line 8168-1 (D) or 8168-2 (D) with a slightly higher voltage is driven to the power supply voltage V across the source transistor. CC The voltage across the absorption transistor is driven by another digital line 8168-1 (D) or 8168-2 (D), which is a reference voltage (e.g., ground). Therefore, the cross-coupled NMOS transistors 8182-1 and 8182-2 and PMOS transistors 8184-1 and 8184-2 act as a sense amplifier pair, amplifying the differential voltage on digital lines 8168-1 (D) and 8168-2 (D) and operating to latch data values ​​sensed from selected memory cells on nodes 8177-1 and / or 8177-2.

[0113] Examples are not limited to Figure 8 The sense amplifier 8170 configuration described herein. As an example, the sense amplifier 8170 may be a current-mode sense amplifier and a single-ended sense amplifier (e.g., a sense amplifier coupled to a digital line). Furthermore, embodiments of this disclosure are not limited to folded digital line architectures, such as... Figure 8 The folded digital line architecture shown in the image.

[0114] The sensing amplifier 8170 can be operated in conjunction with the computing component 8148 to perform various operations using data from the array as input. In several embodiments, the results of the operations can be stored back to the array without transferring data via digital line address access (e.g., without activating column decoding signals, so that data is transferred to circuitry outside the array and sensing circuitry via local I / O lines). Thus, several embodiments of this disclosure can implement the operation and associated computational functions using less power than various prior methods. Furthermore, because several embodiments do not require data transfer across local and global I / O lines to perform computational functions (e.g., between memory and discrete processors), several embodiments can achieve increased (e.g., faster) processing power compared to prior methods.

[0115] The sense amplifier 8170 may further include a balanced circuit system 8174 configurable to balance digital lines 8168-1(D) and 8168-2(D). In this example, the balanced circuit system 8174 includes transistors 8188 coupled between digital lines 8168-1(D) and 8168-2(D). The balanced circuit system 8174 also includes transistors 8188 each having a voltage coupled to a balancing voltage (e.g., V). DD / 2) Transistors 8180-1 and 8180-2 in the first source / drain region, where V DD This is the power supply voltage associated with the array. The second source / drain region of transistor 8180-1 can be coupled to digital line 8168-1 (D), and the second source / drain region of transistor 8180-2 can be coupled to digital line 8168-2 (D). The gates of transistors 8188, 8180-1, and 8180-2 can be coupled together and coupled to the balancing (EQ) control signal line 8186. Therefore, activating EQ enables transistors 8188, 8180-1, and 8180-2, which effectively shorts digital lines 8168-1 (D) and 8168-2 (D) together and shorts to the balancing voltage (e.g., V). CC / 2).

[0116] Although Figure 8 The illustration shows a sense amplifier 8170 including a balancing circuit system 8174, but embodiments are not limited thereto, and the balancing circuit system 8174 may be implemented separately from the sense amplifier 8170, or implemented in conjunction with... Figure 8 The configurations shown may be implemented in different configurations or not at all.

[0117] As further described below, in several embodiments, a sensing circuitry 8138 (e.g., a sensing amplifier 8170 and a computing component 8148) is operable to perform a selected operation, and the result is first stored in one of the sensing amplifier 8170 or the computing component 8148 without transmitting data from the sensing circuitry via local or global I / O lines (e.g., without performing sensing line address access by activating, for example, a column decoding signal).

[0118] However, further to the embodiments described herein, a sensing amplifier is included, and in some embodiments may also include... Figure 8 The sensing circuit system 8138 of the computing components shown can also couple memory cells from multiplexed columns of memory cells in the array to computing components 9148-1...9148-M and / or logic bars 9152-1...9152-N in the data paths of multiple shared I / O lines 9144 local to the array, such as in combination with Figure 10A and 10B As discussed above, in this manner, computing components 10148-1…10148-M and / or logic bars 10152-1…10152-N can be indirectly coupled to the column memory cells via multiple shared I / O lines 10144 through selection logic (in combination with...). Figure 10A and 10B (As discussed).

[0119] The execution of operations (e.g., Boolean logic operations involving data values) is basic and commonly used. Boolean logic operations are used in many higher-level operations. Therefore, the speed and / or power efficiency achievable through improved operations can be translated into speed and / or power efficiency for higher-order functionality.

[0120] like Figure 8 As shown, computing component 8148 may also include a latch, referred to herein as secondary latch 8190. Secondary latch 8190 may be configured and operated in a manner similar to that described above with respect to primary latch 8175, except that the pair of cross-coupled p-channel transistors (e.g., PMOS transistors) included in the secondary latch have their respective sources coupled to a supply voltage 8176-2 (e.g., V). DD The pair of cross-coupled n-channel transistors (e.g., NMOS transistors) of the secondary latch can selectively couple their respective sources to a reference voltage 8176-1 (e.g., ground), thereby continuously enabling the secondary latch. The configuration of the computing component 848 is not limited to... Figure 8 The configuration shown in the document is feasible, and various other embodiments are also possible.

[0121] As described herein, memory devices (e.g., Figure 7A720 in the memory device 720 can be configured to be coupled to a host (e.g., 702) via a data bus (e.g., 756) and a control bus (e.g., 754). Group 7146 in the memory device 720 may include multiple group segments of memory cells. Figure 7B (7150-1...7150-N in the original text). Group 7146 may include multiple columns via memory cells (...). Figure 7B Sensing circuitry systems coupled to multiple arrays (e.g., Figure 7A 7138 and Figure 8 and 9 (Corresponding reference numerals in the figures). The sensing circuitry system may include a sensing amplifier and a computing component coupled to each of the columns (e.g., respectively). Figure 8 (8170 and 8148 in the middle).

[0122] Each group of 7150 segments can share multiple I / O lines with the array 825 locally. Figure 7B Multiple logical bars in the computational unit of the data path (e.g., 744) in the middle (e.g.) Figure 7B The controllers (e.g., 7152-0, 7152-1, ..., 7152-N-1) in the group are associated with this group. As described herein, the controllers coupled to the group (e.g., Figures 7A to 7B The 722 in the array can be configured to direct the movement of data values ​​to shared I / O lines 9144 / 10144 local to the array. Figure 9 and 10A The calculation unit 998 / 10198 in the data path () Figure 9 and 10A Logic bar 10152 in 10B) Figure 10A The computing component 10148 in ) Figure 10A ).

[0123] Memory devices may include logic bars (e.g., Figure 7B 7152 and Figure 10A (10152 in the text), the logic bar has several columns that can correspond to memory cells ( Figure 7B Multiple computational components (e.g., respectively) Figure 10A (10148-1……10148-Z in the original text). For example, combining... Figure 8 Furthermore, the plurality of sensing amplifiers 8170 and / or computing components 8148 in the sensing circuit system 8138 may be selectively coupled (e.g., via...) Figure 9 The column selection circuitry systems 9194-1 and 9194-2 in the series connect to multiple shared I / O lines 9144. Figure 9The column selection circuitry can be configured to selectively sense data in specific columns of the array's memory cells by selectively coupling to multiple (e.g., four, eight, and sixteen, and other possibilities) sense amplifiers and / or computing components.

[0124] In some embodiments, multiple logical bars in a group (e.g., Figure 7B The numbers 7152-1……7152-N in the original text can correspond to the group in the original text. Figure 7B Several group segments 7150-1…7150-N (e.g., quadrants with multiple subarrays). Logic bars may include similar... Figure 8 The computing component 8148 shown in the diagram has a shared I / O line 10144 local to the array 825. Figure 10A Multiple computing components 10148-1...10148-Z in the data path () Figure 10A ). As in Figure 9 As shown, the data values ​​sensed from the array's rows can be selected by column selection logic via multiple shared I / O lines 9144 ( Figure 9 ) Move in parallel to multiple shared I / O lines 944 ( Figure 9 Computation unit 9198 in the data path () Figure 9 Multiple computing components 10148 in ) Figure 10A In some embodiments, the data volume may correspond to a width of at least one thousand bits for multiple shared I / O lines.

[0125] As described herein, a memory cell array may include implementations of DRAM memory cells, wherein a controller is configured to move (e.g., copy, transfer, and / or deliver) data from a source location to a destination location via shared I / O lines in response to a command. In various embodiments, the source location may be in a first set, and the destination location may be on shared I / O line 9144 local to array 825. Figure 9 Computation unit 9198 in the data path () Figure 9 )middle.

[0126] like Figure 8 As described herein, the device can be configured to transfer data from a source location, including specific rows associated with a first plurality of sensing amplifiers and computing components (e.g., Figure 9 (919 in the column address) and column address, move (e.g., copy, transfer and / or send) to shared I / O lines (e.g., Figure 9 (9144 in the middle). Additionally, the device can be configured to move data to a destination location, including with shared I / O line 10194 ( Figure 10A The specific logical bar 10144 associated with the computing unit 10198 in the data path of ) Figure 10AAs the reader will know, each shared I / O line has 10144 (). Figure 10A This can actually include complementary pairs of shared I / O lines (e.g., Figure 9 Shared I / O lines and shared I / O lines*). In some embodiments described herein, 2048 shared I / O lines (e.g., complementary pair shared I / O lines) may be configured as 2048-bit wide shared I / O lines. In some embodiments described herein, 1024 shared I / O lines (e.g., complementary pair shared I / O lines) may be configured as 1024-bit wide shared I / O lines.

[0127] Figure 9 This is a schematic diagram illustrating a circuit system for data movement in a memory device according to several embodiments of the present disclosure. Figure 9 Eight sense amplifiers (e.g., sense amplifiers 0, 1, ... 7 shown at 9170-0, 9170-1 ... ... 9170-7) are shown, each coupled to a corresponding pair of complementary shared I / O lines 9144 (e.g., shared I / O line and shared I / O line*). Figure 9 It also demonstrates eight computing components (e.g., computing components 0, 1...7 shown at 9148-0, 9148-1...9148-7) each coupled to a corresponding sense amplifier (e.g., as shown for sense amplifier 0 at 9170-0) via corresponding conduction gates 9172-1 and 9172-2 and digital lines 9168-1 and 9168-2). For example, the conduction gates can be as follows: Figure 2 The connections shown are controllable by the operation selection signal Pass. For example, the output of the selection logic can be coupled to the gates of conduction gates 9172-1 and 9172-2 and digital lines 9168-1 and 9168-2. Correspondingly, the sense amplifier and computing components can facilitate the formation of the sense circuitry system indicated at 9138-0, 9138-1...9138-7.

[0128] The data values ​​existing on the complementary digital lines 9168-1 and 9168-2 can be loaded into the computing component 9148-0, such as in combination with... Figure 9 As described. For example, when through gates 9172-1 and 9172-2 are enabled, the data values ​​on the complementary digital lines 9168-1 and 9168-2 can be transferred from the sense amplifier to the computing components (e.g., 9170-0 to 9148-0). When the sense amplifier is activated, the data values ​​on the complementary digital lines 9168-1 and 9168-2 can be data values ​​stored in the sense amplifier 9170-0.

[0129] Figure 9 The sensing amplifiers 9170-0, 9170-1...9170-7 in the model can each correspond to... Figure 8The sensor amplifier 8170 shown in the image. Figure 9 The computing components 9148-0, 9148-1...9148-7 shown in the diagram can each correspond to... Figure 8 The computing component 8148 shown in the figure. A combination of a sense amplifier and a computing component can facilitate a sense circuitry system (e.g., 9138-0, 9138-1...9138-7) coupled to a portion of a DRAM memory subarray 9145 with a shared I / O line 9144, which is shared by multiple logic bars in the data path of the shared I / O line 9144.

[0130] For clarity Figure 9 The configurations described in the embodiments are as follows, but the configurations are not limited to these configurations. For example, Figure 9 The configuration described herein for the sense amplifiers 9170-0, 9170-1...9170-7 and the computing components 9148-0, 9148-1...9148-7 and the shared I / O line 9144 is not limited to half of the combination of the sense amplifiers 9170-0, 9170-1...9170-7 and the computing components 9148-0, 9148-1...9148-7 of the sense circuit system being formed above column 9192 of the memory cells (not shown) and half of the combination being formed below column 9192 of the memory cells. The number of such combinations of sense amplifiers and computing components configured to be coupled to the shared I / O line of the sense circuit system is also not limited to eight. Furthermore, the configuration of the shared I / O line 9144 is not limited to a configuration in which it is split into two sets of complementary digital lines 9168-1 and 9168-2 for separate coupling of each of them, nor is the location of the shared I / O line 9144 limited to being within the combination of the sense amplifier and the computing component that form the sense circuit system (e.g., rather than at either end of the combination of the sense amplifier and the computing component).

[0131] Figure 9 The circuit system described also illustrates column selection circuit systems 9194-1 and 9194-2, which are configured relative to a specific column 9192 of the subarray 9145, the associated complementary digital lines 9168-1 and 9168-2, and the shared I / O line 9144 (e.g., as described by...). Figure 7A The controller 722 shown in the diagram guides the data movement operation. For example, column selection circuit system 9194-1 has selection lines 0, 2, 4, and 6 configured to be coupled to corresponding columns, such as column 0 (332-0), column 2, column 4, and column 6. Column selection circuit system 9194-2 has selection lines 1, 3, 5, and 7 configured to be coupled to corresponding columns, such as column 1, column 3, column 5, and column 7. In various embodiments, combined with Figure 3A and Figure 3BThe described column selection circuit system 9194 may represent at least a portion of the functionality embodied in and contained within a multiplexer, such as an eight (8)-way multiplexer, a sixteen (16)-way multiplexer, etc.

[0132] The controller may be coupled to column selection circuitry 9194 to control a selection line (e.g., selection line 0) to access data values ​​stored in the sense amplifier, computing component, and / or present on the pair of complementary digital lines (e.g., 9168-1 and 9168-2, whereby selection transistors 9196-1 and 9196-2 are activated via a signal from selection line 0). Activating selection transistors 9196-1 and 9196-2 (e.g., as directed by the controller) enables coupling of the complementary digital lines 9168-1 and 9168-2 of the sense amplifier 9170-0, computing component 9148-0, and / or column 0 (9192-0) to move data values ​​on digital line 0 and digital line 0* to shared I / O line 9144. For example, the moved data value may be a data value from a specific row 919 stored (cached) in the sense amplifier 9170-0 and / or computing component 9148-0. The data values ​​from each of columns 0 to 7 can be similarly selected by a controller that activates the appropriate selection transistor.

[0133] Furthermore, enabling (e.g., activating) selection transistors (e.g., selection transistors 9196-1 and 9196-2) allows specific sensing amplifiers and / or computing components (e.g., 9170-0 and / or 9148-0, respectively) to couple to shared I / O line 9144, enabling data values ​​stored by the amplifiers and / or computing components to be moved to shared I / O line 9144 (e.g., placed on the shared I / O line and / or transmitted to the shared I / O line). In some embodiments, a column (e.g., column 9192-0) is selected at a time to couple to a specific shared I / O line 9144 to move (e.g., copy, transmit, and / or deliver) the stored data values. Figure 9 In the example configuration, shared I / O line 9144 is described as a shared, differential I / O line pair (e.g., shared I / O line and shared I / O line*). Therefore, selecting column 0 (9192-0) yields two data values ​​(e.g., two bits with values ​​of 0 and / or 1) from rows (e.g., row 919) and / or stored in sense amplifiers and / or computing components associated with complementary digital lines 9168-1 and 9168-2. These data values ​​can be input in parallel with each shared, differential I / O pair (e.g., shared I / O and shared I / O*) of shared differential I / O line 9144.

[0134] Figure 10AA block diagram illustrating how multiple shared I / O lines 10144 in the local data path of array 1025 are coupled to one of multiple segments 10150 of a computing unit 10198 having multiple logic bars 10152-1...10152-N. Figure 10A In an example embodiment, a group segment 10150 (e.g., a group quadrant) with multiple subarrays 10145-1…10145-32 is shown. Figure 10A In this example, thirty-two (32) subarrays are described in group quadrant 10150. However, the embodiment is not limited to this example. This example shows a group segment with 16K columns, which are multiplexed to a shared I / O line 10144 by sixteen (16) multiplexing. Thus, 16K columns are multiplexed to 1K shared I / O lines 10144, such that each group of 16 columns can provide data values ​​that can be moved in parallel to the computing unit 10198 as groups of 1024 (1K) bits. Here, the shared I / O line 10144 provides a 1K bit-wide data path to the computing unit 10198.

[0135] exist Figure 10A In the example, each logic bar 10152-1...10152-N has multiple computing components 10148-1...10148-Z, as shown in the text. Figure 7AThe same as described in the sensing circuit system 7138. In some embodiments, each of the plurality of logic bars 10152-1…10152-N is configured to perform computational functions using a plurality of computing components 10148-1…10148-Z. In some embodiments, each of the plurality of logic bars 10152-1…10152-Z may perform different logic operations using a plurality of computing components 10148-1…10148-Z. For example, in some embodiments, at least one of the plurality of logic bars 10152-1…10152-Z may be configured to perform long shift accelerator operations, such as an eight (8) sixty-four (64)-bit barrel shifter operation. This example may also provide partial reordering in an eight (8)-bit block and support aggregation / dispersion operations in a 256-bit block with eight-bit crossbars. In another instance, in some embodiments, at least one of the multiple logic bars 10152-1…10152-Z can be configured to perform Kogge-Stone acceleration to generate partial carry-lookahead to speed up horizontal addition. In another instance, in some embodiments, at least one of the multiple logic bars 10152-1…10152-Z can be configured to perform “block-of-information” mathematical acceleration. This instance can provide vertical mode acceleration in small bit groups (e.g., 4 or 8-bit blocks of information). In another instance, in some embodiments, the multiple logic bars 10152-1…10152-Z can be configured to act as explicit mask registers to implement Boolean operations as would be used by the compiler. As used herein, a “block-of-information” is intended to reference a bit length smaller than the addressed data line, for example, a 256-bit block of information (within a 128-byte addressable line) addressable to match the bit width to a particular interface. This may require matching a 256-bit interface to a 16K+ column memory array.

[0136] According to an embodiment, the controller associated with the group segment can execute microcode instructions to combine specific access rows in multiple subarrays 10145-1...10145-32 to guide 1K bits of data values ​​in parallel from each multiplexed column to specific computing components 10148-1...10148-Z in specific logic bars 10152-1...10152-N in computing unit 10198.

[0137] According to some embodiments, the butterfly network 10202 can be used to connect a 1K-bit data value to a corresponding one of a plurality of computing components 10148-1...10148-Z in a corresponding one of a plurality of logic bars 10152-1...10152-N. As an example, but not a limitation, the 1K bits of the data value can be moved in parallel to the logic bar associated with each of the 32 subarrays 10145-1...10145-32 in each of the four quadrants of the group segment 10150. In this example, each of the 128 logic bars 10152-1...10152-N having a 1K computing component 10148-1...10148-Z can be included in a computing unit 10198. The data values ​​of multiple computing components 10148-1...10148-Z loaded into the logic bars 10152-1...10152-N of computing unit 10198 can be manipulated according to microcode instructions from the controller to perform operations on the data values, such as AND, OR, NOR, XOR, addition, subtraction, multiplication, division, etc., as described in this document. Figure 7A The sensing circuit system 7138 is the same as described above. As stated above, once the data value is loaded into the calculation unit 10198, it can be processed according to the controller 722 ( Figure 7A The microcode instructions executed control computational operations in the computing unit much faster (e.g., at approximately 2 nanoseconds (ns)) than the data values ​​need to be moved back to subarrays 10145-1…10145-32. Figure 10A In the row of ), for example, computing unit 10198 can be used to perform computational operations at a much faster speed than instance time (e.g., approximately 60 nanoseconds (ns)), starting and accessing subarrays 10145-1...10145-32 ( Figure 10A The rows in ) may require the instance time.

[0138] exist Figure 10A In an example embodiment, multiple computing components 10148-1…10148-Z and / or logic bars 10152-1…10152-N in the computing units 10198 within the data paths of multiple shared I / O lines 10144 have a spacing equal to the spacing of the data paths of the shared I / O lines. According to an embodiment, the spacing of the data paths is a function, for example, the array subarrays 10145-1…10145-32 from the digital lines to the memory cells (…). Figure 10A The spacing is a multiple of the distance between the digital lines and the memory cell array (2×, 4×, etc.). For example, multiple computing components 10148-1……10148-Z and / or logic bars 10152-1……10152-N have a spacing that is an integer multiple of the distance between the digital lines and the memory cell array.

[0139] Figure 10BThis is a block diagram example illustrating multiple arrays coupled to multiple computing components in a computing unit via multiple shared I / O lines in the array's local data path, wherein the computing components have a spacing equal to the spacing of the data path of the shared I / O lines and a multiple of the spacing between the digital lines and the array. Figure 10B Examples illustrate multiple arrays, such as the group quadrants 10150-1 and 10150-2 of the array, which may have memory cells accessed by digital lines 10168.

[0140] exist Figure 10B Examples show quadrants 10150-1 and 10150-2 with a spacing of approximately sixteen thousand (16K) digital lines 10168 width, based on a given feature size (design rule) of the digital line manufacturing process. Multiple shared I / O lines 10144-1, 10144-2…10144-Z are also shown, which may have different spacings as a function of the given feature size (design rule) of the digital line manufacturing process, such as multiples thereof. Figure 10B In this example, the data paths of the multiple shared I / O lines 10144-1, 10144-Z have a spacing approximately sixteen (16) times larger than the spacing of the digital lines 10168. Therefore, in this example, approximately one thousand (1K) shared I / O lines 10144-1…10144-Z are shown, multiplexed to 16K digital lines 10168 via 16:1 multiplexers (e.g., 10204-1…10204-Z and 10206-1…10206-Z). However, the embodiments are not limited to the digital example provided herein, and more or fewer digital lines 10168 may be multiplexed to multiple shared I / O lines 10144-1…10144-Z. For example, shared I / O lines 10144-1...10144-Z may have a spacing other than 16 times (e.g., 16×) of the spacing of digital lines 10168 as set by the given feature dimensions (design rules) of the digital line manufacturing process.

[0141] like Figure 10B As shown in the example, for instance, a computing component among multiple computing components 10148-1…10148-Z and 10149-1…10149-Z can be associated with each shared I / O line 10144-1…10144-Z. The multiple computing components 10148-1…10148-Z and 10149-1…10149-Z can be located in multiple logical bars (e.g., ...) of the computing unit shown as 10198-1 and 10198-2 respectively. Figure 10A Within the range shown in the figure (10152-1, 10152-2...10152-N). Figure 10BAs shown in the example, the computing components, such as multiple computing components 10148-1...10148-Z and 10149-1...10149-Z, associated with each shared I / O line 10144-1...10144-Z, may have a spacing equal to the data path of the shared I / O line 10144-1...10144-Z and therefore the spacing of the digital line 10168 to the array (e.g., 10150-1 and 10150-2). According to various embodiments, since the computing components 10148-1...10148-Z and 10149-1...10149-Z in the data path of shared I / O lines 10144-1...10144-Z are not limited to a one-to-one (e.g., 1× times) relationship with the spacing of digital lines 8168, the computing components 10148-1...10148-Z and 10149-1...10149-Z are not limited to the “vertical” alignment of arrays 10150-1 and 10150-2, and in this example, can be as large as sixteen times (16×). Therefore, the computing components 10148-1...10148-Z and 10149-1...10149-Z in the data path of shared I / O lines 10144-1...10144-Z can be used to perform a more robust set of logical operations on the data values ​​stored therein (e.g., by having a larger footprint and space), such as the long shift acceleration mentioned above, while still being close to arrays 10150-1 and 10150-1 and not turned off in the peripheral region of the array or memory die.

[0142] Although specific embodiments have been illustrated and described herein, those skilled in the art will understand that arrangements calculated to achieve the same results may replace the specific embodiments shown. This disclosure is intended to cover modifications or variations of various embodiments of this disclosure. It should be understood that the above description is illustrative rather than restrictive. Combinations of the above embodiments and other embodiments not specifically described herein will be apparent to those skilled in the art upon review of the above description. The scope of the various embodiments of this disclosure includes other applications using the above structures and methods. Therefore, the scope of the various embodiments of this disclosure should be determined by reference to the appended claims and the full scope of the equivalents granted by those claims.

[0143] In the foregoing detailed description, various features have been grouped together in a single embodiment for the purpose of brevity. This approach of the disclosure should not be construed as reflecting an intention that the disclosed embodiments must use more features than are expressly stated in each claim. In fact, as reflected in the appended claims, the subject matter of the invention lies in less than all the features of a single disclosed embodiment. Therefore, the appended claims are hereby incorporated into the detailed description, wherein each claim is, in itself, a separate embodiment.

Claims

1. A memory device (120) for performing artificial intelligence (AI) operations, comprising: Several memory arrays (125); and Controller (122), comprising: A plurality of activation function registers (232-34…232-39, 332-34…332-39) are configured to define activation functions for AI operations, each of the plurality of activation function registers (232-34…232-39, 332-34…332-39) comprising a plurality of bits, wherein the controller (122) is configured to select a defined activation function by programming the first bit of the plurality of activation function registers to a first state executing an identity function for the AI ​​operation and programming the second bit of the plurality of activation function registers to the first state executing a binary step function for the AI ​​operation, modifying the defined activation function to a custom activation function, and executing the custom activation function for the AI ​​operation without a host by programming the third bit of the plurality of activation function registers to the first state in response to the result of a previous AI operation, wherein a different activation function is executed when the previous AI operation is executed.

2. The memory device of claim 1, wherein the activation function register in the plurality of activation function registers (232-34……232-39, 332-34……332-39) includes one or more bits, wherein the controller (122) is configured to identify a predefined activation function by programming the one or more bits.

3. The memory device of claim 1, wherein the activation function register in the plurality of activation function registers (232-34……232-39, 332-34……332-39) includes one or more bits, wherein the controller (122) is configured to identify the custom activation function by programming the one or more bits.

4. The memory device according to claim 2, wherein the predefined activation function is at least one of the following: identity function, binary step function, logic function, hyperbolic tangent function, arctangent function, soft sign function, inverse square root unit function, modified linear unit function, leakage modified linear unit function, parameterized modified linear unit function, randomized leakage modified linear unit function, exponential linear unit function, or scaled exponential linear unit function.

5. The memory device according to claim 1, wherein the preceding AI operation is a debugging operation.

6. A device for performing artificial intelligence (AI) operations, comprising: Several memory arrays (125); and Controller (122), comprising: A register containing bits, wherein the controller (122) is configured to enable the use of an activation function used in artificial intelligence operations by programming the bits of the register; and A plurality of activation function registers (232-34…232-39, 332-34…332-39) are configured to define activation functions for the AI ​​operation, each of the plurality of activation function registers (232-34…232-39, 332-34…332-39) comprising a plurality of bits, wherein the controller (122) is configured to select a defined activation function, modify the defined activation function to a custom activation function, and execute the custom activation function for the AI ​​operation without a host by programming the first bit of the plurality of activation function registers to execute an identity function for the AI ​​operation and programming the second bit of the plurality of activation function registers to execute a binary step function for the AI ​​operation to the first state, wherein a different activation function is executed when the previous AI operation is executed.

7. The device according to claim 6, wherein the activation function is a predefined activation function.

8. The device of claim 7, wherein the indicator of the activation function register (232-34, 332-34) is programmed to a first state to begin the predefined activation function.

9. The device of claim 8, wherein the indicator of the activation function register (232-34, 332-34) is programmed to a second state to stop the predefined activation function.

10. The device of claim 8, wherein the indicator of the activation function register (232-34, 332-34) is programmed to a second state to prevent execution of the predefined activation function.

11. The device according to claim 6, wherein the activation function is the custom activation function.

12. The device of claim 11, wherein the indicator of the activation function register (232-34, 332-34) is programmed to a first state to begin the custom activation function.

13. The device of claim 12, wherein the indicator of the activation function register (232-34, 332-34) is programmed to a second state to stop the custom activation function.

14. The device of claim 12, wherein the indicator of the activation function register (232-34, 332-34) is programmed to a second state to prevent execution of the custom activation function.

15. A method for performing artificial intelligence (AI) operations, comprising: The use of activation functions in artificial intelligence operations is enabled by programming the bits of the registers through the controller of the memory device; The activation function is executed when the AI ​​operation is performed; and Different activation functions can be selected by programming the first bit of the activation function register to execute the first state of the identity function for the AI ​​operation and programming the second bit of the activation function register to execute the first state of the binary step function for the AI ​​operation; the different activation functions can be modified into custom activation functions; and the custom activation function can be executed by the memory controller without a host by programming the third bit of the activation function register to the first state in response to the result of the AI ​​operation.

16. The method of claim 15, wherein the activation function is a predefined activation function.

Images