Processing unit comprising activation function unit

Abstract

The present disclosure relates to a processing unit comprising an activation function unit. The processing unit may include an activation function unit. Data may be received at a plurality of registers of a processing unit of a memory subsystem. The data may be received at a multiply-accumulate MAC unit coupled to the plurality of registers. A first plurality of operations may be performed at the MAC unit to generate a first output. The first output may be provided to the activation function unit. The first output may be provided from the AFU to the plurality of registers using a bus or signal line that couples the plurality of registers to the AFU.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to a processing unit, and more particularly, to a processing unit including an activation function unit. Background Art

[0002] Various types of electronic devices, such as digital logic circuits and memory systems, can store and process data. A digital logic circuit is an electronic circuit that processes digital signals or binary information that can take two possible values, usually represented as 0 and 1. A digital logic circuit can use logic gates to manipulate and transform digital signals or binary information. Digital logic circuits can be used, for example, in a wide range of electronic devices, including computers, calculators, digital clocks, and many other electronic devices that employ digital processing. Digital logic circuits can be designed to perform specific logic operations on digital inputs to produce digital outputs, and in some examples, can be combined to form more complex circuits to perform more complex operations. A memory device can include one or more memory devices that store data. A memory device can be, for example, a non-volatile memory device and a volatile memory device. In general, a host system can utilize a memory system to store data at a memory device and retrieve data from a memory device. Summary of the invention

[0003] Aspects of the present disclosure relate to a device for a processing unit including an activation function unit, comprising: a memory cell array; a processing unit coupled to the memory cell array, the processing unit comprising: a plurality of registers configured to store data provided by the array; a multiply-accumulate (MAC) unit coupled to the plurality of registers and configured to perform a first plurality of operations on the data; an activation function unit (AFU) coupled to the MAC unit and configured to perform a second plurality of operations on an output of the MAC unit; and a plurality of signal lines coupling the AFU to the plurality of registers and configured to provide the output of the AFU unit to the plurality of registers to perform polynomial operations.

[0004] Another aspect of the present disclosure relates to a method for a processing unit including an activation function unit, comprising: receiving data at a plurality of registers of a processing unit of a memory subsystem; receiving the data at a multiply-accumulate (MAC) unit coupled to the plurality of registers; performing a first plurality of operations at the MAC unit to produce a first output; providing the first output to an activation function unit (AFU); and providing the first output from the AFU to the plurality of registers using a bus that couples the plurality of registers to the AFU.

[0005] Yet another aspect of the present disclosure relates to a device for a processing unit including an activation function unit, comprising: a memory cell array; a processing unit coupled to the memory cell array, the processing unit comprising: a first plurality of registers configured to store data provided by the array; a multiply-accumulate (MAC) unit coupled to the first plurality of registers and configured to perform a first plurality of operations on the data; an activation function unit (AFU) coupled to the MAC unit and configured to perform a second plurality of operations using a first output provided by the MAC unit; a first plurality of signal lines configured to provide a second output of the AFU as an input to the AFU; wherein the AFU is further configured to perform a second plurality of operations using the input. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The present disclosure will be more fully understood from the detailed description given below and from the accompanying drawings of various embodiments of the present disclosure.

[0007] Figure 1 An example computing system including a memory subsystem according to some embodiments of the present disclosure is described.

[0008] Figure 2 is a block diagram of a memory device and a processing unit according to some embodiments of the present disclosure.

[0009] Figure 3 is a block diagram of a processing unit according to some embodiments of the present disclosure.

[0010] Figure 4 is a block diagram of a processing unit implemented using a bus according to some embodiments of the present disclosure.

[0011] Figure 5 The description shows a table of pseudo code for implementing a polynomial function according to several embodiments of the present disclosure.

[0012] Figure 6 The description shows a table of pseudocode for implementing a hard Gaussian error linear activation unit approximation according to several embodiments of the present disclosure.

[0013] Figure 7 The description shows a table of pseudocode for implementing integer Gaussian error linear activation unit approximation according to several embodiments of the present disclosure.

[0014] Figure 8 is a flowchart corresponding to a method for implementing a processing unit including an activation function unit according to some embodiments of the present disclosure.

[0015] Fig. 9 is a block diagram of an example computer system in which embodiments of the present disclosure may operate. DETAILED DESCRIPTION

[0016] Aspects of the present disclosure relate to a processing unit including an activation function unit (AFU). The processing unit may include a plurality of registers, a multiply-accumulate (MAC) unit, and an activation function unit. The plurality of registers may store data provided by an array. The MAC unit may perform a first plurality of operations on the data. The AFU may perform a second plurality of operations on the output of the MAC unit. The processing unit may also include a plurality of signal lines that couple the AFU to the plurality of registers and are configured to provide the output of the AFU unit to the plurality of registers to perform polynomial operations. The polynomial operation may be used as an activation function and / or may be used to approximate an activation function of an artificial neural network (ANN). The ANN may be implemented in a memory subsystem. The memory subsystem may be a storage system, a storage device, a memory module, or a combination thereof. An example of a memory subsystem is a storage system, such as a dynamic random access memory (DRAM). Figure 1 Examples of storage devices and memory modules are described in , etc. In general, a host system may utilize a memory subsystem that includes one or more components, such as a memory device that stores data. The host system may provide data to be stored at the memory subsystem and may request data to be retrieved from the memory subsystem.

[0017] Although some non-limiting examples herein are generally described in terms of applicability to memory subsystems and / or to memory devices, the embodiments are not limited thereto, and aspects of the present disclosure may also be applied to systems on a chip, computing subsystems, data collection and processing, storage, networking, communications, power, artificial intelligence, control, telemetry, sensing and monitoring, digital entertainment, and other types of systems / subsystems and / or devices. Thus, aspects of the present disclosure may be applied to these components in order to calculate and / or implement the activation function of an ANN, as described herein.

[0018] As used herein, ANN can provide learning by forming a probability weight association between input and output. Probability weight association can be provided by multiple nodes that make up ANN. Nodes together with weights, biases and activation functions can be used to generate the output of ANN based on the input of ANN. Multiple nodes of ANN can be grouped to form a layer of ANN. A signal provided from a layer consisting of multiple nodes to another layer consisting of multiple nodes can cause ANN to process input to generate output. Based on the activation of each of the multiple nodes of the layer, a signal is provided from a layer consisting of multiple nodes to another layer. When a specific node receives multiple signals (e.g., spikes) sufficient to reach a threshold, activation may occur. When a specific node reaches a threshold, the specific node will provide its signal to one or more nodes of the subsequent layer of ANN.

[0019] The thresholds and / or combinations of weights, biases, and signals provided by other nodes of the ANN may be defined by an activation function. An activation function may be used to determine when to forward a propagated signal received by a particular node of the ANN. An ANN may use an activation function to use relevant signals and suppress irrelevant signals.

[0020] There are many types of activation functions that can be used for the specific problem being solved by the ANN. For example, the activation function may include a binary step activation function, a sigmoid linear unit (SiLU) activation function, a hyperbolic tangent (TanH) activation function, a rectified linear unit (ReLU) activation function, a leaky ReLU activation function, an exponential linear unit (ELU) activation function, a soft maximum activation function, and a Gaussian error linear unit (GELU) activation unit, among others.

[0021] A processing unit (PU) used to implement an ANN may be configured to implement an activation function. However, implementing one or a limited number of activation functions limits the applicability of the PU to specific problems that may be better solved using a different activation function that the PU does not support.

[0022] To address these and other deficiencies of current approaches, embodiments of the present disclosure allow for the implementation of a PU utilizing an activation function unit. A PU may also be referred to as a Processing-In-Memory (PIM) unit. A PU may perform several operations in a memory subsystem (e.g., a memory device), including operations for implementing or training an ANN utilizing an activation function. An activation function unit (AFU) may be used to implement a greater number of activation functions by performing atomic operations and by utilizing a multiply-accumulate (MAC) unit. The AFU and MAC units may be used to approximate a greater number of activation functions than a PU configured to implement a limited number of activation functions. The output of an ANN may be correct a greater percentage of the time when utilizing a PU that approximates a greater number of activation functions than a PU that implements a single activation function or a limited number of activation functions.

[0023] Figure 1 An example computing system 100 is illustrated that includes a memory subsystem 103 according to some embodiments of the present disclosure. Memory subsystem 103 may include media such as one or more volatile memory devices (e.g., memory device 110), one or more non-volatile memory devices (e.g., memory device 109), or a combination thereof.

[0024] The memory subsystem 103 may be a storage device, a memory module, or a mixture of storage devices and memory modules. Examples of storage devices include solid state drives (SSDs), flash drives, universal serial bus (USB) flash drives, embedded multimedia controller (eMMC) drives, universal flash storage devices (UFS) drives, secure digital (SD) cards, and hard disk drives (HDDs). Examples of memory modules include dual in-line memory modules (DIMMs), small outline DIMMs (SO-DIMMs), and various types of non-volatile dual in-line memory modules (NVDIMMs).

[0025] Computing system 100 may be a computing device such as a desktop computer, a laptop computer, a server, a network server, a mobile device, a vehicle (e.g., an airplane, drone, train, automobile, or other transportation vehicle), a device with Internet of Things (IoT) capabilities, an embedded computer (e.g., an embedded computer included in a vehicle, industrial equipment, or a networked commercial device), or such a computing device that includes a memory and a processing device.

[0026] In other embodiments, the computing system 100 may be deployed on or otherwise included in a computing device, such as a desktop computer, a laptop computer, a server, a network server, a mobile computing device, a vehicle (e.g., an airplane, drone, train, car, or other transportation vehicle), a device with Internet of Things (IoT) capabilities, an embedded computer (e.g., an embedded computer included in a vehicle, industrial equipment, or a networked commercial device), or such computing devices including a memory and a processing device. As used herein, the term "mobile computing device" generally refers to a handheld computing device having a tablet or tablet form factor. For example, a mobile computing device may include a mobile phone, a tablet, an entertainment device, a gaming device, a navigation device, a camera / camcorder, etc. In general, a tablet form factor may include a display screen of approximately between 3 inches and 5.2 inches (measured diagonally), while a tablet form factor may include a display screen of approximately between 5.2 inches and 7 inches (measured diagonally). However, examples of a "mobile computing device" are not limited thereto, and in some embodiments, a "mobile computing device" may refer to an IoT device as well as other types of edge computing devices.

[0027] The computing system 100 may include a host system 102 coupled to one or more memory subsystems 103. In some embodiments, the host system 102 is coupled to memory subsystems 103 of different types. Figure 1An example of a host system 102 coupled to a memory subsystem 103 is illustrated. As used herein, "coupled to" or "coupled with" generally refers to a connection between components, which can be an indirect communication connection or a direct communication connection (e.g., without intervening components), whether wired or wireless, including connections such as electrical connections, optical connections, magnetic connections, and the like.

[0028] The host system 102 may include a processor chipset and a software stack executed by the processor chipset. The processor chipset may include one or more cores, one or more caches, a memory controller (e.g., an SSD controller), and a storage protocol controller (e.g., a PCIe controller, a SATA controller). The host system 102 uses the memory subsystem 103 to, for example, write data to the memory subsystem 103 and read data from the memory subsystem 103.

[0029] Host system 102 includes a processing unit 104. Processing unit 104 may be a central processing unit (CPU) configured to execute an operating system. In some embodiments, processing unit 104 includes a complex instruction set computer architecture, such as x86 or other architecture suitable for use as a CPU for host system 102.

[0030] The host system 102 may be coupled to the memory subsystem 103 via a physical host interface. Examples of the physical host interface include, but are not limited to, a Serial Advanced Technology Attachment (SATA) interface, a Peripheral Component Interconnect Express (PCIe) interface, a Universal Serial Bus (USB) interface, a Fibre Channel, a Serial Attached SCSI (SAS), a Small Computer System Interface (SCSI), a Double Data Rate (DDR) memory bus, a Dual In-line Memory Module (DIMM) interface (e.g., a DIMM slot interface supporting Double Data Rate (DDR)), an Open NAND Flash Interface (ONFI), Double Data Rate (DDR), Low Power Double Data Rate (LPDDR), or any other interface. The physical host interface may be used to transfer data between the host system 102 and the memory subsystem 103. When the memory subsystem 103 is coupled to the host system 102 through a PCIe interface, the host system 102 may further utilize an NVM Express (NVMe) interface to access components (e.g., memory device 109). The physical host interface may provide an interface for passing control, address, data, and other signals between the memory subsystem 103 and the host system 102 . Figure 1 Memory subsystem 103 is illustrated as an example. In general, host system 102 can access multiple memory subsystems via the same communication connection, multiple separate communication connections, and / or a combination of communication connections.

[0031] The memory devices 109, 110 may include any combination of different types of non-volatile memory devices and / or volatile memory devices. Volatile memory devices (e.g., memory device 110) may be, but are not limited to, random access memory (RAM), such as dynamic random access memory (DRAM) and synchronous dynamic random access memory (SDRAM).

[0032] Some examples of non-volatile memory devices (e.g., memory device 109) include non-and (NAND) type flash memory and write-in-place memory, such as a three-dimensional cross-point ("3D cross-point") memory device, which is a cross-point array of non-volatile memory cells. The cross-point array of non-volatile memory can perform bit storage based on body resistance changes in combination with a stackable cross-gate data access array. In addition, compared to many flash-based memories, cross-point non-volatile memory can perform write-in-place operations, where non-volatile memory cells can be programmed without previously erasing the non-volatile memory cells. For example, NAND type flash memory includes two-dimensional NAND (2D NAND) and three-dimensional NAND (3D NAND).

[0033] Each of the memory devices 109, 110 may include one or more arrays of memory cells. One type of memory cell, for example a single level cell (SLC), may store one bit per cell. Other types of memory cells, such as multi-level cells (MLC), three-level cells (TLC), four-level cells (QLC), and five-level cells (PLC), may store multiple bits per cell. In some embodiments, each of the memory devices 109 may include one or more arrays of memory cells, such as SLC, MLC, TLC, QLC, or any combination thereof. In some embodiments, a particular memory device may include an SLC portion and an MLC portion, a TLC portion, a QLC portion, or a PLC portion of a memory cell. The memory cells of the memory device 109 may be grouped into pages, which may refer to a logical unit of a memory device for storing data. For some types of memory (e.g., NAND), pages may be grouped to form blocks.

[0034] Although nonvolatile memory components such as a three-dimensional cross-point array of nonvolatile memory cells and NAND-type memory (e.g., 2D NAND, 3D NAND) are described, the memory device 109 may be based on any other type of nonvolatile memory or storage device, such as read-only memory (ROM), phase-change memory (PCM), self-select memory, other chalcogenide-based memories, ferroelectric transistor random access memory (FeTRAM), ferroelectric random access memory (FeRAM), magnetic random access memory (MRAM), spin transfer torque (STT)-MRAM, conductive bridging RAM (CBRAM), resistive random access memory (RRAM), oxide-based RRAM (OxRAM), non-OR (NOR) flash memory, and electrically erasable programmable read-only memory (EEPROM).

[0035] The memory subsystem controller 105 (or, for simplicity, the controller 105) may communicate with the memory device 109 to perform operations such as reading data, writing data, or erasing data at the memory device 109, and other such operations. The memory subsystem controller 105 may include hardware, such as one or more integrated circuits and / or discrete components, buffer memory, or a combination thereof. The hardware may include digital circuitry with dedicated (i.e., hard-coded) logic for performing the operations described herein. The memory subsystem controller 105 may be a microcontroller, dedicated logic circuitry (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), or other suitable processor.

[0036] The memory subsystem controller 105 may include a processor 106 (e.g., a processing device) configured to execute instructions stored in a local memory 107. In the illustrated example, the local memory 107 of the memory subsystem controller 105 includes an embedded memory configured to store instructions for executing various processes, operations, logic flows, and routines that control the operation of the memory subsystem 103, including handling communications between the memory subsystem 103 and the host system 102.

[0037] In some embodiments, local memory 107 may include memory registers that store memory pointers, fetched data, etc. Local memory 107 may also include read-only memory (ROM) for storing microcode. Figure 1 The example memory subsystem 103 in FIG. 1 is illustrated as including a memory subsystem controller 105, but in another embodiment of the present disclosure, the memory subsystem 103 does not include a memory subsystem controller 105 and may instead rely on external control (e.g., provided by an external host or by a processor or controller separate from the memory subsystem).

[0038] In general, the memory subsystem controller 105 may receive commands or operations from the host system 102 and may convert the commands or operations into instructions or appropriate commands to achieve the desired access to the memory device 109 and / or the memory device 110. The memory subsystem controller 105 may be responsible for other operations such as wear leveling operations, garbage collection operations, error detection and error correction code (ECC) operations, encryption operations, cache operations, and address translation between logical addresses (e.g., logical block addresses (LBAs), namespaces) and physical addresses (e.g., physical block addresses, physical media locations, etc.) associated with the memory device 109. The memory subsystem controller 105 may further include a host interface circuit system to communicate with the host system 102 via a physical host interface. The host interface circuit system may convert commands received from the host system into command instructions to access the memory device 109 and / or the memory device 110, and convert responses associated with the memory device 109 and / or the memory device 110 into information for the host system 102.

[0039] The memory subsystem 103 may also include additional circuitry or components not illustrated. In some embodiments, the memory subsystem 103 may include a cache or buffer (e.g., DRAM) and address circuitry (e.g., row decoder and column decoder) that may receive addresses from the memory subsystem controller 105 and decode the addresses to access the memory device 109 and / or the memory device 110.

[0040] In some embodiments, the memory device 109 includes a local media controller 111 that operates in conjunction with the memory subsystem controller 105 to perform operations on one or more memory cells of the memory device 109. An external controller (e.g., the memory subsystem controller 105) can manage the memory device 109 externally (e.g., perform media management operations on the memory device 109). In some embodiments, the memory device 109 is a managed memory device, which is a raw memory device combined with a local controller (e.g., the local controller 111) for media management within the same memory device package. An example of a managed memory device is a managed NAND (MNAND) device.

[0041] The memory subsystem 103 may include PU control circuitry 108. Although Figure 1Although not shown in order not to obscure the drawings, PU control circuitry 108 may include various circuitry for facilitating aspects of the present disclosure. In some embodiments, PU control circuitry 108 may include specialized circuitry in the form of an ASIC, FPGA, state machine, hardware processing device, and / or other logic circuitry that may allow PU control circuitry 108 to orchestrate and / or control PU 112 to perform multiple operations to approximate multiple activation functions (particularly with respect to a system on a chip) in accordance with the present disclosure.

[0042] In some embodiments, the memory subsystem controller 105 includes at least a portion of the PU control circuit system 108. For example, the memory subsystem controller 105 may include a processor 106 (processing device) that is configured to execute instructions stored in the local memory 107 for performing the operations described herein. In some embodiments, the PU control circuit system 108 is part of the host system 103, the application, or the operating system. The PU control circuit system 108 may reside on the memory subsystem 103 and / or the memory subsystem controller 105. As used herein, the term "resides on" refers to something that is physically located on a specific component. For example, the PU control circuit system 108 "resides on the memory subsystem 103" refers to a state in which the hardware circuit system including the PU control circuit system 108 is physically located on the memory subsystem 103. Herein, the term "resides on" may be used interchangeably with other terms such as "deployed on" or "located on".

[0043] PU control circuitry 108 may provide commands to PU 112 implemented in memory devices 109, 110. Implementing PU 112 in memory devices 109, 110 has the advantage of limiting the movement of data between the arrays of memory devices 109, 110 and PU 112. Limiting the movement of data between the arrays of memory devices 109, 110 and PU 112 may improve the efficiency of operating an ANN (not shown). In various examples, the ANN may be implemented in memory subsystem 103 and / or memory devices 109, 110. PU 112 may implement an activation function and may provide the result of the activation function to the ANN for execution of the ANN. In various examples, the activation function may be used for purposes other than execution of the ANN.

[0044] Figure 2 is a block diagram of a memory device 209 and a processing unit according to some embodiments of the present disclosure. The memory device 209 is labeled corresponding to Figure 1 However, the memory device 209 may be implemented as a volatile memory device or a non-volatile memory device and may also correspond to Figure 1The memory device 209 includes memory banks 221-1, 221-2, 221-3, 221-4, 221-5, 221-6, 221-7, 221-8, 221-9, 221-10, 221-11, 221-12, 221-13, 221-14, 221-15, 221-16 (referred to as memory banks 221). The memory device 209 also includes PUs 212-1, 212-2, 212-3, 212-4, 212-5, 212-6, 212-7, 212-8 (referred to as PUs 212).

[0045] The memory banks 221 may be arranged into groups (e.g., memory bank groups). For example, the memory banks 221-1 and 221-2 may be a first group. The memory banks 221-3 and 221-4 may be a second group. The memory banks 221-5 and 221-6 may be a third group. The memory banks 221-7 and 221-8 may be a fourth group. The memory banks 221-9 and 221-10 may be a fifth group. The memory banks 221-11 and 221-12 may be a sixth group. The memory banks 221-13 and 221-14 may be a seventh group. The memory banks 221-15 and 221-16 may be an eighth group.

[0046] PU 212 may be part of a bank group or may be implemented to receive data from banks in a bank group. For example, PU 212-1 may be part of a first group. PU 212-2 may be part of a second group. PU 212-3 may be part of a third group. PU 212-4 may be part of a fourth group. PU 212-5 may be part of a fifth group. PU 212-6 may be part of a sixth group. PU 212-7 may be part of a seventh group. PU 212-8 may be part of an eighth group.

[0047] PU 212-1 may receive data from memory banks 221-1 and 221-2. PU 212-2 may receive data from memory banks 221-3 and 221-4. PU 212-3 may receive data from memory banks 221-5 and 221-6. PU 212-4 may receive data from memory banks 221-7 and 221-8. PU 212-5 may receive data from memory banks 221-9 and 221-10. PU 212-6 may receive data from memory banks 221-11 and 221-12. PU 212-7 may receive data from memory banks 221-13 and 221-14. PU 212-8 may receive data from memory banks 221-15 and 221-16. PU 212 may be implemented between memory banks 221 and global bus 222. The implementation of the PU per bank group is exemplary. Other examples of PUs and bank groups may be implemented. For example, one PU may be implemented per bank or one PU may be implemented per multiple bank groups, as well as other implementations of PU 212 and bank 221.

[0048] The MAC array 232 may include a plurality of registers and a number of arithmetic and logic processing units. For example, each of the PUs 212 may include a MAC array 232. The MAC array 232 may also be referred to as a MAC unit. The MAC array 232 may include hardware for calculating the product of two numbers and using an accumulator to accumulate the results. The registers of the MAC array 232 that store the accumulated results may also be referred to as accumulators. The PU 212 may also include an AFU 224.

[0049] Available Figure 1 PU 212 may be controlled by PU control circuitry 108 of the PU control circuitry 108. In various examples, PU control circuitry 108 may be part of a command decoder. For example, process-in-memory (PIM) commands may be provided from PU control circuitry 108 to memory device 209 on a standard DDR command bus. The extended command decoder may decode the commands and send control signals to PU 212 in addition to sending DDR commands such as read, write mode register read and write commands.

[0050] AFU 234 may include hardware and / or firmware for performing a plurality of operations. The operations performed by AFU 234 may include atomic operations. As described herein, atomic operations include any basic arithmetic operations, such as addition, subtraction, and multiplication. Atomic operations may include operations not performed by MAC array 232. Figure 5 , 6 Additional examples of operations performed by AFU 234 are provided in and 7.

[0051] Figure 331 is a block diagram of a PU 312 according to some embodiments of the present disclosure. The PU 312 includes registers 331-1 (OP / SHIFT RA A) and 331-2 (OP RA B), a MAC array 332, a shift register 333, and an AFU 334. The PU 312 also includes multiplexers (MUX) 335-1, 335-2, a de-MUX 336, a register 331-3 (OP RA X), and signal lines 327, 328, 329. The signal lines may be referred to herein as "lines".

[0052] In various examples, PU 312 may receive data from multiple banks of a memory device. The data may be stored in registers 331-1, 331-2. For example, data provided by a first register may be stored in register 331-1 and data provided by a second register may be stored in register 331-2. Registers 331-1, 331-2 may provide data to MAC array 332. The data provided by registers 331-1, 331-2 may be referred to as operators (e.g., operator A, operator B). Register 331-1 may be coupled to a clock signal as represented by the inverted triangle shown in register 331-1. The clock signal may be used to enable register 331-1 to right-shift or left-shift the data stored therein.

[0053] MAC array 332 may include multipliers 337, adders 338, and registers 339. Multiplier 337 may perform a multiplication operation using data stored in registers 331-1 and 331-2. Adder 338 may perform an addition operation to accumulate the result of multiplier 337. Adder 338 may store the result in register 339. Register 339 may be reset (e.g., mac_reset). Register 339 may be reset using a clock signal represented by an inverted triangle. In the previous method, a reset signal is used to reset the PU unit including registers 331-1, 331-2. In several examples, a reset signal is used to reset register 339 instead of registers 331-1, 331-2, 331-3.

[0054] The multiplier 337 may be an integer multiplier. The multiplier 337 may multiply two fixed-point values ​​(e.g., operator A and operator B) provided by the registers 331-1, 331-2. The adder 338 may be an integer adder. The adder 338 may add the multiplication result to the value stored in the register 339. The register 339 may also be referred to as an accumulation register or an accumulator. The register 339 may store the accumulation result. The register 333 may reduce the MAC result.

[0055] The data stored in registers 331-1, 331-2 may be referred to as prefetch data. Registers 331-1, 331-2 and the MAC array size may match the prefetch data width and the integer data width. For example, the prefetch width (e.g., the width of the prefetch data) may be 128 bits and the MAC array 332 may provide 8 bits of data, which means that 16 MAC arrays (e.g., 128 / 8=16) may be implemented in the PU 312. There may be 16 instances of the MAC array 332 in the PU 312.

[0056] AFU 334, MUX 335-1, 335-2, de-MUX 336, registers 331-3, and lines 327, 328, 329 enable PU 312 to perform a variety of non-linear activation functions. Various activation functions are more complex and utilize resources of memory devices that may be better served elsewhere. PU 312 may approximate and / or perform various non-linear activation functions, such as GELU, SiLU, ELU, and ReLU using polynomial approximation, as well as other activation functions.

[0057] The AFU 334 may perform atomic functions that may supplement operations supported by the MAC array 332. For example, the AFU 334 may perform operations that the MAC array 332 does not perform. Line 327 enables the output of the AFU 334 to be provided as input to the MAC array 332 via registers 331-1, 331-2. Providing the output of the AFU 334 as input to the MAC array 332 enables the PU 312 to perform polynomial operations. As used herein, a polynomial operation includes multiple operations that reflect a polynomial equation when combined. Lines 328, 329 also enable the output of the AFU 334 to be provided as input to the AFU 334 via registers 331-3. For example, registers 331-3 may store the output of the AFU 334 and the output of the AFU 334 may be provided as input to the AFU 334 using lines 328, 329. Although a single line is shown for lines 327 , 328 , 329 , each of lines 327 , 328 , 329 may include multiple signal lines.

[0058] MUX 335-1, 335-2 and de-MUX 336 enable the output of AFU 334 to be provided as input to MAC array 332 or AFU 334. For example, de-MUX 336 may receive the output of AFU 334 and may cause the output of AFU 334 to be provided to a global bus or register 331-3 and MUX 335-1 of a memory device using lines 327, 329 based on the configuration of de-MUX 336.

[0059] MUX 335-1 may receive data from a memory bank or from de-MUX 336. MUX 335-1 may be configured to provide data received from de-MUX 336 via line 327 to registers 331-1, 331-2. Providing the output of AFU 334 to registers 331-1, 331-2 enables registers 331-1, 331-2 to store the output of AFU 334 as operators A and B.

[0060] The register 331-1 may perform a shift operation on the output of the AFU 334. The register 331-1 may provide the output of the AFU 334 to the MAC array 332 or provide the shifter output of the AFU 334 to the MAC array 332. The MAC array 332 may receive the output of the AFU 334 as operators A and B. The MAC array 332 may perform a plurality of operations using the output of the AFU 334 as input. The output of the MAC array 332 may be provided to the register 333 and from the register 333 to the MUX 335-2.

[0061] DeMUX 336 may also cause the output of AFU 334 to be provided to register 331-3 using line 329. Register 331-1 may provide the output of AFU 334 to MUX 335-2 via line 328. MUX 335-2 may be configured to provide the output of register 333 or the data stored in register 331-3 as input to AFU 334. MUX 335-2, deMUX 336, line 329, and register 331-3 may allow AFU 334 to perform atomic operations on the outputs of different atomic operations. In various examples, the atomic operations may include operations for estimating a nonlinear activation function or a portion of a nonlinear activation function. For example, AFU 334 may perform a -sign() operation or a ReLU6 operation. AFU 334 may include logic blocks that implement functions for clipping an input value or a given maximum or minimum value. AFU 334 may also include logic blocks that implement functions for returning the sign of an input value.

[0062] In various examples, AFU 334 and MUX 335-1, 335-2, de-MUX 336, lines 327, 328, 329, and registers 331-3 may enable PU 312 to combine several operations to construct a non-linear activation function or approximate a non-linear activation function. The non-linear activation function constructed using PU 312 may be provided to the ANN via a global bus or may be stored back to a bank of a memory device, for example. Figure 4 An implementation of the PU 312 that utilizes a bus rather than wires is described. Figure 5 , 6 , 7 provides an example of pseudo code for implementing a non-linear activation function using PU 312.

[0063] Figure 4 4 is a block diagram of a PU 412 implemented using a bus according to some embodiments of the present disclosure. PU 412 includes registers 431-1 and 431-2, a MAC array 432, a shift register 433, and an AFU 434. PU 312 also includes tri-state buffers 441-1, 441-2, 441-3 and buses 442, 443, 444.

[0064] PU 412 may utilize buses 442, 443, 444 to provide the output of AFU 434 as input to itself and / or to provide the output of AFU 434 as input to registers 431-1, 431-2. PU 412 may utilize buses 442, 443, 444 instead of utilizing Figure 3 Lines 327, 328, 329 provide the output of AFU 434 to MAC array 432 and / or AFU 434. Figure 3 The difference between the lines 327, 329, 328 (eg, signal lines) and the buses 442, 443, 444 is that the buses contain multiple signal lines while the lines contain a single signal line. The difference between the lines and the buses 442, 443, 444 may result in the use of the buses 442, 443, 444 instead of the lines in various scenarios.

[0065] In various examples, tri-state buffers 441-1, 441-2, 441-3 may allow data to be provided to buses 442, 443, 444. For example, tri-state buffer 441-1 may be activated or deactivated to allow AFU 434 to provide data to buses 442, 443, or a global bus (not shown). Tri-state buffer 441-2 may be activated to allow register 431-3 to provide data to AFU 434 via bus 444. Tri-state buffer 441-3 may also be activated to allow bus 442 to provide data to registers 431-1 and 431-2.

[0066] Figure 5 The description shows a table 551 of pseudo code for implementing the polynomial function 550 according to several embodiments of the present disclosure. Table 551 shows pseudo code of steps, values ​​associated with the steps, and results of performing the steps.

[0067] The polynomial function 550 is defined as f(x)=a+bs+ax 2 +βx 3=a+bx(1+cx(1+dx)), where x is an input and a, b, c, d, α, and β are constants. At step 1, register X (RAX) may be loaded with a value of x. RAX may be loaded by, for example, reading a value x from a memory array and providing the value x to a MAC array. The MAC array may multiply x by 1 to output the value x. A right shift register (e.g., Figure 3 333 in RA X may avoid shifting the value x. The value x may be provided to the AFU. A command to pass the value x may be provided to the AFU without performing a step using the value x. The output of the AFU (e.g., x) may be stored in RA X.

[0068] At step 2, register A (RA A) may be loaded with the value x. The value x may be read from the memory array or the value x may be provided as an output of the AFU. At step 3, register B (RAB) may be loaded with the value d. The value d may be read from the memory array and stored in RAB. In step 4, the accumulator of the MAC array may be reset to a value of 0. At step 5, the MAC array may be used to perform a multiplication operation to multiply the value x by d, thereby generating the expression y=dx, which may be part of a polynomial function 550 (e.g., a+bx(1+cx(1+dx))).

[0069] At step 6, RA A may store a 1 value. At step 7, RA B may store a 1 value. At step 8, the MAC array may be utilized to add the 1 value to the dx value, thereby generating the expression y=1+dx, which is part of the polynomial function 550 (e.g., a+bc(1+cx(1+dx))). The accumulator of the MAC array is not reset between steps 5 and 8. The accumulator retains the value dx, which allows it to perform an addition operation with the adder and accumulate the 1 value to the dx value. As used herein, Figure 3 The register 339 of the MAC array 332 may be referred to as an accumulator.

[0070] At step 9, RA A loads the y value, which is 1+dx. At step 10, RA B loads the x value. At step 11, the MAC array is reset (e.g., the accumulator of the MAC array is reset). At step 12, the MAC array performs a multiplication operation using the multiplier, so that the output of the accumulator y=x(1+dx).

[0071] At step 13, RA A loads the y value. RA A may load the value y by outputting y from the MAC array. The value y may be provided to the AFU. The AFU may pass the y value without performing additional operations. The y value may be stored in RA A. At step 14, RA B loads the c value. At step 15, the accumulator of the MAC array is reset. At step 16, a multiplication operation may be performed using the MAC array to multiply the c value by the y value (e.g., x(1+dx)). The accumulator of the MAC array may store the value y=cx(1+dx), which is part of the polynomial function 550 (e.g., a+bx(1+cx(1+dx))).

[0072] At step 17, RAA may be loaded with a value of 1. At step 18, RAB may be loaded with a value of 1. At step 19, the MAC array may add the value of 1 to the value stored in the accumulator to generate a value y=1+cx(1+dx).

[0073] At step 20, RA A loads the y value. At step 21, RA B loads the x value. At step 22, the accumulator of the MAC array is reset. At step 23, the MAC array performs a multiplication operation using the x value and the y value, which produces y=x(1+cx(1+dx)).

[0074] At step 24, RA A loads the y value. At step 25, RA B loads the b value. At step 26, the accumulator of the MAC array is reset. At step 27, the MAC array performs a multiplication operation using the b value and the y value, which produces y=bx(1+cx(1+dx)).

[0075] At step 28, RA A loads a value of 1. At step 29, RA B loads a value of a. At step 30, the MAC array performs an addition operation to add the value a to y=bx(1+cx(1+dx)), which is stored in the accumulator and produces y=a+bx(1+cx(1+dx)). The MAC array may perform a multiplication operation before adding a to y=bx(1+cx(1+dx)). For example, the MAC array may multiply the a value with the 1 value to obtain the value a, which may then be added to the value stored in the accumulator (e.g., a register). The MAC array may then output y=a+bx(1+cx(1+dx)) to the AFU. The AFU may output y=a+bx(1+cx(1+dx)) to the global bus. The ANN may be executed using the activation function y=a+bx(1+cx(1+dx)). The output y=a+bx(1+cx(1+dx)) is equal to the activation function 550 (eg, f(x)=a+bx(1+cx(1+dx))).

[0076] Figure 6The description shows a table 661 of pseudo code for implementing a hard GELU (h-GELU) approximation 660 according to several embodiments of the present disclosure. Table 661 shows pseudo code for the steps, values ​​associated with performing the steps, whether the right shift register right shifts the data, and the atomic operations performed by the AFU.

[0077] The h-GELU approximation 660 is expressed as Where x is an input and ReLU6 is an atomic operation performed by the AFU. At step 1, RA X is loaded with the x value (e.g., MAC result x). The value x may also be right shifted before being stored in RA X. The right shift allows the output of the MAC to be in the same format as used to store data in a register or for performing operations using the AFU. For example, the data provided by the MAC may include 24 bits. The right shift may be used to provide 8 bits of data to the AFU. The MAC array may output the value x to the AFU. The AFU may output the x value to RA X. At step 2, RA A may store the value 1.702. At step 3, RA B may store the value x. The value x may be provided from RA X to RA B. For example, the value x may be read from RA X and provided to the AFU. The AFU may avoid performing operations using the value x and may provide the value x to Figure 3 The MUX 335-1 of RA B may provide the value x to RA B. At step 4, the accumulator of the MAC array may be reset. At step 5, the MAC array may perform a multiplication operation using the value x and the value 1.702 such that y=1.702x.

[0078] At step 6, RA A may store a value of 1. At step 7, RA B may store a value of 3. At step 8, the MAC array may multiply the value of 1 by 3, such that the output is the value of 3. The value of 3 may be added to the value stored in the accumulator, i.e., the value 1.702x. The output of the MAC array may be y=1.702x+3. The output of the MAC array may be provided to the AFU. The AFU may perform a ReLU6 operation using 1.702x+3 as input, such that the output of the AFU is y=ReLU6(1.702x+3).

[0079] At step 9, RA A may store the value y (e.g., ReLU6(1.702x+3)). At step 10, RA B may store the value 1 / 6. At step 11, the accumulator of the MAC array may be reset. At step 12, the MAC array may multiply ReLU6(1.702x+3) by 1 / 6 to produce the output

[0080] At step 13, RA A may store a value x. At step 14, RA B may store a value y (e.g., ). At step 15, the accumulator of the MAC array may be reset. At step 16, the MAC array may compare the value x with Multiply to produce output It can be used as an activation function to approximate h-GELU 660. Figure 6 Provides an example of using AFU to perform atomic operations. AFU can be used to approximate activation functions without requiring specific hardware for executing the activation function (e.g., h-GELU).

[0081] Figure 7 Table 751 shows pseudo code for implementing integer GELU (i-GELU) approximation 771 according to several embodiments of the present disclosure. Table 761 shows pseudo code for the steps, values ​​associated with performing the steps, whether the right shift register right shifts the data, and the atomic operations performed by the AFU.

[0082] The i-GELU approximation 771 is defined as in Where x is the input and a and b are constants. At step 1, RA X stores x. At step 2, RA A stores the value At step 3, RA B stores the value x. At step 4, the MAC array accumulator is reset. At step 5, the MAC array adds the value x to Multiply to produce The MAC array can be set to Provided to AFU. AFU can use the value To perform the atomic operation clip(|x| * ). The output of the AFU is

[0083] At step 6, RA A stores y (e.g., ). At step 7, RA B stores the value At step 8, the accumulator of the MAC array is reset. At step 9, the value y is compared with Multiplying together results in the value is stored in the accumulator of the MAC array.

[0084] At step 10, RA A stores 1. At step 11, RA B stores the value At step 12, the MAC array is used to compare the value 1 with Multiplying together results in the value The value to be added to the value stored in the accumulator of the MAC array. Added to value lead to is stored in the accumulator of the MAC array.

[0085] At step 13, RA A stores y. At step 14, RA B stores the value y. The value y is provided from the AFU to RA A and RAB. The AFU receives the value y from the MAC array. At step 15, the accumulator of the MAC array is reset. At step 16, the MAC array multiplies the values ​​stored in RA A and RA B, resulting in is stored in the accumulator of the MAC array.

[0086] At step 17, RA A stores a value of -1. At step 18, RA B stores a value of 1. At step 19, the MAC array multiplies the value -1 by 1 to obtain a value of -1 and adds the value -1 to the value stored in the accumulator. The MAC array Stored in the accumulator.

[0087] At step 20, RA A stores the value y. At step 21, RA B stores the value x. At step 21, the AFU also performs the operation -sign() with the value x, so that the output is -sign(x). At step 22, the accumulator of the MAC array is reset. At step 23, the MAC array multiplies y with -sign(x). Given that x has the value The MAC array combines y with Multiply. Result Stored in the accumulator.

[0088] At step 24, RAA stores the value y. At step 25, RAB stores the value 1. At step 26, the accumulator of the MAC array is reset. At step 27, the MAC array multiplies the values ​​stored in RAA and RAB to produce the output in At step 28, RAA stores the value 1. At step 29, RAB stores the value 1. At step 30, the MAC array multiplies 1*1 to produce an output of 1. Next, the MAC accumulates 1 and y stored in the accumulator to produce an output

[0089] At step 31, RA A stores the value y. At step 32, RA B stores the value x. At step 33, the MAC array accumulator is reset. At step 34, the MAC array multiplies the values ​​stored in RA A and RA B to obtain At step 34, the shift register also right-shifts the output of the MAC array, resulting in the value Provided to the global bus by the AFU. Activation functions can be used To implement ANN.

[0090] Figure 8 is a flowchart corresponding to a method 880 for implementing a processing unit including an activation function unit according to some embodiments of the present disclosure. The method 880 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, a dedicated logic, a programmable logic, a microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 880 is performed by Figure 1 112. Although shown in a particular sequence or order, the order of the processes may be modified unless otherwise specified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0091] The method 880 includes implementing a PU 412 with an AFU 434. At 881, data may be received at a plurality of registers 431-1, 431-2 of a PU 412 of a memory subsystem 103. The PU 412 may be implemented between memory banks 221 of the memory subsystem 103. The memory banks 221 may be organized into memory bank groups. The plurality of registers 431-1, 431-2 may include a first plurality of registers and a second plurality of registers. At 882, data may be received at a MAC unit 432 coupled to the plurality of registers 431-1, 431-2. The first plurality of registers and the second plurality of registers may be coupled to the MAC unit. The MAC unit 432 may also be referred to as a MAC array.

[0092] At 883, a first plurality of operations may be performed at the MAC unit 432 to produce a first output. The first plurality of operations may be performed using a multiplier 437, an adder 438, and a register 439 (referred to as an accumulator). At 884, the first output may be provided to the AFU 434. At 885, the first output may be provided from the AFU 434 to the plurality of registers 431-1, 431-2 using a bus 442 that couples the plurality of registers 431-1, 431-2 to the AFU 434. In other examples, the first output may be provided from the AFU to the plurality of registers using one or more lines that couple the plurality of registers to the AFU.

[0093] The AFU may provide a first output without performing additional operations with the first output. The AFU may provide the first output in response to receiving a command. The command may be received from the PU control circuit system 108. In various examples, the PU, registers, MAC unit, and / or AFU and other components of the PU are described as performing actions. The actions performed by the PU or its components may be coordinated by the PU control circuit system 108. For example, the PU control circuit system 108 may provide a command (e.g., a signal) to the AFU to cause the AFU to provide the first output without performing additional operations with the first output.

[0094] The AFU may provide the first output to an output bus using a tri-state buffer. The tri-state buffer may allow the AFU to control the output bus or a bus that couples the AFU to a plurality of registers. The PU control circuitry 108 may control whether the AFU provides data to an output bus of a memory device or to a bus that couples the AFU to a plurality of registers.

[0095] The MAC unit may perform a second plurality of operations at the MAC unit to generate a second output. The MAC unit may utilize the first output received from the plurality of registers to perform the second plurality of operations. The MAC unit and the AFU may perform the plurality of operations without providing the output to the global bus. The AFU may utilize the second output to perform a third plurality of operations. Various examples of the AFU may pass the output of the MAC unit without performing additional operations. The AFU may also perform operations on the output of the MAC unit before providing its output to the global bus or the plurality of registers. For example, the second output may be provided to the output bus.

[0096] In various examples, a memory device may include a memory cell array and a PU coupled to the memory cell array. The PU may include a plurality of registers configured to store data provided by the array. The plurality of registers may also store data provided by the AFU. The PU may include a MAC unit coupled to the plurality of registers. The MAC unit may perform a first plurality of operations on the data. The PU may also include an AFU that may be coupled to the MAC unit. The AFU may perform a second plurality of operations on the output of the MAC unit. The PU may include a first plurality of wires coupling the AFU to the plurality of registers. The plurality of wires may provide the output of the AFU unit to the plurality of registers to perform a polynomial operation.

[0097] The MAC unit may provide output to the AFU unit via a multiplexer. The multiplexer may allow data to flow from the MAC unit to the AFU or allow data to flow from different multiple registers. The PU may also include a MUX. The MUX may provide the output of the AFU to the MAC unit via multiple registers.

[0098] In various examples, the AFU may provide its own output to a de-MUX. The de-MUX may be part of the PU. The de-MUX may provide the output of the AFU to a global bus or multiple lines.

[0099] The PU may also include a MUX that receives the output and data of the AFU unit. The MUX may provide the output or data of the AFU unit to a plurality of registers. The plurality of registers may store the output of the AFU unit in response to receiving the output of the AFU unit from the MUX. The second plurality of operations performed by the AFU may be atomic operations. The MAC unit may not support atomic operations.

[0100] In various examples, the PU may include a first plurality of registers, a MAC unit, an AFU, and a plurality of wires. The first plurality of registers may store data provided by the array. The MAC unit may be coupled to the first plurality of registers and may perform a first plurality of operations on the data. Data may be received from the first plurality of registers. The PU may include an AFU coupled to the MAC unit and may utilize a first output provided by the MAC unit to perform a second plurality of operations. The MAC unit may provide its own output to the AFU. The PU may include a first plurality of wires. The first plurality of wires may provide a second output of the AFU as an input to the AFU. The first plurality of wires may couple the output of the AFU to an input of the AFU. The AFU may utilize the input to perform a second plurality of operations, wherein the input is the second output of the AFU.

[0101] The first plurality of signal lines may couple the AFU to a second plurality of registers. The second plurality of registers may provide a second output to the AFU via a second plurality of lines coupling the second plurality of registers to the AFU. The second plurality of lines and the second plurality of registers allow the AFU to perform operations on its own outputs. For example, the AFU may utilize the second output to perform a third plurality of operations. The PU may also include a third plurality of lines to provide a third output of the AFU to the first plurality of registers to perform a polynomial operation.

[0102] Fig. 9 is a block diagram of an example computer system in which embodiments of the present disclosure may operate. For example, Fig. 9 An example machine illustrating a computer system 990 within which a set of instructions for causing the machine to perform any one or more of the methodologies discussed herein may be executed. In some embodiments, the computer system 990 may correspond to a host system (e.g., Figure 1 ) that includes, is coupled to, or utilizes a memory subsystem (e.g., Figure 1 The memory subsystem 103 of the controller may be used to execute the operation of the controller (for example, execute the operating system to execute the corresponding Figure 1In some embodiments, the machine may be connected (e.g., using a network) to other machines in a LAN, an intranet, an extranet, and / or the Internet. The machine may operate in the capacity of a server or a client user machine in a client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or a client user machine in a cloud computing infrastructure or environment.

[0103] The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network appliance, a server, a network router, a switch or a bridge, or any machine capable of executing (sequentially or otherwise) a set of instructions that specify actions to be taken by the machine. In addition, while a single machine is described, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[0104] The example computer system 990 includes a processing device 991, a main memory 993 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) (e.g., synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM)), etc.), a static memory 997 (e.g., flash memory, static random access memory (SRAM)), etc.), and a data storage system 998, which communicate with each other via a bus 996.

[0105] The processing device 991 represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit, or the like. More specifically, the processing device may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets or multiple processors implementing a combination of instruction sets. The processing device 991 may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. The processing device 991 is configured to execute instructions 992 for performing the operations and steps discussed herein. The computer system 990 may further include a network interface device 994 to communicate over a network 995.

[0106] The data storage system 998 may include a machine-readable storage medium 999 (also referred to as a computer-readable medium) on which is stored one or more sets of instructions 992 or software embodying any one or more of the methodologies or functions described herein. The instructions 992 may also reside, completely or at least partially, within the main memory 993 and / or within the processing device 991 during execution thereof by the computer system 990, the main memory 993 and the processing device 991 also constituting machine-readable storage media. The machine-readable storage medium 999, the data storage system 998, and / or the main memory 993 may correspond to Figure 1 Memory subsystem 103.

[0107] In one embodiment, instructions 992 include implementing instructions corresponding to a processing unit (e.g., Figure 1 PU 112 of the present invention). Although the machine-readable storage medium 999 is shown as a single medium in the example embodiment, the term "machine-readable storage medium" should be deemed to include a single medium or multiple media storing one or more sets of instructions. The term "machine-readable storage medium" should also be deemed to include any medium capable of storing or encoding a set of instructions for machine execution and causing the machine to perform any one or more of the methodologies of the present disclosure. The term "machine-readable storage medium" should accordingly be deemed to include, but not limited to, solid-state memory, optical media, and magnetic media.

[0108] Some portions of the foregoing detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here and generally conceived to be a self-consistent sequence of operations leading to a desired result. Operations are those requiring physical manipulation of physical quantities. Typically, although not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, combined, compared, and otherwise manipulated. It has proven convenient at times, primarily for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0109] It should be remembered, however, that all of these and similar terms should be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. The present disclosure may involve actions and processes of computer systems or similar electronic computing devices that manipulate and transform data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage systems.

[0110] The present disclosure also relates to an apparatus for performing the operations described herein. This apparatus may be specially constructed for the intended purpose, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. This computer program may be stored in a computer-readable storage medium, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs and magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus.

[0111] The algorithms and displays presented herein are not inherently related to any particular computer or other device. Various general purpose systems may be used in conjunction with programs according to the teachings herein, or it may prove convenient to construct more specialized equipment to perform the methods. The structures of a variety of these systems will appear as set forth in the description below. In addition, the present disclosure is not described with reference to any particular programming language. It should be appreciated that a variety of programming languages ​​may be used to implement the teachings of the present disclosure as described herein.

[0112] The present disclosure may be provided as a computer program product or software, which may include a machine-readable medium having instructions stored thereon, which may be used to program a computer system (or other electronic device) to perform a process according to the present disclosure. A machine-readable medium includes any mechanism for storing information in a form that can be read by a machine (e.g., a computer). In some embodiments, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium, such as a read-only memory ("ROM"), a random access memory ("RAM"), a magnetic disk storage medium, an optical storage medium, a flash memory device, etc.

[0113] In the foregoing description, embodiments of the present disclosure have been described with reference to specific example embodiments thereof. It will be appreciated that various modifications may be made to the present disclosure without departing from the broader spirit and scope of the embodiments of the present disclosure as set forth in the appended claims. Accordingly, the description and drawings should be viewed in an illustrative rather than a restrictive sense.

Claims

1. A device for a processing unit including an activation function unit, comprising: Memory cell arrays (221-1, 221-2, 221-3, 221-4, 221-5, 221-6, 221-7, 221-8, 221-9, 221-10, 221-11, 221-12, 221-13, 221-14, 221-15, 221-16); A processing unit (112, 212-1, 212-2, 212-3, 212-4, 212-5, 212-6, 212-7, 212-8, 312, 412) coupled to the memory cell array, the processing unit comprising: a plurality of registers (331-1, 331-2, 431-2, 431-2) configured to store data provided by the array; a multiply-accumulate MAC unit (232, 332, 432) coupled to the plurality of registers and configured to perform a first plurality of operations on the data; an activation function unit (AFU) (234, 334, 434) coupled to the MAC unit and configured to perform a second plurality of operations on an output of the MAC unit; A plurality of signal lines (327) couple the AFU to the plurality of registers and are configured to provide outputs of the AFU unit to the plurality of registers to perform polynomial operations.

2. The apparatus of claim 1, wherein the MAC unit is further configured to provide the output to the AFU unit via a multiplexer MUX (335-2).

3. The apparatus of claim 1, wherein the processing unit further comprises a MUX (335-1) configured to provide the output of the AFU to the MAC unit via the plurality of registers.

4. The apparatus of claim 1, wherein the AFU is further configured to provide the output of the AFU unit to a de-MUX (336).

5. The apparatus of claim 4, wherein the processing unit further comprises the de-MUX configured to provide the output of the AFU to a global bus or the plurality of signal lines.

6. The apparatus of any one of claims 1 to 4, wherein the processing unit further comprises a MUX (335-1) configured to receive the output of the AFU unit and the data.

7. The apparatus of claim 6, wherein the MUX is further configured to provide the output of the AFU unit or the data to the plurality of registers and wherein the plurality of registers are configured to store the output of the AFU unit in response to receiving the output of the AFU unit from the MUX.

8. The apparatus of any one of claims 1 to 4, wherein the second plurality of operations are atomic operations not supported by the MAC unit.

9. A method for a processing unit including an activation function unit, comprising: receiving data at a plurality of registers (331-1, 331-2, 431-2, 431-2) of a processing unit (112, 212-1, 212-2, 212-3, 212-4, 212-5, 212-6, 212-7, 212-8, 312, 412) of a memory subsystem (103); receiving the data at a multiply-accumulate MAC unit (232, 332, 432) coupled to the plurality of registers; performing a first plurality of operations at the MAC unit to produce a first output; Providing the first output to an activation function unit AFU (234, 334, 434); and The first output is provided from the AFU to the plurality of registers using a bus (442) coupling the plurality of registers to the AFU.

10. The method of claim 9, wherein the AFU provides the first output without performing additional operations using the first output. The method of claim 10 , wherein the AFU provides the first output in response to receiving a command.

12. The method of claim 10, wherein the AFU utilizes a tri-state buffer (441-1) to provide the first output to an output bus and further comprising utilizing the tri-state buffer to provide the first output to the plurality of registers.

13. The method according to any one of claims 9 to 12, further comprising: performing a second plurality of operations at the MAC unit to generate a second output using the first output received from the plurality of registers; performing a third plurality of operations at the AFU using the second output; and The second output is provided to an output bus.

14. A device comprising a processing unit of an activation function unit, comprising: Memory cell array (221-1, 221-2, 221-3, 221-4, 221-5, 221-6, 221-7, 221-8, 221-9, 221-10、221-11、221-12、221-13、221-14、221-15、221-16)； Processing unit (112, 212-1, 212-2, 212-3, 212-4, 212-5, 212-6, 212-7, 212-8, 312, 412), which is coupled to the memory cell array, the processing unit includes: a first plurality of registers (331-1, 331-2, 431-2, 431-2) configured to store data provided by the array; a multiply-accumulate MAC unit (232, 332, 432) coupled to the first plurality of registers and configured to perform a first plurality of operations on the data; an activation function unit (AFU) (234, 334, 434) coupled to the MAC unit and configured to perform a second plurality of operations using a first output provided by the MAC unit; a first plurality of signal lines (329) configured to provide a second output of the AFU as an input to the AFU; Wherein the AFU is further configured to utilize the input to perform a second plurality of operations.

15. The apparatus of claim 14, wherein: The first plurality of signal lines are coupled to a second plurality of registers (331-3, 431-3); The second plurality of registers are configured to provide the second output to the AFU via a second plurality of signal lines (328) coupling the second plurality of registers to the AFU; and Wherein the AFU is further configured to utilize the second output to perform a third plurality of operations.

16. The apparatus of claim 15, wherein the processing unit further comprises a third plurality of signal lines (327) configured to provide a third output of the AFU to the first plurality of registers to perform polynomial operations.

Images