Calculation circuit, memory cell including calculation circuit, and calculation method

Abstract

The invention provides a calculation circuit, a storage device including the calculation circuit, and a calculation method. The computing circuit includes: an input distributor receiving n bits of input data and dividing the input data into a plurality of operational elements based on a data type of the input data, where n is a natural number equal to or greater than 2; an adder tree that performs a multiplication operation between the operation elements; and an accumulator generating a first output value by adding an output value of the adder tree to the value stored in the accumulation register, where the first output value includes sign bits and data bits, and the accumulator includes a first lightweight normalizer configured to normalize the sign bits and the data bits, and a second lightweight normalizer configured to normalize the sign bits and the data bits. And a shift unit that performs a shift on the first output value by comparing a value of the sign bit with a value of m bits among the data bits, where m is a natural number greater than n, less than n, or equal to n.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority and all rights to Korean Patent Application No. 10-2024-0104509, filed on August 6, 2024, with the Korean Intellectual Property Office, the entire contents of which are incorporated herein by reference. Technical Field

[0003] Some example embodiments relate to a computing circuit, a storage device including the computing circuit, and / or a computing method. Background Technology

[0004] High-performance applications are data-intensive and computationally intensive. To perform inference more efficiently in data-intensive deep neural networks, computing systems with massive computational and storage capabilities are desired.

[0005] Processing in Memory (PIM) type storage devices are being developed to perform some computational operations of computing systems through internal processing. PIM computing can reduce the computational load on computing systems.

[0006] Such PIM calculations need to support calculations on various data types, such as floating-point (FP) and integer (INT) data, and are expected to be performed within a short timeframe. Therefore, research is actively underway on computing devices capable of performing calculations on various types of data within a short timeframe. Summary of the Invention

[0007] Some example embodiments provide computing circuitry, storage devices including the computing circuitry, and / or computing methods capable of performing computations on various types of data in a short period of time.

[0008] However, the aspects of the exemplary embodiments are not limited to those set forth herein. Some of the above and other aspects of the exemplary embodiments will become more apparent to those skilled in the art to which the inventive concept belongs from the detailed description given below.

[0009] According to some example embodiments, a computing circuit is provided, including: an input distributor configured to receive n-bit input data and divide the input data into a plurality of operands based on the data type of the input data, where n is a natural number equal to or greater than 2; an adder tree configured to perform multiplication operations between operands; and an accumulator configured to generate a first output value by adding the output value of the adder tree to a value stored in an accumulator register. The first output value includes a sign bit and data bits, and the accumulator includes a first lightweight normalizer configured to perform a shift operation on the first output value by comparing the value of the sign bit with the values ​​of m bits in the data bits, where m is a natural number greater than, less than, or equal to n.

[0010] Alternatively or additionally, according to some example embodiments, a storage device is provided, comprising: a memory cell array configured to store data; and a processing-in-memory (PIM) device configured to be supplied with data from the memory cell array and configured to perform arithmetic operations, wherein the PIM device includes: an input distributor configured to receive n-bit input data and divide the input data into a plurality of operands based on the data type of the input data, where n is a natural number equal to or greater than 2; an adder tree configured to perform multiplication operations between operands; and an accumulator configured to generate a first output value by adding the output value of the adder tree to a value stored in an accumulator register. The first output value includes a sign bit and data bits, and the accumulator includes a first lightweight normalizer configured to perform a shift operation on the first output value by comparing the value of the sign bit with the values ​​of m bits in the data bits, where m is a natural number greater than, less than, or equal to n.

[0011] Alternatively or additionally, according to some example embodiments, a computation method is provided, comprising: receiving n-bit input data and dividing the input data into a plurality of operands based on the data type of the input data, wherein n is a natural number equal to or greater than 2; performing a multiplication operation between the operands using an adder; generating a first output value including a sign bit and data bits by adding the value of the result of the multiplication operation to a value stored in an accumulator register; and performing a shift operation on the first output value by comparing the value of the sign bit with the value of m bits in the data bits, wherein m is a natural number greater than, less than, or equal to n.

[0012] It should be noted that the effects of some example embodiments are not limited to those described above, and other effects of some example embodiments will be apparent from the following description. Attached Figure Description

[0013] The above and other aspects and / or features of some exemplary embodiments will become more apparent from the detailed description of exemplary embodiments with reference to the accompanying drawings, wherein:

[0014] Figure 1 This is a diagram illustrating an example computing circuit that performs a multiply-accumulate (MAC) operation on floating-point (FP) data;

[0015] Figure 2 This is a diagram showing example FP type data;

[0016] Figure 3 This is a diagram illustrating a storage system according to some example embodiments;

[0017] Figure 4 yes Figure 3 A block diagram of a memory chip;

[0018] Figure 5 It is shown Figure 3 A block diagram of a PIM device;

[0019] Figure 6 It is shown Figure 5 A block diagram of the Arithmetic Logic Unit (ALU);

[0020] Figure 7 yes Figure 6 Detailed block diagram of the ALU;

[0021] Figure 8 This shows the input to Figure 6 A table of example input data for the ALU;

[0022] Figures 9 to 11 It is used for explanation Figure 7 A diagram illustrating the operation of a lightweight normalizer;

[0023] Figure 12 This is a diagram showing example FP16 type data;

[0024] Figure 13 This is a diagram showing the data for example BF16 type;

[0025] Figure 14 It is a diagram used to interpret the ALU that performs FP16 × FP16 or BF16 × BF16 operations;

[0026] Figure 15 This is a graph showing example FP8 type data;

[0027] Figure 16 This is a graph showing example FP8 type data;

[0028] Figure 17 This is a diagram showing the ALU performing FP8 × FP8 operations;

[0029] Figure 18 This is a diagram showing example MXINT8 and FP16 data;

[0030] Figure 19 This is a diagram showing the ALU performing MXINT8 × FP16 operations;

[0031] Figure 20 This is a diagram showing an example of INT8 type data;

[0032] Figure 21 This is a diagram showing the ALU performing an INT8 × INT8 operation;

[0033] Figure 22 This is a diagram showing an example of INT4 type data;

[0034] Figure 23 This is a diagram showing the ALU performing the INT4 × INT4 operation;

[0035] Figure 24 This is a flowchart illustrating a calculation method according to some example embodiments;

[0036] Figure 25 This is a diagram illustrating the effect of an ALU according to some example embodiments; and

[0037] Figure 26 This is a diagram illustrating a storage device according to some example embodiments. Detailed Implementation

[0038] Some example embodiments will be described with reference to the accompanying drawings.

[0039] Figure 1 This is a diagram of an example computing circuit that performs a multiply-accumulate (MAC) operation on floating-point (FP) data. Figure 2 This is a diagram showing example FP type data.

[0040] refer to Figure 1 The computing circuit 10 may include adder 11 and multiplier 12 for performing multiplication operations on FP type data A and FP type data B, and subtractor 13, shifter 14, adder 15, normalizer 16 and exponent updater 17 for accumulating the results of the multiplication operations.

[0041] In some example embodiments, data A and data B may have Figure 2 The configuration shown.

[0042] refer to Figure 2Data A may include: a sign bit S, indicating whether the real number represented by data A is negative or positive; an exponent bit Exp.A, representing the exponent part of data A; and a data bit Man.A, representing the fractional part of data A. Since data A is of type FP, the data bit Man.A can be the mantissa.

[0043] Similarly, data B may include: a sign bit S, indicating whether the data B representing a real number is negative or positive; an exponent bit "Exp.B", representing the exponent part of data B; and a data bit "Man.B", representing the fractional part of data B. Since data B is of type FP, the data bit "Man.B" can be the mantissa.

[0044] by Figure 2 The process of storing a binary number, for example, 0.01101, in the form shown is as follows.

[0045] As an illustrative example, the binary number 0.01101 can be expressed as 1.101 × 2. -2 However, the example embodiment is not necessarily limited to this. In this case, the sign bit S can be stored as 0 to indicate that the binary number is positive, the exponent bit "Exp.a" can be stored as a bit corresponding to -2 according to a standard such as a variable standard (or, alternatively, a predetermined standard), and the data bit "Man.A" can be stored as a bit corresponding to 101 according to a standard.

[0046] Therefore, when Figure 2 When storing data A and data B as shown, it can be done as follows: Figure 1 The multiplication of data A and data B is performed by adding the exponent bits "Exp.A" of data A and "Exp.B" of data B using adder 11, and multiplying the data bits "Man.A" of data A and "Man.B" of data B using multiplier 12.

[0047] The result of the operation of adder 11 can be stored in register RG1, and the result of the operation of multiplier 12 can be stored in register RG2.

[0048] After that, it can be used Figure 1 The computational circuit 10 performs the accumulation operation as follows: it adds the result of multiplying data A and data B to the existing accumulation result (for example, if the current iteration is the kth iteration, then it is the accumulation result up to the (k-1)th iteration).

[0049] In some example embodiments, subtractor 13 calculates the difference between the accumulated exponent bits stored in register RG3 up to the current iteration and the exponent bits produced by the multiplication of data A and data B stored in register RG1. This subtraction operation is used to determine the difference in the exponents used to align the two values ​​to be added.

[0050] Subsequently, shifter 14 uses the result of subtraction 13 (e.g., aligning the bits of the two values ​​to be added) to shift the bits of the accumulated data bits stored in register RG4 up to the current iteration, or to shift the bits of the data bits resulting from the multiplication of data A and B stored in register RG2. Then, adder 15 is used to add the two values.

[0051] Subsequently, normalizer 16 performs normalization on the data bits. Here, normalization is the process of converting the result of adder 15 into a form such as 1.XXXX, where different X can be '0' or '1'.

[0052] For example, if the result of adder 15 is 0.001011, normalizer 16 first uses encoder 16a to search for the position of the first "1" below the decimal point. Then, based on the detected position of the "1", normalizer 16 uses shifter 16b to perform a shift to convert 0.001011 to 1.011, and uses rounding circuit 16c to remove unnecessary zeros by rounding if needed or desired.

[0053] In this example, since a 3-shift is performed through normalization, the normalizer 16 provides relevant information to the exponent updater 17 so that -3 is reflected in the exponent bit.

[0054] Then, the result of the accumulated exponent bits can be stored in register RG3, and the result of the accumulated data bits can be stored in register RG4.

[0055] Because whenever in Figure 1 Normalization is performed when a new value is added during the accumulation step indicated by the dashed line, making it potentially difficult to perform the accumulation operation within a short timeframe using the computing circuit 10. Alternatively or additionally, if the computing circuit 10 is used in a memory-in-process (PIM) device of a storage system, performing the accumulation operation within a unit loop becomes challenging, as a unit loop is the basic unit of operation for the storage device. Therefore, in addition to Figure 1 In addition to the registers shown, additional registers may be required or expected to support accumulation operations.

[0056] Additionally or alternatively, in the case of computing circuit 10, since there is no logic for actively allocating input data based on data type, inefficient computation may occur depending on the input data type.

[0057] The following describes computing circuits (e.g., PIM devices) that are able to overcome or at least partially overcome or improve these and other problems and perform calculations on various types of data in a short period of time.

[0058] Figure 3 This is a diagram illustrating a storage system according to some example embodiments.

[0059] refer to Figure 3 The storage system MS may include a host 100, a memory controller 200, and a storage device 300.

[0060] The host 100 can send commands CMD and addresses ADDR to the storage device 300 via the memory controller 200. Alternatively or additionally, the host 100, including the memory controller 200, can send commands CMD and addresses ADDR to the storage device 300. The host 100 can exchange data signals DQ with the memory controller 200.

[0061] For example, host 100 can send a write command CMD, address ADDR, and data signal DQ to memory controller 200. In response, memory controller 200 can send the write command CMD and address ADDR to storage device 300.

[0062] The memory controller 200 can send a data signal DQ to the storage device 300 to write data to the storage device 300. The storage device 300 can write data to the memory cell corresponding to the write command CMD and address ADDR received from the memory controller 200.

[0063] For example, host 100 can send a read command CMD and an address ADDR to memory controller 200. Memory controller 200 can then send the read command CMD and address ADDR to storage device 300. Storage device 300 can read data from the storage cell corresponding to the read command CMD and address ADDR received from memory controller 200 and send the read data as a data signal DQ to memory controller 200. Then, memory controller 200 can send the data signal DQ back to host 100.

[0064] For example, the memory controller 200 may store the PIM instruction set in the storage device 300 before sending the read command CMD and address ADDR to the storage unit 300. The PIM instruction set may include at least one of a variety of setup commands defined by standards.

[0065] Using the PIM instruction set, storage device 300 can read data from a storage cell corresponding to a PIM address, which is generated independently of at least a portion of the address ADDR received from memory controller 200. Storage device 300 can then perform PIM operations based on the read data.

[0066] In some example embodiments, storage cell 300 may include dynamic random access memory (DRAM), but the example embodiments are not limited thereto. Alternatively or additionally, in some example embodiments, storage device 300 may be implemented as and / or may include one or more of a variety of random access memories, such as static random access memory (SRAM), magnetic random access memory (MRAM), phase-change random access memory (PRAM), ferroelectric random access memory (FRAM), and resistive random access memory (RRAM).

[0067] In some example embodiments, storage device 300 may be or may include (or may be included in) high-bandwidth memory (HBM), but the example embodiments are not limited thereto.

[0068] Storage device 300 may include a plurality of storage chips 300a to 300n. For convenience, only storage chip 300a will be described below, but the following description may also apply to other storage chips; each of the plurality of storage chips 300a to 300n may store the same amount and / or different amounts of data; the example embodiments are not limited thereto.

[0069] The memory chip 300a may include multiple memory banks BA, error correction code (ECC) block EOE, and logic circuit 305.

[0070] Each of the memory banks BA may include a memory cell array MCA, a row decoder 360, a sense amplifier and a write driver 385, and a column decoder 370.

[0071] A memory cell array (MCA) may include multiple memory cells arranged in the row and column directions. Each memory cell may be connected to one of a plurality of word lines (WL) (e.g., row) and one of a plurality of bit lines (BL) (e.g., column).

[0072] The row decoder 360 can operate in response to control from the logic circuit 305. Based on the command CMD and row address RA received from the logic circuit 305, the row decoder 360 can activate a word line WL selected from multiple word lines WL as the access target.

[0073] The sense amplifier and write driver 385 can operate in response to control from the logic circuit 305. The sense amplifier and write driver 385 can be connected to the memory cell via each bit line of a plurality of bit lines BL.

[0074] Column decoder 370 can operate in response to control from logic circuit 305. Column decoder 370 can be connected to a sense amplifier and a write driver 385. Based on the command CMD and column address CA received from logic circuit 305, column decoder 370 can select one or more bit lines BL from the bit lines BL.

[0075] The memory chip 300a may include multiple ECC block EOEs. The ECC block EOEs can be connected to the corresponding memory bank BA.

[0076] The ECC block EOE can use ECC to perform error correction encoding on data sent to the storage bank BA. The ECC block EOE can use ECC to perform error correction decoding on data received from the storage bank BA.

[0077] The logic circuit 305 can store the PIM instruction set received from the host 100 via the memory controller 200. The PIM instruction set may include at least one of a variety of setting commands defined by a standard.

[0078] When the logic circuit 305 receives the command CMD and address ADDR from the memory controller 200, the logic circuit 305 can determine the operating mode of the memory chip 300a based on whether the PIM instruction set is present.

[0079] If there is no PIM instruction set in the logic circuit 305, the memory chip 300a can write data to the memory bank BA based on the write command CMD, address ADDR and data signal DQ received from the memory controller 200, or read data stored in the memory bank BA based on the read command and address received from the memory controller 200.

[0080] If the PIM instruction set exists in logic circuit 305, then memory chip 300a can perform PIM operations. Logic circuit 305 may include PIM device 313 and control logic 310. PIM device 313 can execute PIM commands corresponding to the PIM instruction set based on the command CMD and address ADDR received from memory controller 200.

[0081] Control logic 305 can receive commands CMD and addresses ADDR from memory controller 200. Control logic 305 can control PIM device 313 to execute PIM commands corresponding to the PIM instruction set based on the received commands CMD and addresses ADDR.

[0082] Control logic 305 may include mode register group 312. Mode register group 312 may include information about preset modes received from memory controller 200.

[0083] For example, mode register group 312 may include information about the operating mode of memory chip 300a and / or the mode for reporting errors generated in memory chip 300a to host 100. However, the information included in mode register group 312 is not particularly limited and may be modified as appropriate.

[0084] When PIM device 313 receives read command CMD and address ADDR from memory controller 200, PIM device 313 can read data DATA from the selected memory cell in the selected memory bank BA. In this case, ECC block EOE can receive data DATA from the selected memory cell.

[0085] The ECC block EOE can use ECC to perform error correction decoding on data DATA from the selected storage unit. If the error correction of data DATA is successful, the PIM device 313 can receive the corrected data DATA1 from the ECC block EOE.

[0086] The ECC block EOE can determine, based on the result of error correction decoding, that error correction of the data DATA is impossible. In this case, the ECC block EOE can generate an error correction failure code. The ECC block EOE can then send the error correction failure code to the logic circuit 305.

[0087] Figure 4 yes Figure 3 Block diagram of memory chip 300a.

[0088] refer to Figure 4 The memory chip 300a may include control logic 310, address register 320, memory bank control logic 330, row address multiplexer 340, refresh address generator 345, column address latch 350, row decoder 360, column decoder 370, sense amplifier unit 385, input / output (I / O) gate circuit 390, memory cell array MCA, ECC engine EOE, and data I / O buffer 395.

[0089] A memory cell array (MCA) may include multiple memory cells (MCs) for storing data; the data may be or may include one or more of text data, image data, computer instruction data, etc., and the example embodiments are not limited thereto. For example, the memory cell array (MCA) may include first memory cell arrays (BA1) to eighth memory cell arrays (BA8). Each of the first memory cell arrays (BA1) to the eighth memory cell arrays (BA8) may include multiple word lines (WL), multiple bit lines (BTL), and multiple memory cells (MCs) arranged at the intersections between the word lines (WL) and the bit lines (BTL).

[0090] The memory cell array MCA may include the first memory cell array BA1 to the eighth memory cell array BA8. Figure 4 The memory chip 300a is shown as comprising eight memory bank arrays, for example, first memory bank arrays BA1 to eighth memory bank arrays BA8, but the example embodiment is not limited thereto. That is, the memory chip 300a may include any number of memory bank arrays.

[0091] Control logic 310 can control the operation of memory chip 300a. For example, control logic 310 can generate control signals CTL1 and CTL2 to control memory chip 300a to perform write or read operations. Control logic 310 may include a command decoder 311 for decoding the command CMD received from host 100 and a mode register 312 for setting the operating mode of memory chip 300a.

[0092] For example, the command decoder 311 can generate a control signal corresponding to the command CMD by decoding one or more of the write enable signal, row address strobe signal, column address strobe signal, chip select signal, etc. The control logic 310 can also or alternatively receive a clock signal and a clock enable signal to drive the memory chip 300a synchronously.

[0093] Alternatively, control logic 310 may control refresh address generator 345 to generate refresh line address REF_ADDR in response to refresh command.

[0094] Address register 320 can receive address ADDR from host 100. For example, address register 320 can receive address ADDR including bank address BANK_ADDR, row address ROW_ADDR, and column address COL_ADDR. Address register 320 can provide bank address BANK_ADDR to bank control logic 330, row address ROW_ADDR to row address multiplexer 340, and column address COL_ADDR to column address latch 350.

[0095] The memory bank control logic 330 can generate memory bank control signals in response to the memory bank address BANK_ADDR received from the address register 320. In response to these memory bank control signals, the memory bank row decoders corresponding to the memory bank address BANK_ADDR among the first to eighth memory bank row decoders 360a to 360h can be activated, and the memory bank column decoders corresponding to the memory bank address BANK_ADDR among the first to eighth memory bank column decoders 370a to 370h can also be activated.

[0096] The row address multiplexer 340 can receive the row address ROW_ADDR from the address register 320 and the refresh row address REF_ADDR from the refresh address generator 345. The row address multiplexer 340 can selectively output either the row address ROW_ADDR received from the address register 320 or the refresh row address REF_ADDR received from the refresh address generator 345 as the row address RA. The row address RA output from the row address multiplexer 340 can be applied to each of the first memory bank row decoders 360a to the eighth memory bank row decoders 360h.

[0097] The refresh address generator 345 can generate a refresh row address REF_ADDR to refresh the memory cell MC. The refresh address generator 345 can provide the refresh row address REF_ADDR to the row address multiplexer 340. Therefore, the memory cell MC aligned with the word line WL corresponding to the refresh row address REF_ADDR can be refreshed.

[0098] Column address latch 350 can receive column address COL_ADDR from address register 320 and temporarily store the received column address COL_ADDR. Additionally or alternatively, in burst mode, column address latch 350 can incrementally increment the received column address COL_ADDR. Column address latch 350 can apply the temporarily stored or incrementally incremented column address COL_ADDR to each of the first memory bank column decoders 370a to the eighth memory bank column decoder 370h.

[0099] Row decoder 360 may include first bank row decoders 360a to eighth bank row decoders 360h, respectively connected to the first bank arrays BA1 to the eighth bank arrays BA8. Column decoder 370 may include first bank column decoders 370a to eighth bank column decoders 370h, respectively connected to the first bank arrays BA1 to the eighth bank arrays BA8. Readout amplifier unit 385 may include first bank readout amplifiers 385a to eighth bank readout amplifiers 385h, respectively connected to the first bank arrays BA1 to the eighth bank arrays BA8.

[0100] The bank row decoders activated by the bank control logic 330 in the first bank row decoders 360a to the eighth bank row decoders 360h can decode the row address RA output from the row address multiplexer 340 and activate the word line corresponding to the row address RA. For example, the activated bank row decoder can apply a word line drive voltage to the word line WL corresponding to the row address RA.

[0101] The memory bank column decoders 370a to 370h, activated by the memory bank control logic 330, can activate the memory bank read amplifiers corresponding to the memory bank address BANK_ADDR and the column address COL_ADDR through the input / output gate circuit 390.

[0102] I / O gating circuit 390 may include circuitry for gating input / output data, input data masking logic, a read data latch for storing data output from the first memory array BA1 to the eighth memory array BA8, and a write driver for writing data to the first memory array BA1 to the eighth memory array BA8.

[0103] The codeword CW to be read from one of the first memory arrays BA1 to the eighth memory array BA8 can be detected by the corresponding memory readout amplifiers 385a to 385h and stored in the read data latch.

[0104] The ECC engine EOE can perform ECC decoding on the codeword CW stored in the read data latch. If an error is detected in the data of the codeword CW, the ECC engine EOE can provide a correction data signal DQ to the external memory controller through the data I / O buffer 395.

[0105] The data signal DQ to be written to one of the first memory arrays BA1 to the eighth memory array BA8 can be provided to the ECC engine EOE, and the ECC engine EOE can generate a parity bit based on the data signal DQ and provide the data signal DQ and the parity bit to the I / O gating circuit 390. The I / O gating circuit 390 can write the data signal DQ and the parity bit to a subpage of one of the first memory arrays BA1 to the eighth memory array BA8 via a write driver.

[0106] The data I / O buffer 395 can receive a data signal DQ and a data strobe signal DQS from an external source. In some example embodiments, the data input / output buffer 395 may include a first data I / O buffer (e.g., a data buffer) that receives the data signal DQ from an external source and a second data input / output buffer (e.g., a data strobe buffer) that receives the data strobe signal DQS from an external source.

[0107] During a write operation, the data I / O buffer 395 can buffer or drive the data signal DQ (e.g., writing data) and provide it to the ECC engine EOE. During a read operation, the data I / O buffer 395 can buffer or drive the data signal DQ provided by the ECC engine EOE (e.g., reading data) and deliver it externally.

[0108] Figure 5 It is shown Figure 3 A block diagram of a PIM device. Figure 6 It is shown Figure 5 A block diagram of the arithmetic logic unit (ALU).

[0109] refer to Figure 5 The PIM device 313 may include multiple ALUs 316, such as first to P ALUs 316-1 to 316-P of a single instruction multiple data (SIMD) structure, and multiple accumulator registers 318, such as first to P accumulator registers 318 corresponding to first to P ALUs 316-1 to 316-P respectively, where P is an integer greater than or equal to 2.

[0110] The ALU 316 can perform MAC operations using input data IDATA. For example, the ALU 316 can perform a multiplication operation on the input data IDATA, add the result of the multiplication operation to the previously calculated data stored in the corresponding accumulator register 318, and store the result of the addition in the corresponding accumulator register 318. If necessary or desired, the ALU 316 can output the accumulated calculation result data as output data ODATA.

[0111] In some example embodiments, the input data IDATA received by the ALU 316 may be from... Figure 3 The data DATA1 is provided by the storage BA, but the example embodiment is not limited to this.

[0112] Accumulator register 318 can provide previously calculated data required by ALU 316 to perform MAC operations, and store newly calculated data received from ALU 316.

[0113] refer to Figure 6 The ALU 316 may include an input distributor 316a, an adder tree 316b, an accumulator 316c, a normalizer 316d, and an exponent controller 316e.

[0114] The input allocator 316a can receive input data IDATA and divide the n-bit data into multiple operands based on the data type of the input data IDATA, and provide the operands to the adder tree 316b, where n is an integer greater than or equal to 2.

[0115] The adder tree 316b can use multiple adders to perform multiplication operations between operands received from the input distributor 316a.

[0116] Accumulator 316c can perform an accumulation operation by adding the output value of adder tree 316b to the value stored in accumulator register 318.

[0117] Normalizer 316d can normalize the result of the accumulation operation of accumulator 316c, and if external output is required, the normalized result is output as output data ODATA.

[0118] The exponent controller 316e can receive input data IDATA and manage the value stored in the exponent bits during MAC operations.

[0119] Figure 7 yes Figure 6 Detailed block diagram of the ALU. Figure 8 This shows the input to Figure 6 A table of sample input data for the ALU. Figures 9 to 11 It is used for explanation Figure 7 A diagram illustrating the operation of a lightweight normalizer.

[0120] refer to Figure 7 The adder tree 316b may include multiple multipliers MUL1, MUL2, MUL3 and MUL4, multiple adders ADD1, ADD2, ADD3, ADD4, ADD5 and ADD6, a subtractor SUB, multiple static shifters SBS1, SBS2 and SBS3, multiple dynamic shifters DBS1 and DBS2, and a lightweight normalizer (“LWNorm”) LWN1.

[0121] The accumulator 316c may include multiple dynamic shifters DBS3 and DBS4, adder ADD7, and lightweight normalizer LWN2.

[0122] The ALU 316 can perform MAC operations on input data of various data types, such as... Figure 8 As shown.

[0123] Figure 6 The input allocator 316a can divide or split n-bit input data into multiple operands based on the data type of the input data, where n is a natural number greater than or equal to 2. Figure 6 The input distributor 316a can provide operands as input to the multipliers MUL1, MUL2, MUL3 and MUL4 and the adders ADD1 and ADD2 in the adder tree 316b.

[0124] Through repeated testing, the researchers who developed this invention have verified that when n is 32, the computational efficiency of the ALU 316 is improved or maximized, taking into account the input data. Therefore, examples with n=32 will be used below to explain some exemplary embodiments, but these exemplary embodiments are not necessarily limited thereto.

[0125] Figure 6The input distributor 316a can divide the data bits and exponent bits of the input data into multiple operands and can provide the operands to the adder tree 316b.

[0126] For example, refer to Figure 7 The input distributor 316a can provide the data bit "int" to the multipliers MUL1, MUL2, MUL3 and MUL4, and can provide the exponent bit "exp" to the adders ADD1 and ADD2.

[0127] Multipliers MUL1, MUL2, MUL3, and MUL4 can perform multiplication operations on the input operand "int" and output the result.

[0128] Adders ADD1 and ADD2 can perform operations on... Figure 7 The input operand "exp" is used to perform an addition operation and output the result.

[0129] The static shifter SBS1 can perform a first-order shift on the output of the multiplier MUL1. Here, the first-order shift can be determined based on the data type of the input data provided to the ALU 316.

[0130] For example, the number of bits shifted by the static shifter SBS1 during an FP16×FP16 operation performed by the ALU 316 may be different from the number of bits shifted during a BF16×BF16 (or brain floating-point) operation performed by the ALU 316.

[0131] The static shifter SBS2 can perform a second bit shift on the output of the multiplier MUL3. Here, the second bit count can be determined based on the data type of the input data provided to the ALU 316.

[0132] Adder ADD4 adds the output of static shifter SBS1 to the output of multiplier MUL2. Adder ADD5 adds the output of static shifter SBS2 to the output of multiplier MUL4.

[0133] The subtractor SUB can calculate the difference between the output of adder ADD1 and the output of adder ADD2.

[0134] If the input data is of type MXINT8, adder ADD3 can add the scaled bits of the input data to the output of subtractor SUB. The output of adder ADD3 can be provided to exponent controller 316e and reflected in the exponent bits of the calculated data.

[0135] Dynamic shifter DBS1 can receive the exponent information of the calculated data from the exponent controller 316e, and can perform shifting on the output of adder ADD4 or static shifter SBS1 based on the received exponent information. Dynamic shifter DBS2 can receive the exponent information of the calculated data from the exponent controller 316e, and can perform shifting on the output of adder ADD5 based on the received exponent information.

[0136] Adder ADD6 can perform addition operations on the outputs of adder ADD4, static shifter SBS1, or dynamic shifter DBS1, and adder ADD5 or dynamic shifter DBS2.

[0137] In some example embodiments, depending on the data type of the input data, at least some components within the adder tree 316b (e.g., at least some of the multipliers MUL1, MUL2, MUL3, and MUL4, adders ADD1, ADD2, ADD3, ADD4, ADD5, and ADD6, subtractor SUB, static shifters SBS1, SBS2, and SBS3, and dynamic shifters DBS1 and DBS2) can be disabled and not perform computations. This will be explained in detail later.

[0138] The lightweight normalizer LWN1 can perform lightweight normalization on the output of adder ADD6.

[0139] Here, lightweight normalization involves comparing the value of the sign bit of the output of adder ADD6 with the values ​​of m bits of the data bits of the output of adder ADD6, and performing a shift, where m is a natural number that can be greater than, equal to, or less than n. In some example embodiments, lightweight normalization involves performing a shift if all m bits have the same value as the sign bit.

[0140] Lightweight normalization will be described in more detail below. First, refer to... Figure 9 The data X1 has a zero sign bit S and m bits within the data set that are not all zero. In this case, the lightweight normalizer LWN1 does not perform a shift. Therefore, the exponent bits of the calculated data can remain unchanged.

[0141] refer to Figure 10 Data X2 has a zero sign bit S and m zero bits within the data bits. In this case, the lightweight normalizer LWN1 performs an m-shift on data X2 and outputs data X3. As a result, the exponent bit of the calculated data changes by -m.

[0142] refer to Figure 11Data X4 has a zero sign bit S and two sets of m bits, all of which are zero. In this case, the lightweight normalizer LWN1 performs two rounds of m-shifts on data X4 and outputs data X5. As a result, the exponent bits of the calculated data change by -2m.

[0143] Instead of the above reference, a lightweight normalization operation is performed that examines only the values ​​of m bits within the data. Figure 1 The normalization operation described allows the ALU 316 to perform MAC operations faster.

[0144] and Figure 1 Compared to the result of the regular normalization operation, the lightweight normalization operation may result in an increase in the number of bits in the data, thus requiring a slightly larger accumulator register 318. However, this lightweight normalization operation can be performed more efficiently than... Figure 1 The normalization operation is performed much faster. Alternatively, simulation results show that the error in the computation from the lightweight normalization operation is almost identical to that from... Figure 1 The error in the calculation result of the normalization operation. Therefore, compared with using Figure 1 Compared to computations performed by traditional normalization operations, lightweight normalization operations can achieve similar accuracy in a relatively short time. This will be described in more detail later.

[0145] Furthermore, through repeated testing, researchers have verified that when m is 8, the data error rate is minimized, and computational efficiency is improved or maximized. Therefore, examples with m=8 will be used below to explain some exemplary implementations.

[0146] The adder ADD7 included in accumulator 316c can add the previously calculated result stored in accumulator register 318 to the result from adder tree 316b.

[0147] For this purpose, accumulator 316c may include dynamic shifter DBS3 that performs shift on the output of adder tree 316b and dynamic shifter DBS4 that performs shift on the value stored in accumulator register 318.

[0148] Dynamic shifters DBS3 and DBS4 are used to align the bits between the output of adder tree 316b and the value stored in accumulator register 318, so that adder ADD7 can perform addition operations.

[0149] The lightweight normalizer LWN2 performs lightweight normalization on the output of adder ADD7. The operation of lightweight normalizer LWN2 is similar to that of lightweight normalizer LWN1 described earlier, so its redundant explanation is omitted.

[0150] Normalizer 316d can perform operations on the output of accumulator 316c or accumulator register 318. Figure 1 The normalization operation. For example, when external output of computation result data is required (e.g., when computation result data needs to be sent to host 100 or stored in a storage cell array), the ALU 316 can perform the normalization operation. Figure 1 Normalization operations are performed, and lightweight normalization operations are performed during MAC operations to improve computational efficiency.

[0151] This will be referenced later. Figures 12 to 23 Describe the MAC operations of the ALU 316 for various data types.

[0152] Figure 12 This is a diagram showing example FP16 type data. Figure 13 This is a graph showing example BF16 type data. Figure 14 This is a diagram of the ALU 316 used to interpret and perform FP16×FP16 or BF16 BF16 operations.

[0153] First refer to Figure 12 FP16 data can include a sign bit S, five exponent bits a_exp and ten data bits (a_man0 and a_man1).

[0154] Figure 6 The input allocator 316a can divide 32-bit data into multiple operands, such as a_exp, a_man0, a_man1, b_exp, b_man0, and b_man1.

[0155] Figure 6 The input allocator 316a can then provide the partitioned operands to Figure 14 The adder tree 316b.

[0156] Here, the operands corresponding to the data bits—for example, a_man0 and b_man0—can be provided to multiplier MUL1, the operands corresponding to the data bits—for example, a_man0 and b_man1—can be provided to multiplier MUL2, the operands corresponding to the data bits—for example, a_man1 and b_man0—can be provided to multiplier MUL3, and the operands corresponding to the data bits—for example, a_man1 and b_man1—can be provided to multiplier MUL4. Additionally, the operands corresponding to the exponent bits—for example, a_exp and b_exp—can be provided to adder ADD1.

[0157] refer to Figure 14When the ALU 316 performs an FP16 × FP16 operation, dynamic shifters DBS1 and DBS2 can be disabled, and adders ADD2, SUB, and ADD3 can also be disabled. Figure 14 In the diagram, enabled components are represented by solid lines, while disabled components are represented by dashed lines.

[0158] Adder tree 316b can be used Figure 14 Enabled component execution in Figure 12 The FP16 × FP16 operation shown can then be performed with a first lightweight normalization.

[0159] Multiplier MUL1 performs a multiplication operation on the high data bits a_man0 and b_man0, and outputs the result of the multiplication operation. Multiplier MUL2 performs a multiplication operation on the high data bits a_man0 and b_man1, and outputs the result of the multiplication operation.

[0160] In order to add the results from multipliers MUL1 and MUL2 using adder ADD4, the result from multiplier MUL1 needs to be shifted for bit alignment. Therefore, considering Figure 12 The divided operands are shifted by the static shifter SBS1.

[0161] Multiplier MUL3 performs a multiplication operation on the low data bits a_man1 and the high data bits b_man0, and outputs the result of the multiplication operation. Multiplier MUL4 performs a multiplication operation on the low data bits a_man1 and the low data bits b_man1, and outputs the result of the multiplication operation.

[0162] Similarly, in order to add the results from multipliers MUL3 and MUL4 using adder ADD5, the result from multiplier MUL3 needs to be shifted for bit alignment. Therefore, considering Figure 12 The divided operational elements are shifted by the static shifter SBS2.

[0163] Similarly, in order to add the results from adders ADD4 and ADD5 using adder ADD6, the result from adder ADD4 needs to be shifted for bit alignment. Therefore, considering Figure 12 The divided operands are shifted by the static shifter SBS3.

[0164] The output of adder ADD6 is as follows: Figure 12 The result of the multiplication of data bits performed in the FP16 × FP16 operation.

[0165] The lightweight normalizer LWN1 performs the first lightweight normalization on the output of adder ADD6 and outputs the result of the first lightweight normalization to accumulator 316c.

[0166] Simultaneously, adder ADD1 performs addition on exponent bits a_exp and b_exp, and outputs the result of the addition operation. The output of adder ADD1 can be reflected as the exponent bits of the result of the FP16 × FP16 operation through exponent controller 316e.

[0167] Accumulator 316c can be used Figure 14 The enabled component in the process adds the result from adder tree 316b to the value stored in accumulator register 318, performs a second lightweight normalization, and stores the result of the second lightweight normalization in accumulator register 318.

[0168] The previously calculated accumulated value stored in the accumulator register 318 and the result value output from the adder tree 316b can have different digit counts (or exponent values). Therefore, in order to perform an addition operation using the adder ADD7, digit alignment is required first.

[0169] One of the dynamic shifters DBS3 and DBS4 can perform the shift under the control of the exponent controller 316e. For example, if the result value output from the adder tree 316b requires bit alignment to perform an addition operation using adder ADD7, dynamic shifter DBS3 can perform a shift on the result value output from the adder tree 316b. Additionally, if the previously calculated accumulated value stored in the accumulator register 318 requires bit alignment to perform an addition operation using adder ADD7, dynamic shifter DBS4 can perform a shift on the previously calculated accumulated value stored in the accumulator register 318.

[0170] The lightweight normalizer LWN2 performs a second lightweight normalization on the output of adder ADD7 and stores the result of the second lightweight normalization as a new accumulated value in the accumulated register 318.

[0171] refer to Figure 13 BF16 type data may include a sign bit S, eight exponent bits a_exp and seven data bits (a_man0 and a_man1).

[0172] Figure 6 The input allocator 316a can divide 32-bit data into multiple operands, such as a_exp, a_man0, a_man1, b_exp, b_man0, and b_man1, as shown below. Figure 13 As shown.

[0173] Figure 6The input distributor 316a can provide the partitioned operands to the adder tree 316b, such as Figure 14 As shown, the ALU 316 can perform the BF16 × BF16 operation using the aforementioned method (the only difference being how the 32-bit data is divided into operands) and add the result of the BF16 × BF16 operation to the existing computation result data.

[0174] Figure 15 This is a diagram showing example FP8 type data. Figure 16 This is a diagram showing example FP8 type data. Figure 17 This is a diagram illustrating the ALU 316 performing FP8 × FP8 operations.

[0175] First refer to Figure 15 FP8 type data (“FP8 (E4M3)”) may include a sign bit S, four exponent bits a_exp0 and three data bits a_man0.

[0176] Figure 6 The input allocator 316a can divide 32-bit data into multiple operands, such as a_exp0, a_man0, b_exp0, b_man0, a_exp1, a_man1, b_exp1, and b_man1, as shown below. Figure 15 As shown.

[0177] Then, Figure 6 The input allocator 316a can provide the partitioned operands to the adder tree 316b, such as Figure 17 As shown.

[0178] Here, the operands corresponding to the data bits—for example, a_man0 and b_man0—can be provided to multiplier MUL2, and the operands corresponding to the data bits—for example, a_man1 and b_man1—can be provided to multiplier MUL4. Additionally, the operands corresponding to the exponent bits—for example, a_exp0 and b_exp0—can be provided to adder ADD1, and the operands corresponding to the exponent bits—for example, a_exp1 and b_exp1—can be provided to adder ADD2.

[0179] like Figure 17 As shown, when the ALU 316 performs an FP8 × FP8 operation, static shifters SBS1, SBS2, and SBS3, multipliers MUL1 and MUL3, and adders ADD3, ADD4, and ADD5 can be disabled. Figure 17 In the diagram, enabled components are represented by solid lines, while disabled components are represented by dashed lines.

[0180] Adder tree 316b can be used Figure 17 The enabled components in the code perform FP8 (E4M3) × FP8 (E4M3) operations, such as... Figure 15 As shown in the diagram, and then a first lightweight normalization can be performed.

[0181] exist Figure 15 In some of the example embodiments shown, unlike the previous example embodiments, the data bits are not divided into high and low bits. Therefore, static shifters SBS1, SBS2, and SBS3 are not used. Instead, dynamic shifters DBS1 and DBS2 perform bit alignment of the multiplication result data before the addition operation is performed by adder ADD6.

[0182] Dynamic shifters DBS1 and DBS2 can perform shifts under the control of exponent controller 316e based on the results of exponent bit operations using adders ADD1 and ADD2 and subtractor SUB.

[0183] Other features are similar to the corresponding features in the previous embodiments, therefore, their redundant explanations will be omitted.

[0184] Accumulator 316c can be used Figure 17 The enabled component in the process adds the result from adder tree 316b to the value stored in accumulator register 318, performs a second lightweight normalization, and stores the result of the second lightweight normalization in accumulator register 318.

[0185] refer to Figure 16 FP8 type data (“FP8 (E5M2 )”) may include a sign bit S, five exponent bits a_exp0 and two data bits a_man0.

[0186] Figure 6 The input allocator 316a can divide 32-bit data into multiple operands, such as a_exp0, a_man0, b_exp0, b_man0, a_exp1, a_man1, b_exp1, and b_man1, as shown below. Figure 16 As shown.

[0187] Then, Figure 6 The input allocator 316a can provide the partitioned operands to the adder tree 316b, such as Figure 17 As shown, the ALU 316 can perform the FP8 (E5M2) × FP8 (E5M2) operation using the aforementioned method, and add the result of the FP8 (E5M2) × FP8 (E5M2) operation to the existing calculation result data.

[0188] Figure 18This is a diagram showing example MXINT8 and FP16 data. Figure 19 This is a diagram showing the ALU 316 performing MXINT8 ×FP16 operations.

[0189] refer to Figure 18 MXINT8 data can include an 8-bit scaling factor and eight data bits a0. FP16 data can include a sign bit S, five exponent bits b_exp0, and ten data bits (b_man0 and b_man1).

[0190] Figure 6 The input allocator 316a can divide 32-bit data into multiple operands, such as scaling factors a0, b_exp0, b_man0, b_man1, a1, b_exp1, b_man2, and b_man3, as shown below. Figure 18 As shown.

[0191] Figure 6 The input allocator 316a can then provide the partitioned operands to the adder tree 316b, such as Figure 19 As shown in the image.

[0192] Here, the operands corresponding to the data bits—for example, a0 and b_man0—can be provided to multiplier MUL1, to multiplier MUL2, to multiplier MUL3, and to multiplier MUL4. Additionally, the operands corresponding to the exponent bits—for example, b_exp0 and b_exp1—can be provided to subtractor SUB, and the operands corresponding to the scaling factor can be provided to adder ADD3.

[0193] like Figure 19 As shown, when the ALU 316 performs MXINT8 × FP16 operations, adders ADD1 and ADD2 can be disabled. Figure 19 In the diagram, enabled components are indicated by solid lines, while disabled components are indicated by dashed lines.

[0194] Adder tree 316b can be used Figure 19 The enabled components in the code perform MXINT8 × FP16 operations, such as... Figure 18 As shown in the diagram, and then a first lightweight normalization can be performed.

[0195] Accumulator 316c can be used Figure 19The enabled component in the process adds the result from adder tree 316b to the value stored in accumulator register 318, performs a second lightweight normalization, and stores the result of the second lightweight normalization in accumulator register 318.

[0196] Figure 20 This is a diagram showing an example of INT8 type data. Figure 21 This is a diagram showing the ALU 316 performing INT8 × INT8 operations.

[0197] refer to Figure 20 INT8 data can include eight data bits a0.

[0198] Figure 6 The input allocator 316a can divide 32-bit data into multiple operands, such as a0, b00, b01, a1, b10, and b11, as shown below. Figure 20 As shown. That is to say, the input distributor 316a can choose not to divide the data bits of the first INT8 type data into operands, and can divide the data bits of the second INT8 type data into two 4-bit operands.

[0199] Figure 6 The input allocator 316a can then provide the partitioned operands to the adder tree 316b, such as Figure 21 As shown in the image.

[0200] Here, the operands corresponding to the data bits—for example, a0 and b00—can be provided to multiplier MUL1, the operands corresponding to the data bits—for example, a0 and b01—can be provided to multiplier MUL2, the operands corresponding to the data bits—for example, a1 and b10—can be provided to multiplier MUL3, and the operands corresponding to the data bits—for example, a1 and b11—can be provided to multiplier MUL4.

[0201] Since INT data types do not have an exponent, the adders ADD1, ADD2, ADD3, and the subtractor SUB, which are associated with exponent operations, can be disabled. Additionally, the static shifter SBS3, dynamic shifters DBS1, DBS2, DBS3, and DBS4, lightweight normalizers LWN1 and LWN2, and normalizer 316d can also be disabled, as normalization is not necessary for INT operations. Figure 21 In the diagram, enabled components are indicated by solid lines, while disabled components are indicated by dashed lines.

[0202] Adder tree 316b can be used Figure 21 The enabled components in the code perform INT8 × INT8 operations, such as... Figure 20 As shown in the image.

[0203] Accumulator 316c can be used Figure 21 The enabled component in the process adds the result from adder tree 316b to the value stored in accumulator register 318 and stores the result of the addition in accumulator register 318.

[0204] Figure 22 This is a diagram showing an example of INT4 type data. Figure 23 This is a diagram showing the ALU 316 performing the INT4 × INT4 operation.

[0205] refer to Figure 22 INT4 data can include four data bits a0.

[0206] Figure 6 The input allocator 316a can divide 32-bit data into multiple operands, such as a0, b0, a1, b1, a2, b2, a3, and b3, as shown below. Figure 22 As shown.

[0207] Then, Figure 6 The input allocator 316a can provide the partitioned operands to the adder tree 316b, such as Figure 23 As shown.

[0208] Here, the operands corresponding to the data bits—for example, a0 and b0—can be provided to multiplier MUL1, the operands corresponding to the data bits—for example, a1 and b1—can be provided to multiplier MUL2, the operands corresponding to the data bits—for example, a2 and b2—can be provided to multiplier MUL3, and the operands corresponding to the data bits—for example, a3 and b3—can be provided to multiplier MUL4.

[0209] Figure 22 and Figure 23 The operation of ALU 316 in the embodiment is similar to the aforementioned INT operation; therefore, its redundant explanation will be omitted. Figure 23 In the diagram, enabled components are represented by solid lines, while disabled components are represented by dashed lines.

[0210] Adder tree 316b can be used Figure 23 The enabled components in the code perform INT4 × INT4 operations, such as... Figure 22 As shown in the image.

[0211] Accumulator 316c can be used Figure 23 The enabled component in the process adds the result from adder tree 316b to the value stored in accumulator register 318 and stores the result of the addition in accumulator register 318.

[0212] Figure 24 This is a flowchart illustrating a calculation method according to some example embodiments.

[0213] refer to Figure 24 The n-bit input data is divided into multiple operands (S100) based on its data type.

[0214] For example, Figure 6 The input allocator 316a can divide 32-bit data into multiple operation elements based on the data type of the 32-bit data, as previously described.

[0215] Then, the adder is used to perform multiplication between operands (S200).

[0216] For example, Figure 6 The adder tree 316b can perform multiplication between operands in any of the aforementioned ways.

[0217] Then, the accumulation operation is performed using a lightweight normalizer (S300).

[0218] For example, Figure 6 The accumulator 316c can use a lightweight normalizer to perform the accumulation operation in any of the aforementioned ways.

[0219] Figure 25 This is a diagram illustrating the effect of an ALU according to some example embodiments.

[0220] Figure 25 It records the input of arbitrary data into the ALU (“P”) of this disclosure while incrementing the accumulation count. Figure 1 The table shows the variation of the root mean square error (RMSE) obtained from the ALU (“Q”). As previously described, simulations of the ALU of this disclosure were performed under the conditions of n=32 and m=8, where efficiency was maximized.

[0221] refer to Figure 25 As can be seen, even with a continuous increase in the number of accumulations, there is no significant difference in RMSE between the two ALUs (e.g., "P" and "Q"). In other words, the ALU disclosed herein can perform MAC operations quickly and accurately for various data types.

[0222] Figure 26 This is a diagram illustrating a storage device according to some example embodiments.

[0223] Figure 26 Showing it Figure 3 Storage device 300 is implemented as an example of HBM. Figure 26 HBM is conceptual, and its actual implementation can be related to... Figure 26 The configurations shown are different.

[0224] refer to Figure 26 The HBM 1200 can connect to host devices (e.g., memory controllers) via the HBM protocol according to the Joint Electronic Devices Engineering Committee (JEDEC) standard. The HBM protocol is a high-performance random access memory (RAM) interface for three-dimensional (3D) stacked memories (e.g., DRAM). Additionally, the HBM 1200 can connect to host devices via the PIM protocol according to the JEDEC standard.

[0225] The PIM protocol is the interface for the PIM device 1220 used with HBM 1200. Compared to other DRAM technologies (e.g., DDR4, GDDR5, etc.), HBM 1200 typically achieves wider bandwidth while consuming significantly less power and occupying a significantly smaller form factor.

[0226] The HBM 1200 may include multiple channels CH1 to CH8 with independent interfaces, and can therefore have high bandwidth. The HBM 1200 may include multiple bare dies 2100 and 2200. For example, the HBM 1200 may include a logical bare die (or buffer bare die) 2100 and one or more core bare dies 2200 stacked on the logical bare die 2100.

[0227] Figure 26 An example is shown in which the first core die 2210 to the fourth core die 2240 are stacked on an HBM 1200, but the number of core dies 2200 can vary. Core dies 2200 may be referred to as memory dies.

[0228] Each of the first core die 2210 to the fourth core die 2240 may include one or more channels. Figure 26 An example is shown in which each of the first core die 2210 to the fourth core die 2240 includes two channels, so that the HBM 1200 has a total of eight channels, for example, the first channel CH1 to the eighth channel CH8.

[0229] For example, the first core die 2210 may include a first channel CH1 and a third channel CH3, the second core die 2220 may include a second channel CH2 and a fourth channel CH4, the third core die 2230 may include a fifth channel CH5 and a seventh channel CH7, and the fourth core die 2240 may include a sixth channel CH6 and an eighth channel CH8.

[0230] The logic die 2100 may include interface circuitry 2110 for communicating with a host device. Through the interface circuitry 2110, the logic die 2100 can communicate with the host device (e.g., ...). Figure 3The memory controller 200 receives commands, addresses, and data.

[0231] Host device (e.g., Figure 3 The memory controller 200 can send commands, addresses, and data via buses corresponding to channels CH1 through CH8. The buses can be configured to be separate for channels CH1 through CH8, or some of the buses can be shared by at least two of channels CH1 through CH8. Interface circuitry 2110 can deliver commands, addresses, and data to channels of the host device requesting memory operations or arithmetic processing.

[0232] In some example embodiments, each of the core bare die 2200 or each of the first channel CH1 to the eighth channel CH8 may include the PIM device 1220.

[0233] The host device can provide commands, addresses, and data, enabling the execution of multiple arithmetic tasks or at least some of the kernels within the HBM 1200, and allowing arithmetic processing to be performed in the PIM device 1220 on a channel specified by the host device. For example, when a received command or address indicates arithmetic processing, the PIM device 1220 on the corresponding channel can perform the arithmetic processing using data read from the corresponding channel and write the result of the arithmetic processing back to the corresponding channel. In another example, when a received command or address indicates memory operation, a data access operation can be performed.

[0234] In some example embodiments, each of the first channel CH1 to the eighth channel CH8 may include multiple storage units, and the PIM device 1220 for each of the first channel CH1 to the eighth channel CH8 may be equipped with one or more processing elements, for example... Figure 5 The ALU 316, as previously described. For example, the number of processing elements in each of the first channels CH1 to the eighth channel CH8 can be equal to the number of memory banks, or if the number of processing elements in each of the first channels CH1 to the eighth channel CH8 is less than the number of memory banks, then a processing element can be shared by at least two memory banks. The PIM device 1220 of each of the first channels CH1 to the eighth channel CH8 can execute kernel instructions offloaded by the host device.

[0235] The logic die 2100 may include a through-silicon via (TSV) region 2120, an HBM physical layer interface (HBM PHY) region 2130, and a serializer / deserializer (SERDES) region 2140.

[0236] TSV region 2120 is or includes a region in which a TSV for communication with core bare die 2200 is formed and in which buses corresponding to first channel CH1 to eighth channel CH8 are arranged. If each of the first channel CH1 to eighth channel CH8 has, for example, 128-bit bandwidth, the TSV may include components for 1024-bit data I / O.

[0237] HBM PHY region 2130 may include multiple I / O circuits for communication between memory controller 200 and channels CH1 through CH8. For example, HBM PHY region 2130 may include one or more interconnect circuits for connecting memory controller 200 to channels CH1 through CH8. HBM PHY region 2130 may include physical or electrical layers and logic layers providing signals, frequencies, timing, drives, detailed operating parameters, and functions required for effective communication between memory controller 200 and channels CH1 through CH8. HBM PHY region 2130 can perform memory docking operations, such as selecting each of channels CH1 through CH8 to correspond to a row and column of a memory cell, writing data to a memory cell, or reading data written to a memory cell. HBM PHY region 2130 may support features of the HBM protocol and / or PIM protocol according to JEDEC standards.

[0238] SERDES region 2140 provides a SERDES interface, and in some example embodiments, SERDES region 2140 may be adapted to JEDEC standards as the processing throughput of the host device's processor increases and the demand for memory bandwidth increases. SERDES region 2140 may include a SERDES transmitter section, a SERDES receiver section, and a controller section.

[0239] The SERDES transmitter section may include parallel-to-serial circuitry and a transmitter. The SERDES transmitter section can receive a parallel data stream and serialize the received parallel data stream. The SERDES receiver section may include a receiver amplifier, equalizer, clock, data recovery circuitry, and serial-to-parallel circuitry. The SERDES receiver section can receive a serial data stream and parallelize the received serial data stream. The controller section may include error detection circuitry, error correction circuitry, and registers, such as First-In-First-Out (FIFO), or a queue.

[0240] Any of the elements and / or functional blocks disclosed above may include or be implemented in processing circuitry, such as hardware including logic circuitry; hardware / software combinations, such as a processor executing software; or combinations thereof. For example, processing circuitry may more specifically include, but is not limited to, a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field-programmable gate array (FPGA), a system-on-a-chip (SoC), a programmable logic unit, a microprocessor, an application-specific integrated circuit (ASIC), etc. Processing circuitry may include electronic components, such as at least one of transistors, resistors, capacitors, etc. Processing circuitry may include electrical components, such as logic gates including at least one of AND gates, OR gates, NAND gates, NOT gates, etc.

[0241] While some exemplary embodiments have been described above with reference to the accompanying drawings, this disclosure is not limited to the above exemplary embodiments and can be embodied in various other forms. Those skilled in the art will understand that the invention can be implemented in other specific forms without changing the technical spirit or essential features of the inventive concept. Therefore, it should be understood that the above exemplary embodiments are illustrative in all respects and not restrictive. Furthermore, the exemplary embodiments are not necessarily mutually exclusive. For example, some exemplary embodiments may include one or more features described with reference to one or more accompanying drawings, and may also include one or more other features described with reference to one or more other accompanying drawings.

Claims

1. A computing circuit, comprising: An input allocator is configured to receive n bits of input data and divide the input data into multiple operands based on the data type of the input data, where n is a natural number equal to or greater than 2. Adder trees are configured to perform multiplication operations between operands; and The accumulator is configured to generate a first output value by adding the output value of the adder tree to the value stored in the accumulator register. in The first output value includes the sign bit and the data bits, and The accumulator includes a first lightweight normalizer configured to perform a shift on a first output value by comparing the value of the sign bit with the values ​​of m bits in the data bits, where m is a natural number greater than, less than, or equal to n.

2. The computing circuit according to claim 1 further includes: The exponent controller is configured to update the exponent value based on the result of the shift.

3. The computing circuit according to claim 2 further includes: The normalizer is configured to perform normalization on the output of the accumulator using the exponent controller.

4. The computing circuit according to claim 1, wherein, In response to input data including floating-point (FP) data, the input allocator is configured to allocate the exponent bits of the FP data as first operands, allocate some data bits of the FP data as second operands, and allocate the remaining data bits of the FP data as third operands.

5. The computing circuit according to claim 1, wherein, In response to input data including integer (INT) data, the input allocator is configured to allocate some data bits of the INT data as first operands and allocate the remaining data bits of the INT data as second operands.

6. The computing circuit according to claim 1, wherein, The operands include exponent bits and data bits, and The adder tree includes multiple first adders configured to perform addition on the value of the exponent bits, a first subtractor configured to perform subtraction on the output of the first adders, multiple first multipliers configured to perform multiplication on the value of the data bits, and multiple second adders configured to perform addition on the output of the first multipliers.

7. The computing circuit according to claim 6, wherein, Some of the first adder, first subtractor, multiple first multipliers, and second adder are configured to be disabled, depending on the data type.

8. The computing circuit according to claim 6, wherein, Data types include MXINT8, and The adder tree also includes a third adder configured to add a scaling factor of MXINT8 to the output of the first subtractor.

9. The computing circuit according to claim 6, wherein, The adder tree also includes multiple first static shifters and multiple dynamic shifters, the multiple first static shifters being configured to perform multiple bit shifts on the output of the first multiplier, and the multiple dynamic shifters being configured to perform shifts on the output of the second adder based on the output of the exponent controller.

10. The computing circuit according to claim 9, wherein, The adder tree also includes a second static shifter configured to perform a multi-bit shift on the output of the second adder.

11. The computing circuit according to claim 10, wherein, The adder tree also includes a third adder, configured to perform addition on the output of the second adder.

12. The computing circuit according to claim 1, wherein, The result of multiplication between operands includes a sign bit and data bits, and The adder tree also includes a second lightweight normalizer configured to perform a shift operation on the result of multiplication between operands by comparing the value of the sign bit with the values ​​of m bits in these data bits.

13. The computing circuit according to claim 1, wherein, The first lightweight normalizer is configured to perform an m-shift on the first output value in response to m bits in the data bits having the same value as the sign bit.

14. The computing circuit according to claim 1, wherein, The accumulator further includes: a first dynamic shifter configured to perform a shift on the output of the adder tree based on the output of the exponent controller; a second dynamic shifter configured to perform a shift on the value stored in the accumulator register based on the output of the exponent controller; and an adder configured to add the outputs of the first dynamic shifter and the second dynamic shifter.

15. The computing circuit according to claim 1, wherein, n is 32.

16. The computing circuit according to claim 15, wherein, m is 8.

17. A storage device, comprising: A storage cell array, configured to store data; as well as A processing-in-memory (PIM) device is configured to be provided with data from a memory cell array and to perform arithmetic operations. in The PIM device includes: an input distributor configured to receive n-bit input data and divide the input data into multiple operands based on the data type of the input data, where n is a natural number equal to or greater than 2; an adder tree configured to perform multiplication operations between operands; and an accumulator configured to generate a first output value by adding the output value of the adder tree to a value stored in an accumulator register. The first output value includes the sign bit and the data bits, and The accumulator includes a first lightweight normalizer configured to perform a shift on a first output value by comparing the value of the sign bit with the values ​​of m bits in the data bits, where m is a natural number greater than, less than, or equal to n.

18. The storage device according to claim 17, wherein, The PIM device also includes an index controller, configured to update the index value based on the result of the shift.

19. The storage device according to claim 18, wherein, The PIM device also includes a normalizer configured to perform normalization on the output of the accumulator using an exponent controller and output the normalized result to the storage cell array.

20. A calculation method, comprising: It receives n bits of input data and divides the input data into multiple operands based on the data type of the input data, where n is a natural number equal to or greater than 2; Use an adder to perform multiplication between operands; Generate a first output value including a sign bit and data bits, said first output value being generated by adding the value of the result of the multiplication operation to a value stored in an accumulator register; and The first output value is shifted by comparing the value of the sign bit with the values ​​of m bits in the data bits, where m is a natural number greater than, less than, or equal to n.

Images