Matrix multiplier and operation method of matrix multiplication device including matrix multiplier

Abstract

A matrix multiplier and a method of operating a matrix multiplication device including the matrix multiplier are provided. A matrix multiplier includes: an input vector scaler that generates a first quantized scaled input vector based on a first input vector, a plurality of common scaling coefficients, and first to Rth multiplication scaling coefficients; a first data type converter generating a first fixed-point quantization scaled input vector based on the first quantization scaled input vector; an element array comprising a first processing element and a second processing element, the first processing element generating a first fixed-point output element based on a first fixed-point quantization scaled input vector and a first plurality of quantization sign bits, a second processing element generates a second fixed-point output element based on the first fixed-point quantization scaled input vector and a second plurality of quantization sign bits; and a second data type converter that generates and outputs the first output element and the second output element by converting data types of the first fixed-point output element and the second fixed-point output element.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of Korean Patent Application No. 10-2024-0019936 filed on February 8, 2024, in the Korean Intellectual Property Office, and Korean Patent Application No. 10-2023-0143978 filed on October 25, 2023, in the Korean Intellectual Property Office, the entire contents of each of which are incorporated herein by reference. Technical Field

[0003] Various example embodiments relate to a semiconductor device. More particularly, the present disclosure relates to a matrix multiplier that performs matrix multiplication and a matrix multiplication device including the matrix multiplier. Background Art

[0004] With the recent development of artificial intelligence technology, the computational complexity of artificial intelligence models has increased dramatically. Therefore, various technologies are being studied to shorten the running time of artificial intelligence models.

[0005] Typically, most or a significant amount of the operation time of an AI model is spent on matrix multiplication. For example, an AI model may spend most of its runtime computing the output matrix by performing multiplication of the input matrix and the weight matrix. Therefore, various algorithms such as binary coded quantization (BCQ) are being studied to perform multiplication of the input matrix and the weight matrix with less computation. Summary of the invention

[0006] Various example embodiments seek to provide a matrix multiplier and / or a matrix multiplication device including a matrix multiplier configured to perform matrix multiplication at a faster speed and / or with fewer computations.

[0007] According to various example embodiments, a matrix multiplier may include: an input vector scaler configured to generate a first quantized scaled input vector based on a first input vector, a plurality of common scaling coefficients, and a first multiplication scaling coefficient to an Rth multiplication scaling coefficient, wherein R is an integer greater than or equal to 2; a first data type converter configured to generate a first fixed-point quantized scaled input vector based on the first quantized scaled input vector; a processing element array including a first processing element configured to generate a first fixed-point output element based on the first fixed-point quantized scaled input vector and a first plurality of quantization sign bits; and a second processing element configured to generate a second fixed-point output element based on the first fixed-point quantized scaled input vector and a second plurality of quantization sign bits; and a second data type converter configured to generate a first output element and a second output element by converting the data types of the first fixed-point output element and the second fixed-point output element, respectively, and configured to output a first output vector including the first output element and the second output element.

[0008] Alternatively or additionally, according to various example embodiments, an operating method of a matrix multiplication device may include: receiving first to Nth weights from an external device, where N is an integer greater than or equal to 2, generating first to Nth common scaling coefficients, first to Rth multiplication scaling coefficients, and first to (N×R)th quantization sign bits by performing uniform binary coded quantization on the first to Nth weights, where R is an integer greater than or equal to 2, receiving first to Nth input elements from an external device, generating first to (N×R)th quantization scaled input elements by performing quantization scaling on the first to Nth input elements based on the first to Nth common scaling coefficients and the first to Rth multiplication scaling coefficients, and outputting a first output element generated based on the first to (N×R)th quantization sign bits and the first to (N×R)th quantization scaled input elements.

[0009] Alternatively or additionally, according to various example embodiments, a matrix multiplier may include: an input vector scaler configured to generate a first multiplication scaled input vector based on a first input vector and a first multiplication scaling coefficient to an Rth multiplication scaling coefficient, where R is an integer greater than or equal to 2; a first data type converter configured to generate a first fixed-point multiplication scaled input vector based on the first multiplication scaled input vector; a processing element array including a first processing element configured to generate a first fixed-point partial product based on the first fixed-point multiplication scaled input vector and a first plurality of quantization sign bits; a second data type converter configured to generate a first partial product by converting a data type of the first fixed-point partial product; and a common scaler configured to generate a first output element based on the product of the first partial product and a first common scaling coefficient and configured to output a first output vector including the first output element. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a block diagram illustrating a matrix multiplication device according to some example embodiments.

[0011] Figure 2 is shown to be implemented as Figure 1 Schematic diagram of the operation of a matrix multiplication device where the input matrix is ​​directly multiplied by the weight matrix.

[0012] Figure 3 and Figure 4 It is shown Figure 1 A diagram of the operation of the uniform BCQ circuit.

[0013] Figure 5is a diagram showing the binary coding quantization operation performed on each column of the weight matrix Figure 1 A diagram of the operation of the uniform BCQ circuit.

[0014] Figure 6 It shows that according to Figure 5 Schematic diagram of the operation of a matrix multiplier of an embodiment.

[0015] Figure 7 It shows that according to Figure 4 Embodiment of Figure 1 Block diagram of the configuration of a matrix multiplier.

[0016] Figure 8 It is shown Figure 7 Block diagram of the configuration of the input vector scaler.

[0017] Fig. 9 is shown in more detail Figure 8 Schematic diagram of the operation of the multiplication and scaling circuit.

[0018] Fig.10 is shown in more detail by Figure 8 A diagram of a method in which a multiplication-scaling circuit performs a multiplication-scaling operation.

[0019] Fig.11 It is shown Figure 7 A diagram of a configuration of a first data type converter.

[0020] Fig.12 It is shown Fig.11 A diagram of the operation of the index extraction circuit.

[0021] Fig.13 It is shown Fig.11 A diagram of the operation of the data type conversion circuit.

[0022] Fig.14 is shown in more detail Figure 7 A block diagram of a configuration of a processing element array.

[0023] Fig.15 is shown in more detail Figure 7 Block diagram of some operations of the matrix multiplier.

[0024] Fig.16 is shown in more detail Fig.15 A block diagram of the operation of a first processing element row.

[0025] Fig.17 It is shown that the Fig.16 A diagram of the configuration of one of the processing elements.

[0026] Fig.18 It is shown Figure 7 FIG. 2 is a diagram of the operation of a second data type converter.

[0027] Fig.19 It is shown Figure 1 A flow chart of the operation of the matrix multiplication device.

[0028] Fig. 20 is shown in more detail Fig.19 Flow chart of step S150.

[0029] Fig.21 It is shown Figure 1 A flow chart of the operation of the matrix multiplication device.

[0030] Fig. 22 is shown in more detail Fig.21 Flow chart of step S250.

[0031] Fig.23 is a diagram showing some example embodiments Figure 1 A diagram of the operation of the BCQ circuit.

[0032] Fig.24 It shows that according to Fig.23 Schematic diagram of an approximate weight matrix of an embodiment of FIG.

[0033] Fig.25 It is shown Fig.24 Schematic diagram of the quantization symbol bit matrix.

[0034] Fig.26 is a diagram showing implementation according to some example embodiments Figure 1 Block diagram of the configuration of a matrix multiplier.

[0035] Fig. 27 is a diagram showing a method according to an embodiment of the present invention. Fig.26 Block diagram of the configuration of the input vector scaler.

[0036] Fig.28 is shown in more detail Fig. 27 Schematic diagram of the operation of a common scaling circuit.

[0037] Fig.29 is a diagram showing a method according to an embodiment of the present invention. Fig.26 Block diagram of the configuration of the input vector scaler.

[0038] Fig.30 is shown in more detail Fig.29 Schematic diagram of the operation of a common scaling circuit.

[0039] Fig.31 is shown in more detail Fig.29 Schematic diagram of the operation of the multiplication and scaling circuit.

[0040] Fig.32 is a diagram showing a method according to an embodiment of the present invention. Fig.26 Block diagram of the configuration of the input vector scaler.

[0041] Fig.33 is shown in more detail Fig.32 Schematic diagram of the operation of the multiplication and scaling circuit.

[0042] Fig.34 is shown in more detail Fig.32 A diagram of the operation of the quantization scaling circuit.

[0043] Fig.35 It is shown Fig.26 A block diagram of a configuration of a first data type converter.

[0044] Fig.36 is shown in more detail Fig.26 A block diagram of a configuration of a processing element array.

[0045] Fig.37 is shown in more detail Fig.36 A block diagram of the operation of a processing element.

[0046] Fig.38 It is shown that the Fig.36 A diagram of the configuration of one of the processing elements.

[0047] Fig.39 It is shown Fig.26 FIG. 2 is a diagram of the operation of a second data type converter.

[0048] Fig.40 Yes means Figure 1 A flow chart of the operation of the matrix multiplication device.

[0049] Fig.41 is shown in more detail Fig.40 Flow chart of step S350.

[0050] Fig.42 It is shown Figure 1 Flowchart of the operation of the matrix multiplier.

[0051] Figure 43-45 The embodiment is shown in more detail Fig.42 Flow chart of step S440.

[0052] Fig.46 is shown in more detail Fig.42 Flow chart of step S450.

[0053] Fig.47 is a diagram showing a method according to an embodiment of the present invention. Figure 1A diagram of the operation of the BCQ circuit.

[0054] Fig.48 It is shown that the Figure 1 Block diagram of the configuration of a matrix multiplier.

[0055] Fig.49 is a diagram showing a method according to an embodiment of the present invention. Fig.48 Block diagram of the configuration of the input vector scaler.

[0056] Fig.50 is shown in more detail Fig.49 Schematic diagram of the operation of the multiplication and scaling circuit.

[0057] Fig.51 is shown in more detail Fig.49 Schematic diagram of the operation of a common scaling circuit.

[0058] Fig.52 is a diagram showing a method according to an embodiment of the present invention. Fig.48 Block diagram of the configuration of the input vector scaler.

[0059] Fig.53 is shown in more detail Fig.52 Schematic diagram of the operation of a common scaling circuit.

[0060] Fig.54 The embodiment according to the present invention is shown in more detail. Fig.52 Schematic diagram of the operation of the multiplication and scaling circuit.

[0061] Fig.55 is a diagram showing various exemplary embodiments Fig.48 Block diagram of the configuration of the input vector scaler.

[0062] Fig.56 is shown in more detail Fig.55 Schematic diagram of the operation of the multiplication and scaling circuit.

[0063] Fig.57 is shown in more detail Fig.55 A diagram of the operation of the quantization scaling circuit.

[0064] Fig.58 is shown in more detail Fig.48 A diagram of the operation of an array of processing elements.

[0065] Fig.59 It is shown that the systolic array is implemented Fig.26 Block diagram of an array of processing elements.

[0066] Fig.60 is shown in more detail Fig.59 A diagram of a configuration of a processing element;

[0067] Fig.61 is a diagram showing various exemplary embodiments Figure 1 Schematic diagram of the operation of a matrix multiplier.

[0068] Fig.62 It is shown Fig.61 Graph of the full input matrix of .

[0069] Fig.63 It is shown Fig.61 Graph of the full weight matrix of .

[0070] Fig.64 It is shown Fig.61 Graph of the full output matrix of .

[0071] Fig.65 is a block diagram illustrating a neural processing system implemented according to an embodiment.

[0072] Fig.66 It is shown by Fig.65 Block diagram of an artificial intelligence model driven by a neural processing system. DETAILED DESCRIPTION

[0073] Hereinafter, some example embodiments will be described clearly and in detail so that those skilled in the art can easily practice the present disclosure. Details such as detailed configuration and structure are provided only to promote a general understanding of various example embodiments. Therefore, without departing from the scope of the present invention, those of ordinary skill in the art may modify the various example embodiments described herein. In addition, for the sake of clarity and simplicity, the description of certain functions and structures (such as well-known functions and structures) may be omitted.

[0074] The components in the following figures or specific embodiments may be connected to other components except those components shown in the figures or described in the specific embodiments. The terms used herein are defined in view of the functions of the present disclosure and are not limited to specific functions. The definition of terms can be determined based on the details described in the specific embodiments. The components described with reference to the terms such as drivers or blocks used in the detailed description can be implemented in the form of software, hardware, or a combination thereof. Illustratively, software can be machine code, firmware, embedded code, and application software. For example, hardware can include circuits, electronic circuits, processors, computers, IC cores, pressure sensors, inertial sensors, micro-electromechanical systems (MEMS), passive devices, or a combination thereof.

[0075] Hereinafter, for a more concise explanation, matrices are referred to with square brackets "[", "]", sets are referred to with curly brackets "{", "}", and vectors are referred to with arrows. However, the scope of the inventive concept is not limited to this representation method.

[0076] Figure 1 is a block diagram illustrating a matrix multiplication device according to some example embodiments. Figure 1 The matrix multiplication device MMD (eg, a matrix multiplier device or a matrix multiplication device) may include a matrix multiplier 100 and a uniform BCQ circuit UBC. The matrix multiplication device MMD may receive an input matrix XM.

[0077] The input matrix XM may include multiple input vectors. Each of the multiple input vectors may include multiple input elements. For example, the input matrix XM may be expressed as the following equation 1.

[0078] (Equation 1)

[0079]

[0080] Referring to Equation 1, XM can represent the input matrix XM, to can represent the first to hth input vectors respectively, and x 11 to x hn can represent different input elements. For example, x 11 to x 1n It can be represented by the first input vector (for example, ), and x h1 to x hn can represent the input vector (e.g., ) in the input element.

[0081] Hereinafter, for a more concise description, an example embodiment in which the dimension of each of the input vectors included in the input matrix XM is "n" will be representatively described. That is, hereafter, an example embodiment in which each of the input vectors includes "n" input elements will be representatively described. For example, hereafter, an example embodiment in which the input matrix XM includes "n" columns will be representatively described. Here, "h" may be the same as "n", greater than or less than "n".

[0082] In some example embodiments, each of the input elements included in the input matrix XM may have a 16-bit floating point (FP16) and / or 32-bit floating point (FP32) data type. However, example embodiments are not limited thereto.

[0083] The matrix multiplication device MMD may receive a weight matrix WM. The weight matrix WM may include a plurality of weights. For example, the weight matrix WM may be expressed as the following equation 2.

[0084] (Equation 2)

[0085]

[0086] Referring to Equation 2, WM can represent the weight matrix WM, and w 11 tow nm Each of w can represent a different weight. For example, w ij The weights may be arranged in the i-th row and j-th column of the weight matrix WM. The weight matrix WM may have "n" rows and "m" columns. Here, "m" may be the same as "n", greater than or less than "n".

[0087] In some example embodiments, each of the weights included in the weight matrix WM may have a 16-bit floating point FP16 or 32-bit floating point FP32 data type. However, example embodiments are not limited thereto.

[0088] The uniform BCQ circuit UBC may perform uniform binary coded quantization (BCQ) for the weight matrix WM. For example, the uniform BCQ circuit UBC may determine a plurality of common scaling factors CSC, a plurality of multiplication scaling factors MSC, and a plurality of quantization sign bits QSB based on the weight matrix WM.

[0089] More specifically, the uniform BCQ circuit UBC can convert each of the multiple weights into multiple combinations of a common scaling coefficient CSC, a multiplication scaling coefficient MSC, and a quantization sign bit QSB. For example, the uniform BCQ circuit UBC can approximate each of the weights of the weight matrix WM based on multiple combinations of the common scaling coefficient CSC, the multiplication scaling coefficient MSC, and the quantization sign bit QSB. That is, the unified BCQ circuit can determine the common scaling coefficient CSC, the multiplication scaling coefficient MSC, and the quantization sign bit QSB representing the quantization value of the weight for each weight. The common scaling coefficient CSC, the multiplication scaling coefficient MSC, and the quantization sign bit QSB can be used to determine the quantization value of the weight. This will be described in more detail below.

[0090] In some example embodiments, each of the plurality of quantization sign bits QSB may represent '0' or '1'.

[0091] In various example embodiments, each of the plurality of common scaling coefficients CSC may have the same data type as the weights of the weight matrix WM. The data type may define a set of possible values, a set of allowed operations on the values, and / or a representation of the values. For example, each of the plurality of common scaling coefficients CSC may have an FP16 or FP32 data type. However, example embodiments are not limited thereto. Figure 3 The operation of the uniform BCQ circuit UBC is described in more detail.

[0092] The matrix multiplier 100 may receive a plurality of common scaling coefficients CSC, a plurality of multiplication scaling coefficients MSC, and a plurality of quantization sign bits QSB. The matrix multiplier 100 may perform matrix multiplication on an input matrix XM and a weight matrix WM based on a plurality of common scaling coefficients CSC, a plurality of multiplication scaling coefficients MSC, and a plurality of quantization sign bits QSB. For example, the matrix multiplier 100 may multiply the input matrix XM by a weight matrix WM approximated by a plurality of common scaling coefficients CSC, a plurality of multiplication scaling coefficients MSC, and a plurality of quantization sign bits QSB to generate an output matrix YM.

[0093] The output matrix YM may include multiple output vectors. Each of the multiple output vectors may include multiple output elements. For example, the output matrix YM may be expressed as the following equation 3.

[0094] (Equation 3)

[0095]

[0096] In this case, YM can represent the output matrix YM, and y 11 To hm can represent different output elements. For example, y 11 To 1m It can be represented as included in the first output vector (for example, ) and y h1 To hm can represent the h-th output vector (e.g., ) in the output element.

[0097] In some example embodiments, "n" and "m" may be the same integer. For example, the weight matrix WM may be implemented as a square matrix. In this case, the dimension of the output vector included in the output matrix YM may be the same as the dimension of the input vector. However, example embodiments are not limited thereto.

[0098] In some example embodiments, when the matrix multiplication device MMD calculates the output matrix YM by directly multiplying the input matrix XM with the weight matrix WM, the matrix multiplication device MMD may have to use a very large amount of computing resources and / or time for processing. In this case, the operating speed of the matrix multiplication device MMD may be reduced. Figure 2 The operation of the matrix multiplication device MMD which computes the output matrix YM by directly multiplying the input matrix XM with the weight matrix WM is described in more detail.

[0099] On the other hand, when the matrix multiplication device MMD calculates the output matrix YM by multiplying the input matrix XM and the weight matrix WM (wherein the output matrix YM is approximated by a plurality of common scaling coefficients CSC, a plurality of multiplication scaling coefficients MSC, and a plurality of quantization sign bits QSB), the amount of calculation of the matrix multiplication device MMD can be reduced, for example, can be greatly reduced. The operation of the matrix multiplication device MMD to calculate the output matrix YM based on a plurality of common scaling coefficients CSC, a plurality of multiplication scaling coefficients MSC, and a plurality of quantization sign bits QSB will be described in more detail below with reference to the accompanying drawings.

[0100] Figure 2 is shown to be implemented as Figure 1 Schematic diagram of the operation of the matrix multiplication device, where the input matrix is ​​directly multiplied by the weight matrix. Figure 1 and Figure 2 , the matrix multiplication device MMD can calculate the output matrix YM by directly multiplying the input matrix XM with the weight matrix WM.

[0101] To calculate an output element included in the output matrix YM, the matrix multiplication device MMD may perform floating point multiplication "n" times and then perform floating point summation "n-1" times. For example, the matrix multiplication device MMD may calculate one of the output elements (e.g., y) included in the output matrix YM in a manner similar to the following equation 4: 11 ).

[0102] (Equation 4)

[0103] y 11 =x 11 × 11 +x 12 × 21 +…+x 1n × n1

[0104] In this way, to compute the value corresponding to an input vector (e.g., ) of an output vector (e.g., ), the matrix multiplication device MMD may have to perform "m×n" floating-point multiplication operations and perform floating-point sum operations "m×(n-1)" times. In this case, the operation speed of the artificial intelligence model driven by the matrix multiplication device MMD will be reduced due to the excessive amount of calculations performed by the matrix multiplication device MMD.

[0105] Figure 3 and Figure 4 It is shown Figure 1 Operation of the uniform BCQ circuit. Figure 3 is shown as a tree, such as a binary tree; example embodiments are not limited thereto. First, reference is made to Figure 1 and Figure 3 , the horizontal axis represents the magnitude of the weights included in the weight matrix WM.

[0106] The uniform BCQ circuit UBC may approximate one or more of the weights included in the weight matrix WM as a plurality of quantum levels (QL).

[0107] The uniform BCQ circuit UBC may determine the number of the plurality of quantum levels QL based on a predetermined BCQ resolution. For example, the uniform BCQ circuit UBC may approximate each of the plurality of weights to 2 R (where R is the BCQ resolution) quantum levels QL. 2 can be determined based on a combination of a zero point value ZPV (null point value), R quantization scaling coefficients QSC, and R quantization sign bits QSB. R Each of the quantum levels QL.

[0108] Hereinafter, for a more concise explanation, an example embodiment in which R is "3" will be representatively described; however, the example embodiment is not limited thereto. For example, the uniform BCQ circuit UBC may approximate one or more of the weights included in the weight matrix WM to the first quantum level QL1 to the eighth quantum level QL8. In this case, each of the first quantum level QL1 to the eighth quantum level QL8 may be determined based on the following equation 5.

[0109] (Equation 5)

[0110]

[0111] Referring to Equation 5, QL can represent the quantum level, ZPV can represent the zero-point value ZPV of the uniform BCQ, α 1 To α 3 The first quantization scaling coefficient QSC1 to the third quantization scaling coefficient QSC3 may be represented respectively, and b 1 To b 3 The first to third quantization symbol bits QSB1 to QSB3 may be represented respectively. In this case, the first to eighth quantum levels QL1 to QL8 may correspond to different combinations of the first to third quantization symbol bits QSB1 to QSB3.

[0112] For example, further reference Figure 4 , the first quantum level QL1 may correspond to the case where the first quantization symbol bit QSB1 to the third quantization symbol bit QSB3 are all "1". In this case, the first quantum level QL1 may correspond to "ZPV-α 1 -α 2 -α 3Similarly, the eighth quantum level QL8 may correspond to the case where the first quantization symbol bit QSB1 to the third quantization symbol bit QSB3 are all "0". In this case, the eighth quantum level QL8 may correspond to "ZPV+α 1 +α 2 +α 3 In this way, the seventh quantum level QL7 may correspond to the case where the first quantization symbol bit QSB1 to the third quantization symbol bit QSB3 are "1", "0", and "0", respectively. In this case, the seventh quantum level Q17 may correspond to "ZPV-α 1 +α 2 +α 3 However, example embodiments are not limited thereto.

[0113] Continue to refer Figure 3 , the intervals between multiple quantum levels QL can be "uniform" to each other. For example, the intervals between multiple quantum levels QL can be equal to each other. In this case, multiple quantization scaling coefficients QSC can form a geometric sequence with a common ratio of "2". For example, the second quantization scaling coefficient QSC2 can be twice the first quantization scaling coefficient QSC1, and the third quantization scaling coefficient QSC3 can be twice the second quantization scaling coefficient QSC2.

[0114] Each of the plurality of quantization scaling coefficients QSC may be represented as a product of the same common scaling coefficient CSC but a different multiplication scaling coefficient MSC. For example, the first quantization scaling coefficient QSC1 may be represented as a product of the common scaling coefficient CSC and the first multiplication scaling coefficient MSC1, the second quantization scaling coefficient QSC2 may be represented as a product of the common scaling coefficient CSC and the second multiplication scaling coefficient MSC2, and the third quantization scaling coefficient QSC3 may be represented as a product of the common scaling coefficient CSC and the third multiplication scaling coefficient MSC3.

[0115] The multiplication scaling coefficient MSC corresponding to each of the plurality of quantization scaling coefficients QSC may be a different consecutive power of 2. For example, the k-th multiplication scaling coefficient MSCk may be 2 k-1 For a more detailed example, the first multiplication scaling factor may be 2 0 , the second multiplication scaling factor MSC2 can be 2 1 , and the third multiplication scaling factor MSC3 may be 2 2 .

[0116] Therefore, 2 can be determined based on a combination of the zero point value ZPV, a common scaling factor CSC, R multiplication scaling factors MSC and R quantization sign bits QSB. R Each of the quantum levels QL. For example, Equation 5 described above may also or alternatively be expressed as Equation 6 below.

[0117] (Equation 6)

[0118]

[0119] Referring to Equation 5 and Equation 6, 2 0 to 2 2 The first to third multiplication scaling coefficients MSC1 to MSC3 may be represented as different powers of 2, and s may represent a common scaling coefficient CSC.

[0120] The uniform BCQ circuit UBC may approximate each of the weights included in the weight matrix WM as a quantum level having the closest value among the plurality of quantum levels QL. For example, when the weight “w 11 When the size of " is closest to the seventh quantum level QL7 among the first quantum level QL1 to the eighth quantum level QL8, the uniform BCQ circuit UBC can change the weight "w 11 ” is approximately the seventh quantum level QL7. In this case, the uniform BCQ circuit UBC can convert the weight “w 11 ” is expressed as a combination of a zero point value ZPV, a common scaling coefficient CSC, a plurality of multiplication scaling coefficients MSC and a plurality of quantization sign bits QSB corresponding to the seventh quantum level QL7. Similarly, the uniform BCQ circuit UBC can approximate the weights included in the weight matrix WM based on the zero point value ZPV, a plurality of common scaling coefficients CSC, a plurality of multiplication scaling coefficients MSC and a plurality of quantization sign bits QSB.

[0121] However, hereinafter, for a more concise description, an example embodiment in which the zero point value ZPV is "0" will be representatively described. For example, an example embodiment in which the uniform BCQ circuit UBC symmetrically performs uniform binary coded quantization on each of the weights included in the weight matrix WM will be typically described below. Specifically, an example embodiment in which the uniform BCQ circuit UBC approximates the weight matrix WM based on a plurality of common scaling coefficients CSC, a plurality of multiplication scaling coefficients MSC, and a plurality of quantization sign bits QSB will be described. Figures 47 to 58 The operation of the matrix multiplication device MMD when the zero point value ZPV is not "0" (eg, the operation of the matrix multiplication device MMD when the uniform BCQ circuit UBC asymmetrically performs uniform binary coded quantization) is described in more detail.

[0122] For a more concise description, Figure 3 In the example embodiment, the BCQ resolution is representatively described as "3", but the example embodiment is not limited thereto. For example, depending on the type of artificial intelligence model driven by the matrix multiplication device MMD, the BCQ resolution may be determined as an integer greater than or equal to "2".

[0123] In an example embodiment, when the artificial intelligence model driven based on the matrix multiplication device MMD is or includes (or is included in) a large language model (LLM), the BCQ resolution may be “3.” However, example embodiments are not limited thereto.

[0124] In example embodiments, when the artificial intelligence model based on the matrix multiplication device MMD driving is or includes an image object recognition model (or is included in the image object recognition model), the BCQ resolution may be “2.” However, example embodiments are not limited thereto.

[0125] In an example embodiment, the uniform BCQ circuit UBC may perform a uniform binary coded quantization operation for each column of the weight matrix WM. For example, the uniform BCQ circuit UBC may determine a different common scaling coefficient CSC for each column of the weight matrix WM. In this case, the common scaling coefficient CSC for the weights included in the first column of the weight matrix WM may be different from the common scaling coefficient CSC for the weights included in the second column of the weight matrix WM. Figures 5 to 22 An example embodiment in which the uniform BCQ circuit UBC performs a binary coded quantization operation on each column of the weight matrix WM is described in more detail. However, example embodiments are not limited thereto.

[0126] In various example embodiments, the uniform BCQ circuit UBC may perform a uniform binary coded quantization operation for each row of the weight matrix WM. For example, the uniform BCQ circuit UBC may determine a different common scaling coefficient CSC for each row of the weight matrix WM. In this case, the common scaling coefficient CSC for the weights included in the first row of the weight matrix WM may be different from the common scaling coefficient CSC for the weights included in the second row of the weight matrix WM. Figure 23 to Figure 58 An example embodiment in which the uniform BCQ circuit UBC performs a binary coded quantization operation for each row of the weight matrix WM is described in more detail. However, example embodiments are not limited thereto.

[0127] Figure 5 The binary coded quantization operation is shown for each column of the weight matrix. Figure 1 The operation of the uniform BCQ circuit. Figures 1 to 5 , the uniform BCQ circuit UBC can perform a uniform binary coding quantization operation on each column of the weight matrix WM.

[0128] First, the weight matrix WM can be expressed as the following Equation 7.

[0129] (Equation 7)

[0130]

[0131] in this case, can represent the j-th column vector of the weight matrix WM. For example, Can include w 1j tow nj . “j” may be an integer greater than or equal to 1 and less than or equal to m.

[0132] The BCQ circuit 200 may perform a binary coded quantization operation on each column of the weight matrix WM based on Equation 8 below.

[0133] (Equation 8)

[0134]

[0135] Referring to Equation 8, R can represent the BCQ resolution. k_cj The k-th quantized scaling coefficient corresponding to the j-th column vector of the weight matrix WM may be represented.

[0136] It can be expressed as corresponding to α k_cj A quantized symbol vector of . For example, The quantization symbol vector corresponding to the k-th quantization scaling coefficient of the j-th column vector of the weight matrix WM may be represented. may include multiple values ​​of powers of multiple corresponding quantization sign bits equal to -1, where the multiple corresponding quantization sign bits correspond to the kth quantization scaling factor of the jth column vector. These values ​​may be -1 or 1. More specifically, It can be expressed as the following equation 9.

[0137] (Equation 9)

[0138]

[0139] b 1_k_cj To b n_k_cj Each of b can represent a different quantization sign bit QSB. More specifically, b 1_k_cj To b n_k_cj Each of may be the kth quantized sign bit of the weights arranged in different rows of the weight matrix WM. i_k_cj It can represent the kth quantized sign bit of the weight at the i-th column and j-th row of the weight matrix WM. 1_k_cj To b n_k_cj Each of may be either "0" or "1". For each weight in the weight matrix WM, there may be R quantization scaling coefficients and R quantization sign bits QSB.

[0140] The calculation of Equation 8 may be accomplished using a circuit (e.g., an arithmetic logic unit) that adds or subtracts values ​​based on the value of the quantized sign bit. Thus, to accomplish this, the circuit may receive the quantized sign bit vector A plurality of quantization sign bits corresponding to the kth quantization scaling coefficient of the jth column vector may be included. More specifically, It can be expressed as the following equation 9.1.

[0141] (Equation 9.1)

[0142]

[0143] At the same time, α k_cj It can be expressed as the product of a common scaling factor and a multiplicative scaling factor. For example, α k_cj It can be expressed as 2 k-1 ×s cj Therefore, Equation 8 described above may alternatively or additionally be expressed as the following Equation 10.

[0144] (Equation 10)

[0145]

[0146] Refer to Equation 10, 2 k-1 can represent the kth multiplication scaling factor, and s cj The common scaling coefficient of the j-th column vector of the weight matrix WM can be represented.

[0147] For example, the uniform BCQ circuit UBC may approximate each weight of the weight matrix WM based on one or more common scaling coefficients CSC (e.g., "s" values), a plurality of multiplication scaling coefficients MSC (e.g., powers of 2), and a plurality of quantization sign bits QSB (e.g., "b" values). The same common scaling coefficient CSC may be used for each column, but different common scaling coefficients CSC may be used for different columns.

[0148] In this way, various example embodiments in which a plurality of common scaling coefficients CSC, a plurality of multiplication scaling coefficients MSC, and a plurality of quantization sign bits QSB may be generated for a uniform BCQ circuit UBC based on a weight matrix WM. The uniform BCQ circuit UBC may provide a plurality of common scaling coefficients CSC, a plurality of multiplication scaling coefficients MSC, and a plurality of quantization sign bits QSB to a matrix multiplier 100. Hereinafter, the operation of the matrix multiplier 100 performing a matrix multiplication operation will be described based on a plurality of common scaling coefficients CSC, a plurality of multiplication scaling coefficients MSC, and a plurality of quantization sign bits QSB.

[0149] Figure 6 It shows that according to Figure 5Schematic diagram of the operation of a matrix multiplier of various example embodiments. Figures 1 to 6 , the matrix multiplier 100 can calculate the output matrix YM based on a plurality of common scaling coefficients CSC, a plurality of multiplication scaling coefficients MSC and a plurality of quantization sign bits QSB.

[0150] Hereinafter, for a more concise explanation, the matrix multiplier 100 will be representatively described as follows: ) and the first column vector of the approximate weight matrix WM (e.g., ) to calculate the output elements of the first row and first column of the output matrix YM (e.g., y 11 However, example embodiments are not limited thereto.

[0151] Referring to Equation 10, the matrix multiplier 100 can calculate y according to the following Equation 11 11 .

[0152] (Equation 11)

[0153]

[0154] Referring to Equation 11, first, the matrix multiplier 100 may perform the multiplication of the matrix vectors for each input vector (eg, ) to calculate the product of multiple multiplication scaling factors (eg, powers of 2). For example, the matrix multiplier 100 may multiply each multiplication scaling factor by multiple input elements (eg, x) included in the input vector. 11 to x 1n ) to generate a multiplication scaling input vector (hereinafter referred to as "multiplication scaling input vector MSX"). In this case, the multiplication scaling input vector MSX may include a plurality of multiplication scaling input elements (hereinafter referred to as "multiplication scaling input elements MSIE").

[0155] The data types of the plurality of input elements included in the input vector may be floating point. In this case, the matrix multiplier 100 may have to perform a total of (R×n) power-of-two floating point multiplication operations in order to calculate one output element (e.g., y 11 ).

[0156] In various example embodiments, floating point multiplication operations of powers of 2 can be performed with less computation (e.g., reduced computation). For example, the matrix multiplier 100 can perform floating point multiplication operations of powers of 2 by changing the exponent portion of the floating point data type. In the case of input elements represented in floating point using base 2, multiplication using a multiplication scaling factor that is a power of 2 can be efficiently implemented by simple addition of the corresponding exponents. Therefore, this allows efficient multiplication. Referring to FIG. Fig.10 The operation of the matrix multiplier 100 to perform power-of-two floating-point multiplication operations is described in more detail.

[0157] Thereafter, the matrix multiplier 100 may be configured to perform the multiplication by multiplying each of the plurality of scaled input vectors MSX by a different quantized symbol vector (eg, In the following, for a more concise description, the j-th column of the weight matrix WM and the i-th input vector (e.g., ) will be referred to as “PSPij”.

[0158] In some example embodiments, a multiplication scaling input vector MSX and a quantization symbol vector (e.g., a quantization symbol vector) may be calculated by sequentially adding or subtracting values ​​of a plurality of multiplication scaling elements MSIE based on a plurality of quantization symbol bits QSB. ). This may be based on a plurality of received quantized sign bits (e.g., in the form of a quantized sign bit vector ). Thereafter, the matrix multiplier 100 may calculate the partial product PSP by adding the result of multiplying the multiplication scaled input vector MSX with the quantized symbol vector. Therefore, in order to calculate one partial product PSP, the matrix multiplier 100 may have to perform floating point summation operations "R×n" times and multiple floating point summation operations "R-1" times.

[0159] In various example embodiments, when the data type of each of the plurality of input elements is converted to a fixed point, the matrix multiplier 100 may calculate the partial product PSP with fewer calculations. For example, when the data type of each of the plurality of input elements is a fixed point data type corresponding to the same exponent value, the matrix multiplier 100 may have to perform a fixed point sum operation "R×n" times and a fixed point sum operation "R-1" times to calculate the partial product PSP. Figures 7 to 22 A detailed operation of the matrix multiplier 100 to convert the data type of each of the plurality of input elements into a fixed point is described in more detail.

[0160] The matrix multiplier 100 can be implemented by multiplying the partial product PSP11 with a common scaling factor (e.g., s c1 ) to compute an output element (e.g., y11 For example, the matrix multiplier 100 may be implemented by performing a partial product PSP11 and a common scaling factor (eg, s c1 ) to compute the output element (e.g., y 11 ).

[0161] As described above, RxN floating-point multiplications of powers of 2 can be efficiently performed by simple addition of exponents. As a result, matrix multiplier 100 can compute one output element (e.g., y) by performing a full floating-point multiplication only once. 11 ). In this case, as with the previous reference Figure 2 The difference described above can reduce or minimize the number of floating-point multiplication operations performed by the matrix multiplier 100. Figure 5 and Figure 6 The various exemplary embodiments shown can improve the operating speed of the matrix multiplication device MMD.

[0162] Figure 7 It is shown that according to Figure 4 Various example embodiments of Figure 1 A block diagram of the configuration of a matrix multiplier. Figures 1 to 7 , the matrix multiplier 100 may include a multiplication scaling coefficient buffer 110, an input vector sealer 120, a first data type converter 130, a quantization sign bit buffer 140, a processing element array 150, a second data type converter 160, a common scaling coefficient buffer 170 and a common sealer 180.

[0163] The multiplication scaling factor buffer 110 may store a plurality of multiplication scaling factors MSC provided from the uniform BCQ circuit UBC. The multiplication scaling factor buffer 110 may provide the plurality of multiplication scaling factors MSC to the input vector scaler 120.

[0164] The input vector scaler 120 may receive an input matrix XM. For example, the input vector scaler 120 may receive a plurality of input vectors (eg, to ).

[0165] The input vector scaler 120 may perform multiplication scaling on the input matrix XM based on a plurality of multiplication scaling coefficients MSC. For example, the input vector scaler 120 may generate a plurality of scaled input vectors MSX based on a plurality of input vectors. In this case, the plurality of scaled input vectors MSX may correspond to a plurality of input vectors (e.g., to ). In the following, corresponding to to The scaled input vectors will be referred to as to

[0166] In an example embodiment, a plurality of multiplication scaling input vectors MSX may be included in a scaling input matrix. In this case, the row size of the multiplication scaling input matrix may be an integer multiple of the row size of the input matrix XM, and the column size of the multiplication scaling input matrix may be the same as the column size of the input matrix XM. However, example embodiments are not limited thereto.

[0167] Each of the plurality of multiplication-scaled input vectors MSX may be implemented as a row vector having a dimension "R" times the dimension of the corresponding input vector. For example, corresponding to the first input vector (e.g., ) scales the input vector (e.g., ) may include “R×n” scaling input elements MSIE, as shown in Equation 12 below.

[0168] (Equation 12)

[0169]

[0170] Referring to Equation 12, the multiplication scaling factor included in the first input vector may be generated by multiplying each of the plurality of input elements included in the first input vector by a number of multiplication scaling factors corresponding to the BCQ resolution (eg, R multiplication scaling factors). Multiple multiplication scaling of input elements in MSIE.

[0171] In this way, the second to h-th multiplications scale the input vector (e.g., to ) may include "R×n" multiplication scaling input elements. For the sake of simplicity, the multiplication scaling input elements included in the second to the h-th scaling input vectors (eg, to ) in each of the multiplication scaling input elements. Figures 8 to 10 The configuration and operation of the input vector sealer 120 is described in more detail.

[0172] In an example embodiment, the data type of each multiplication scaling input element MSIE may be a floating point. However, example embodiments are not limited thereto.

[0173] The first data type converter 130 may receive a plurality of multiplication scaling input vectors MSX. For example, the first data type converter 130 may receive first to h-th multiplication scaling input vectors (eg, to ). Each of the first to h-th multiplication scaling input vectors may include a plurality of multiplication scaling input elements MSIE.

[0174] The first data type converter 130 may extract an exponent EXP from each of the plurality of multiplication-scaled input vectors MSX. For example, the first data type converter 130 may extract an exponent EXP from the first multiplication-scaled input vector Extract the first exponent and scale the input vector from the second multiplication Extract the second index.

[0175] The first data type converter 130 may provide the extracted exponent EXP to the second data type converter 160. The first data type converter 130 may convert the data types of the plurality of multiplication scaling input vectors MSX into fixed point. For example, the first data type converter 130 may receive the plurality of multiplication scaling input vectors MSX and output a plurality of fixed-point multiplication scaling input vectors MSXfxp. In some examples, the first data type converter 130 may generate the first to hth fixed-point multiplication scaling input vectors (e.g., to ).

[0176] More specifically, the first data type converter 130 converts the data type of each of the plurality of multiplication scaling input elements MSIE included in the multiplication scaling input vector MSX into a fixed point based on the extracted exponent. In this case, the first fixed-point multiplication scaling input vector (e.g., ) may include the elements of the above equation 12 converted into fixed-point form (hereinafter, these elements may be referred to as fixed-point multiplication scaling input elements MSIEfxp). Figures 11 to 13 The configuration and operation of the first data type converter 130 will be described in more detail.

[0177] The quantization sign bit buffer 140 may store a plurality of quantization sign bits QSB provided from the uniform BCQ circuit UBC. The quantization sign bit buffer 140 may provide a plurality of quantization sign bits QSB to the processing element array 150.

[0178] The processing element array 150 may receive a plurality of quantization sign bits QSB and a plurality of fixed-point multiplication scaling input vectors MSXfxp and may calculate a plurality of fixed-point partial products PSPfxp based on a plurality of fixed-point scaling input elements MSIEfxp and a plurality of quantization sign bits QSB included in the plurality of fixed-point scaling input vectors MSXfxp.

[0179] The processing element array 150 may include a plurality of processing elements arranged in the row direction and the column direction. Each of the plurality of processing elements may calculate a different fixed-point partial product PSPfxp. Figures 14 to 17A more detailed configuration and operation of each of processing element array 150 will be described in more detail.

[0180] Second data type converter 160 may receive a plurality of exponents EXP from first data type converter 130. Second data type converter 160 may receive a plurality of fixed-point partial products PSPfxp from processing element array 150. Second data type converter 160 may convert the data types of the plurality of fixed-point partial products PSPfxp into floating points based on the plurality of exponents EXP. In some examples, second data type converter 160 may output the plurality of partial products PSP having a floating-point data type. Fig.18 The detailed configuration and operation of the second data type converter 160 will be described in more detail.

[0181] The common scaling coefficient buffer 170 may store a plurality of common scaling coefficients CSC provided from the uniform BCQ circuit UBC. The common scaling coefficient buffer 170 may provide the plurality of common scaling coefficients CSC to the common scaler 180.

[0182] The common scaler 180 may receive a plurality of common scaling coefficients CSC and a plurality of partial products PSP. The common scaler 180 may scale the plurality of partial products PSP based on the plurality of common scaling coefficients CSC. For example, the common scaler 180 may generate a plurality of output vectors included in the output matrix YM by multiplying each of the plurality of partial products PSP by the corresponding common scaling coefficient CSC. Fig.15 The detailed configuration and operation of the common sealer 180 will be described in more detail.

[0183] Figure 8 It is shown Figure 7 A block diagram of the configuration of the input vector scaler. Figures 1 to 8 , the input vector scaler 120 may include a first multiplication scaling circuit 121 to an hth multiplication scaling circuit 12h.

[0184] Each of the first multiplication and scaling circuit 121 to the hth multiplication and scaling circuit 12h can receive a different input vector. For example, each of the first multiplication and scaling circuit 121 to the hth multiplication and scaling circuit 12h can receive the first to the hth input vectors (eg, to ).

[0185] Each of the first multiplication and scaling circuit 121 to the hth multiplication and scaling circuit 12h may sequentially receive a plurality of input elements. For example, the first multiplication and scaling circuit 121 may sequentially receive x 11 to x 1N , and the hth multiplication and scaling circuit 12h can sequentially receive x h1to x Hn The multiplication-scaling circuits 121 to 12h may operate in parallel with each other.

[0186] Each of the first to h-th multiplication scaling circuits 121 to 12h may sequentially receive a plurality of multiplication scaling coefficients MSC from the multiplication scaling coefficient buffer 110. For example, each of the first to h-th multiplication scaling circuits 121 to 12h may receive first to R-th multiplication scaling coefficients MSC1 to MSCR (e.g., 2 0 to 2 R -1 ).

[0187] The first multiplication-scaling circuit 121 to the hth multiplication-scaling circuit 12h may respectively generate first to hth multiplication-scaling input vectors (eg, to ). For example, the first multiplication scaling circuit 121 may be based on x 11 to x 1n and the first multiplication scaling coefficient MSC1 to the Rth multiplication scaling coefficient MSCR to generate a first multiplication scaling input vector (eg, ).

[0188] Fig. 9 is shown in more detail Figure 8 In the following, for a more concise description, reference will be made to Figures 1 to 9 1 to representatively describe the operation of the first multiplication-scaling circuit 121. However, example embodiments are not limited thereto, and the second to h-th multiplication-scaling circuits 122 to 12h may operate in a similar manner.

[0189] The first multiplication and scaling circuit 121 may receive a first input vector (eg, Specifically, the first multiplication and scaling circuit 121 may sequentially receive x 11 to x 1n The first multiplication scaling circuit 121 may sequentially receive the first to R-th multiplication scaling coefficients MSC1 to MSCR. The first multiplication scaling circuit 121 may generate a plurality of multiplication scaling input elements MSIE by multiplying each of the received input elements by the first to R-th multiplication scaling coefficients MSCR.

[0190] For example, the first multiplication and scaling circuit 121 can be configured by converting the input element “x 11 " is multiplied by the first multiplication scaling coefficient MSC1 to the Rth multiplication scaling coefficient MSCR to generate a plurality of multiplication scaling input elements MSIE11_1 to MSIE11_R. Fig. 9 Shown as diagonal stripes.

[0191] Similarly, the first multiplication and scaling circuit 121 can be implemented by converting the input element “x 12 " respectively multiply the first multiplication scaling coefficient MSC1 to the Rth multiplication scaling coefficient MSCR to generate the multiplication scaling input elements MSIE12_1-MSIE12_R. This Fig. 9 Shown as a dot pattern.

[0192] In this way, the first multiplication and scaling circuit 121 will be able to sequentially calculate the values ​​corresponding to x 13 to x 2N Multiple multiplications scale the input elements MSIE.

[0193] The first multiplication scaling circuit 121 may sequentially output a plurality of multiplication scaling input elements MSIE. For example, the first multiplication scaling circuit 121 may provide a plurality of multiplication scaling input elements MSIE to the first data type converter 130.

[0194] Specifically, according to various exemplary embodiments of the present disclosure, the first multiplication and scaling circuit 121 may be based on an input element (eg, x 11 ) to generate a plurality of multiplicative scaling input elements (eg, multiplicative scaling input elements (x) shown as diagonal stripes 11 In some cases, the first multiplication scaling circuit 121 can generate multiple multiplication scaling input elements MSIE by reusing one input element. This can be achieved by using a buffer or a memory. Therefore, according to various example embodiments, the input reuse of the matrix multiplier 100 can be increased or maximized, and therefore, the number of times the matrix multiplier 100 receives input elements from the outside can be reduced or minimized. In this case, the number of times the matrix multiplier 100 accesses an external memory device that stores input elements can be reduced or minimized, so the operating efficiency and operating speed of the matrix multiplication device MMD can be improved.

[0195] Fig.10 is shown in more detail by Figure 8 1 is a diagram of a method for performing a multiplication and scaling operation by a multiplication and scaling circuit of FIG. 1 . In the following, for a more concise description, reference will be made to FIG. Figures 1 to 10 1 to representatively describe a method in which the first multiplication scaling circuit 121 generates one multiplication scaled input element MSIE. However, example embodiments are not limited thereto.

[0196] The first multiplication scaling circuit 121 may receive an input element IE and a multiplication scaling coefficient MSC.

[0197] The input element IE may have a floating point data type. Floating point arithmetic may use an integer (significant number or mantissa) of fixed precision and an integer exponent of a fixed base (e.g., base 2) to represent a number. For example, the input element IE may include a sign portion SP (e.g., indicating whether the input element IE is positive or negative), an exponent portion EXPP (e.g., indicating the exponent of the input element IE) and a mantissa portion MTSP (e.g., indicating the mantissa (also referred to as the significand) of the input element IE). The input element IE may be a floating point data type of base 2.

[0198] The multiplication scaling factor MSC may be a power of 2. For example, the multiplication scaling factor MSC may be 2 0 to 2 R-1 one.

[0199] The first multiplication scaling circuit 121 can generate a multiplication scaling input element MSIE by multiplying the input element IE by the multiplication scaling coefficient MSC. In this case, the first multiplication scaling circuit 121 can change the value of the exponent part EXPP of the input element IE to generate the multiplication scaling input element MSIE. For example, the first multiplication scaling circuit 121 can perform a floating-point multiplication operation on a power of 2 by increasing the value of the exponent part EXPP of the input element IE by a size corresponding to the multiplication scaling coefficient MSC.

[0200] For example, when the first multiplication scaling circuit 121 calculates the input element IE and the kth multiplication scaling coefficient MSCk (eg, 2 k -1 ), the first multiplication scaling circuit 121 may generate a multiplication scaling input element MSIE by increasing the value of the exponent EXPP of the input element IE by “k-1”.

[0201] For a more detailed example, when the lowest three digits of the exponent EXPP of the input element IE are "100" (for example, indicating 2 4 ) and the first multiplication scaling circuit 121 calculates the input element IE and the second multiplication scaling coefficient MSC2 (eg, 2 1 ), the product of 2 may be obtained by changing the lowest three bits of the exponent EXPP of the input element IE of the first multiplication scaling circuit 121 to "101" (for example, by increasing the value of the exponent part EXPP by "1") (representing 2 5 ) to generate the multiplicative scaling input element MSIE.

[0202] Therefore, according to various exemplary embodiments of the present disclosure, the input vector scaler 120 will be able to generate the first to h-th multiplication-scaled input vectors (ie, to ). However, example embodiments are not limited thereto.

[0203] Fig.11 It is shown Figure 7 A diagram of a configuration of a first data type converter. Figures 1 to 11 , the first data type converter 130 may include first to h-th exponent extraction circuits 131_1 to 131_h and first to h-th data type conversion circuits 132_1 to 132_h.

[0204] Each of the first to h-th exponent extraction circuits 131_1 to 131_h may receive a different multiplication scaling input vector MSX. For example, the first to h-th exponent extraction circuits 131_1 to 131_h may receive first to h-th multiplication scaling input vectors (eg, to ).

[0205] Each of the first exponent extraction circuit 131_1 to the hth exponent extraction circuit 131_h can extract an exponent from a plurality of received multiplication scaling input elements MSIE. For example, the first exponent extraction circuit 131_1 can extract an exponent from the first multiplication scaling input vector (eg, ), and the second exponent extraction circuit 131_2 can extract the first exponent EXP1 from the multiplication scaling input element MSIE included in the second multiplication scaling input vector (eg, ) in the multiplication scaling input element MSIE. In this way, the first exponent extraction circuit 131_1 to the h-th exponent extraction circuit 131_h can extract the first exponent EXP1 to the h-th exponent EXPh, respectively.

[0206] The first to h-th exponent extraction circuits 131_1 to 131_h may provide the extracted exponents to the first to h-th data type conversion circuits 132_1 to 132_h, respectively. Alternatively or additionally, each of the first to h-th exponent extraction circuits 131_1 to 131_h may provide the extracted exponents to the second data type converter 160. Fig.12 Detailed operations of the first to h-th exponent extraction circuits 131_1 to 131_h will be described in more detail.

[0207] The first data type conversion circuit 132_1 to the hth data type conversion circuit 132_h can receive the first exponent EXP1 to the hth exponent EXPh respectively. The first data type conversion circuit 132_1 to the hth data type conversion circuit 132_h can receive the first to the hth multiplication scaling input vectors (eg, to ).

[0208] Each of the first data type conversion circuit 132_1 to the hth data type conversion circuit 132_h can convert the data type of the received multiplication scaling input vector into a fixed point based on the received exponent. The first data type conversion circuit 132_1 to the hth data type conversion circuit 132_h can respectively output the first fixed-point multiplication scaling input vector to the hth fixed-point multiplication scaling input vector (e.g., to ). For example, the first data type conversion circuit 132_1 may convert each of the multiplication scaling input elements MSIE included in the first multiplication scaling input vector MSX into a fixed-point data type based on the first exponent EXP1. Fig.13 The detailed operations of the first to h-th data type conversion circuits 132_1 to 132_h are described in more detail.

[0209] Fig.12 It is shown Fig.11 FIG. 1 is a diagram of an operation of an exponent extraction circuit of FIG. 1 . Hereinafter, for a more concise description, the operation of the first exponent extraction circuit 131_1 will be representatively described. However, example embodiments are not limited thereto.

[0210] refer to Figures 1 to 12 , the first exponent extraction circuit 131_1 may receive a plurality of multiplication scaling input elements MSIE. For example, the first exponent extraction circuit 131_1 may receive the above reference Fig. 9 The multiplication scaling input elements MSIE11_1 to MSIE1n_R are described.

[0211] The data type of each of the plurality of multiplication scaling input elements MSIE may be a floating point. For example, each of the multiplication scaling input elements MSIE11_1 to MSIE1n_R may include a sign part SP, an exponent part EXPP, and a mantissa part MTSP. The floating point format may use base 2.

[0212] The first exponent extraction circuit 131_1 may identify the maximum value among the values ​​of the exponent part EXPP of each of the received plurality of multiplication scaling input elements MSIE. In this case, the first exponent extraction circuit 131_1 may determine the identified value as the first exponent EXP1. That is, the first exponent extraction circuit 131_1 may extract the maximum exponent among the exponents of the multiplication scaling input elements MSIE11_1 to MSIE1n_R as the first exponent EXP1.

[0213] For a more concise description, Fig.12Various embodiments in which the first exponent extraction circuit 131_1 extracts the maximum value among the values ​​of the exponent part EXPP of the plurality of multiplication scaling input elements MSIE are representatively described in, but the scope of the present disclosure is not limited thereto. For example, the first exponent extraction circuit 131_1 may be implemented to extract the minimum value among the values ​​of the exponent part EXPP of the plurality of multiplication scaling input elements MSIE.

[0214] Fig.13 It is shown Fig.11 Hereinafter, for a more concise description, the operation of the first data type conversion circuit 132_1 will be representatively described. However, example embodiments are not limited thereto.

[0215] refer to Figures 1 to 13 The first data type conversion circuit 132_1 may receive a plurality of multiplication scaling input elements MSIE. For example, the first data type conversion circuit 132_1 may receive multiplication scaling input elements MSIE11_1 to MSIE1n_R.

[0216] The first data type conversion circuit 132_1 may receive a first exponent EXP1. The first data type conversion circuit 132_1 may convert the data types of the plurality of received multiplication scaling input elements MSIE into fixed points based on the first exponent EXP1. That is, the first data type conversion circuit 132_1 may convert each of the multiplication scaling input elements MSIE11_1 to MSIE1n_R into fixed-point multiplication scaling input elements MSIEfxp11_1 to MSIEfxp1n_R. However, hereinafter, for a more concise description, the operation of the first data type conversion circuit 132_1 that converts the multiplication scaling input element MSIE11_1 into the fixed-point multiplication scaling input element MSIEfxp11_1 will be representatively described.

[0217] The first data type conversion circuit 132_1 may shift the mantissa part (hereinafter, referred to as the first mantissa part MTSPa) of the multiplication scaling input element MSIE11_1 toward the least significant bit (LSB) direction by as much as the difference between the exponent of the multiplication scaling input element MSIE11_1 and the first exponent EXP1. For example, when the difference between the exponent of the multiplication scaling input element MSIE11_1 and the first exponent EXP1 is "4", the first data type conversion circuit 132_1 may insert "four" "0" bits into the most significant bit (MSB) position of the first mantissa part MTSPa.

[0218] The first data type conversion circuit 132_1 can determine the mantissa part (hereinafter referred to as the second mantissa part MTSPb) of the fixed-point multiplication scaling input element MSIEfxp11_1 based on the shifted first mantissa part MTSPa. For example, the first data type conversion circuit 132_1 can cut off the low bits of the shifted first mantissa part MTSPa according to the code length of the second mantissa part MTSPb. Alternatively, the first data type conversion circuit 132_1 can round the lower bits of the shifted first mantissa part MTSPa based on various types of rounding algorithms such as "nearest even rounding" according to the code length of the second mantissa part MTSPb. However, the example embodiments are not limited to this. That is, the mantissa part of the fixed-point multiplication scaling input element MSIEfxp11_1 can be set to a predefined maximum length. This may require the shifted first mantissa part MTSPa to be shortened and / or rounded.

[0219] In example embodiments, the code length of the first mantissa part MTSPa may be “10 bits” or “23 bits.” However, example embodiments are not limited thereto.

[0220] In example embodiments, the code length of the second mantissa part MTSPb may be “7 bits.” However, example embodiments are not limited thereto.

[0221] In an example embodiment, the data type of each of the plurality of fixed-point multiplication scaling input elements MSIEfxp may be INT8 (8-bit integer). However, example embodiments are not limited thereto. The fixed-point multiplication scaling input elements MSIEfxp may each include a sign portion and a second mantissa portion of the corresponding multiplication scaling input elements MSIE11_1 to MSIE1n_R.

[0222] Fig.14 is shown in more detail Figure 7 A block diagram of the configuration of the processing element array. Figures 1 to 14 , the processing element array 150 may include a plurality of processing elements PE arranged in the row direction and the column direction. Hereinafter, for a more concise description, it is assumed that the plurality of processing elements PE are arranged along h rows and m columns. In addition, the processing element placed in the i-th row and the j-th column of the processing element array 150 will be referred to as "PEij". For example, the processing element arranged in the first row and the second column of the processing element array 150 will be referred to as "PE12".

[0223] The processing element array 150 may include first to h-th processing element rows PER1 to PERh. Each of the first to h-th processing element rows PER1 to PERh may include a plurality of processing elements PE. For example, the first processing element row PER1 may include processing elements PE11 to PE1m.

[0224] The processing element array 150 may include first to m-th processing element columns PEC1 to PECm. Each of the first to m-th processing element columns PEC1 to PECm may include a plurality of processing elements PE. For example, the first processing element column PEC1 may include processing elements PE11 to PEh1.

[0225] Each of the plurality of processing element rows PER may receive a different fixed-point multiplication scaling input vector MSXfxp. For example, the first processing element row PER1 to the hth processing element row PERh may receive the first to the hth fixed-point multiplication scaling input vectors (ie, to ).

[0226] The processing elements included in the same processing element row may receive the same fixed-point multiplication scaling input vector. For example, each of the processing elements PE11 to PE1m may receive a first fixed-point multiplication scaling input vector (eg, ).

[0227] Each of the plurality of processing element columns PEC may receive a different quantization sign bit vector from the quantization sign bit buffer 140 For example, the first processing element column PEC1 to the mth processing element column PECm may receive the first to the mth quantization symbol bit vectors (eg, to ).

[0228] The first to mth quantized symbol bit vectors (eg, to ) may include multiple quantization sign bits QSB of different columns of the weight matrix WM. For example, the j-th quantization sign bit vector (e.g., ) may include a plurality of quantization sign bits QSB of the j-th column vector of the weight matrix WM. More specifically, the j-th quantization sign bit vector (eg, ) may include reference to Equation 9.1 described above to The quantization sign bit QSB included in . to Each of may include n quantization sign bits QSB. Therefore, each quantization sign bit vector (e.g., ) may include nxR quantization sign bits QSB. Fig.16 Describing the first to mth quantized symbol bit vectors (eg, to ).

[0229] The processing elements included in the same processing element column may receive the same quantization sign bit vector. For example, each of the processing elements PE11 to PEh1 may receive a first quantization sign bit vector (eg, ), and each of the processing elements PE12 to PEh2 may receive a second quantized symbol bit vector (eg, ). That is, the processing elements included in the same processing element column may receive the same quantization sign bit QSB.

[0230] Each of the plurality of processing elements PE may calculate a different fixed-point partial product PSPfxp based on the received fixed-point multiplication scaling input vector MSXfxp and the quantization sign bit vector. That is, according to some example embodiments, one processing element PE may calculate one fixed-point partial product PSPfxp. For example, a processing element PEij may calculate a fixed-point partial product PSPfxpij. Below, a method of calculating a fixed-point partial product PSPfxp in each processing element PE is described in more detail.

[0231] Fig.15 is shown in more detail Figure 7 1 is a block diagram of some operations of the matrix multiplier 100. In the following, the matrix multiplier 100 will be representatively described for the first fixed-point multiplication scaling input vector (e.g., ), but example embodiments are not limited thereto.

[0232] For example, the matrix multiplier 100 may scale the input vectors for the second to h-th fixed-point multiplications in a similar manner (eg, to ) to perform the operation.

[0233] refer to Figures 1 to 15 , the first processing element row PER1 may receive a first fixed-point multiplication scaling input vector (ie, ). For example, each of the processing elements PE11 to PE1m may receive fixed-point multiplication scaling input elements MSIEfxp11_1 to MSIEfxp1n_R.

[0234] Each processing element PE11 to PE1m may receive a corresponding quantization symbol bit vector. For example, the processing elements PE11 to PE1m may receive the first to mth quantization symbol bit vectors (eg, to ).

[0235] Each processing element PE11 to PE1m may calculate a corresponding fixed-point partial product PSPfxp. For example, the processing elements PE11 to PE1m may calculate fixed-point partial products PSPfxp11 to PSPfxp1m, respectively.

[0236] The second data type converter 160 may receive the first exponent EXP1. The second data type converter 160 may receive the fixed-point partial products PSPfxp11 to PSPfxp1m from the first processing element row PER1. The second data type converter 160 may convert the fixed-point partial products PSPfxp11 to PSPfxp1m into floating-point data types based on the first exponent EXP1. For example, the second data type converter 160 may convert the fixed-point partial products PSPfxp11 to PSPfxp1m into partial products PSP11 to PSP1m, respectively.

[0237] The common scaler 180 may include first to m-th multiplication circuits MUL1 to MULm. The first to MULm multiplication circuits MUL1 to MULm may receive partial products PSP11 to PSP1m, respectively. The first to MULm multiplication circuits MUL1 to MULm may receive first to m-th common scaling coefficients CSC1 to CSCm, respectively, from the common scaling coefficient buffer 170. Each of the first to m-th common scaling coefficients CSC1 to CSCm may correspond to a different column vector of the weight matrix WM. For example, as described above with reference to Equations 9 to 11, the first to m-th common scaling coefficients CSC1 to CSCm may correspond to s c1 to cm .

[0238] Each of the first to m-th multiplication circuits MUL1 to MULm may calculate an output element based on the received partial product and the common scaling factor. For example, the first multiplication circuit MUL1 may calculate y by multiplying the partial product PSP11 by the first common scaling factor CSC1. 11 Similarly, the second multiplication circuit MUL2 can calculate y by multiplying the partial product PSP12 by the second common scaling coefficient CSC2. 12 .

[0239] In this manner, the matrix multiplier 100 may calculate a matrix corresponding to the first input vector (eg, ) output element (e.g., y 11 To 1m ). Similarly, the matrix multiplier 100 may use the processing elements included in the second processing element row PER2 to calculate the matrix corresponding to the second input vector (eg, ) output element (e.g., y 21 To 2m ).

[0240] In an example embodiment, the first to m-th multiplication operation circuits MUL1 to MULm may further generate a plurality of output elements by multiplying the first to m-th common scaling coefficients CSC1 to CSCm by partial products (e.g., partial products PSP21 to PSP2m) generated based on the second processing element row PER2, respectively. For example, the first multiplication operation circuit MUL1 may generate an output element “y” by multiplying the partial product PSP21 by the first common scaling coefficient CSC1. 21 ". In some examples, the first to m-th multiplication operation circuits MUL1 to MULm may respectively multiply the first to m-th common scaling coefficients CSC1 to CSCm by partial products generated based on the first to m-th processing element columns PEC1 to PECm. That is, the first to m-th multiplication circuits MUL1 to MULm may operate on partial products output from processing elements of each row (e.g., each multiplication circuit may operate on a corresponding column of the processing element array). However, example embodiments are not limited to a specific implementation method of the common sealer 180.

[0241] In an example embodiment, when the number of columns of the weight matrix WM is greater than the number of columns of the processing element array 150 or the number of rows of the weight matrix WM is greater than the number of rows of the processing element array 150, the matrix multiplier 100 may calculate the output elements based on various tiling techniques. Figure 61 to Figure 64 Let's describe the tiling technique in more detail.

[0242] Fig.16 is shown in more detail Fig.15 That is, hereinafter, reference will be made to Figures 1 to 16 However, example embodiments are not limited thereto, and the second to h-th processing element rows PER2 to PERh may also operate in a similar manner.

[0243] The first processing element row PER1 may receive a first fixed-point multiplication scaling input vector (eg, ). For example, the first processing element row PER1 may sequentially receive fixed-point multiplication scaling input elements MSIEfxp11_1 to MSIEfxp1n_R. In this case, the plurality of fixed-point multiplication scaling input elements MSIEfxp11_1 to MSIEfxp1n_R may correspond to the fixed-point multiplication scaling input elements MSIEfxp11_1 to MSIEfxp1n_R described above. Fig. 9 Each of the multiple multiplication scaling input elements MSIE11_1 to MSIE1n_R is described.

[0244] Processing elements PE11 to PE1m may receive first to mth quantized symbol bit vectors (eg, arrive ). The first to mth quantized symbol bit vectors (eg, to ) may include multiple quantization sign bits QSB generated based on different column vectors of the weight matrix WM.

[0245] For a more detailed example, the first quantized symbol bit vector (eg, ) may include b 1_1_c1 To b n_R_c1 In this case, b 1_1_c1 To b 1_R_c1 The input elements MSIEfxp11_1 to MSIEfxp11_R (eg, x 11 fixed-point multiplication scales the input elements); and b 2_1_c1 To b 2_R_c1 The input elements MSIEfxp12_1 to MSIEfxp12_R (eg, x 12 ) Similarly, b n_1_c1 To b n_R_c1 The input elements MSIEfxp1n_1 to MSIEfxp1n_R (eg, x 1n Fixed-point multiplication scales the input elements).

[0246] The processing element PE11 may sequentially receive a plurality of pairs of quantization sign bits QSB and fixed-point multiplication scaling input elements MSIEfxp corresponding to each other. Each pair may include a quantization sign bit QSB and a corresponding fixed-point multiplication scaling input element MSIEfxp. For example, the processing element PE11 may sequentially receive the quantization sign bits QSB and the fixed-point multiplication scaling input elements MSIEfxp corresponding to each other. 11 Fixed-point scaling multiplication scaling input element MSIEfxp corresponding to b 1_1_c1 To b 1_R_c1 , and then sequentially receive the same 12 Fixed-point scaling multiplication scaling input element MSIEfxp corresponding to b 2_1_c1 To b 2_R_c1 In this way, the processing element PE11 can sequentially receive the same 1n Fixed-point multiplication scales the input element MSIEfxp corresponding to b 1_1_c1 To b n_R_c1 .

[0247] The processing element PE11 may calculate the fixed-point partial product PSPfxp11 based on the order of the quantization sign bit QSB and the received plurality of fixed-point multiplication scaling input elements MSIEfxp. For example, the processing element PE11 may calculate the fixed-point partial product PSPfxp11 by subtracting a value obtained by summing the fixed-point multiplication scaling input elements corresponding to the quantization sign bit QSB indicating "0" among the fixed-point multiplication scaling input elements MSIEfxp11_1 to MSIEfxp1n_R from a value obtained by summing the fixed-point multiplication scaling input elements corresponding to the quantization sign bit QSB indicating "0".

[0248] Similarly, processing element PE1j may receive a bit vector including the jth quantized symbol bit vector (eg, ) in the quantization sign bit (e.g., b 1_1_cj To b n_R_cj ). The processing element PE1j may receive a scaling input vector including scaling the input vector in a first fixed-point multiplication (eg, ) in multiple fixed-point multiplications to scale the input elements. The processing element PE1j may be based on the j-th quantized symbol bit vector (e.g., ) and the first fixed-point multiplication scales the input vector (e.g., ) to compute the fixed-point partial product PSPfxp1j.

[0249] For a more detailed example, the processing element PE may calculate the fixed-point partial product PSPfxp according to Equation 13 below.

[0250] (Equation 13)

[0251] PSPfxp=∑MSIEfxp×(-1) QSB

[0252] Referring to Equation 13, MSIEfxp and QSB may represent a pair of a fixed-point multiplication scaling input element MSIEfxp and a quantization sign bit QSB provided to the processing element PE. That is, the processing element PE may calculate the fixed-point partial product PSPfxp by sequentially performing a calculation of adding or subtracting the value of the fixed-point multiplication scaling input element MSIEfxp based on the received quantization sign bit QSB. However, example embodiments are not limited thereto.

[0253] Fig.17 is a diagram showing implementation according to various exemplary embodiments Fig.16 A diagram of the configuration of one of the processing elements of FIG. Figures 1 to 17 The processing element PE may include an arithmetic logic unit ALU and an accumulation register REG_ACC.

[0254] The arithmetic logic unit ALU may include first to third input terminals TI1 to TI3 and an output terminal TO. The first input terminal TI1 may sequentially receive a plurality of fixed-point multiplication scaling input elements MSIEfxp. The second input terminal TI2 may sequentially receive a plurality of quantization sign bits QSB. The third input terminal TI3 may be connected to an accumulation register REG_ACC.

[0255] When the value received by the second input terminal TI2 is "0", the arithmetic logic unit ALU can store the sum of the values ​​received by the first input terminal TI1 and the third input terminal TI3 in the accumulation register REG_ACC through the output terminal TO. On the other hand, when the value received by the second input terminal TI2 is "1", the arithmetic logic unit ALU can store the value obtained by subtracting the value provided to the first input terminal TI1 from the value provided to the third input terminal TI3 in the accumulation register REG_ACC via the output terminal TO. That is, when a quantization sign bit QSB equal to "0" is received at the second terminal TI2, the arithmetic logic unit ALU can calculate the sum of the values ​​input to the first terminal TI1 and the third terminal TI3, and output the sum through the output terminal TO. Similarly, when a quantization sign bit QSB equal to "1" is received at the second terminal TI2, the arithmetic logic unit ALU can subtract the value input to the first terminal TI1 from the value input to the third terminal TI3, and output the resulting value through the output terminal TO. The accumulator register may store the most recently received value output from the arithmetic logic unit ALU and feed the value back to the third input terminal TI3.

[0256] In this way, the processing element PE will be able to store the fixed-point partial product PSPfxp calculated according to the above-described equation 13 in the accumulation register REG_ACC. In this case, the fixed-point partial product PSPfxp stored in the accumulation register REG_ACC may be provided to the second data type converter 160. However, example embodiments are not limited to a specific method in which the processing element PE performs calculations and a specific configuration method of the processing element PE.

[0257] In example embodiments, the processing element PE may not include any "1-bit adder" for performing a multiplication operation on the multiplication scaling coefficient MSC. That is, according to some example embodiments, since the multiplication scaling input element MSIE is provided to the processing element array 150, each processing element PE may be implemented to not perform calculations on the scaling coefficient MSC. In this case, instead of each processing element PE including a circuit element for performing a multiplication operation on the multiplication scaling coefficient MSC, only one multiplication scaling circuit 121 may be included for each processing element row PER, so the size and production cost of the matrix multiplier 100 may be reduced.

[0258] Fig.18 It is shown Figure 7 1. In the following, for a more concise description, a method of receiving a first input vector (eg, ) generates a fixed-point partial product PSPfxp and outputs an operation of the second data type converter 160 of the partial product PSP. However, example embodiments are not limited thereto.

[0259] refer to Figures 1 to 18 , the second data type converter 160 may receive the first exponent EXP1 from the first data type converter 130. The second data type converter 160 may receive the fixed-point partial product PSPfxp from one processing element PE. In this case, the fixed-point partial product PSPfxp may include a sign part SP and a mantissa part MTSP.

[0260] The second data type converter 160 may generate a partial product PSP by converting the data type of the fixed-point partial product PSPfxp to a floating point. For example, the second data type converter 160 may add (e.g., insert) the exponent part EXPP to the fixed-point partial product PSPfxp. The second data type converter 160 may determine the exponent part EXPP of the partial product PSP so that the exponent part EXPP of the partial product PSP corresponds to the first exponent EXP1.

[0261] Second data type converter 160 may add a plurality of '0' bits to the least significant bit (LSB) position of mantissa part MTSP of partial product PSP according to the code length difference between the mantissa part of partial product PSP and the mantissa part of fixed point partial product PSPfxp. However, example embodiments are not limited thereto.

[0262] Fig.19 It is shown Figure 1 Flowchart of the operation of the matrix multiplication device. In the following, reference is made to Figures 1 to 19 , will describe receiving an input vector (e.g., ) and output an output vector (e.g., )'s matrix multiplication device MMD operation.

[0263] In operation S110, the matrix multiplication device MMD may receive a weight matrix WM. For example, the uniform BCQ circuit UBC may receive a plurality of weights (eg, w 11 tow nm ).

[0264] In operation S120, the matrix multiplication device MMD performs uniform binary coded quantization on the weight matrix WM to generate multiple multiplication scaling coefficients MSC, multiple common scaling coefficients CSC, and multiple quantization sign bits QSB. For example, the uniform BCQ circuit UBC can generate multiple multiplication scaling coefficients MSC, one common scaling coefficient CSC, and multiple quantization sign bits QSB for each weight. The uniform BCQ circuit UBC can provide multiple multiplication scaling coefficients MSC, multiple common scaling coefficients CSC, and multiple quantization sign bits QSB to the matrix multiplier 100. In this case, multiple multiplication scaling coefficients MSC can be stored in the multiplication scaling coefficient buffer 110, multiple common scaling coefficients CSC can be stored in the common scaling coefficient buffer 170, and multiple quantization sign bits QSB can be stored in the quantization sign bit buffer 140. However, example embodiments are not limited thereto.

[0265] In operation S130, the matrix multiplication device MMD may receive an input vector (eg, For example, the matrix multiplier 100 may receive a plurality of input elements (eg, x 11 to x 1n ).

[0266] In example embodiments, the matrix multiplication device MMD may perform operation S130 regardless of the order of steps S110 to S120. For example, the matrix multiplication device MMD may perform operation S130 after steps S110 to S120, or between steps S110 and S120.

[0267] In operation S140, the matrix multiplication device MMD may generate a multiplication scaling input vector MSX by performing multiplication scaling on the input vector based on a plurality of multiplication scaling coefficients MSC. For example, the input vector scaler 120 may perform multiplication scaling on each of a plurality of input elements based on a plurality of multiplication scaling coefficients MSC provided from the multiplication scaling coefficient buffer 110. More specifically, the input vector scaler 120 may multiply each of a plurality of input elements with a plurality of multiplication scaling coefficients MSC to generate a plurality of multiplication scaling input elements MSIE.

[0268] In operation S150, the matrix multiplication device MMD may generate a plurality of partial products PSP based on elements (eg, a plurality of multiplication scaling input elements MSIE) included in the multiplication scaling input vector MSX and a plurality of quantization sign bits QSB. Fig. 20 Operation S150 will be described in more detail.

[0269] In operation S160, the matrix multiplication device MMD may generate an output vector (eg, For example, the matrix multiplier 100 may generate an output element (eg, y) by multiplying the partial product PSP11 by the first common scaling factor CSC1. 11 Similarly, the matrix multiplier 100 may generate an output element (eg, y) by multiplying the partial product PSP12 by the second common scaling factor CSC2. 12 ). In this manner, the matrix multiplier 100 may generate a plurality of output elements included in an output vector.

[0270] Fig. 20 is shown in more detail Fig.19 Flow chart of operation S150. Figures 1 to 20 , operation S150 may include the following operations S151 to S153.

[0271] In operation S151, the matrix multiplier 100 may convert the data type of the multiplication scaling input vector MSX into a fixed point and generate a fixed-point multiplication scaling input vector MSXfxp. For example, the first data type converter 130 may convert each data type of a plurality of multiplication scaling input elements MSIE into a fixed point and generate a plurality of fixed-point multiplication scaling input elements MSIEfxp.

[0272] In operation S152, the matrix multiplier 100 may generate a plurality of fixed-point partial products PSPfxp based on a plurality of quantization sign bits QSB and a fixed-point multiplication scaling input vector MSXfxp. For example, the matrix multiplier 100 may generate a fixed-point partial product PSPfxp by sequentially adding or subtracting each of the fixed-point multiplication scaling input elements MSIEfxp based on a plurality of quantization sign bits QSB.

[0273] In operation S153, the matrix multiplier 100 may change the data type of the plurality of fixed-point partial products PSPfxp to floating points. For example, the second data type converter 160 may generate a plurality of partial products PSP based on the plurality of fixed-point partial products PSPfxp.

[0274] Fig.21 It is shown Figure 1 Flow chart of the operation of the matrix multiplier. In the following, reference is made to Figures 1 to 18 and Fig.21 , will describe receiving an input vector (e.g., ) and generates an input element (e.g., y 11 )'s matrix multiplication device MMD operation.

[0275] In operation S210, the matrix multiplication device MMD may receive first to nth weights. For example, the uniform BCQ circuit UBC may receive weights corresponding to a column of the weight matrix WM (eg, w 11 tow n1 ).

[0276] In operation S220, the matrix multiplication device MMD may generate first to Rth multiplication scaling coefficients MSC, one common scaling coefficient CSC, and first to (N×R)th quantization sign bits QSB by performing uniform binary coded quantization on first to nth weights.

[0277] In operation S230, the matrix multiplication device MMD may receive first to nth input elements. For example, the matrix multiplier 100 may receive a plurality of input elements (eg, x 11 to x 1n ).

[0278] In example embodiments, the matrix multiplication device MMD may perform operation S230 regardless of the order of operations S210 and S220. For example, the matrix multiplication device MMD may perform operation S230 after operations S210 and S220, or between operations S210 and S220.

[0279] In operation S240, the matrix multiplication device MMD may generate first to (N×R)th multiplication scaling input elements by performing multiplication scaling on the first to nth input elements based on the first to Rth multiplication scaling coefficients MSC. For example, the input vector sealer 120 may generate "R" multiplication scaling input elements MSIE for each single input element based on "R" scaling coefficients MSC.

[0280] In operation S250, the matrix multiplication device MMD may generate a partial product PSP based on the first to (N×R)th scaling input elements and the first to (N×R)th quantization sign bits QSB. For example, the processing element PE may calculate the partial product PSP by adding or subtracting each of the first to (N×R)th scaling input elements based on the first to (N×R)th quantization sign bits QSB.

[0281] In operation S260, the matrix multiplication device MMD may output an output element (eg, y 11 ). For example, the common sealer 180 may generate an output element by multiplying the partial product PSP by the common scaling coefficient CSC. In this case, the generated output element may correspond to the sum of the multiplications of the weights and input elements previously received through operations S210 and S230, respectively.

[0282] Fig. 22 is shown in more detail Fig.21 Flow chart of operation S250. Figures 1 to 18 and Figure 21 to Figure 22 , operation S250 may include operations S251 to S253.

[0283] In operation S251, the matrix multiplier 100 may generate first to (N×R)th fixed-point multiplication scaling input elements MSIEfxp by converting the data types of the first to (N×R)th multiplication scaling input elements MSIE into fixed points. For example, the first data type converter 130 may convert the first to (N×R)th multiplication scaling input elements MSIE into first to (N×R)th fixed-point multiplication scaling input elements MSIEfxp, respectively.

[0284] In operation S252, the matrix multiplier 100 may calculate the fixed-point partial product PSPfxp by subtracting a value obtained by adding the corresponding fixed-point multiplication scaling input element MSIEfxp to the quantization sign bit QSB representing "1" from a value obtained by summing the corresponding fixed-point multiplication scaling input element MSIEfxp to the quantization sign bit QSB representing "0". For example, the processing element PE may calculate the fixed-point partial product PSPfxp in the manner described above with reference to Equation 13.

[0285] In operation S253 , the matrix multiplier 100 may output a partial product PSP generated by converting the data type of the fixed-point partial product PSPfxp into a floating point. For example, the second data type converter 160 may provide the partial product PSP to the common scaler 180 .

[0286] Fig.23 is a diagram showing some example embodiments Figure 1 A diagram of the operation of the BCQ circuit. Figures 1 to 4 and Fig.15 , the uniform BCQ circuit UBC can perform a binary coded quantization operation for each row of the weight matrix WM.

[0287] First, the weight matrix WM can be expressed as the following Equation 14.

[0288] (Equation 14)

[0289]

[0290] Referring to Equation 14, It can represent the i-th row vector of the weight matrix WM. For example, Can include w i1 tow im .

[0291] The uniform BCQ circuit UBC may perform a binary coded quantization operation on each row of the weight matrix WM based on the following Equation 15.

[0292] (Equation 15)

[0293]

[0294] Referring to Equation 15, R can represent the BCQ resolution. k_ri The k-th quantized scaling coefficient of the i-th row vector of the weight matrix WM may be represented. It can be expressed as corresponding to α k_ri quantized symbol vector of . For example, The quantization symbol vector corresponding to the k-th quantization scaling coefficient of the ith row vector of the weight matrix WM may be represented. Can be used with is calculated in a similar manner (see Equation 8); however, instead of being calculated over the columns of WM, it is calculated over the rows of WM.

[0295] At the same time, α k_ri It can be expressed as the product of a common scaling factor and a multiplicative scaling factor. For example, α k_ri It can be expressed as 2 k-1 ×s ri Therefore, Equation 15 described above can be expressed as the following Equation 16.

[0296] (Equation 16)

[0297]

[0298] Refer to Equation 16, 2 k-1 The k-th multiplication scaling factor MSCk can be represented, and s ri The common scaling coefficient CSC of the i-th row vector may be represented.

[0299] Fig.24 It shows that according to Fig.23 Schematic diagram of the approximate weight matrix of various example embodiments. Figures 1 to 4 as well as Fig.23 and Fig.24 , the uniform BCQ circuit UBC can perform uniform binary coded quantization for each row of the weight matrix WM.

[0300] Quantized symbol vector can be implemented as a row vector with the same dimension as the number of columns of the weight matrix WM. For example, the quantized symbol vector It can be expressed as the following equation 17.

[0301] (Equation 17)

[0302]

[0303] b 1_k_ri To b m_k_ri Each of b can represent a different quantization sign bit QSB. More specifically, b 1_k_ri To b m_k_ri Each of b may be a quantization sign bit QSB for weights arranged in different columns of the weight matrix WM. That is, b 1_k_ri To b m_k_ri Each of can be either "0" or "1".

[0304] and Figure 5 and Figure 6 As in the embodiment of the present invention, multiplication based on the quantization sign bit QSB can be implemented by adding or subtracting a value based on the corresponding quantization sign bit QSB. as input. A plurality of quantization sign bits corresponding to the k-th quantization scaling coefficient of the i-th row vector may be included. More specifically, It can be expressed as the following equation 17.1.

[0305] (Equation 17.1)

[0306]

[0307] That is, the uniform BCQ circuit UBC can approximate each row vector included in the weight matrix WM based on a combination of multiple quantization sign bits QSB, a common scaling coefficient CSC and multiple multiplication scaling coefficients MSC. In this case, the matrix multiplier 100 can calculate the output matrix YM corresponding to the product of the input matrix XM and the weight matrix WM with a smaller amount of calculation.

[0308] For a more detailed example, in the following, for a more concise description, the output elements of the first row and the first column of the output matrix YM (e.g., y 11 ) of the matrix multiplier 100. However, example embodiments are not limited thereto.

[0309] The uniform BCQ circuit UBC may provide a plurality of quantization sign bits QSB (eg, “b” values), a plurality of common scaling coefficients CSC (eg, “s” values), and a plurality of multiplication scaling coefficients MSC (eg, powers of 2) to the matrix multiplier 100 .

[0310] The matrix multiplier 100 may calculate the output matrix YM based on a plurality of quantization sign bits QSB, a plurality of common scaling coefficients CSC, and a plurality of multiplication scaling coefficients MSC. For example, the matrix multiplier 100 may calculate the output matrix YM by combining the above reference equations 14 to 17.1 and Fig.24 The described approach is to multiply the approximate weight matrix WM with the input matrix XM to calculate the output matrix YM.

[0311] For a more detailed example, the matrix multiplier 100 may calculate y according to the following equation 18: 11 .

[0312] (Equation 18)

[0313]

[0314] In an example embodiment, the matrix multiplier 100 may calculate the output elements by first performing quantization scaling on the input elements based on the common scaling coefficient CSC, and then accumulating the quantization scaled input elements (hereinafter referred to as "quantization scaled input elements QSIE") based on a plurality of quantization sign bits QSB. In this case, the matrix multiplier 100 may use the same "quantization scaled input elements (QSIE)" to calculate the output elements included in an output vector (e.g., ) in different output elements. For example, the matrix multiplier 100 can 0 ×s r1 × 11 "For y 11 To 1n Therefore, according to various exemplary embodiments of the present disclosure, the amount of calculation of the matrix multiplier 100 and the number of accesses to the memory device outside the matrix multiplier 100 can be reduced or minimized. Figures 25 to 46 The detailed configuration and operation of the matrix multiplier 100 will be described in more detail.

[0315] Meanwhile, continue to refer to Fig.24 and Equations 14 to 17.1 for quantizing the symbol vector (e.g., to ) and the first quantized scaling coefficients (e.g., α 1_r1 To α 1_rn ) multiplied by the quantized sign bit vector (e.g., to ) can be expressed as a first quantization symbol bit matrix QSBM_1. For example, the first quantization symbol bit matrix QSBM_1 can be expressed as the following equation 19.

[0316] (Equation 19)

[0317]

[0318] In a similar manner, for quantizing symbol vectors (e.g., to ) and the Rth quantized scaling coefficients (e.g., α R_r1 To α R_rn ) multiplied by the quantized sign bit vector (e.g., to ) can be expressed as the Rth quantization sign bit matrix QSBM_R.

[0319] In an example embodiment, the first quantization symbol bit matrix QSBM_1 to the Rth quantization symbol bit matrix QSBM_R may be implemented using the same number of rows and columns as the weight matrix WM. For example, the number of rows of the first quantization symbol bit matrix QSBM_1 may be "n", and the number of columns may be "m". Fig.25 The first to R-th quantization symbol bit matrices QSBM_1 to QSBM_R will be described in more detail.

[0320] Fig.25 It is shown Fig.24 An illustration of the quantization symbol bit matrix of . Figures 1 to 4 and Figure 23 to Figure 25 The uniform BCQ circuit UBC may perform uniform binary coded quantization on the weight matrix WM using a plurality of quantization sign bit matrices QSBM, a plurality of common scaling coefficients CSC and a plurality of multiplication scaling coefficients MSC.

[0321] The number of quantization symbol bit matrices QSBM may be determined according to the BCQ resolution (eg, “R”). For example, the uniform BCQ circuit UBC may generate first to Rth quantization symbol bit matrices QSBM_1 to QSBM_R by performing binary coded quantization on the weight matrix WM.

[0322] Each of the first quantization symbol bit matrix QSBM_1 to the R-th quantization symbol bit matrix QSBM_R may be implemented using the same number of rows and columns as the weight matrix WM. For example, the number of rows of each of the first quantization symbol bit matrix QSBM_1 to the R-th quantization symbol bit matrix QSBM_R may be "n", and the number of columns may be "m". In this case, the quantization symbol bits QSB arranged in the i-th row and the j-th column of the k-th quantization symbol bit matrix QSBM_k may be "b j_k_ri ”.

[0323] Each weight of the weight matrix WM can be approximated by the quantization symbol bits QSB arranged at corresponding positions of the first quantization symbol bit matrix QSBM_1 to the Rth quantization symbol bit matrix QSBM_R. For example, the weights of the weight matrix WM can be approximated based on the quantization symbol bits QSB arranged at corresponding positions of the first quantization symbol bit matrix QSBM_1 to the Rth quantization symbol bit matrix QSBM_R (for example, b j_1_ri To b j_R_ri ) to approximate the quantized sign bits in the i-th row and i-th column in the weight matrix WM (e.g., w ij )’s weights in the i-th row and j-th column.

[0324] Each of the first quantization symbol bit matrix QSBM_1 to the Rth quantization symbol bit matrix QSBM_R may correspond to a plurality of common scaling coefficients QSC. Fig.23 and Fig.24 As described, when the uniform BCQ circuit UBC performs a uniform binary coding quantization operation for each row of the weight matrix WM, different rows of the weight matrix WM will be approximated based on different common scaling coefficients QSC; and the weights included in the same row of the weight matrix WM will be approximated based on the same common scaling coefficient QSC.

[0325] Therefore, all quantization sign bits (eg, b 1_k_ri To b m_k_ri ) may correspond to a common scaling factor "s ri In this way, the first to n-th rows of each of the first to k-th quantization symbol bit matrices QSBM_1 to QSBM_k may correspond to the common scaling coefficients "s r1 ” to “s rn For a more detailed example, the first to nth rows of the first quantization symbol bit matrix QSBM_1 may correspond to “s r1 ” to “s rn ”.

[0326] Each of the first to R quantization symbol bit matrices QSBM_1 to QSBM_R may correspond to a different multiplication scaling coefficient MSC. For example, each of the plurality of quantization symbol bits QSB included in the first quantization symbol bit matrix QSBM_1 may correspond to a first multiplication scaling coefficient MSC1 (eg, 2 0 ), and each of the plurality of quantization sign bits QSB included in the second quantization sign bit matrix QSBM_2 may correspond to a second multiplication scaling coefficient MSC2 (eg, 2 1 ).

[0327] In this way, the weight matrix WM (e.g., w ij) can be approximated based on the quantization sign bits arranged in the i-th row and j-th column of the first quantization sign bit matrix QSBM_1 to the R-th quantization sign bit matrix QSBM_R; a common scaling coefficient CSC corresponding to the i-th row of the first quantization sign bit matrix QSBM_1 to the R-th quantization sign bit matrix QSBM_R; and a plurality of multiplication scaling coefficients MSC corresponding to the first quantization sign bit matrix QSBM_1 to the R-th quantization sign bit matrix QSBM_R, respectively.

[0328] More specifically, each of the plurality of weights in the i-th row may be approximated based on R scaling coefficients MSC, one common scaling coefficient CSC (eg, corresponding to the i-th row), and R quantization sign bits QSB according to Equation 20 below.

[0329] (Equation 20)

[0330]

[0331] The following will refer to Figure 26 to Figure 46 The operation of the matrix multiplier 100 according to the weights approximated based on the multiplication scaling factor MSC, the common scaling factor CSC and the quantization sign bit QSB will be described in more detail.

[0332] Fig.26 is a diagram showing implementation according to some example embodiments Figure 1 A block diagram of the configuration of a matrix multiplier. Figures 1 to 4 and Figure 23 to Figure 26 , Figure 1 The matrix multiplier 100 can be implemented as Fig.26 A matrix multiplier 200 is provided.

[0333] Matrix multiplier 200 may include a multiplication scaling factor buffer 210 , a common scaling factor buffer 270 , an input vector scaler 220 , a first data type converter 230 , a quantization sign bit buffer 240 , a processing element array 250 , and a second data type converter 260 .

[0334] The multiplication scaling factor buffer 210 may store a plurality of multiplication scaling factors MSC provided from the uniform BCQ circuit UBC. The multiplication scaling factor buffer 210 may provide the plurality of multiplication scaling factors MSC to the input vector scaler 220.

[0335] The common scaling coefficient buffer 270 may store a plurality of common scaling coefficients CSC provided from the uniform BCQ circuit UBC. The common scaling coefficient buffer 270 may provide the plurality of common scaling coefficients CSC to the input vector scaler 220.

[0336] The input vector scaler 220 may receive an input matrix XM. For example, the input vector scaler 120 may receive a plurality of input vectors (eg, to ).

[0337] The input vector scaler 220 may perform quantization scaling on the input matrix XM based on a plurality of common scaling coefficients CSC and a plurality of multiplication scaling coefficients MSC. For example, the input vector scaler 220 may generate a plurality of quantization scaling input vectors QSX based on a plurality of input vectors. In this case, the plurality of quantization scaling input vectors may correspond to a plurality of input vectors (e.g., to ). In the following, corresponding to to The quantized scaled input vector QSX can be referred to as to

[0338] In an example embodiment, a plurality of quantization scaling input vectors may be included in a quantization scaling input matrix. In this case, the row size of the quantization scaling input matrix may be an integer multiple of the row size of the input matrix XM, and the column size of the quantization scaling input matrix may be the same as the column size of the input matrix XM. However, example embodiments are not limited thereto.

[0339] Each of the plurality of quantized scaled input vectors QSX may be implemented as a row vector having a dimension "R" times that of the corresponding input vector. For example, corresponding to the first input vector (eg, ) of the first quantized scaled input vector (e.g., ) may include “R×n” quantization scaling input elements QSIE, as shown in Equation 21 below.

[0340] (Equation 21)

[0341]

[0342] Referring to Equation 21, the quantized scaling factor MSC included in the first input vector may be generated by multiplying each of the plurality of input elements included in the first input vector by a corresponding multiplication scaling factor MSC and a corresponding common scaling factor CSC. Multiple quantized scaled input elements in QSIE.

[0343] In this way, the second to h-th quantized scaled input vectors (eg, to ) may include "R×n" quantization scaling input elements QSIE. For the sake of simplicity, the elements included in the second to h-th quantization scaling input vectors (eg, to ) in each of the quantization scaling input elements QSIE. Figures 27 to 34 The configuration and operation of the input vector sealer 220 will be described in more detail.

[0344] In an example embodiment, the data type of the plurality of input elements may be a floating point. In this case, the data type of each of the plurality of quantization scaling input elements QSIE may be a floating point. However, example embodiments are not limited thereto.

[0345] In an example embodiment, the code length of each of the plurality of input elements may be 16 bits or 32 bits. However, example embodiments are not limited thereto.

[0346] In an exemplary embodiment, the code length of each of the plurality of common scaling coefficients CSC may be 16 bits or 32 bits. However, exemplary embodiments are not limited thereto.

[0347] The first data type converter 230 may receive a plurality of quantization scaling input vectors QSX. For example, the first data type converter 230 may receive first to h-th quantization scaling input vectors (eg, to ). Each of the first to h-th quantization scaling input vectors may include a plurality of quantization scaling input elements QSIE.

[0348] The first data type converter 230 can extract the exponent EXP from each of the plurality of quantization scaled input vectors QSX. For example, the first data type converter 230 can extract the exponent EXP from the first quantization scaled input vector QSX. Extract the first index and scale the input vector from the second quantization The first data type converter 230 may provide the extracted exponent EXP to the second data type converter 260 .

[0349] The first data type converter 230 may convert the data types of the plurality of quantization scaling input vectors QSX into fixed points. For example, the first data type converter 230 may receive the plurality of quantization scaling input vectors QSX and output a plurality of fixed-point quantization scaling input vectors QSXfxp. That is, the first data type converter 230 may generate the first to h-th fixed-point quantization scaling input vectors (e.g., to ).

[0350] More specifically, the first data type converter 230 may convert the data type of each of the plurality of quantization scaling input elements QSIE included in the plurality of quantization scaling input vectors QSX into a fixed point based on the extracted index. In this case, the first fixed point quantization scaling input vector (e.g., ) may include elements of the above equation 21 converted into fixed-point form (hereinafter, they may be referred to as fixed-point quantized scaled input elements QSIEfxp). Fig.35 The configuration and operation of the first data type converter 230 will be described in more detail.

[0351] The quantization sign bit buffer 240 may store a plurality of quantization sign bits QSB provided from the uniform BCQ circuit UBC. The quantization sign bit buffer 240 may provide the plurality of quantization sign bits QSB to the processing element array 250.

[0352] The processing element array 250 may receive a plurality of quantization sign bits and a plurality of fixed-point quantization scaling input vectors QSXfxp. The processing element array 250 may generate a fixed-point output matrix YMfxp based on a plurality of fixed-point quantization scaling input elements QSIEfxp and a plurality of quantization sign bits QSB. The fixed-point output matrix YMfxp may be expressed as the following equation 22.

[0353] (Equation 22)

[0354]

[0355] Referring to Equation 22, YM_fxp can represent the fixed-point output matrix YM_fxp, and to Can represent different output vectors with fixed-point data types. y′ 11 to y′ hm Each of can represent a different output element with a fixed-point data type.

[0356] The processing element array 250 may include a plurality of processing elements arranged in the row direction and the column direction. Each of the plurality of processing elements may calculate and output a different fixed-point output element of the equation 22 described above. Figure 36 to Figure 38 A more detailed configuration and operation of each of the processing element arrays will be described in more detail.

[0357] Second data type converter 260 may receive a plurality of exponents EXP from first data type converter 230. Second data type converter 260 may receive a fixed-point output matrix YMfxp from processing element array 250. For example, second data type converter 260 may receive a plurality of fixed-point output elements from processing element array 250.

[0358] The second data type converter 260 can convert the data type of the fixed-point output matrix YMfxp into a floating point based on the plurality of exponents EXP. That is, the second data type converter 260 can output the output matrix YM having a floating point data type. For example, the second data type converter 260 can generate a plurality of output elements by converting the data type of each of the plurality of received fixed-point output elements into a floating point. Fig.39 The detailed configuration and operation of the second data type converter 160 will be described in more detail.

[0359] Fig. 27 is a diagram showing various exemplary embodiments Fig.26 A block diagram of the configuration of the input vector scaler. Figures 1 to 4 and Figure 23 to Figure 27 , the input vector scaler 220 may include a multiplication scaler SCL_multiple and a common scaler SCL_common. The multiplication scaler SCL_multiple may include first to h-th multiplication scaling circuits 221_1 to 221_h. The common scaler SCL_common may include first to h-th common scaling circuits 222_1 to 222_h.

[0360] Each of the first to h-th multiplication and scaling circuits 221_h may receive a different input vector. For example, the first to h-th multiplication and scaling circuits 221_h may receive first to h-th input vectors (eg, to ).

[0361] That is, each of the first multiplication and scaling circuit 221_1 to the hth multiplication and scaling circuit 221_h may sequentially receive a plurality of input elements of the corresponding input vector. For example, the first multiplication and scaling circuit 221_h may sequentially receive x 11 to x 1n , and the hth multiplication-scaling circuit 221_h can sequentially receive x h1 to x hn .

[0362] The multiplication scaler SCL_multiple may sequentially receive a plurality of multiplication scaling coefficients MSC from the multiplication scaling coefficient buffer 210. For example, each of the first to the h-th multiplication scaling circuits 221_1 to 221_h may sequentially receive a plurality of multiplication scaling coefficients MSC from the multiplication scaling coefficient buffer 210. For a more detailed example, each of the first to the h-th multiplication scaling circuits 221_1 to 221_h may receive a first to an R-th multiplication scaling coefficient MSCR (e.g., 2 0to 2 R-1 ).

[0363] Each of the first to h-th multiplication scaling circuits 221_1 to 221_h can perform multiplication scaling on the received input elements based on a plurality of multiplication scaling coefficients MSC. That is, the first to h-th multiplication scaling circuits 221_1 to 221_h can generate first to h-th multiplication scaling input vectors (e.g., to ). For example, the first multiplication scaling circuit 221_1 may be based on x 11 to x 1n and the first multiplication scaling coefficient MSC1 to the Rth multiplication scaling coefficient MSCR to generate a first multiplication scaling input vector (eg, ). In this case, the first to h-th multiplications scale the input vector (e.g., to ) may include multiple multiplication scaling input elements (hereinafter referred to as "MSIE"). Since the specific configuration and operation of each of the first multiplication scaling circuit 221_1 to the hth multiplication scaling circuit 221_h are similar to those of the above reference Fig. 9 and Fig.10 The configuration and operation of the first multiplication-scaling circuit 121 are not described, and thus a detailed description thereof will be omitted.

[0364] In an example embodiment, the first multiplication scaling circuit 221_1 to the hth multiplication scaling circuit 221_h may calculate the multiplication scaling input element by increasing the exponential part EXPP of the input element, as described above with reference to FIG. Fig.10 In this case, the amount of calculation of the first to h-th multiplication-scaling circuits 221_1 to 221_h can be reduced or minimized.

[0365] Each of the first to the hth common scaling circuits 222_1 to 222_h may receive a different multiplication scaling input vector. For example, the first to the hth common scaling circuits 222_1 to 221_h may receive first to hth multiplication scaling input vectors (eg, to ).

[0366] Each of the first to the hth common scaling circuits 222_1 to 222_h may sequentially receive a plurality of multiplication scaling input elements of a corresponding multiplication scaling input vector. For example, the first common scaling circuit 222_1 may sequentially receive the above reference vectors. Fig. 9 The multiplication scaling input elements MSIE11_1 to MSIE1n_R are described.

[0367] The common scaler SCL_common may sequentially receive a plurality of common scaling coefficients CSC from the common scaling coefficient buffer 270. For example, each of the first to hth common scaling circuits 222_1 to 222_h may sequentially receive a plurality of common scaling coefficients CSC from the common scaling coefficient buffer 270.

[0368] For example, each of the first to hth common scaling circuits 222_1 to 222_h may be identical to the common scaling coefficient CSC received from the common scaling coefficient buffer 270. For example, the common scaling coefficient CSC sequentially received by the first common scaling circuit 222_1 may be identical to the common scaling coefficient CSC sequentially received by the second common scaling circuit 222_2.

[0369] The order in which each of the first public scaling circuit 222_1 to the hth public scaling circuit 222_h receives the common scaling coefficient CSC may be the same. For example, the common scaling coefficient CSC first received by the first public scaling circuit 222_1 may be the same as the common scaling coefficient CSC first received by the second public scaling circuit 222_2. Similarly, the common scaling coefficient CSC received second by the first public scaling circuit 222_1 may be the same as the common scaling coefficient CSC received second by the second public scaling circuit 222_2.

[0370] Each of the first to the h-th common scaling circuits 222_1 to 222_h may perform common scaling on the received multiplication scaling input elements MSIE based on a plurality of common scaling coefficients CSC. That is, each of the first to the h-th common scaling circuits 222_1 to 222_h may generate a first quantization scaling input vector to an h-th quantization scaling input vector (e.g., to ). For example, the first common scaling circuit 222_1 may scale the input elements MSIE11_1 to MSIE1n_R based on the multiplication and the above reference Fig. 9 The plurality of common scaling coefficients CSC described above are used to generate a first quantized scaling input vector (eg, ). The following will refer to Fig.28 The operations of the first to hth common scaling circuits 222_1 to 222_h will be described in more detail.

[0371] Fig.28 is shown in more detail Fig. 27 In the following, for a more concise description, reference will be made to Figures 1 to 4 and Figure 23 to Figure 28The operation of the first common scaling circuit 222_1 is representatively described. However, example embodiments are not limited thereto, and the second through h-th common scaling circuits 222_2 through 222_h may also operate in a similar manner.

[0372] The first common scaling circuit 222_1 may receive a first multiplication scaling input vector (eg, ). That is, the first common scaling circuit 222_1 may sequentially receive the multiplication scaling input elements MSIE11_1 to MSIE1n_R.

[0373] The first common scaling circuit 222_1 may sequentially receive a plurality of common scaling coefficients CSC. For example, the first common scaling circuit 222_1 may sequentially receive S r1 To S rn In this case, S r1 To S rn Each of S may correspond to a different input element. r1 To S rn can correspond to x respectively 11 to x 1n .

[0374] The first common scaling circuit 222_1 may scale the input vector (eg, ) performs common scaling to generate a first quantized scaled input vector (e.g., ). That is, the first common scaling circuit 222_1 may multiply each of the multiplication scaling input elements MSIE11_1 to MSIE1n_R by the corresponding common scaling coefficient CSC to generate a plurality of quantization scaling input elements QSIE. For example, the first common scaling circuit 222_1 may multiply each of the multiplication scaling input elements MSIE11_1 to MSIE11_R by the common scaling coefficient “S r1 " to generate the value for x 11 More specifically, the first common scaling circuit 222_1 may multiply each of the scaling input elements MSIE11_1 to MSIE11_R with a common scaling factor “S r1 " are multiplied to generate multiple quantized scaled input elements QSIE (e.g., "α 1_r1 × 11 ” to “α R_r1 × 11 ”) (shown as diagonal stripes).

[0375] Similarly, the first common scaling circuit 222_1 may multiply and scale each of the input elements MSIE12_1 to MSIE12_R with the common scaling factor “s r2 " to generate the value for x 12 Multiple quantized scaled input elements QSIE (e.g., “α 1_r2 × 12 ” to “a R_r2 × 12 ”) (shown as a dot pattern).

[0376] In this way, the first common scaling circuit 222_1 may be able to sequentially calculate the values ​​corresponding to x 13 to x 1n Multiple quantized scaled input elements QSIE.

[0377] The first common scaling circuit 222_1 may sequentially output a plurality of quantized scaling input elements QSIE. For example, the first common scaling circuit 222_1 may provide the plurality of quantized scaling input elements QSIE to the first data type converter 230.

[0378] That is to say, according to Fig. 27 and Fig.28 According to various example embodiments, the input vector sealer 220 may generate a quantized scaled input vector by performing multiplicative scaling on each of a plurality of input vectors and then performing common scaling.

[0379] Fig.29 is a diagram showing various exemplary embodiments Fig.26 A block diagram of the configuration of the input vector scaler. Figures 1 to 4 , Figure 23 to Figure 26 and Fig.29 , the input vector scaler 220 may include a common scaler SCL_common and a multiplication scaler SCL_multiple. The common scaler SCL_common may include first to h-th common scaling circuits 223_1 to 223_h. The multiplication scaler SCL_multiple may include first to h-th multiplication scaling circuits 224_1 to 224_h.

[0380] Each of the first to h-th common scaling circuits 223_1 to 223_h may receive a different input vector. For example, the first to h-th common scaling circuits 223_1 to 223_h may receive first to h-th input vectors (eg, to ).

[0381] Each of the first to h-th common scaling circuits 223_1 to 223_h may sequentially receive a plurality of input elements of a corresponding input vector. For example, the first common scaling circuit 223_1 may sequentially receive x 11 to x 1n , and the hth common scaling circuit 223_h can sequentially receive x h1 to x hn .

[0382] The common scaler SCL_multiple may sequentially receive a plurality of common scaling coefficients CSC from the common scaling coefficient buffer 270. For example, each of the first to hth common scaling circuits 223_1 to 223_h may sequentially receive a plurality of common scaling coefficients CSC from the common scaling coefficient buffer 270.

[0383] The common scaling coefficients CSC received by each of the first to hth common scaling circuits 223_1 to 223_h from the common scaling coefficient buffer 270 may be the same. The order in which each of the first to hth common scaling circuits 223_1 to 223_h receives the common scaling coefficients CSC may be the same. The relationship between the common scaling coefficients CSC received by each of the first to hth common scaling circuits 223_1 to 223_h is similar to that described above with reference to Fig. 27 The relationship described hereof will therefore be omitted in detail.

[0384] Each of the first to h-th common scaling circuits 223_1 to 223_h may perform common scaling on a received input vector based on a plurality of common scaling coefficients CSC. That is, each of the first to h-th common scaling circuits 223_1 to 223_h may generate a first to h-th common scaling input vector (e.g., to ). For example, the first common scaling circuit 223_1 can be 11 to x 1n The first common scaling input vector (eg, ) (hereinafter referred to as "CSIE") in multiple common scaling input elements. Fig.30 The operations of the first to hth common scaling circuits 223_1 to 223_h will be described in more detail.

[0385] Each of the first to h-th multiplication and scaling circuits 224_1 to 224_h may receive a different common scaling input vector. For example, the first to h-th multiplication and scaling circuits 224_1 to 224_h may receive first to h-th multiplication and scaling input vectors (eg, to ).

[0386] Each of the first multiplication scaling circuit 224_1 to the hth multiplication scaling circuit 224_h may sequentially receive a plurality of common scaling input elements of a corresponding scaling input vector. For example, the first multiplication scaling circuit 224_1 may sequentially receive a plurality of common scaling input elements included in the first common multiplication scaling input vector (eg, ) in multiple common scaling input elements CSIE.

[0387] The multiplication scaler SCL_multiple may sequentially receive a plurality of multiplication scaling coefficients MSC from the multiplication scaling coefficient buffer 210. For example, each of the first to the h-th multiplication scaling circuits 224_1 to 224_h may sequentially receive a plurality of multiplication scaling coefficients MSC from the multiplication scaling coefficient buffer 210. For a more detailed example, each of the first to the h-th multiplication scaling circuits 224_1 to 224_h may receive a first to an R-th multiplication scaling coefficient MSCR (e.g., 2 0 to 2 R-1 ).

[0388] Each of the first multiplication and scaling circuit 224_1 to the hth multiplication and scaling circuit 224_h can perform multiplication and scaling on the received common scaling input vector based on multiple multiplication and scaling coefficients MSC. That is, the first multiplication and scaling circuit 224_1 to the hth multiplication and scaling circuit 224_h can perform multiplication and scaling on the first to the hth common scaling input vectors to generate the first quantization scaling input vector to the hth quantization scaling input vector (for example, to ).

[0389] For example, the first multiplication and scaling circuit 224_1 may include a first common multiplication and scaling input vector (eg, ) is multiplied by the first multiplication scaling coefficient MSC1 to the Rth multiplication scaling coefficient MSCR to generate a first quantized scaling input vector (eg, ). The following will refer to Fig.31 The operations of the first to h-th multiplication-scale circuits 224_1 to 224_h are described in more detail.

[0390] Fig.30is shown in more detail Fig.29 In the following, for a more concise description, reference will be made to Figures 1 to 4 , Figure 23 to Figure 26 , Fig.29 and Fig.30 The operation of the first common scaling circuit 223_1 is representatively described as follows. However, example embodiments are not limited thereto, and the second through h-th common scaling circuits 223_2 through 223_h may also operate in a similar manner.

[0391] The first common scaling circuit 223_1 may receive a first input vector (eg, That is, the first common scaling circuit 223_1 can sequentially receive x 11 to x 1n .

[0392] The first common scaling circuit 223_1 may sequentially receive a plurality of common scaling coefficients CSC. For example, the first common scaling circuit 223_1 may sequentially receive S r1 To S rn .

[0393] The first common scaling circuit 223_1 can scale the first input vector (eg, ) performs common scaling to generate a first common scaled input vector (e.g., ). For example, the first common scaling circuit 223_1 can be 11 to x 1n Multiply by S r1 To S rn To generate common scaling input elements CSIE11 to CSIE1n.

[0394] Fig.31 is shown in more detail Fig.29 In the following, for a more concise description, reference will be made to Figures 1 to 4 , Figure 23 to Figure 26 , Fig.29 and Fig.31 1 to representatively describe the operation of the first multiplication scaling circuit 224_1. However, example embodiments are not limited thereto, and the second to h-th multiplication scaling circuits 224_2 to 224_h may also operate in a similar manner.

[0395] The first multiplication and scaling circuit 224_1 may receive a first common scaling input vector (eg, ). That is, the first multiplication and scaling circuit 224_1 can sequentially receive the above reference Fig.30The common scaling input elements CSIE11 to CSIE1n described above are shown in FIG.

[0396] The first multiplication scaling circuit 224_1 may generate a plurality of quantization scaling input elements QSIE by multiplying each of the received common scaling input elements CSIE11 to CSIE1n by first to Rth multiplication scaling coefficients MSC1 to MSCR.

[0397] For example, the first multiplication scaling circuit 224_1 may generate the common scaling input element CSIE11 by multiplying each of the first to R-th multiplication scaling coefficients MSCR. 11 More specifically, the first multiplication scaling circuit 224_1 can be configured by combining the common scaling input element CSIE11 with 2 0 to 2 R-1 Multiply each of them to generate the corresponding 1_r1 × 11 ” to “α R_r1 × 11 ” multiple quantization scaling input elements QSIE (shown as diagonal stripes).

[0398] Similarly, the first multiplication scaling circuit 224_1 may generate a plurality of quantization scaling input elements QSIE (shown as a dot pattern) by multiplying the common scaling input element CSIE12 with each of the first to Rth multiplication scaling coefficients MSC1 to MSCR.

[0399] In this way, the first multiplication and scaling circuit 224_1 will be able to sequentially calculate the values ​​corresponding to x 13 to x 1n Multiple quantized scaled input elements QSIE.

[0400] In an example embodiment, the first multiplication scaling circuit 224_1 to the hth multiplication scaling circuit 224_h may calculate the quantized scaled input element by increasing the exponential part EXPP of the multiplication scaling input element, as described above with reference to FIG. Fig.10 In this case, the amount of calculation of the first to h-th multiplication-scaling circuits 224_1 to 224_h can be reduced or minimized.

[0401] The first multiplication scaling circuit 224_1 may sequentially output a plurality of quantization scaling input elements QSIE. For example, the first multiplication scaling circuit 224_1 may provide the plurality of quantization scaling input elements QSIE to the first data type converter 230.

[0402] That is to say, according to Figure 29 to Figure 31 In various example embodiments, the input vector sealer 220 may perform common scaling on each of the plurality of input vectors and then perform multiplicative scaling to generate a plurality of quantized scaled input vectors QSX.

[0403] Fig.32 is a diagram showing various exemplary embodiments Fig.26 A block diagram of the configuration of the input vector scaler. Figures 1 to 4 , Figure 23 to Figure 26 and Fig.32 , the input vector scaler 220 may include a multiplication scaling circuit 225 and a quantization scaler SCL_quantization. The quantization scaler SCL_quantization may include a first quantization scaling circuit 226_1 to an h-th quantization scaling circuit 226_h.

[0404] The multiplication scaling circuit 225 may sequentially receive a plurality of multiplication scaling coefficients MSC from the scaling coefficient buffer 210. For example, the multiplication scaling circuit 225 may sequentially receive a first multiplication scaling coefficient MSC1 to an Rth multiplication scaling coefficient MSCR (eg, 2 0 to 2 R-1 ).

[0405] The multiplication and scaling circuit 225 may sequentially receive a plurality of common scaling coefficients CSC from the common scaling coefficient buffer 270. For example, the multiplication and scaling circuit 225 may sequentially receive S r1 To S rn .

[0406] The multiplication scaling circuit 225 may generate a plurality of quantization scaling coefficients QSC based on a plurality of multiplication scaling coefficients MSC and a plurality of common scaling coefficients CSC. For example, the multiplication scaling circuit 225 may generate a plurality of quantization scaling coefficients QSC by multiplying each of a plurality of common scaling coefficients CSC by a plurality of multiplication scaling coefficients MSC. Fig.33 The operation of the multiplication-scaling circuit 225 will be described in more detail.

[0407] The quantization scaler SCL_quantization may receive a plurality of quantization scaling coefficients QSC. For example, each of the first quantization scaling circuit 226_1 to the hth quantization scaling circuit 226_h may receive a plurality of quantization scaling coefficients QSC from the multiplication scaling circuit 225. In this case, the quantization scaling coefficients QSC received by each of the first quantization scaling circuit 226_1 to the hth quantization scaling circuit 226_h may be the same.

[0408] Each of the first multiplication and scaling circuit 226 to the hth multiplication and scaling circuit 226_h can receive a different input vector. For example, the first quantization and scaling circuit 226_1 to the hth quantization and scaling circuit 226_h can respectively receive the first to the hth input vectors (eg, to ). That is, each of the first quantization scaling circuit 226_1 to the hth quantization scaling circuit 226_h may sequentially receive a plurality of input elements of the corresponding input vector. For example, the first common scaling circuit 226_1 may sequentially receive x 11 to x 1n .

[0409] Each of the first quantization scaling circuit 226_1 to the hth quantization scaling circuit 226_h can perform quantization scaling on the received input vector based on a plurality of quantization scaling coefficients QSC. For example, the first quantization scaling circuit 226_1 to the hth quantization scaling circuit 226_h can respectively generate first to hth quantization scaling input vectors (e.g., to ). The following will refer to Fig.34 The operations of the first to hth quantization scaling circuits 226_1 to 226_h will be described in more detail.

[0410] Fig.33 is shown in more detail Fig.32 A diagram of the operation of the multiplication and scaling circuit. Figures 1 to 4 , Figure 23 to Figure 26 as well as Fig.32 and Fig.33 The multiplication and scaling circuit 225 may sequentially receive the first multiplication and scaling coefficient MSC1 to the Rth multiplication and scaling coefficient MSCR (eg, 2 0 to 2 R-1 ). The multiplication and scaling circuit 225 may sequentially receive S r1 To S rn .

[0411] The multiplication and scaling circuit 225 can be realized by converting S r1 To S rn Each of the quantized scaling factors MSC is multiplied by each of the multiplication scaling factors MSC to generate a plurality of quantized scaling factors QSC (eg, α 1_r1 To α R_rn ).

[0412] For example, the multiplication and scaling circuit 225 can be realized by setting S r1 The first row vector of the weight matrix WM is generated by multiplying the first multiplication scaling coefficient MSC1 to the Rth multiplication scaling coefficient (eg, ) of the quantized scaling factor (e.g., α 1_r1 To αR_rn )(shown as diagonal stripes).

[0413] Similarly, the multiplication and scaling circuit 225 can be realized by converting S r2 The second row vector of the weight matrix WM is generated by multiplying the first multiplication scaling coefficient MSC1 to the Rth multiplication scaling coefficient MSCR (for example, ) of the quantization scaling factor QSC (e.g., α 1_r2 To α R_r2 )(shown as a dot pattern).

[0414] In this way, the first common scaling circuit 222_1 will be able to sequentially calculate the values ​​corresponding to S r3 To S rn Multiple quantization scaling coefficients QSC.

[0415] In an example embodiment, the multiplication scaling circuit 225 may calculate the quantization scaling factor QSC by increasing the exponent EXPP of the common scaling factor CSC, similar to the above reference. Fig.10 Described.

[0416] Fig.34 is shown in more detail Fig.32 In the following, for a more concise description, reference will be made to Figures 1 to 4 , Figure 23 to Figure 26 and Figure 32 to Figure 34 The operation of the first quantization scaling circuit 226_1 is representatively described. However, example embodiments are not limited thereto, and the second quantization scaling circuit 226_2 to the h-th quantization scaling circuit 226_h may also operate in a similar manner.

[0417] The first quantization scaling circuit 226_1 may receive a first input vector (eg, That is, the first quantization scaling circuit 226_1 may sequentially receive x 11 to x 1n .

[0418] The first quantization scaling circuit 226_1 may receive a plurality of quantization scaling coefficients QSC. For example, the first quantization scaling circuit 226_1 may sequentially receive the above reference Fig.33 Description of α 1_r1 To α R_rn .

[0419] The first quantization scaling circuit 226_1 may perform quantization scaling on a plurality of input elements based on a plurality of quantization scaling coefficients QSC. For example, the first quantization scaling circuit 226_1 may generate a plurality of quantization scaling input elements QSIE by multiplying each of the plurality of quantization scaling coefficients QSC by a corresponding input element.

[0420] For example, the first quantization scaling circuit 226_1 can be configured by setting α 1_r1 To α R_r1 With x 11 Multiply to generate x 11 Multiple quantized scaling input elements QSIE (shown as diagonal stripes).

[0421] Similarly, the first quantization scaling circuit 226_1 can be 1_r2 To α R_r2 With x 12 Multiply to generate x 12 Multiple quantized scaling input elements QSIE (shown as a dot pattern).

[0422] In this way, the first quantization scaling circuit 226_1 can be based on x 13 to x 1n To generate multiple quantized scaled input elements QSIE.

[0423] That is to say, according to Figure 32 to Figure 34 In various example embodiments, the input vector scaler 220 may generate a plurality of quantized scaled input vectors by generating a plurality of quantized scale coefficients and then performing quantized scale on each of the plurality of input vectors based on the plurality of generated quantized scale coefficients. In this case, as with the above reference Figures 27 to 31 Unlike the various embodiments described, the input vector sealer 220 may include only one multiplication and scaling circuit. However, example embodiments are not limited thereto.

[0424] refer to Figures 27 to 34 , the input vector scaler 220 may be based on an input element (e.g., x 11 ) to generate multiple quantized scaled input elements (e.g., quantized scaled input elements QSIE, shown as diagonal stripes). In other words, the input vector sealer 220 can generate multiple quantized scaled input elements QSIE by reusing one input element. This can be achieved by using a buffer or a memory. Therefore, according to various example embodiments, the input reuse of the matrix multiplier 200 can be increased or maximized, and therefore, the number of times the matrix multiplier 200 receives input elements from the outside can be reduced or minimized. In this case, the number of times the matrix multiplier 200 accesses an external memory device that stores input elements can be reduced or minimized, and therefore the operating efficiency and operating speed of the matrix multiplication device MMD can be improved.

[0425] Fig.35 It is shown Fig.26 A block diagram of a configuration of a first data type converter. Figure 1 , Figure 4 and Figures 23 to 35, the first data type converter 230 may include first to h-th exponent extraction circuits 231_1 to 231_h and first to h-th data type conversion circuits 232_1 to 232_h.

[0426] Each of the first index extraction circuit 231_1 to the hth index extraction circuit 231_h can receive a different quantization scaling input vector QSX. For example, the first index extraction circuit 231_1 to the hth index extraction circuit 231_h can respectively receive the first quantization scaling input vector to the hth quantization scaling input vector (e.g., to ).

[0427] Each of the first index extraction circuit 231_1 to the hth index extraction circuit 231_h may extract an index from a plurality of quantization scaling input elements QSIE included in the received quantization scaling input vector QSX. For example, the first index extraction circuit 231_1 may sequentially receive the quantization scaling input elements QSIE included in the first quantization scaling input vector QSX. The first index extraction circuit 231_1 can extract the first quantized scaled input vector QSIE from the first quantized scaled input vector QSIE. In this way, the first to h-th index extraction circuits 231_1 to 231_h can extract the first to h-th indexes EXP1 to EXPh, respectively.

[0428] The first to h-th exponent extraction circuits 231_1 to 231_h may provide the extracted exponents to the first to h-th data type conversion circuits 232_2 to 232_h, respectively. In addition, each of the first to h-th exponent extraction circuits 231_1 to 231_h may provide each of the extracted exponents to the second data type converter 260.

[0429] The detailed method of extracting the index from the received element by each of the first index extraction circuit 231_1 to the hth index extraction circuit 231_h is similar to the above reference numeral 231_1. Fig.11 and Fig.12 The operation of the exponent extraction circuit is described, and thus a detailed description thereof will be omitted.

[0430] The first data type conversion circuit 232_1 to the hth data type conversion circuit 232_h can receive the first index EXP1 to the hth index EXPh respectively. The first data type conversion circuit 232_1 to the hth data type conversion circuit 232_h can receive the first quantization scaling input vector to the hth quantization scaling input vector (for example, to ).

[0431] Each of the first data type conversion circuit 232_1 to the hth data type conversion circuit 232_h can convert the data type of the received quantization scaled input vector into a fixed point based on the received exponent. That is, the first data type conversion circuit 232_1 to the hth data type conversion circuit 232_h can respectively output the first to the hth fixed-point quantization scaled input vectors (e.g., to ). For example, the first data type conversion circuit 232_1 may convert the first quantized scaled input vector (eg, ) converts each of the quantized scaled input elements QSIE in fixed-point format.

[0432] The specific method in which each of the first data type conversion circuits 232_1 to the hth data type conversion circuit 232_h converts the data type of the quantized scaling input element QSIE into a fixed point is similar to the method described above with reference to Fig.11 and Fig.13 The operations of the data types described are not described, so their detailed description will be omitted.

[0433] Fig.36 is shown in more detail Fig.26 A block diagram of the configuration of the processing element array. Figures 1 to 4 as well as Figures 23 to 36 , the processing element array 250 may include a plurality of processing elements PE arranged in the row direction and the column direction. Hereinafter, for a more concise description, it is assumed that the plurality of processing elements PE are arranged with h rows and m columns. In addition, the processing element arranged in the i-th row and the j-th column of the processing element array 150 will be referred to as "PEij". For example, the processing element arranged in the first row and the second column of the processing element array 150 will be referred to as "PE12".

[0434] The processing element array 250 may include first to h-th processing element rows PER1 to PERh. Each of the first to h-th processing element rows PER1 to PERh may include a plurality of processing elements PE. For example, the first processing element row PER1 may include processing elements PE11 to PE1m.

[0435] The processing element array 150 may include first to hth processing element columns PEC1 to PECm. Each of the first to mth processing element columns PEC1 to PECm may include a plurality of processing elements PE. For example, the first processing element column PEC1 may include processing elements PE11 to PEh1.

[0436] Different processing element rows may receive different fixed-point quantization scaling input vectors QSXfxp. For example, the first processing element row PER1 to the hth processing element row PERh may receive the first to the hth fixed-point quantization scaling input vectors (eg, to ).

[0437] The processing elements included in the same processing element row may receive the same fixed-point multiplication scaling input vector. For example, each of the processing elements PE11 to PE1m may receive a first fixed-point quantization scaling input vector (eg, ).

[0438] Different processing element columns may receive different quantization sign bits QSB. For example, the first processing element column PEC1 to the mth processing element column PECm may receive the first plurality of quantization sign bits QSBs_1 to the mth plurality of quantization sign bits QSBs_m, respectively.

[0439] Processing elements arranged in the same processing element column may receive a plurality of the same quantization sign bits QSB. For example, each of processing elements PE11 to PEh1 may receive a first plurality of quantization sign bits QSBs_1, and each of processing elements PE12 to PEh2 may receive a second plurality of quantization sign bits QSBs_2.

[0440] Each of the plurality of processing elements PE may calculate a different fixed-point output element based on the received plurality of fixed-point quantized scaled input elements QSIEfxp and the plurality of quantized sign bits QSB. That is, according to some example embodiments, one processing element PE may calculate one fixed-point output element. For example, a processing element PEij may calculate y′ ij In the following, the fixed-point output elements calculated in each processing element PE will be described in more detail.

[0441] Each of the first processing element row PER1 to the hth processing element row PERh may calculate a different fixed-point output vector. For example, the first processing element row PER1 to the hth processing element row PERh may respectively output the first to the hth fixed-point output vectors (eg, to ).

[0442] Processing elements arranged in the same processing element row and different processing element columns can calculate different fixed-point output elements. For example, processing elements PE11 to PE1m can calculate y′ 11 to y′ 1m In a similar manner, processing elements PE21 to PE2m can calculate y′ respectively. 21 to y′ 2m, and the processing elements PEh1 to PEhm can calculate y′ respectively h1 to y′ hm .

[0443] Fig.37 is shown in more detail Fig.36 The following is a block diagram of the operation of the processing element. Figures 1 to 4 , Figures 23 to 37 However, example embodiments are not limited thereto, and the second to h-th processing element rows PER2 to PERh may also operate in a similar manner.

[0444] The first processing element row PER1 may receive a first fixed-point quantized scaled input vector (eg, For example, the first processing element row PER1 may sequentially receive a plurality of fixed-point quantized scaled input elements QSIEfxp. That is, the first processing element row PER1 may sequentially receive the above referenced elements using a fixed-point format. Fig.28 , Fig.31 and Fig.34 Describes multiple quantization scaled input elements QSIE.

[0445] Processing elements PE11 to PE1m may receive first to m-th quantization sign bits QSBs_1 to QSBs_m, respectively. For example, processing element PE11 may receive first to m-th quantization sign bits QSBs_1 and processing element PE12 may receive second to m-th quantization sign bits QSBs_2.

[0446] The first plurality of quantization sign bits QSBs_1 may include the bits arranged above the reference Fig.25 The quantization symbol bits QSB in the first column of the first quantization symbol bit matrix QSBM_1 to the Rth quantization symbol bit matrix QSBM_R are described. For example, the first plurality of quantization symbol bits QSBs_1 are arranged in the first quantization symbol bit matrix QSBM_1 to the Rth quantization symbol bit matrix QSBM_R (for example, b 1_1_r1 To b 1_R_rn ) in the first column of each of .

[0447] More specifically, the processing element PE11 may sequentially receive the quantization sign bits (eg, b ) arranged in the first row and the first column of each of the first to R th quantization sign bit matrices QSBM_1 to QSBM_R. 1_1_r1 To b 1_R_r1 ); then sequentially receiving the quantization symbol bits (eg, b ) arranged in the second row and the first column of each of the first quantization symbol bit matrix QSBM_1 to the Rth quantization symbol bit matrix QSBM_R;1_1_r2 To b 1_R_r2 In this way, the processing element PE11 may sequentially receive the quantization sign bits (eg, b ) arranged in the nth row and the first column of each of the first to Rth quantization sign bit matrices QSBM_1 to QSBM_R. 1_1_rn To b 1_R_rn ).

[0448] That is, the processing element PE11 may sequentially receive pairs of quantization sign bits QSB and fixed-point quantization scaling input elements QSIEfxp corresponding to each other.

[0449] The processing element PE11 may calculate a fixed-point output element (eg, y′) based on the order in which the quantization sign bit QSB and the plurality of fixed-point multiplication scaling input elements QSIEfxp are received. 11 For example, the processing element PE11 may calculate the fixed-point output element y′ by subtracting a second value obtained by summing the fixed-point multiplication scaling input elements corresponding to the quantization sign bit QSB representing “1” among the fixed-point quantization scaling input elements QSIEfxp from a first value obtained by summing the fixed-point quantization scaling input elements corresponding to the quantization sign bit QSB representing “0” among the plurality of fixed-point scaling input elements QSIEfxp. 11 .

[0450] In this manner, processing element PE1j may receive the j-th plurality of quantization sign bits QSBs_j (eg, b 1_1_cj To b n_R_cj ). The processing element PE1j may receive a first fixed-point quantized scaled input vector (eg, ) in a plurality of fixed-point quantization scaling input elements QSIEfxp. The processing element PE1j may scale the input vector (eg, ) to calculate the output element "y′ 1j ”.

[0451] For a more detailed example, the processing element PE may calculate the fixed-point output element OEfxp according to Equation 23 below.

[0452] (Equation 23)

[0453] OEfxp=∑QSIEfxp×(-1) QSB

[0454] Referring to Equation 23, OEfxp may represent a fixed-point output element, and QSIEfxp and QSB may represent a pair of a quantization sign bit QSB and a fixed-point quantization scaling input element QSIEfxp provided to the processing element PE. That is, the processing element PE may calculate the fixed-point output element by sequentially performing a calculation of adding or subtracting a value of the fixed-point quantization scaling input element QSIEfxp based on the received quantization sign bit QSB. However, example embodiments are not limited thereto.

[0455] Fig.38 The present invention is implemented according to various exemplary embodiments. Fig.36 A diagram of the configuration of one of the processing elements of FIG. Figures 1 to 4 and Figures 23 to 38 The processing element PE may include an arithmetic logic unit ALU and an accumulation register REG_ACC.

[0456] The arithmetic logic unit ALU may include first to third input terminals TI1 to TI3 and an output terminal TO. The first input terminal TI1 may sequentially receive a plurality of fixed-point quantization scaling input elements QSIEfxp. The second input terminal TI2 may sequentially receive a plurality of quantization sign bits QSB. The third input terminal TI3 may be connected to an accumulation register REG_ACC.

[0457] In example embodiments, each fixed-point quantization scaling input element QSIEfxp may have a code length of 8 bits, and the arithmetic logic unit ALU may be configured to receive data in units of 8 bits through the first input terminal TI1 .

[0458] When the value received by the second input terminal TI2 is "0", the arithmetic logic unit ALU can store the sum of the values ​​received by the first input terminal TI1 and the third input terminal TI3 in the accumulation register REG_ACC through the output terminal TO. On the other hand, when the value received by the second input terminal TI2 is "1", the arithmetic logic unit ALU can store the value obtained by subtracting the value provided to the first input terminal TI1 from the value provided to the third input terminal TI3 in the accumulation register REG_ACC. That is, when a quantization sign bit QSB equal to "0" is received at the second terminal TI2, the arithmetic logic unit ALU can calculate the sum of the values ​​input to the first terminal TI1 and the third terminal TI3, and output the sum through the output terminal TO. Similarly, when a quantization sign bit QSB equal to "1" is received at the second terminal TI2, the arithmetic logic unit ALU can subtract the value input to the first terminal TI1 from the value input to the third terminal TI3, and output the resulting value through the output terminal TO. The accumulator register may store the most recently received value output from the arithmetic logic unit ALU and feed the value back to the third input terminal TI3.

[0459] In this way, the processing element PE may be able to store the fixed-point output element OEfxp calculated according to the above-described equation 23 in the accumulation register REG_ACC. In this case, the fixed-point output element OEfxp stored in the accumulation register REG_ACC may be provided to the second data type converter 260. However, example embodiments are not limited to a specific method in which the processing element PE performs calculations and a specific configuration method of the processing element PE.

[0460] In an example embodiment, the fixed-point output element OEfxp may have a code length of 8 bits or more. For example, the fixed-point output element OEfxp may have a code length long enough to represent the cumulative size of multiple fixed-point quantized scaled input elements QSIEfxp.

[0461] In an example embodiment, the accumulation register REG_ACC may have a size of 8 bits or more. For example, the accumulation register REG_ACC may have a size large enough to store the fixed-point output element OEfxp.

[0462] In example embodiments, the fixed-point output element OEfxp may have a code length of 10 bits to 12 bits or more. However, example embodiments are not limited thereto.

[0463] In example embodiments, the accumulation register REG_ACC may have a size of 10 bits to 12 bits. However, example embodiments are not limited thereto.

[0464] In an example embodiment, a plurality of fixed-point quantization scaling input elements QSIEfxp received by the arithmetic logic unit ALU may correspond to the same exponent value. In this case, the arithmetic logic unit ALU may be able to perform the operation of equation 23 described above without considering the position value of each of the plurality of fixed-point quantization scaling input elements QSIEfxp. Therefore, the arithmetic logic unit ALU may calculate the fixed-point output element OEfxp with a reduced or minimal amount of calculation.

[0465] In an example embodiment, the processing element PE may not include a "1-bit adder" for performing a multiplication operation on the multiplication scaling coefficient MSC. That is, according to some example embodiments, since the quantized scaling input element QSIE is provided to the processing element array 250, each processing element PE may be implemented to not perform calculations on the multiplication scaling coefficient MSC. In this case, instead of each processing element PE including a circuit element for performing a multiplication calculation on the multiplication scaling coefficient MSC, each processing element row PER may include only one multiplication scaling circuit (for example, when referring to the above reference). Figures 27 to 31220); or there may be only one multiplication scaling circuit for the entire processing element array 250 (e.g., when the input vector scaler 220 is implemented according to the various example embodiments described above); Figure 32-34 The various example embodiments described herein are used to implement the input vector scaler 220). Therefore, the size and production cost of the matrix multiplier 200 may be reduced.

[0466] Fig.39 It is shown Fig.26 2 is a diagram of the operation of the second data type converter. Hereinafter, for a more concise description, the operation of the second data type converter 260 for one fixed-point output element OEfxp will be representatively described. However, example embodiments are not limited thereto. For example, the second data type converter 260 can operate in a similar manner for any fixed-point output element OEfxp.

[0467] refer to Figures 1 to 4 and Figures 23 to 39 , the second data type converter 260 can receive the fixed-point output element OEfxp. For example, the second data type converter 260 can receive y′ 11 to y′ hn one.

[0468] The second data type converter 260 may receive the first to h-th exponents EXP1 to EXPh from the first data type converter 230. In this case, the first to h-th exponents EXP1 to EXPh may correspond to the first fixed-point output vectors (eg, ) to the hth fixed-point output vector (e.g., ).

[0469] The fixed-point output element OEfxp may include a sign part SP and a mantissa part MTSP. The second data type converter 260 may generate an output element OE by converting a data type of the fixed-point output element OEfxp into a floating point.

[0470] The method of converting the data type of the fixed-point output element OEfxp by the second data type converter 260 is similar to that of the above reference. Fig.18 The method in which the second data type converter 160 converts the data type of the fixed-point partial product PSPfxp is not described, and thus a detailed description thereof will be omitted.

[0471] Fig.40 It is shown Figure 1 Flow chart of the operation of the matrix multiplier. In the following, reference is made to Figures 1 to 4 and Figures 23 to 40 , will describe receiving an input vector (e.g., ) and output an output vector (e.g., )'s matrix multiplication device MMD operation.

[0472] In operation S310, the matrix multiplication device MMD may receive a weight matrix WM. For example, the uniform BCQ circuit UBC may receive a plurality of weights (eg, w 11 tow nm ).

[0473] In operation S320, the matrix multiplication device MMD may generate a plurality of multiplication scaling coefficients MSC, a plurality of common scaling coefficients CSC, and a plurality of quantization sign bits QSB by performing uniform binary coded quantization on the weight matrix WM. For example, the uniform BCQ circuit UBC may approximate each weight based on a plurality of multiplication scaling coefficients MSC, a common scaling coefficient CSC, and a plurality of quantization sign bits QSB. The uniform BCQ circuit UBC may provide a plurality of generated multiplication scaling coefficients MSC, a plurality of common scaling coefficients CSC, and a plurality of quantization sign bits QSB to the matrix multiplier 200. In this case, a plurality of multiplication scaling coefficients MSC may be stored in a multiplication scaling coefficient buffer 210, a plurality of common scaling coefficients CSC may be stored in a common scaling coefficient buffer 270, and a plurality of quantization sign bits QSB may be stored in a quantization sign bit buffer 240. However, example embodiments are not limited thereto.

[0474] In operation S330, the matrix multiplication device MMD may receive an input vector (eg, For example, the matrix multiplier 200 may receive a plurality of input elements (eg, x 11 to x 1n ).

[0475] In example embodiments, the matrix multiplication device MMD may perform operation S330 regardless of the order of operations S310 to S320. For example, the matrix multiplication device MMD may perform operation S330 and then perform operations S310 to S320, or perform operation S330 between operations S310 and S320.

[0476] In operation S340, the matrix multiplication device MMD can generate a quantized scaled input vector QSX by performing quantized scaling on the input vector based on multiple multiplication scaling coefficients MSC and multiple common scaling coefficients CSC. For example, the input vector scaler 220 can generate multiple quantized scaled input elements QSIE by performing quantized scaling on each of multiple input elements based on multiple multiplication scaling coefficients MSC and multiple common scaling coefficients CSC.

[0477] In operation S350, the matrix multiplication device MMD may generate an output vector (eg, For example, the matrix multiplier 200 may generate an output element (eg, y) by performing a calculation of sequentially adding or subtracting a plurality of quantization scaling input elements QSIE included in the quantization scaling input vector QSX based on the first plurality of quantization sign bits QSBs_1. 11 ). In this manner, the matrix multiplier 200 may generate a plurality of output elements included in an output vector.

[0478] Fig.41 is shown in more detail Fig.40 Flow chart of operation S350. Figures 1 to 4 and Figure 23 to Figure 41 , operation S350 may include operations S351 to S353.

[0479] In operation S351, the matrix multiplier 200 may generate a fixed-point quantization scaling input vector QSXfxp by converting the data type of the quantization scaling input vector QSX into a fixed point. That is, the matrix multiplier 200 may generate a fixed-point quantization scaling input vector QSXfxp based on the quantization scaling input vector QSX. For example, the first data type converter 230 may generate a plurality of fixed-point quantization scaling input elements QSIEfxp by converting each data type of the plurality of quantization scaling input elements QSIE into a fixed point.

[0480] In operation S352, the matrix multiplier 100 may generate a fixed-point output vector (eg, ). For example, the processing element array 150 may generate a fixed-point output vector by performing a calculation of sequentially adding or subtracting a plurality of fixed-point quantization scaling input elements QSIEfxp based on a plurality of quantization sign bits QSB.

[0481] More specifically, the processing element array 250 may generate a fixed-point output element (eg, y′) by performing a calculation of sequentially adding or subtracting a plurality of quantized scaled input elements QSIE based on the first plurality of quantized sign bits QSBs_1. 11 Similarly, the processing element array 250 may generate a fixed-point output element (eg, y′) by performing a calculation of sequentially adding or subtracting a plurality of quantized scaled input elements QSIE based on the second plurality of quantized sign bits QSBs_2. 12 ). In this manner, processing element array 250 can calculate the output vectors included in the fixed-point output vector (e.g., ) in multiple fixed-point output elements OEfxp.

[0482] In operation S353, the matrix multiplier 200 may convert the data type of the fixed-point output vector (eg, ) is converted to a floating point. For example, the second data type converter 260 may receive a plurality of fixed-point output elements OEfxp included in a fixed-point output vector. The second data type converter 260 may convert the plurality of fixed-point output elements OEfxp into a plurality of output elements OE, respectively.

[0483] Fig.42 It is shown Figure 1 Flow chart of the operation of the matrix multiplier. In the following, reference is made to Figures 1 to 4 and Figures 23 to 42 , will describe receiving an input vector (e.g., ) and generates an input element (e.g., y 11 )'s matrix multiplication device MMD operation.

[0484] In operation S410, the matrix multiplication device MMD may receive first to nth weights. Operation S410 is similar to the above reference Fig.21 Operation S210 is not described, and thus a detailed description thereof will be omitted.

[0485] In operation S420, the matrix multiplication device MMD may generate first to Rth multiplication scaling coefficients MSC, first to nth common scaling coefficients CSC, and first to (N×R)th quantization sign bits QSB by performing uniform binary coding quantization on the first to nth weights. Fig.21 Unlike the described S220 (which generates only a single common scaling coefficient CSC for the first to nth weights), the uniform BCQ circuit UBC may generate the first to nth common scaling coefficients CSC based on the first to nth weights.

[0486] In operation S430, the matrix multiplication device MMD may receive first to nth input elements. Operation S430 is similar to the above reference numerals. Fig.21 Operation S230 is not described, and thus a detailed description thereof will be omitted.

[0487] In example embodiments, the matrix multiplication device MMD may perform operation S430 regardless of the order of operations S410 to S420. For example, the matrix multiplication device MMD may perform operation S430 and then perform operations S410 and S420, or perform operation S430 between operations S410 and S420.

[0488] In operation S440, the matrix multiplication device MMD may generate first to (N×R)th quantized scaled input elements QSIE by performing quantization scaling on the first to Nth input elements based on the first to Rth multiplication scaling coefficients MSC and the first to nth common scaling coefficients CSC. For example, the input vector scaler 220 may generate the first to (N×R)th quantized scaled input elements QSIE by performing quantization scaling on the first to Nth input elements based on the first to Rth multiplication scaling coefficients MSC and the first to nth common scaling coefficients CSC. Figures 27 to 34 The various methods described perform quantization scaling on the first to Nth input elements to generate the first to (N×R)th quantization scaling input elements QSIE. Figure 43 to Figure 45 A more detailed operation of the input vector sealer 220 is described in more detail.

[0489] In operation S450, the matrix multiplication device MMD may generate an output element (eg, y) based on the first to (N×R)th quantization scaling input elements QSIE and the first to (N×R)th quantization sign bits QSB. 11 ). For example, the matrix multiplier 200 can generate one output element (e.g., y) by calculating a value obtained by subtracting a value obtained by summing the quantization scaling input elements QSIE corresponding to the quantization sign bit QSB indicating "1" among the first to (N×R) quantization scaling input elements QSIE from the sum of the quantization scaling input elements QSIE corresponding to the quantization sign bit QSB indicating "0" among the first to (N×R) quantization scaling input elements QSIE. 11 ).

[0490] Figure 43 to Figure 45 is a more detailed illustration of the implementation according to an exemplary embodiment Fig.42 Flow chart of operation S440. Fig.43 Description When based on the above reference Figure 27 to Figure 28 The various exemplary embodiments described herein implement operation S440 when the input vector scaler 220 is implemented, and reference will be made to Fig.44 Description When based on the above reference Figure 29 to Figure 31 The various exemplary embodiments described herein implement operation S440 when the input vector scaler 220 is implemented, and will be referred to as Fig.45 Description When based on the above reference Figure 32 to Figure 34 Various exemplary embodiments are described to implement operation S440 when the vector scaler 220 is input.

[0491] First reference Figures 1 to 4 and Figure 23 to Figure 43 , operation S440 may be implemented as the following operation S440a. Operation S440a may include operations S441a and S442a.

[0492] In operation S441a, the input vector sealer 220 may generate first to (N×R)th multiplication scaling input elements MSIE by multiplying each of the first to Nth input elements by the first to Rth multiplication scaling coefficients MSC. For example, the first multiplication scaling circuit 221_1 may generate first to Rth multiplication scaling input elements MSIE by multiplying the first input element by the first multiplication scaling coefficients MSC1 to the Rth multiplication scaling coefficients MSCR; and generate (R+1)th to (2R)th multiplication scaling input elements MSIE by multiplying the second input element by the first scaling coefficients MSC1 to the Rth scaling coefficients MSCR.

[0493] In operation S442a, the input vector sealer 220 may multiply each of the first to (N×R)th multiplication scaled input elements MSIE by a corresponding one of the first to nth common scaling coefficients CSC to generate the first to (N×R)th quantization scaled input elements QSIE. For example, the first common scaling circuit 222_1 may generate the first to (N×R)th quantization scaled input elements QSIE by multiplying the first to Rth multiplication scaled input elements MSIE (e.g., the multiplication scaled input elements MSIE11_1 to MSIE11_R) by the first common scaling coefficient CSC (e.g., S r1 ) to generate the first to Rth quantized scaled input elements QSIE (eg, x 11 Similarly, the first common scaling circuit 222_1 may be configured to generate a quantized scaled input element by combining the (R+1)th to (2R)th multiplication scaled input elements MSIE (eg, the multiplication scaled input elements MSIE12_1 to MSIE12_R) with the second common scaling coefficient CSC (eg, S r2 ) are multiplied to generate the (R+1)th to (2R)th quantized scaling input elements QSIE (e.g., x 12 quantized scaling of input elements).

[0494] Reference now Figures 1 to 4 , Figures 23 to 42 and Fig.44 , operation S440 may be implemented as the following operation S440b. Operation S440b may include operations S441b and S442b.

[0495] In operation S441b, the input vector sealer 220 may generate first to nth common scaling input elements CSIE by multiplying the first to Nth input elements by the first to nth common scaling coefficients CSC, respectively. For example, the first common scaling circuit 223_1 may generate first to nth common scaling input elements CSIE by multiplying the first to Nth input elements by the first to nth common scaling coefficients CSC, respectively.

[0496] In operation S442b, the input vector sealer 220 may generate the first to (N×R)th quantized scaling input elements QSIE by multiplying each of the first to Nth common scaling input elements CSIE by the first to Rth multiplication scaling coefficients MSC. For example, the first multiplication scaling circuit 224_1 may generate the first to Rth quantized scaling input elements QSIE by multiplying the first common scaling input element CSIE by the first to Rth multiplication scaling coefficients MSC; and generate the (R+1)th to (2R)th quantized scaling input elements QSIE by multiplying the second common scaling input element CSIE by the first to Rth multiplication scaling coefficients MSC.

[0497] Reference now Figures 1 to 4 , Figures 23 to 42 and Fig.45 , operation S440 may be implemented as the following operation S440c. Operation S440c may include operations S441c and S442c.

[0498] In operation S441c, the input vector sealer 220 may generate the first to (N×R)th quantization scaling coefficients QSC by multiplying each of the first to Rth multiplication scaling coefficients MSC by the first to nth common scaling coefficients CSC. For example, the multiplication scaling circuit 225 may generate the first to (N×R)th quantization scaling coefficients QSC (e.g., α ) by calculating the product of different combinations (e.g., each combination) of the first to Rth multiplication scaling coefficients MSC and the first to nth common scaling coefficients CSC. 1_r1 To α R_rn ).

[0499] In operation S442c, the input vector scaler 220 may generate the first to (N×R)th quantized scaled input elements QSIE by multiplying each of the first to (N×R)th quantized scale coefficients QSC by a corresponding one of the first to nth input elements. For example, the first quantized scaler 226_1 may generate the first to (N×R)th quantized scaled input elements QSIE by multiplying each of the first to (N×R)th quantized scale coefficients QSC by a corresponding one of the first to nth input elements. 11 ) and the first to Rth quantization scaling coefficients QSC (e.g., α 1_r1 To α R_r1 ) to generate the first to Rth quantization scaling input elements QSIE. Similarly, the first quantization scaling circuit 226_1 can generate the first to Rth quantization scaling input elements QSIE by multiplying the second input element (eg, x 12 ) and the (R+1)th to (2R)th quantization scaling coefficients QSC (e.g., α 1_r2 To α R_r2 ) are multiplied to generate the (R+1)th to (2R)th quantized scaling input elements QSIE.

[0500] Fig.46 is shown in more detail Fig.42Flow chart of operation S450. Figures 1 to 4 and Figures 23 to 46 , operation S450 may include operations S451 to S453.

[0501] In operation S451, the matrix multiplier 100 may generate first to (N×R)th fixed-point quantized scaling input elements QSIEfxp by converting the data types of the first to (N×R)th quantized scaling input elements QSIE into fixed points. For example, the first data type converter 230 may convert the first to (N×R)th quantized scaling input elements QSIE into first to (N×R)th fixed-point quantized scaling input elements QSIEfxp, respectively.

[0502] In operation S452, the matrix multiplier 200 may calculate a fixed-point output element OEfxp by subtracting a value obtained by summing the fixed-point quantization scaling input elements QSIE corresponding to the quantization sign bit QSB representing "1" among the first to (N×R) fixed-point quantization scaling input elements QSIEfxp from the sum of the fixed-point quantization scaling input elements QSIE corresponding to the quantization sign bit QSB representing "0" among the first to (N×R) fixed-point quantization scaling input elements QSIEfxp. For example, the processing element PE11 may calculate a fixed-point output element OEfxp (e.g., y′ 11 ).

[0503] In operation S453, the matrix multiplier 200 may generate an output element by converting the data type of the fixed-point output element OEfxp into a floating point. For example, the second data type converter 260 may receive the fixed-point output element OEfxp (eg, y′ 11 ) and outputs the output element OE (e.g., y 11 ).

[0504] Fig.47 is a diagram showing various exemplary embodiments Figure 1 The operation of the BCQ circuit. Figures 1 to 4 and Fig.47 , the uniform BCQ circuit UBC can perform a binary coded quantization operation for each row of the weight matrix WM.

[0505] The uniform BCQ circuit UBC can be used to make use of the zero vector (e.g., ) and a plurality of quantization scaling coefficients (e.g., α) and a plurality of quantization symbol vectors (e.g., ) to approximate each row vector of the weight matrix WM. For example, the uniform BCQ circuit UBC can perform a uniform binary coding quantization operation for each row of the weight matrix WM based on the following equation 24. Figures 23 to 46 Describe the differences in the operation of the matrix multiplication device MMD.

[0506] (Equation 24)

[0507]

[0508] Referring to Equation 24, It can represent the zero point vector of the i-th row vector of the weight matrix WM. can be implemented as a row vector with the same dimension as the number of columns of the weight matrix WM. Each element of can have the same value. For example, It can be expressed as the following equation 25.

[0509] (Equation 25)

[0510]

[0511] Referring to Equation 25, ZPVi can be expressed as That is, the zero-point values ​​ZPVi corresponding to the weights included in the i-th row of the weight matrix WM can be the same.

[0512] Therefore, each of the plurality of weights included in the weight matrix WM may be approximated based on the plurality of quantization scaling coefficients QSC and the plurality of quantization sign bits QSB according to Equation 26 below.

[0513] (Equation 26)

[0514]

[0515] That is, according to various exemplary embodiments of the present disclosure, a weight may be approximated based on a zero point value ZPV, R multiplication scaling coefficients MSC, a common scaling coefficient CSC, and R quantization sign bits QSB.

[0516] In this way, the uniform BCQ circuit UBC can approximate the weight matrix WM based on a plurality of zero point values ​​ZPV, a plurality of multiplication scaling coefficients MSC, a plurality of common scaling coefficients CSC and a plurality of quantization sign bits QSB. In this case, as previously described with reference to Figure 1 Different from what is described, the uniform BCQ circuit UBC may also provide a plurality of zero point values ​​ZPV to the matrix multiplier 100 .

[0517] The following will refer to Figure 48-58The operation of the matrix multiplier 100 based on weights approximated on the basis of the zero point value ZPV, the multiplication scaling factor MSC, the common scaling factor CSC and the quantization sign bit QSB is described in more detail.

[0518] Fig.48 is a diagram showing implementation according to various exemplary embodiments Figure 1 A block diagram of the configuration of a matrix multiplier. Figures 1 to 4 as well as Fig.47 and Fig.48 , Figure 1 The matrix multiplier 100 can be implemented as Fig.48 The matrix multiplier 300 is provided.

[0519] The matrix multiplier 300 may include a multiplication scaling coefficient buffer 310 , a common scaling coefficient buffer 370 , an input vector scaler 320 , a first data type converter 330 , a quantization sign bit buffer 340 , a processing element array 350 , and a second data type converter 360 .

[0520] Each component of matrix multiplier 300 may perform the same operations as described above with reference to Figures 23 to 46 The components of the matrix multiplier 200 described above operate similarly. Figures 23 to 46 The differences between the matrix multiplier 300 and the matrix multiplier 200 are described.

[0521] The multiplication scaling coefficient buffer 310 may store a plurality of multiplication scaling coefficients MSC and a plurality of zero-point scaling coefficients ZPSC provided from the uniform BCQ circuit UBC. The multiplication scaling coefficient buffer 210 may provide a plurality of multiplication scaling coefficients MSC and a plurality of zero-point scaling coefficients ZPSC to the input vector scaler 220.

[0522] In an example embodiment, each of the plurality of zero-point scaling coefficients ZPSC may be “1.” Hereinafter, for a more concise description, various example embodiments in which each of the plurality of zero-point scaling coefficients ZPSC is “1” will be representatively described. However, example embodiments are not limited thereto.

[0523] The common scaling coefficient buffer 370 may store a plurality of common scaling coefficients CSC and a plurality of zero point values ​​ZPC provided from the uniform BCQ circuit UBC. The common scaling coefficient buffer 370 may provide the plurality of common scaling coefficients CSC and a plurality of zero point values ​​ZPV to the input vector scaler 320.

[0524] The input vector scaler 320 can receive the input matrix XM. The input vector scaler 320 can generate multiple quantized scaling input vectors QSX based on multiple common scaling coefficients CSC, multiple zero point values ​​ZPV, multiple zero point scaling coefficients ZPSC and multiple multiplication scaling coefficients MSC. In other words, the input vector scaler 320 can also generate multiple quantized scaling input vectors QSX based on multiple zero point values ​​ZPV and multiple zero point scaling coefficients ZPSC.

[0525] Each of the plurality of quantized scaled input vectors QSX may be implemented as a row vector having a dimension "R+1" times that of the corresponding input vector. For example, corresponding to the first input vector (eg, ) of the first quantized scaled input vector (e.g., ) may include “(R+1)×n” quantization scaling input elements QSIE, as shown in Equation 27 below.

[0526] (Equation 27)

[0527]

[0528] Referring to Equation 27, some of the plurality of quantized scaled input elements QSIE included in the first scaled input vector may correspond to a value obtained by multiplying each of the plurality of input elements included in the first input vector by a corresponding zero point value ZPV.

[0529] Similarly, the input vector sealer 320 may generate second to h-th quantized scaled input vectors (eg, to ). For the sake of simplicity, the second to h-th quantized scaling input vectors (eg, to ) in each of the quantization scaling input elements QSIE. Figures 49 to 57 The specific method by which the input vector sealer 320 generates a plurality of quantized scaled input vectors QSX is described in more detail.

[0530] The first data type converter 330 may receive a plurality of quantization scaling input vectors QSX. The first data type converter 330 may convert the data types of the plurality of quantization scaling input vectors QSX into fixed points. That is, the first data type converter 330 may generate first to hth fixed-point quantization scaling input vectors (e.g., to ).

[0531] The quantization sign bit buffer 340 may store a plurality of quantization sign bits QSB and a plurality of zero point correction bits ZPCB provided from the uniform BCQ circuit UBC. The quantization sign bit buffer 140 may provide a plurality of quantization sign bits QSB and a plurality of zero point correction bits ZPCB to the processing element array 350.

[0532] In an example embodiment, each of the plurality of zero point correction bits ZPCB may be “0”. Hereinafter, for a more concise description, various example embodiments in which each of the plurality of zero point correction bits ZPSB is “0” will be representatively described. However, example embodiments are not limited thereto.

[0533] The processing element array 350 can receive multiple quantization sign bits QSB and multiple zero point correction bits ZPCB and multiple fixed point quantization scaling input vectors QSXfxp. The processing element array 350 can generate a fixed point output matrix YMfxp based on multiple quantization sign bits QSB, multiple zero point correction bits ZPCB and multiple fixed point quantization scaling input elements.

[0534] The processing element array 350 may include a plurality of processing elements arranged in a row direction and a column direction. Each of the plurality of processing elements may calculate a different fixed-point output element. Fig.58 The detailed configuration and operation of processing element array 350 are described in further detail.

[0535] The second data type converter 360 may receive a plurality of fixed-point output elements from the processing element array 350. The second data type converter 360 may convert the data type of the fixed-point output matrix YMfxp into a floating point based on a plurality of exponents EXP. That is, the second data type converter 360 may output an output matrix YM having a floating point data type.

[0536] Fig.49 is a diagram showing various exemplary embodiments Fig.48 A block diagram of the configuration of the input vector scaler. Figures 1 to 4 and Figure 47 to Figure 49 , the input vector scaler 220 may include a multiplication scaler SCL_multiple and a common scaler SCL_common. The multiplication scaler SCL_multiple may include first to h-th multiplication scaling circuits 321_1 to 321_h. The common scaler SCL_common may include first to h-th common scaling circuits 322_1 to 322_h.

[0537] The components of input vector scaler 320 may perform the same operations as described above with reference to Fig. 27 and Fig.28 The operation of the input vector scaler 220 described above is similar to the operation of the input vector scaler 220 described above. Figures 23 to 46 The differences between input vector sealer 320 and input vector sealer 220 are described.

[0538] The multiplication scaler SCL_multiple may sequentially receive a plurality of multiplication scaling coefficients MSC and a plurality of zero-point scaling coefficients ZPSC from the multiplication scaling coefficient buffer 310. For example, each of the first to hth multiplication scaling circuits 321_1 to 321_h may sequentially receive a plurality of multiplication scaling coefficients MSC and a plurality of zero-point scaling coefficients ZPSC from the multiplication scaling coefficient buffer 310.

[0539] Each of the first to h-th multiplication scaling circuits 321_1 to 321_h can perform multiplication scaling on the received input vector based on a plurality of multiplication scaling coefficients MSC and a plurality of zero-point scaling coefficients ZPSC. For example, the first to h-th multiplication scaling circuits 321_1 to 321_h can generate first to h-th multiplication scaling input vectors (e.g., to ). The following will refer to Fig.50 The operations of the first to hth common scaling circuits 322_1 to 322_h will be described in more detail.

[0540] The common scaler SCL_common may sequentially receive a plurality of common scaling coefficients CSC and a plurality of zero point values ​​ZPV from the common scaling coefficient buffer 370. For example, each of the first to hth common scaling circuits 322_1 to 322_h may sequentially receive a plurality of common scaling coefficients CSC and a plurality of zero point values ​​ZPV from the common scaling coefficient buffer 270.

[0541] Each of the first to h-th scaling circuits 322_1 to 322_h may perform common scaling on the received multiplication scaling input vector MSX based on a plurality of common scaling coefficients CSC and a plurality of zero point values ​​ZPV. For example, the first to h-th common scaling circuits 322_1 to 322_h may generate a first quantization scaling input vector to an h-th quantization scaling input vector (e.g., to ). The following will refer to Fig.51 The operations of the first to hth common scaling circuits 322_1 to 322_h are described in more detail.

[0542] Fig.50 is shown in more detail Fig.49 In the following, for a more concise description, reference will be made to Figures 1 to 4 and Figures 47 to 50The operation of the first multiplication-scaling circuit 321_1 is representatively described. However, example embodiments are not limited thereto, and the second through h-th multiplication-scaling circuits 321_2 through 321_h may also operate in a similar manner.

[0543] The first multiplication and scaling circuit 321_1 may receive a first input vector (eg, That is, the first multiplication scaling circuit 321_1 may sequentially receive a plurality of input elements (eg, x 11 to x 1n ).

[0544] The first multiplication scaling circuit 321_1 may sequentially receive the zero-point scaling coefficient ZPSC and first to R-th multiplication scaling coefficients MSC1 to MSCR.

[0545] The first multiplication scaling circuit 321_1 may generate multiplication scaling input elements MSIE11_0 to MSIE1n_R by multiplying each of the received plurality of input elements by a zero-point scaling coefficient ZPSC and first to R-th multiplication scaling coefficients MSC1 to MSCR.

[0546] For example, the first multiplication and scaling circuit 321_1 can convert the input element “x 11 " is multiplied by the zero-point scaling factor ZPSC to generate the multiplication scaling input element MSIE11_0, and by 11 ” is multiplied by the first multiplication scaling coefficient MSC1 to the Rth multiplication scaling coefficient MSCR to generate multiplication scaling input elements MSIE11_1 to MSIE11_R (shown as diagonal stripes).

[0547] Similarly, the first multiplication and scaling circuit 321_1 can be configured to multiply the input element “x 12 " is multiplied by the zero point scaling coefficient ZPSC to generate the multiplication scaling input element MSIE12_0; and by 12 ” is multiplied by the first multiplication scaling coefficient MSC1 to the Rth multiplication scaling coefficient MSCR to generate multiplication scaling input elements MSIE12_1 to MSIE12_R (shown as a dot pattern).

[0548] In this way, the first multiplication scaling circuit 321_1 can sequentially calculate the values ​​corresponding to x 13 to x 1nIn this case, because the zero-point scaling coefficient ZPSC is equal to "1", the multiplication scaling input elements (e.g., multiplication scaling input elements MSIE11_0 to MSIE1n_0) generated based on the zero-point scaling coefficient ZPSC may be the same as the corresponding input elements. For example, the multiplication scaling input elements MSIE11_0 to MSIE1n_0 may be respectively the same as x 11 to x 1n same.

[0549] Fig.51 is shown in more detail Fig.49 In the following, for a more concise description, reference will be made to Figures 1 to 4 and Figures 47 to 51 The operation of the first common scaling circuit 322_1 is representatively described as follows. However, example embodiments are not limited thereto, and the second through h-th common scaling circuits 322_2 through 322_h may also operate in a similar manner.

[0550] The first common scaling circuit 322_1 may receive a first multiplication scaling input vector (eg, That is, the first common scaling circuit 322_1 can sequentially receive the above reference Fig.50 Describes the common scaling input elements MSIE11_0-MSIE1n_R.

[0551] The first common scaling circuit 322_1 may sequentially receive a plurality of zero point values ​​ZPV and a plurality of common scaling coefficients CSC. For example, the first common scaling circuit 322_1 may sequentially receive a first row vector of a weight matrix WM (eg, )’s first zero point value ZPV1 and S r1 R instances of , and can sequentially receive the second row vector of the weight matrix WM (e.g., )'s second zero point value ZPV2 and S r2 R instances of .

[0552] The first common scaling circuit 322_1 may generate a first quantized scaling input vector (eg, ).

[0553] For example, the first common scaling circuit 322_1 may generate a quantized scaling input element QSIE based on the product of the multiplication scaling input element MSIE11_0 and the first zero point value ZPV1; and may generate a quantized scaling input element QSIE based on S r1The product with each of the multiplicative scaling input elements MSIE12_1 to MSIE12_R generates R quantized scaling input elements QSIE (shown as diagonal stripes).

[0554] Similarly, the first common scaling circuit 322_1 may generate a quantized scaling input element QSIE based on the product of the multiplication scaling input element MSIE12_0 and the second zero point value ZPV2; and may generate a quantized scaling input element QSIE based on S r2 The product of each of the multiplicative scaling input elements MSIE12_1 to MSIE12_R generates R quantized scaling input elements QSIE (shown as a dot pattern).

[0555] As a result, the first quantization scales the input vector (eg, ) may include a quantization scaling input element QSIE corresponding to the product of multiple zero point values ​​ZPV and multiple input elements.

[0556] Fig.52 is a diagram showing various exemplary embodiments Fig.48 A block diagram of the configuration of the input vector scaler. Figures 1 to 4 , Fig.47 and Fig.48 as well as Fig.52 , the input vector scaler 320 may include a common scaler SCL_common and a multiplication scaler SCL_multiple. The common scaler SCL_common may include a first common scaling circuit 323_1 to an hth common scaling circuit 323_h. The multiplication scaler SCL_multiple may include a first multiplication scaling circuit 324_1 to an hth multiplication scaling circuit 324_h.

[0557] The components of input vector scaler 320 may perform the same operations as described above with reference to Fig.29 and Fig.31 The operation of the input vector scaler 220 described above is similar to the operation of the input vector scaler 220 described above. Figure 29 to Figure 31 The differences between input vector sealer 320 and input vector sealer 220 are described.

[0558] The first to h-th common scaling circuits 323_1 to 323_h may receive first to h-th input vectors (eg, to For example, the first common scaling circuit 323_1 may sequentially receive x 11 to x 1n , and the hth common scaling circuit 323_h can sequentially receive x h1 to x hn .

[0559] The common scaler SCL_common may sequentially receive a plurality of common scaling coefficients CSC and a plurality of zero point values ​​ZPV from the common scaling coefficient buffer 370. For example, each of the first to hth common scaling circuits 323_1 to 323_h may sequentially receive a plurality of common scaling coefficients CSC and a plurality of zero point values ​​ZPV from the common scaling coefficient buffer 370.

[0560] Each of the first to the hth common scaling circuits 323_1 to 322_h may perform common scaling on a received input vector based on a plurality of common scaling coefficients CSC and a plurality of zero point values ​​ZPV. For example, the first to the hth common scaling circuits 323_1 to 322_h may generate a first to an hth common scaling input vector (e.g., to ). The following will refer to Fig.53 The operations of the first to hth common scaling circuits 323_1 to 323 — h are described in more detail.

[0561] The multiplication scaler SCL_multiple may sequentially receive a plurality of multiplication scaling coefficients MSC and a plurality of zero-point scaling coefficients ZPSC from the multiplication scaling coefficient buffer 310. For example, each of the first to hth multiplication scaling circuits 324_1 to 324_h may sequentially receive a plurality of multiplication scaling coefficients MSC and a plurality of zero-point scaling coefficients ZPSC from the multiplication scaling coefficient buffer 310.

[0562] Each of the first multiplication and scaling circuit 324_1 to the hth multiplication and scaling circuit 324_h can perform multiplication and scaling on the received common scaling input vector CSX based on the multiplication scaling coefficient MSC and the plurality of zero-point scaling coefficients ZPSC. For example, the first multiplication and scaling circuit 324_1 to the hth multiplication and scaling circuit 324_h can respectively generate the first quantization scaling input vector to the hth quantization scaling input vector (e.g., to ). The following will refer to Fig.54 The operations of the first to h-th multiplication-scale circuits 324_1 to 324_h are described in more detail.

[0563] Fig.53 is shown in more detail Fig.52 In the following, for a more concise description, reference will be made to Figures 1 to 4 , Fig.47 and Fig.48 as well as Fig.52 and Fig.53The operation of the first common scaling circuit 323_1 is representatively described as follows. However, example embodiments are not limited thereto, and the second through h-th common scaling circuits 323_2 through 323_h may also operate in a similar manner.

[0564] The first common scaling circuit 323_1 may receive a first input vector (eg, That is, the first common scaling circuit 323_1 can sequentially receive x 11 to x 1n The first common scaling circuit 323_1 may sequentially receive a plurality of zero point values ​​ZPV and a plurality of common scaling coefficients CSC. For example, the first common scaling circuit 323_1 may sequentially receive a first row vector of a weight matrix WM (eg, )’s first zero point value ZPV1 and S r1 R instances of , and can sequentially receive the second row vector of the weight matrix WM (e.g., )'s second zero point value ZPV2 and S r2 R instances of .

[0565] The first common scaling circuit 323_1 may generate a first common scaling input vector (eg, ).

[0566] For example, the first common scaling circuit 322_1 may generate common scaling input elements CSIE11_0 to CSIE1n_R. For example, the first common scaling circuit 323_1 may generate common scaling input elements CSIE11_0 to CSIE1n_R based on x 11 The product of the first zero point value ZPV1 generates a common scaled input element CSIE11_0, and based on x 11 With S r1 The product of R instances of CSIE12_1 to generate R common scaled input elements CSIE12_1-CSIE12_R (shown as diagonal stripes).

[0567] Similarly, the first common scaling circuit 323_1 can be based on x 12 The product of the second zero point value ZPV2 generates a common scaled input element CSIE12_0, and based on x 12 With S r2 The product of R instances of CSIE12_1 to generate R common scaled input elements CSIE12_1-CSIE12_R (shown as a dot pattern).

[0568] As a result, the first common scaled input vector (eg, ) may include common scaling input elements CSIE11_0 to CSIE1n_0 corresponding to the products of multiple zero point values ​​ZPV and multiple input elements.

[0569] Fig.54 is a more detailed description of various exemplary embodiments. Fig.52 In the following, for a more concise description, reference will be made to Figures 1 to 4 , Fig.47 and Fig.48 as well as Figure 52 to Figure 54 The operation of the first multiplication scaling circuit 324_1 is representatively described. However, example embodiments are not limited thereto, and the second to h-th multiplication scaling circuits 324_2 to 324_h may also operate in a similar manner.

[0570] The first multiplication and scaling circuit 324_1 may receive a first common scaling input vector (eg, ). For example, the first multiplication and scaling circuit 324_1 may receive the above reference Fig.53 The first multiplication scaling circuit 324_1 may repeatedly receive the zero-point scaling coefficient ZPSC and the first multiplication scaling coefficients MSC1 to the Rth multiplication scaling coefficients MSCR.

[0571] The first multiplication scaling circuit 324_1 may generate a first quantized scaling input vector (eg, ).

[0572] For example, the first multiplication and scaling circuit 324_1 may generate a quantized scaling input element QSIE (eg, “ZPV1×x 11 ”); and generating R quantized scaling input elements QSIE (shown as diagonal stripes) by multiplying the multiplication scaling input elements CSIE11_1 to CSIE11_R with the first multiplication scaling coefficient MSC1 to the Rth multiplication scaling coefficient MSCR, respectively.

[0573] Similarly, the first multiplication and scaling circuit 324_1 may generate a quantized scaling input element QSIE (eg, ZPV2×x ) based on the product of the common scaling input element CSIE12_0 and the zero scaling coefficient ZPSC. 12); and generating R quantized scaling input elements QSIE (shown as a dot pattern) by multiplying the multiplication scaling input elements CSIE12_1 to CSIE12_R with the first multiplication scaling coefficient MSC1 to the Rth multiplication scaling coefficient MSCR, respectively.

[0574] As a result, the first quantization scales the input vector (eg, ) may include a quantization scaling input element QSIE corresponding to the product of a plurality of zero point values ​​ZPV and each of a plurality of input elements.

[0575] Fig.55 is a diagram showing various exemplary embodiments Fig.48 A block diagram of the configuration of the input vector scaler. Figures 1 to 4 , Fig.47 and Fig.48 as well as Fig.55 , the input vector scaler 320 may include a multiplication scaling circuit 325 and a quantization scaler SCL_quantization. The quantization scaler SCL_quantization may include a first quantization scaling circuit 326_1 to an h-th quantization scaling circuit 326_h.

[0576] The components of input vector scaler 320 may perform the same operations as described above with reference to Figure 32 to Figure 34 The operation of the input vector scaler 220 described above is similar to that of the input vector scaler 220 described above. Figure 32 to Figure 34 The differences between input vector sealer 320 and input vector sealer 220 are described.

[0577] The multiplication scaling circuit 325 may sequentially receive a plurality of multiplication scaling coefficients MSC and a plurality of zero-point scaling coefficients ZPSC from the multiplication scaling coefficient buffer 310 .

[0578] The multiplication scaling circuit 325 may sequentially receive a plurality of common scaling coefficients CSC and a plurality of zero point values ​​ZPV from the common scaling coefficient buffer 370 .

[0579] Each of the first multiplication and scaling circuit 326_1 to the hth multiplication and scaling circuit 326_h can receive a different input vector. For example, the first quantization and scaling circuit 326_1 to the hth quantization and scaling circuit 326_h can respectively receive the first to the hth input vectors (e.g., to For a more detailed example, the first quantization scaling circuit 326_1 may sequentially receive x 11 to x 1n .

[0580] Each of the first to h-th quantization scaling circuits 326_1 to 326_h may perform quantization scaling on a received input vector based on a plurality of quantization scaling coefficients QSC and a plurality of zero point values ​​ZPV. For example, the first to h-th quantization scaling circuits 326_1 to 326_h may generate first to h-th quantization scaling input vectors (e.g., to ). The following will refer to Fig.57 The operation of the first to h-th quantization scaling circuits 326_1 to 326_h will be described in more detail.

[0581] Fig.56 is shown in more detail Fig.55 The operation of the multiplication and scaling circuit. Figures 1 to 4 , Fig.47 and Fig.48 as well as Fig.55 and Fig.56 , the multiplication scaling circuit 325 may repeatedly receive the zero-point scaling coefficient ZPSC and the first to Rth multiplication scaling coefficients MSC1 to MSCR.

[0582] The multiplication and scaling circuit 325 may sequentially receive a plurality of zero point values ​​ZPV and a plurality of common scaling coefficients CSC. For example, the multiplication and scaling circuit 325 may sequentially receive a first row vector (eg, )’s first zero point value ZPV1 and S r1 R instances of , and can sequentially receive the second row vector of the weight matrix WM (e.g., )’s second zero point value ZPV2 and s r2 R instances of .

[0583] The multiplication scaling circuit 325 may sequentially output multiple quantization scaling coefficients QSC and multiple zero point values ​​ZPV based on the order in which the multiple multiplication scaling coefficients MSC and multiple zero point scaling coefficients ZPSC are received and the order in which the multiple common scaling coefficients CSC and multiple zero point values ​​ZPV are received.

[0584] For example, the multiplication scaling circuit 325 can generate the first zero point value ZPV1 by multiplying the zero point scaling coefficient ZPSC by the first zero point value ZPV1; and by respectively multiplying the first to Rth multiplication scaling coefficients MSCR by the zero point scaling coefficient ZPSC. r1 The first row vector of the weight matrix WM is generated by multiplying the R instances of ) of multiple quantized scaling coefficients QSC (e.g., α 1_r1 To α R_r1 )(shown as diagonal stripes).

[0585] Similarly, the multiplication and scaling circuit 325 can generate the second zero point value ZPV2 by multiplying the zero point scaling coefficient ZPSC by the second zero point value ZPV2; and by multiplying the first multiplication scaling coefficient MSC1 to the Rth multiplication scaling coefficient MSCR by s r2 The second row vector of the weight matrix WM is generated by multiplying the R instances of ) of multiple quantized scaling coefficients QSC (e.g., α 1_r2 To α R_r2 )(shown as a dot pattern).

[0586] In this manner, the multiplication scaling circuit 325 can sequentially output a plurality of zero point values ​​ZPV and a plurality of quantization scaling coefficients QSC.

[0587] Fig.57 is shown in more detail Fig.55 In the following, for a more concise description, reference will be made to Figures 1 to 4 , Fig.47 and Fig.48 as well as Figure 55 to Figure 57 The operation of the first quantization scaling circuit 326_1 is representatively described. However, example embodiments are not limited thereto, and the second through h-th multiplication scaling circuits 326_2 through 326_h may also operate in a similar manner.

[0588] The first quantization scaling circuit 326_1 may receive a first input vector (eg, That is, the first quantization scaling circuit 326_1 may sequentially receive x 11 to x 1n .

[0589] The first quantization scaling circuit 326_1 may sequentially receive a plurality of zero point values ​​ZPV and a plurality of quantization scaling coefficients QSC. For example, the first quantization scaling circuit 326_1 may sequentially receive the above reference Fig.56 Described are multiple zero point values ​​ZPV and multiple quantization scaling coefficients QSC.

[0590] The first quantization scaling circuit 326_1 may scale the first input vector (eg, ) to generate a first quantized scaled input vector (eg, ).

[0591] For example, the first quantization scaling circuit 326_1 can be configured by setting x 11 With the first zero value ZPV1 and α 1_r1 To α R_r1 Multiply to generate x 11Multiple quantized scaled input elements QSIE. (shown as diagonal stripes)

[0592] Similarly, the first quantization scaling circuit 326_1 can be configured by converting x 12 With the second zero value ZPV2 and α 1_r2 To α R_r2 Multiply to generate x 12 Multiple quantized scaled input elements QSIE. (shown as diagonal stripes)

[0593] As a result, the first quantization scales the input vector (eg, ) may include a quantization scaling input element QSIE corresponding to the product of multiple zero point values ​​ZPV and multiple input elements.

[0594] Fig.58 is shown in more detail Fig.48 A diagram of the operation of an array of processing elements. Figures 1 to 4 and Figures 48 to 58 , the processing element array 350 may include a plurality of processing element rows PER. Each of the plurality of processing element rows PER may include a plurality of processing elements. However, hereinafter, for a more concise description, the operation of the first processing element row PER1 will be representatively described. However, example embodiments are not limited thereto.

[0595] The first processing element row PER1 may include processing elements PE11 to PE1m. The first processing element row PER1 may receive a first fixed-point multiplication scaling input vector (eg, ). For example, each of the processing elements PE11 to PE1m may sequentially receive a plurality of fixed-point quantized scaled input elements QSIEfxp. That is, the processing elements PE11 to PE1m may sequentially receive the fixed-point format of the above reference. Fig.51 , Fig.54 and Fig.57 In this case, some of the plurality of fixed-point quantized scaled input elements QSIEfxp received by each of the processing elements PE11 to PE1m may correspond to a product of a zero-point value ZPV and an input element.

[0596] Each of the processing elements PE11 to PE1m may receive a plurality of quantization sign bits QSB and a plurality of zero point correction bits ZPCB. For example, the processing element PE11 may receive a first plurality of quantization sign bits QSBs_1 and a plurality of zero point correction bits ZPCB. The processing element PE12 may receive a second plurality of quantization sign bits QSBs_2 and a plurality of zero point correction bits ZPCB.

[0597] For a more detailed example, the processing element PE11 may receive a zero point correction bit ZPCB and then receive a bit corresponding to x 11 R quantization sign bits QSB (for example, b 1_1_r1 To b 1_R_r1 ). Thereafter, the processing element PE11 may receive a zero point correction position ZPCB and then receive a value corresponding to x 12 R quantization sign bits QSB (for example, b 1_1_r2 To b 1_R_r2 ). In this way, the processing element PE11 can sequentially receive the first plurality of quantization sign bits QSBs_1 and the plurality of zero point correction bits ZPCB.

[0598] The processing element PE11 may calculate a fixed-point output element (eg, y′) based on the order in which the quantization sign bit QSB and the plurality of zero correction bits ZPCB are received and the order in which the plurality of fixed-point quantization scaling input elements QSIEfxp are received. 11 ). For example, similar to the reference above Fig.37 and Fig.38 As described, the processing element PE11 can calculate the fixed-point output element (e.g., y′) based on whether the quantization sign bit QSB and the zero correction bit ZPCB corresponding to the received fixed-point quantized scaled input element QSIEfxp represent “0” or “1”. 11 ).

[0599] Therefore, according to some example embodiments, the product of the zero point value ZPV and the input element may be calculated in the matrix multiplier 300. In this case, the matrix multiplication device MMD may be able to calculate the product of the input matrix XM and the weight matrix WM even if it does not include a separate multiplier for calculating the product of the zero point value ZPV and the input element.

[0600] In other words, according to some example embodiments, even if the uniform BCQ circuit UBC asymmetrically performs uniform binary coded quantization for the weight matrix WM, the matrix multiplier 300 may be able to calculate the product of the input matrix XM and the weight matrix with a small amount of calculation. Therefore, according to some example embodiments, the versatility of the matrix multiplication device MMD may be increased.

[0601] Fig.59 It is shown that the systolic array is implemented Fig.26 A block diagram of an array of processing elements. Figures 1 to 4 , Figures 23 to 48 and Fig.59 , the processing element array 250 may include a plurality of processing elements PE arranged in row and column directions. The plurality of processing elements PE may operate in a systolic array scheme.

[0602] The processing element array 250 may be implemented to sequentially propagate a plurality of fixed-point quantized scaled input elements QSIEfxp in a row direction. For example, the first processing element row PER1 may sequentially propagate a plurality of fixed-point quantized scaled input elements QSIEfxp in a row direction. ) in multiple fixed-point quantized scaled input elements QSIEfxp.

[0603] For a more detailed example, the processing element PE11 may receive one fixed-point quantization scaled input element QSIEfxp at a first time point. The processing element PE11 may transfer the fixed-point quantization scaled input element QSIEfxp to the processing element PE21 positioned adjacent to the processing element PE11 in the row direction at a second time point after the first time point. In this way, the processing elements PE included in the first processing element row PER1 may sequentially transfer a plurality of fixed-point quantization scaled input elements QSIEfxp provided from the first data type converter 230 to adjacent processing elements.

[0604] The processing element array 250 may be implemented to sequentially propagate a plurality of quantization sign bits QSB in a column direction. For example, the first processing element column PEC1 may sequentially propagate a first plurality of quantization sign bits QSBs_1 in a column direction.

[0605] For a more detailed example, the processing element PE11 may receive one quantization sign bit QSB at a first time point. The processing element PE11 may transmit the quantization sign bit QSB to the processing element PE12 positioned adjacent to the processing element PE11 in the column direction at a second time point after the first time point. In this way, the processing elements PE included in the first processing element column PEC1 may sequentially transmit the first plurality of quantization sign bits QSBs_1 provided from the quantization sign bit buffer 240 to the adjacent processing elements.

[0606] Multiple processing elements PE may generate different fixed-point output elements OEfxp. The processing element array 250 may be implemented to sequentially propagate the fixed-point output elements OEfxp in the column direction. For example, the fixed-point output element OEfxp (eg, y′) calculated by the processing element PE11 11 ) can be transmitted sequentially in the column direction. In this way, the fixed-point output element OEfxp can be transmitted to the second data type converter 260. The method of propagating the fixed-point output element OEfxp is similar to the method of transmitting the quantization sign bit QSB described above, so its detailed description will be omitted.

[0607] That is, each processing element included in the processing element array 250 may be implemented to receive one or more of the fixed-point output element OEfxp, the quantization sign bit QSB, and the fixed-point quantization scaling input element QSIEfxp from the adjacently placed processing element. On the other hand, each of the processing elements included in the processing element array 250 may be implemented to transmit one or more of the fixed-point output element OEfxp, the quantization sign bit QSB, and the fixed-point quantization scaling input element QSIEfxp to the adjacently placed processing element. Fig.60 A more detailed configuration of a processing element PE operating in a systolic array scheme will be described in more detail.

[0608] To explain more concisely, Fig.59 Various example embodiments are representatively shown in which the fixed-point output element OEfxp, the quantization sign bit QSB, and the fixed-point quantization scaling input element QSIEfxp are all propagated in a systolic array method, but the exemplary embodiments are not limited thereto. For example, the processing element array 250 may be implemented to propagate only one or two of the fixed-point output element OEfxp, the quantization sign bit QSB, and the fixed-point quantization scaling input element QSIEfxp in a systolic array scheme.

[0609] In an example embodiment, each of the processing elements included in the processing element array 250 may operate in response to the same control clock signal. In this case, each of the plurality of processing elements may propagate a fixed-point output element OEfxp, a quantization sign bit QSB, and / or a fixed-point quantization scaling input element QSIEfxp to another processing element at the same time point. However, example embodiments are not limited thereto.

[0610] For a more concise description, Fig.59 Representatively, example embodiments are shown, wherein reference is made to Figures 23 to 48 The processing element array 250 is described as operating in a systolic array scheme, but example embodiments are not limited thereto. For example, according to some embodiments, reference Figures 5 to 22 The processing element array 150 described or referenced Figures 49 to 58 The depicted processing element array 350 may also operate as a systolic array type.

[0611] Fig.60 is shown in more detail Fig.59 A diagram of the configuration of the processing element; reference Figures 1 to 4 , Figures 23 to 48 as well as Fig.59 and Fig.60The processing element PE may include an arithmetic logic unit ALU, an accumulation register REG_ACC, a quantization scaling input element register REG_QSIE, a quantization sign bit register REG_QSB and an output register REG_OUT. For a more concise description, the above reference will be omitted. Fig.38 A detailed description of the configuration and operation of the arithmetic logic unit ALU and the accumulator register REG_ACC is described.

[0612] Hereinafter, various exemplary embodiments are representatively described in which the arithmetic logic unit ALU, the accumulation register REG_ACC, the quantization scaling input element register REG_QSIE, the quantization sign bit register REG_QSB and the output register REG_OUT operate in response to the same control clock signal. However, exemplary embodiments are not limited thereto.

[0613] The quantization scaling input element register REG_QSIE may sequentially receive a plurality of quantization scaling input elements QSIEfxp and transmit the received quantization scaling input elements QSIEfxp to the adjacent processing element PE and the first input terminal TI1 after one cycle of the control clock signal has passed.

[0614] The quantization sign bit register REG_QSB may sequentially receive a plurality of quantization sign bits QSB. The quantization sign bit register REG_QSB may receive one quantization sign bit QSB and transmit the received one quantization sign bit QSB to the adjacent processing element PE and the second input terminal TI2 after one cycle of the control clock signal has passed.

[0615] The output register REG_OUT may receive the fixed-point output element OEfxp from the accumulator register REG_ACC. For example, the output register REG_OUT may transmit the fixed-point output element OEfxp to an adjacent processing element PE or the second data type converter 260.

[0616] That is, the output register REG_OUT can provide the fixed-point output element OEfxp to the output register of another adjacent processing element, or directly provide the fixed-point output element OEfxp to the second data type converter 260. For example, the fixed-point output element OEfxp (e.g., y′) calculated by the processing element PE11 h1 ) may be sequentially transmitted to the second data type converter 260 through the processing elements PE21 to PEh1. However, example embodiments are not limited thereto.

[0617] For a more concise description, Fig.60Various example embodiments in which each of the registers included in the processing element PE receives and outputs data at each cycle of the control clock signal are described in detail, but the example embodiments are not limited to a specific operation method of the registers in response to the control clock signal.

[0618] Fig.61 is a diagram showing various exemplary embodiments Figure 1 A diagram of the operation of the matrix multiplication device. Figures 1 to 48 and Fig.61 , the matrix multiplication device MMD can receive the full input matrix FXM. The full input matrix FXM can include the above reference Figures 1 to 48 The input matrix XM described. For example, the full input matrix FXM may include multiple input matrices XM.

[0619] The matrix multiplication device MMD can receive the full weight matrix FWM. The full weight matrix FWM can include the above reference Figures 1 to 48 The weight matrix WM described. For example, the full weight matrix FWM may include multiple weight matrices WM.

[0620] The uniform BCQ circuit UBC may perform a uniform binary coded quantization operation for each of the plurality of weight matrices WM included in the full weight matrix FWM. For example, the uniform BCQ circuit UBC may generate a plurality of quantization sign bits QSB, a plurality of common scaling coefficients CSC, and a plurality of scaling coefficients MSC from each of the plurality of weight matrices WM.

[0621] The matrix multiplier 100 may receive a plurality of quantization sign bits QSB, a plurality of common scaling coefficients CSC and a multiplication scaling coefficient MSC. The matrix multiplier 100 may perform matrix multiplication on a full input matrix FXM and a full weight matrix FWM based on the plurality of quantization sign bits QSB, a plurality of common scaling coefficients CSC and a multiplication scaling coefficient MSC.

[0622] The matrix multiplier 100 can perform matrix multiplication on the full input matrix FXM and the full weight matrix FWM by one of various tiling techniques. For example, the matrix multiplier 100 can calculate the full output matrix FYM by sequentially calculating the product of multiple input matrices XM and multiple weight matrices WM, and then combining the calculation results to calculate the full output matrix FYM.

[0623] Fig.62 It is shown Fig.61 An illustration of the full input matrix of . Figures 1 to 48 , Fig.61 and Fig.62, the full input matrix FXM may include a plurality of input matrices XM. In other words, the full input matrix FXM may be tiled with a plurality of input matrices XM arranged in the row direction and the column direction. Hereinafter, for a more concise description, the input matrix arranged in the i-th row and the j-th column of the full input matrix FXM will be referred to as "XM_ij".

[0624] In an exemplary embodiment, the above reference Figures 1 to 48 The described input matrix XM may be one of a plurality of input matrices XM included in the full input matrix FXM.

[0625] In an example embodiment, the multiple input matrices XM included in the full input matrix FXM can have the same row size and column size. For example, each of the multiple input matrices XM can include "n" input elements for each row. Each of the multiple input matrices XM can include "h" input elements for each column.

[0626] The row size of the full input matrix FXM can be an integer multiple of the row size of each of the multiple input matrices XM. For example, a row of the full input matrix FXM can include "N" input elements. In this case, "N" can be an integer multiple of "n".

[0627] The column size of the full input matrix FXM can be an integer multiple of the column size of each of the multiple input matrices XM. For example, a column of the full input matrix FXM can include "H" input elements. In this case, "H" can be an integer multiple of "h".

[0628] Fig.63 It is shown Fig.61 A diagram of the full weight matrix of . Figures 1 to 48 , Fig.61 and Fig.63 , the full weight matrix FXM may include multiple weight matrices WM. In some examples, the full weight matrix FWM may be tiled with multiple weight matrices WM arranged in the row direction and the column direction. In the following, for a more concise description, the weight matrix arranged in the i-th row and j-th column of the full weight matrix FWM will be referred to as "WM_ij".

[0629] In an exemplary embodiment, the above reference Figures 1 to 48 The described weight matrix WM may be one of a plurality of weight matrices WM included in the full input matrix FXM.

[0630] In an example embodiment, each of the multiple weight matrices WM included in the full weight matrix FWM may have the same row size and column size. For example, each of the multiple input matrices WM may include "m" weights for each row. Each of the multiple weight matrices WM may include "n" weights for each column.

[0631] The row size of the full weight matrix FWM may be an integer multiple of the row size of each of the plurality of input matrices WM. For example, a row of the full weight matrix FWM may include "M" weights. In this case, "M" may be an integer multiple of "m".

[0632] The column size of the full weight matrix FWM may be an integer multiple of the row size of each of the plurality of weight matrices WM. For example, a column of the full weight matrix FWM may include "N" weights. In this case, "N" may be an integer multiple of "n".

[0633] The uniform BCQ circuit UBC may perform a uniform binary coding quantization operation for each of the plurality of weight matrices WM included in the full weight matrix FWM. In this case, the plurality of common scaling coefficients CSC generated based on the weight matrix WM_11 may be different from the plurality of common scaling coefficients CSC generated based on the weight matrix WM_12. Similarly, the plurality of quantization sign bits QSB generated based on the weight matrix WM_11 may be different from the plurality of quantization sign bits QSB generated based on the weight matrix WM_12. The specific method in which the uniform BCQ circuit UBC performs a uniform binary coding quantization operation for each weight matrix WM is similar to the method described above with reference to Figures 1 to 48 The method described herein is not intended to be used herein, and thus its detailed description will be omitted.

[0634] Fig.64 It is shown Fig.61 A diagram of the full output matrix of . Figures 1 to 48 as well as Fig.61 and Fig.64 , the full output matrix FYM can correspond to the product of the full input matrix FXM and the full weight matrix FWM.

[0635] The full output matrix FYM may include a plurality of sub-matrices FYM_sub arranged in row and column directions. Hereinafter, for a more concise description, a sub-matrix arranged in the i-th row and j-th column of the full output matrix FYM will be referred to as "FYM_sub_ij".

[0636] Each of the multiple submatrices FYM_sub can have the same row size and column size. The row size of each of the multiple submatrices FYM_sub can be the same as the row size of the input matrix XM. The column size of each of the multiple submatrices FYM_sub can be the same as the column size of the weight matrix WM. For example, each of the multiple submatrices FYM_sub can include "m" output elements for each row. Each of the multiple submatrices FYM_sub can include "h" output elements for each column.

[0637] The row size of the full output matrix FYM may be the same as the row size of the full weight matrix FWM. For example, the row size of the full output matrix FYM may be "M".

[0638] The column size of the full output matrix FYM may be the same as the column size of the full input matrix FXM. For example, the column size of the full output matrix FYM may be "H".

[0639] The matrix multiplication device MMD may calculate the full output matrix FYM in units of submatrices FYM_sub. For example, the matrix multiplication device MMD may calculate a submatrix FYM_sub by respectively adding the products of a plurality of tiled input matrices XM and a plurality of tiled weight matrices WM.

[0640] For a more detailed example, when "N" is 3 times "n", the matrix multiplier 100 can calculate the submatrix FYM_sub_11 by sequentially calculating the product of the input matrix XM_11 and the weight matrix WM_11, the product of the input matrix XM_12 and the weight matrix WM_21, and the product of the input matrix XM_13 and the weight matrix WM_31 and adding them. In this case, each product of the tiled input matrix and the tiled weight matrix may correspond to the above reference Figures 1 to 48 The output matrix YM described above. In some examples, the matrix multiplication device MMD can calculate a first output matrix based on the product of the input matrix XM_11 and the weight matrix WM_11, calculate a second output matrix based on the product of the input matrix XM_12 and the weight matrix WM_21, and calculate a third output matrix based on the product of the input matrix XM_13 and the weight matrix WM_31. Then, the matrix multiplication device MMD can calculate a submatrix FYM_sub_11 by adding the first output matrix to the third output matrix described above. However, example embodiments are not limited thereto.

[0641] That is, the matrix multiplication device MMD can be implemented to calculate a submatrix (e.g., a portion of the full output matrix FYM) by accumulating multiple output matrices. For example, the matrix multiplication device MMD can be implemented to temporarily store multiple output matrices in a memory device (e.g., an external volatile memory device (e.g., an SRAM device)), and then accumulate the stored output matrices to calculate a submatrix. In this way, the matrix multiplication device MMD can sequentially calculate multiple submatrices FYM_sub to calculate the full output matrix FYM.

[0642] Fig.65 is a block diagram illustrating a neural processing system implemented according to an example embodiment. Fig.65, the neural processing system 2000 may include a central processing unit 2100, a neural processing unit 2200, a volatile memory device 2300, a non-volatile memory device 2400, and a user interface 2500. The central processing unit 2100, the neural processing unit 2200, the volatile memory device 2300, the volatile memory device 2400, and the user interface 2500 may be connected to each other via a bus BUS.

[0643] The central processing unit 2100 may control the overall operation of the neural processing system 2000. For example, the central processing unit 2100 may control each component of the neural processing system 2000 to operate an artificial intelligence model.

[0644] In an example embodiment, the artificial intelligence model executed by the neural processing system 2000 may be or include or be included in one or more of any type of artificial intelligence models, such as a language model, an image recognition model, an image generation model, a weather analysis model, etc. For example, the artificial intelligence model executed by the neural processing system 2000 may be or include or be included in one or more of any type of artificial intelligence models, such as GPT-3, GPT-4, Pangu, GShard, Megatron-LM, etc. However, example embodiments are not limited thereto.

[0645] In an example embodiment, the artificial intelligence model executed by the neural processing system 2000 may perform an inference operation and / or a training operation. However, example embodiments are not limited thereto.

[0646] Each artificial intelligence model may include multiple processing layers. Each of the multiple processing layers may be implemented to receive layer input data to generate layer output data. In this case, the generated layer output data may be used as layer input data for another processing layer. For example, the layer output data generated from the first processing layer may be used as layer input data for the second processing layer. Fig.66 Describe the AI ​​model and processing layers in more detail.

[0647] Each of the multiple processing layers can transform the layer input data into the layer output data based on the matrix multiplication calculation. For example, each of the multiple processing layers can generate an output matrix corresponding to the layer output data by multiplying the input matrix corresponding to the layer input data by the weight matrix. However, the scope of the present disclosure is not limited to this, and each of the multiple processing layers can generate output data by transforming the input matrix corresponding to the layer input data in any way. For example, each of the multiple processing layers can be implemented to generate layer output data by sequentially multiplying the input matrix corresponding to the layer input data by multiple weight matrices, or transforming the input matrix into layer output data based on any transformation parameters. In other words, the example embodiments are not limited to the specific manner in which each of the multiple processing layers transforms the layer input data.

[0648] The neural processing unit 2200 may include a matrix multiplier 2210. The matrix multiplier 2210 may perform at least some of the calculations included in the plurality of processing layers. For example, the matrix multiplier 2210 may perform matrix multiplication calculations included in the plurality of processing layers.

[0649] In an example embodiment, matrix multiplication calculations may account for the majority of the processing load required / used by the neural processing system 2000 to perform each of the multiple processing layers.

[0650] In an example embodiment, the matrix multiplier 2210 may be implemented as described above with reference to Figures 1 to 58 In this case, the neural processing unit 2200 can perform calculations included in multiple processing layers with a smaller amount of calculations. Therefore, the neural processing system 2000 including the matrix multiplication device MMD according to various example embodiments of the present disclosure can drive an artificial intelligence model at a faster speed and using less power.

[0651] The volatile memory device 2300 may be used as an operation memory of the neural processing unit 2200. For example, the volatile memory device 2300 may temporarily store data generated during the operation of the neural processing unit 2200.

[0652] In an example embodiment, the neural processing unit 2200 may access the volatile memory device 2300 to perform calculations included in a plurality of processing layers. For example, the neural processing unit 2200 may be implemented to read parameters stored in the volatile memory device 2300 to perform calculations for layer input data, or may be implemented to temporarily store intermediate data generated during calculations in the volatile memory device 2300.

[0653] In an example embodiment, the computation speed of the neural processing unit 2200 may be higher than the access speed of the neural processing unit 2200 to the volatile memory device 2300. Therefore, due to the communication speed between the neural processing unit 2200 and the volatile memory device 2300, the operation speed of the artificial intelligence model may be bottlenecked.

[0654] In an exemplary embodiment, when the matrix multiplication device 2210 is implemented as above referenced Figures 1 to 58 When the matrix multiplication device MMD is described, an output element can be calculated based on a processing element PE included in the matrix multiplier MMD. In this case, since an output element can be generated even without combining the output results of multiple processing elements PE, the number of accesses to the volatile memory device 2300 of the neural processing unit 2200 can be reduced or minimized, and the artificial intelligence model can be driven based on a smaller size processing element array (e.g., fewer processing elements (PE)). Therefore, according to some example embodiments, the operating speed of the artificial intelligence model can be increased, and the production cost of the neural processing system 2000 can be reduced or minimized.

[0655] In an exemplary embodiment, when the matrix multiplication device 2210 is implemented as above referenced Figures 1 to 58 When describing the matrix multiplication device MMD, each processing element PE may not include a "1-bit adder" for performing a multiplication operation on the multiplication scaling coefficient MSC. In some examples, according to some example embodiments, the result of scaling the input elements by the multiplication scaling circuit is provided to the processing element array, thereby reducing the size and production cost of each processing element PE included in the matrix multiplication device MMD.

[0656] In example embodiments, the volatile memory device 2300 may be implemented using any type of volatile memory, such as one or more of dynamic random access memory (DRAM), static random access memory (SRAM), and the like.

[0657] In example embodiments, the volatile memory device 2300 may be used as a buffer memory, an operation memory, or a cache memory of the central processing unit 2100. However, example embodiments are not limited thereto.

[0658] Volatile memory device 2400 may store data used for the operation of neural processing system 2000. For example, volatile memory device 2400 may store various types of data, such as parameters for an operating system (OS) of neural processing system 2000, parameters for driving an artificial intelligence model, etc. However, example embodiments are not limited thereto.

[0659] The central processing unit 2100 may communicate with, for example, one or more users through the user interface 2500. The central processing unit 2100 may provide the model input data provided by the user to the volatile memory device 2300 or the neural processing unit 2200 through the user interface 2500. The central processing unit 2100 may return the model output data generated by the artificial intelligence model based on the model input data to the user through the user interface 2500.

[0660] Fig.66 It is shown by Fig.65 Block diagram of an artificial intelligence model driven by a neural processing system. Fig.65 and Fig.66 , the neural processing system 2000 can drive the artificial intelligence model AIM.

[0661] The artificial intelligence model AIM may receive model input data MID. The artificial intelligence model AIM may include first to Lth processing layers PL_1 to PL_L.

[0662] The artificial intelligence model AIM can generate model output data MOD by sequentially converting model input data MID through the first processing layer PL_1 to the Lth processing layer PL_L. For example, the first processing layer PL_1 can receive the model input data MID and can generate the second layer input data LID_2. The second processing layer PL_2 can receive the second layer input data LID_2 and can generate the third layer input data LID_3. In this way, the Lth processing layer PL_L can receive the Lth layer input data LID_L and can generate the model output data MOD.

[0663] Each of the first processing layer PL_1 to the Lth processing layer PL_L may transform (or convert) the received data into data to be output by various types of calculations. For example, a matrix multiplication calculation may be included in the calculation performed by the first processing layer PL_1 to transform the model input data MID into the second layer input data LID_2. Similarly, each of the first processing layer PL_1 to the Lth processing layer PL_L may have to perform a matrix multiplication calculation to transform the received layer input data. However, the scope is not limited to this, and some of the processing layers in the first processing layer PL_1 to the Lth processing layer PL_L may not perform a matrix multiplication operation.

[0664] In example embodiments, the matrix multiplication calculation performed by each of the first to L-th processing layers PL_1 to PL_L may be performed by the matrix multiplication device 2210 .

[0665] In an exemplary embodiment, if the matrix multiplier 2210 is implemented as above referenced Figures 1 to 48The matrix multiplication device MMD described above can output the result of the matrix multiplication calculation at a faster speed. Therefore, according to various exemplary embodiments, the operation speed of the artificial intelligence model AIM can be improved.

[0666] For a more concise description, Fig.66 Various example embodiments including a plurality of processing layers in which the artificial intelligence model AIM operates in series are representatively described in the specification, but the example embodiments are not limited thereto. For example, the artificial intelligence model AIM may also include processing layers that operate in parallel with at least some of the first processing layer PL_1 to the Lth processing layer PL_L described above. For example, the example embodiments are not limited to the specific implementation method of the artificial intelligence model AIM.

[0667] Any of the elements and / or functional blocks disclosed above may include or be implemented in a processing circuit, such as hardware including a logic circuit; a hardware / software combination, such as a processor executing software; or a combination thereof. For example, the processing circuit may more specifically include, but is not limited to, a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a system on a chip (SoC), a programmable logic unit, a microprocessor, an application specific integrated circuit (ASIC), etc. The processing circuit may include electrical components, such as at least one of a transistor, a resistor, a capacitor, etc. The processing circuit may include electrical components, such as a logic gate, which includes at least one of an AND gate, an OR gate, a NAND gate, a NOT gate, etc.

[0668] The content described above is a specific example embodiment for realizing various inventive concepts. The inventive concept may include not only the example embodiment described above, but also other example embodiments that can be simply changed in design or easily modified. In addition, the inventive concept may also include technologies that can be easily modified and implemented using various example embodiments. Therefore, the scope of the inventive concept should not be limited to the example embodiments described above, but should be defined by the appended claims and the claims and equivalents of the present disclosure. In addition, the example embodiments are not necessarily mutually exclusive. For example, some example embodiments may include one or more features described with reference to one or more drawings, and may also include one or more other features described with reference to one or more other drawings.

Claims

1. A matrix multiplier, comprising: an input vector sealer configured to generate a first quantized scaled input vector based on the first input vector, a plurality of common scaling coefficients, and first to Rth multiplication scaling coefficients, where R is an integer greater than or equal to 2; A first data type converter configured to generate a first fixed-point quantized scaled input vector based on the first quantized scaled input vector; an array of processing elements, comprising a first processing element and a second processing element, wherein the first processing element is configured to generate a first fixed-point output element based on the first fixed-point quantization scaled input vector and a first plurality of quantization sign bits, and the second processing element is configured to generate a second fixed-point output element based on the first fixed-point quantization scaled input vector and a second plurality of quantization sign bits; as well as A second data type converter is configured to generate a first output element and a second output element by converting the data types of the first fixed-point output element and the second fixed-point output element, respectively, and is configured to output a first output vector including the first output element and the second output element.

2. The matrix multiplier according to claim 1, wherein: The first input vector includes a first input element and a second input element, and The first quantized scaling input vector comprises: a first plurality of quantized scaled input elements configured to be generated based on the first input element, a first common scale factor corresponding to one of the plurality of common scale factors, and the first to Rth multiplicative scale factors; and A second plurality of quantized scaled input elements is configured to be generated based on the second input elements, a second common scale factor corresponding to one of the plurality of common scale factors, and the first to Rth multiplicative scale factors.

3. The matrix multiplier according to claim 2, wherein: The input vector scaler comprises: a multiplication scaling circuit configured to generate first to Rth multiplication scaling input elements based on the first input element and the first to Rth multiplication scaling coefficients, and configured to generate (R+1)th to (2R)th multiplication scaling input elements based on the second input element and the first to Rth multiplication scaling coefficients; and A common scaling circuit is configured to generate the first plurality of quantized scaled input elements based on the product of the first common scaling coefficient and each of the first to R-th multiplication scaled input elements, and is configured to generate the second plurality of quantized scaled input elements based on the product of the second common scaling coefficient and each of the (R+1)-th to (2R)-th multiplication scaled input elements.

4. The matrix multiplier according to claim 2, wherein: The input vector scaler comprises: a common scaling circuit configured to generate a first common scaling input element based on a product of the first input element and the first common scaling factor, and configured to generate a second common scaling input element based on a product of the second input element and the second common scaling factor; and A multiplication scaling circuit is configured to generate the first plurality of quantized scaling input elements based on the product of the first common scaling input element and each of the first to R-th multiplication scaling coefficients, and is configured to generate the second plurality of quantized scaling input elements based on the product of the second common scaling input element and each of the first to R-th multiplication scaling coefficients.

5. The matrix multiplier according to claim 2, wherein: The input vector scaler comprises: a multiplication scaling circuit configured to generate first to Rth quantization scaling factors based on a product of the first common scaling factor and each of the first to Rth multiplication scaling factors, and configured to generate (R+1)th to (2R)th quantization scaling factors based on a product of the second common scaling factor and each of the first to Rth multiplication scaling factors; and A first quantization scaling circuit is configured to generate the first plurality of quantization scaling input elements based on the first input element and the first to R-th quantization scaling coefficients, and is configured to generate the second plurality of quantization scaling input elements based on the second input element and the (R+1)-th to (2R)-th quantization scaling coefficients.

6. The matrix multiplier of claim 5, wherein: The input vector sealer is further configured to generate a second quantized scaled input vector based on a second input vector, the plurality of common scaling coefficients, and the first to R-th multiplication scaling coefficients, The first data type converter is further configured to generate a second fixed-point quantized scaled input vector based on the second quantized scaled input vector, The processing element array further includes a third processing element and a fourth processing element, wherein the third processing element is configured to generate a third fixed-point output element based on the second fixed-point quantization scaled input vector and the first plurality of quantization sign bits, and the fourth processing element is configured to generate a fourth fixed-point output element based on the second fixed-point quantization scaled input vector and the second plurality of quantization sign bits, and The second data type converter is further configured to generate a third output element and a fourth output element by converting the data types of the third fixed-point output element and the fourth fixed-point output element, respectively, and is configured to output a second output vector including the third output element and the fourth output element.

7. The matrix multiplier of claim 6, wherein: The second input vector includes a third input element and a fourth input element, and The input vector scaler also includes a second quantization scaling circuit, which is configured to generate a third plurality of quantization scaling input elements included in the second quantization scaling input vector based on the third input element and the first to R-th quantization scaling coefficients, and is configured to generate a fourth plurality of quantization scaling input elements included in the second quantization scaling input vector based on the fourth input element and the (R+1)-th to (2R)-th quantization scaling coefficients.

8. The matrix multiplier of claim 6, wherein: The first processing elements are arranged in a first processing element row and a first processing element column of the processing element array, The second processing elements are arranged in the first processing element row and the second processing element column of the processing element array, The third processing element is arranged in a second processing element row and the first processing element column of the processing element array, and The fourth processing element is arranged in the second processing element row and the second processing element column of the processing element array.

9. The matrix multiplier of claim 5, wherein: The first common scaling factor has a floating point data type, and The multiplication scaling circuit is configured to generate the first to Rth quantization scaling coefficients by changing a value of an exponential part of the first common scaling coefficient based on the first to Rth multiplication scaling coefficients.

10. The matrix multiplier of claim 1, wherein: The first to R-th multiplicative scaling factors form a geometric sequence having a common ratio of 2.

11. The matrix multiplier of claim 1, wherein: The input vector sealer is configured to generate the first quantized scaled input vector further based on a plurality of zero points, The first processing element is configured to generate the first fixed-point output element further based on a plurality of zero-point correction bits, wherein each zero-point correction bit corresponds to the plurality of zero points, and The second processing element is configured to generate the second fixed-point output element based on the plurality of zero-point correction bits.

12. A method for operating a matrix multiplication device, comprising: Receiving a first weight to an Nth weight from an external device, where N is an integer greater than or equal to 2; generating first to Nth common scaling coefficients, first to Rth multiplication scaling coefficients, and first to (N×R)th quantization sign bits by performing uniform binary coded quantization on the first to Nth weights, wherein R is an integer greater than or equal to 2; receiving a first input element to an Nth input element from the external device; generating first to (N×R)th quantized scaled input elements by performing quantized scaling on the first to Nth input elements based on the first to Nth common scaling coefficients and the first to Rth multiplicative scaling coefficients; as well as A first output element generated based on the first to (N×R)th quantization sign bits and the first to (N×R)th quantization scaling input elements is output.

13. The operating method according to claim 12, wherein: Generating the first quantized scaling input element to the (N×R)th quantized scaling input element comprises: generating first to (N×R)th quantized scaling coefficients by multiplying each of the first to Rth multiplicative scaling coefficients by the first to Nth common scaling coefficients; and The first to (N×R)th quantization scaling input elements are generated by multiplying each of the first to (N×R)th quantization scaling coefficients by a corresponding one of the first to Nth input elements.

14. The operating method according to claim 13, wherein: The first to Nth common scaling factors have floating point data types, and The first to R-th multiplication scaling factors differ from each other by a power of 2.

15. The operating method according to claim 12, wherein: Outputting the first output element includes: Generate first to (N×R)th fixed-point quantized scaling input elements by converting data types of the first to (N×R)th quantized scaling input elements to fixed points; calculating a first fixed-point output element based on the first to (N×R)th quantization sign bits and the first to (N×R)th fixed-point quantization scaling input elements; and The first output element generated by converting the data type of the first fixed-point output element to a floating point is output.

16. The operating method according to claim 15, wherein The first fixed-point output element corresponds to a value obtained by subtracting the second value from the first value, The first value is obtained by adding a first plurality of quantization scaling input elements from among the first quantization scaling input element to the (N×R)th quantization scaling input element, wherein the first plurality of quantization scaling input elements correspond to a first plurality of quantization sign bits representing “0” from among the first quantization sign bit to the (N×R)th quantization sign bit, and The second value is obtained by adding a second plurality of quantization scaling input elements from the first quantization scaling input element to the (N×R)th quantization scaling input element, wherein the second plurality of quantization scaling input elements correspond to a second plurality of quantization sign bits representing “1” from the first quantization sign bit to the (N×R)th quantization sign bit.

17. A matrix multiplier comprising: an input vector sealer configured to generate a first multiplication scaled input vector based on the first input vector and first to Rth multiplication scale factors, where R is an integer greater than or equal to 2; a first data type converter configured to generate a first fixed-point multiplication-scaled input vector based on the first multiplication-scaled input vector; an array of processing elements, comprising a first processing element configured to generate a first fixed-point partial product based on the first fixed-point multiplication scaled input vector and a first plurality of quantization sign bits; a second data type converter configured to generate a first partial product by converting a data type of the first fixed-point partial product; as well as A common sealer is configured to generate a first output element based on a product of the first partial product and a first common scaling factor, and is configured to output a first output vector including the first output element.

18. The matrix multiplier of claim 17, wherein: The first input vector includes a first input element to an Nth input element, where N is an integer greater than or equal to 2, The input vector scaler includes a multiplication scaling circuit, which is configured to generate first to (N×R)th multiplication scaling input elements included in the first multiplication scaling input vector by multiplying each of the first to Nth input elements with the first to Rth multiplication scaling coefficients.

19. The matrix multiplier of claim 17, wherein: The array of processing elements further includes a second processing element configured to generate a second fixed-point partial product based on the first fixed-point multiplication scaled input vector and a second plurality of quantized sign bits, The second data type converter is further configured to generate a second partial product by converting a data type of the second fixed-point partial product, and The common sealer is further configured to generate a second output element included in the first output vector based on a product of the second partial product and a second common scaling coefficient.

20. The matrix multiplier of claim 17, wherein: The first fixed-point multiplication-scaled input vector includes a first plurality of fixed-point multiplication-scaled input elements and a second plurality of fixed-point multiplication-scaled input elements, and The first processing element is configured to generate the first fixed-point partial product based on subtracting a second value from a first value, wherein the first value corresponds to the sum of the first plurality of fixed-point multiplication scaling input elements, and the first plurality of fixed-point multiplication scaling input elements respectively correspond to quantization sign bits representing “0” among the first plurality of quantization sign bits, The second value corresponds to the sum of the second plurality of fixed-point multiplication scaling input elements, and the second plurality of fixed-point multiplication scaling input elements respectively correspond to quantization sign bits representing “1” among the first plurality of quantization sign bits.

Images