Hardware accelerator for lookup table-based general matrix operations

Abstract

This provides a hardware accelerator for lookup table-based general matrix operations. [Solution] The hardware accelerator 100 includes a weight buffer that stores a weight matrix in which the weights of the target model have been quantized, an input buffer that stores an input matrix including floating-point-based input activation values ​​for the target model, and a matrix processing unit 130 that performs matrix operations between the weight matrix and the input matrix. The matrix processing unit includes a lookup table generator that calculates each calculation result value corresponding to all possible cases between the quantum weights and input activation values ​​that can occur during matrix operations and generates a lookup table containing the calculation result values, and a plurality of processing elements that extract calculation result values ​​corresponding to the quantum weights from the lookup table during matrix operations and generate a partial sum.

Description

[Technical Field]

[0001] This invention relates to a hardware accelerator capable of performing general matrix operations between floating-point numbers and integers based on a lookup table. [Background technology]

[0002] Generative Language Models (LPs) are attracting attention in fields such as natural language understanding, natural language generation, and reasoning. However, LPs are generally large in scale, which presents many challenges in their application to services. Even practical LPs often have billions of weights, making them burdensome in terms of cost, resources, and call time when provided to users. Therefore, research is being conducted extensively to lighten practical LPs for economical services.

[0003] One such method is post-training quantization. Post-training quantization simplifies the weight reduction process by reducing the bit precision of the weights of a pre-trained generative language model to 8 bits or 4 bits without fine-tuning the weights. This method is attracting attention because it is simple and has a short processing time.

[0004] However, when quantizing a generative language model, it is common to quantize only the weights of the model. In this case, the weights are converted to integers through quantization, but the input activation values ​​used in matrix operations with these weights have a problem in that they maintain the existing floating-point format. [Overview of the project] [Problems that the invention aims to solve]

[0005] The present invention aims to provide a hardware accelerator for lookup table-based general matrix operations that can provide general matrix operations between floating-point and integer numbers based on a lookup table. [Means for solving the problem]

[0006] A hardware accelerator according to one embodiment of the present invention relates to a hardware accelerator for a look-up table (LUT) based general matrix multiply (GEMM), and includes a weight buffer that stores at least a portion of a weight matrix containing integer quantum weights of a target model, at least a portion of an input matrix containing floating-point number-based input activation values ​​for the target model, and a matrix processing unit (MPU) that performs matrix operations between the weight matrix and the input matrix through communication between the weight buffer and the input buffer, wherein the matrix processing unit includes a look-up table generator that calculates the calculation result values ​​corresponding to all possible cases between the quantum weights and the input activation values ​​that can occur during the matrix operation and generates a look-up table containing the calculation result values, and a plurality of processing elements (PEs) that extract the calculation result values ​​corresponding to the quantum weights from the look-up table during the matrix operation and generate a partial sum.

[0007] Furthermore, the means of solving the above-mentioned problems do not constitute a complete list of all features of the present invention. The various features of the present invention and their advantages and effects can be understood in more detail by referring to the following specific embodiments. [Effects of the Invention]

[0008] According to a hardware accelerator for lookup table-based general matrix operations according to one embodiment of the present invention, general matrix operations between floating-point and integer numbers can be provided based on a lookup table. [Brief explanation of the drawing]

[0009] [Figure 1] A block diagram showing a hardware accelerator for lookup table-based general matrix operations according to one embodiment of the present invention. [Figure 2] A block diagram showing multiple read accumulators included within a processing element according to one embodiment of the present invention. [Figure 3] This is a schematic diagram illustrating matrix operations between a binary matrix and an input matrix according to one embodiment of the present invention. [Figure 4] This is a schematic diagram illustrating lookup table generation and matrix operations using the lookup table according to one embodiment of the present invention. [Figure 5] This is a schematic diagram illustrating the generation of a flip-flop-based lookup table according to one embodiment of the present invention. [Figure 6] This is a schematic diagram showing a half-lookup table according to one embodiment of the present invention. [Figure 7] This is a schematic diagram illustrating the generation of a lookup table using one embodiment of the present invention. [Figure 8] A block diagram showing a hardware accelerator based on one embodiment of the present invention. [Modes for carrying out the invention]

[0010] Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings. However, regardless of the reference numbers, the same or similar components will be given the same reference numbers, and duplicate descriptions thereof will be omitted. In the following description, the suffixes "module" and "unit" for components are given or mixed only for ease of specification writing, and do not have meanings or roles that distinguish them from each other. That is, the term "unit" used in the present invention means a hardware component such as software, FPGA, or ASIC, and the "unit" plays a certain role. However, the "unit" is not limited to the meaning of software or hardware. The "unit" may be configured to be in an addressable storage medium or may be configured to reproduce one or more processors. Thus, by way of example, the "unit" includes components such as software components, object-oriented software components, class components, and task components, and processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "units" may be combined into a smaller number of components and "units" or further separated into additional components and "units".

[0011] Also, in describing the embodiments disclosed in this specification, if it is determined that a specific description of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. Also, the accompanying drawings are only for facilitating understanding of the embodiments disclosed in this specification, and the technical idea disclosed in this specification is not limited by the accompanying drawings, and should be understood to include all modifications, equivalents, or alternatives included in the idea and technical scope of the present invention.

[0012] A generative language model performs natural language processing (NLP) and can generate a new text corresponding to the input text as the output text. Generally, generative language models can be applied in various fields such as text generation in speech-to-text (STT), translation, chatbots, document summarization, and question answering.

[0013] A generative language model can be implemented as a probability model that predicts the next token from a sequence of input tokens corresponding to the input text. After learning the sequence of tokens contained in the text through training data, it can generate the output text by calculating the conditional probability for the next token and selecting the most likely token.

[0014] At this time, a generative language model can be implemented by leveraging recurrent neural networks (RNNs), long short-term memory (LSTMs), gated recurrent units (GRUs), transformers, etc. Recently, it has become common to implement generative language models using transformers. There are models such as GPT (Generative Pre-trained Transformer), BERT (Bidirectional Encoder Representations from Transformer), and T5 (Text-To-Text Transfer Transformer) in generative language models.

[0015] On the other hand, since generative language models are generally very large in scale, they require a lot of hardware and energy resources during learning or inference, and there are problems such as difficulty in distribution and maintenance. Therefore, for an economical and efficient service, it is necessary to lightweight the generative language model, and a method of quantizing the weights of the generative language model for lightweighting has been proposed.

[0016] When quantizing a generative language model, both the model's weights and input activations can be quantized. However, because weights occupy a large proportion of the model's size, maintaining accuracy becomes difficult when quantizing input activations. Therefore, it is common practice to quantize only the weights. In this case, the weights are converted to integers through quantization, but the input activations, which perform matrix operations with the weights, maintain their existing floating-point (FP) format. Consequently, FP-INT operations are necessary to utilize the generative language model.

[0017] Traditionally, since GPUs and other devices that execute generative language models lack physical FP-INT operators, acceleration kernels have been proposed to support such operations. However, these conventional acceleration kernels have problems such as memory bank conflicts and inefficient operations due to the application of bit serial techniques.

[0018] In contrast, a hardware accelerator for general matrix operations based on a lookup table according to one embodiment of the present invention can increase the efficiency of bit serial structures through lookup table-based operations and, because it includes a hardware structure specialized for lookup table-based matrix operations, it can solve problems such as memory bank collisions. A hardware accelerator according to one embodiment of the present invention will be described below with reference to Figure 1.

[0019] Figure 1 is a block diagram showing a hardware accelerator for lookup table-based general matrix operations according to one embodiment of the present invention.

[0020] Referring to Figure 1, a hardware accelerator 100 according to one embodiment of the present invention may include a weight buffer 110, an input buffer 120, a matrix processing unit (MPU) 130, a scale and accumulator 140, and an output buffer 150.

[0021] The weight buffer 110 can store a weight matrix containing integer quantum weights (W) obtained by quantizing the weights of a target model, such as a generative language model. That is, the target model can be quantized to generate quantum weights in advance, and at least a portion of these quantum weights can be sequentially provided and stored as a weight matrix.

[0022] Depending on the embodiment, it is also possible to generate a weight matrix by quantizing the weights of the target model based on binary code quantization (BCQ). When binary code quantization is used, the weights of the target model can be represented by a linear combination of q binary weights W. In this case, each binary weight W may be multiplied by a scale factor and an offset O may be added. That is, when binary code quantization is performed, q binary matrices representing one target model can be generated, and the weight buffer 110 can store q binary matrices containing the binary weights W as a weight matrix. In this case, each binary weight W can be set to {-1, -1} or {0, 1}, so when binary code quantization is applied, matrix operations such as dot products can be converted to simple addition or subtraction.

[0023] The input buffer 120 can store an input matrix containing floating-point-based input activation values ​​for the target model. Here, at least a portion of the input matrix can be sequentially stored in the input buffer 120 for matrix operations with the weight matrix.

[0024] In other words, the size of the overall weight matrix for the target model may be larger than the capacity of the weight buffer 110, or the size of the input matrix may be larger than the capacity of the input buffer 120. In this case, the hardware accelerator 100 can apply tiling to divide the weight matrix or input matrix of the target model into processable units called tiles, and then sequentially load these tiles into the respective weight buffers 110 and input buffers 120 to perform matrix operations on the whole.

[0025] The matrix processing unit 130 can perform matrix operations between the weight matrix and the input matrix via communication with the weight buffer 110 and the input buffer 120. If quantum weights in the weight buffer 110 are generated via binary coding quantization, the matrix operation with the input matrix can be converted to addition or subtraction between input activation values. In this case, instead of performing FP-INT operations during the matrix operation, it is possible to perform FP-FP operations.

[0026] Furthermore, the matrix processing unit 130 can pre-perform calculations for each input activation value and store them in a lookup table. Then, instead of performing the actual calculations on the input activation values, it can read the calculation results from the lookup table. Specifically, the matrix processing unit 130 can include a lookup table generator 131 and a plurality of processing elements (PEs) 132.

[0027] The lookup table generator 131 can determine the calculation result value corresponding to all possible cases between the quantum weights and input activation values ​​that can occur during matrix operations, and generate a lookup table containing said calculation result values.

[0028] Here, the lookup table generator 131 can utilize a hyperparameter (μ) to generate the lookup table. The hyperparameter (μ) may be a hyperparameter and can correspond to a coefficient of the binary weight required to generate each key value of the lookup table. In this case, as the hyperparameter increases, the number of operations that a single lookup table can substitute can increase, but as the hyperparameter increases, the size of the lookup table also increases. That is, as the hyperparameter increases, the memory size occupied by a single lookup table can increase, and in this case, the increase in memory size increases exponentially. Therefore, it is necessary to optimize and set the hyperparameter.

[0029] For example, binary matrix B ∈ {-1,1} 4×6 And, input matrix x∈R 6 It can be generated as follows:

number

[0030] Here, B·x T When performing matrix operations, if the hyperparameter is 3, operations such as (x1-x2-x3) and (-x4-x5+x6) can appear repeatedly. By pre-calculating these repeated operations and storing them in a lookup table, unnecessary repetition of operations can be prevented. In this case, a lookup table can be generated as shown in Table 1 below. [Table 1]

[0031] In other words, it is possible to generate a full lookup table by setting a key value for each binary weight combination (binary pattern), and then generating a value by adding or subtracting the input activation values ​​corresponding to each key value.

[0032] Here, we can consider that the lookup table generator 131 divides the binary matrix into small regions containing columns of hyperparameters (μ), and then generates a lookup table corresponding to each of those small regions. In this case, the lookup table generator 131 calculates the number of all possible combinations of the binary weights of the hyperparameters (μ) (2 μ Each key value corresponding to the given value can be assigned, and the result of a matrix operation between the binary weights of the hyperparameters corresponding to that key value and the input matrix can be generated and assigned as a value. Subsequently, a lookup table can be generated from each key-value pair.

[0033] On the other hand, referring to Figure 1, the lookup table generator 131 can be provided for each sub-region, and when an input matrix is ​​input from the input buffer 120, it can generate a lookup table corresponding to that sub-region. That is, once the hyperparameter (μ) is determined, the lookup table generator 131 generates the possible overall key values ​​(2 μ Since the values ​​corresponding to each key value can be calculated and the corresponding value values ​​can be generated by referring to the input matrix, each lookup table can also be generated without referring to the binary matrix.

[0034] Multiple processing elements 132 may be included within the matrix processing unit 130, and during matrix operations, calculation result values ​​corresponding to quantum weights can be extracted from a lookup table to generate a partial sum. Here, multiple processing elements 132 can be arranged to have a systolic array structure, and for data accumulation, the partial sum of each processing element 132 can be input to other processing elements.

[0035] Referring to Figure 1, each sub-region may include a lookup table generator 131, and once a lookup table L is generated, the processing elements 131 at the positions corresponding to that sub-region in the systolic array structure can share the same lookup table L from the lookup table generator 131.

[0036] Specifically, processing element 132 can generate a key value from the pattern of binary weights W of hyperparameters that appear in the binary matrix, and use this key value to extract the corresponding value, which is the calculation result value, from the lookup table L. Thereafter, the calculation result value can be added to the partial sum P transmitted from the previous processing element 132 to generate a partial sum P, and the generated partial sum P can be transmitted to the next processing element 132. In this way, the partial sums (P) are accumulated to finally generate the output data. Here, as shown in Figure 1, processing element 132 can receive the lookup table L in the column direction and transmit the partial sum P in the row direction.

[0037] Furthermore, referring to Figure 2(a), each processing element 132 can include multiple Read Accumulators (RACs) R1, R2, and R3. Here, each Read Accumulator R1, R2, and R3 can be provided corresponding to each row assigned to the processing element and can generate a partial sum corresponding to that row.

[0038] Here, each of the read accumulators R1, R2, and R3 can share the lookup table L of the processing element 132, and in some embodiments, multiple read accumulators R1, R2, and R3 can access the lookup table L simultaneously. This allows the read accumulators R1, R2, and R3 to simultaneously read the calculation results corresponding to the binary weights contained in each row from the lookup table L, thereby generating the respective partial sums corresponding to each row in parallel. Here, when each of the read accumulators R1, R2, and R3 receives the input of partial sums P11, P12, and P13 from the previous read accumulator, they can generate partial sums P21, P22, and P23 by adding the calculation result values ​​and transmit them to the next read accumulator.

[0039] Furthermore, referring to Figure 2(b), each read accumulator R1 can extract the calculation result value using a multiplexer M. That is, the total value value contained in the lookup table L can be input to the multiplexer M, and the read accumulator R1 can input a key value using the enable signal of the multiplexer M and receive the value value corresponding to that key value. Here, the key value can be generated from the binary weights W of the hyperparameters extracted from the weight matrix, and the extracted calculation result value can be added to the previous partial sum P11 to generate the partial sum P12 of the read accumulator R1.

[0040] FIG. 3 shows the matrix operation B·x between the binary matrix B i and the input matrix x T to generate the output vector y. Here, the lookup table generator 131 can be regarded as dividing the binary matrix Bi into respective small regions each including two columns, and then, as shown in FIG. 4(a), respective lookup tables LUT0, LUT1, LUT2, LUT3 corresponding to the respective small regions can be generated.

[0041] Referring to FIG. 4(b), the first processing element corresponding to the first lookup table LUT0 can be regarded as including four read accumulators, and the remaining lookup tables LUT1, LUT2, LUT3 can also be regarded as including respective corresponding processing elements and four read accumulators included in the processing elements. Therefore, each read accumulator for the first lookup table LUT0 can receive two binary weight inputs corresponding to respective rows in the weight matrix.

[0042] Specifically, the first read accumulator in the first processing element can input 3 as a key value. Referring to the first lookup table LUT0, the operation result value corresponding to the value is x0 + x1, which corresponds to y 0 0. In a similar manner, the second read accumulator can input 2 as a key value corresponding to the next row. Referring to the first lookup table LUT0, the value is x0 - x1, which corresponds to y 0 1.

[0043] Here, the operation result value y 0 0 generated by the first read accumulator can be transmitted to the second read accumulator, and the second read accumulator is y 0 0 + y 1A partial sum can be generated by performing a y operation. The second read accumulator can then transmit the partial sum to the third read accumulator, which then performs a y operation on the partial sum. 2 By adding 0, the partial sum y 0 0+y 1 0+y 2 0 can be calculated and transmitted to the fourth read accumulator. Finally, the fourth read accumulator receives the transmitted partial sum y 0 0+y 1 0+y 2 0 to y 3 Add 0 to get the partial sum y 0 0+y 1 0+y 2 0+y 3 It is possible to generate 0, and referring to Figure 4(c), the last partial sum corresponds to the output vector y0. In a similar manner, the output vectors y1, y2, and y3 can be obtained by calculating partial sums from the remaining processing elements, and the final output vector can be generated. Here, each can be processed in parallel, so the output vector can be obtained quickly.

[0044] The scale and accumulator 140 can receive partial sums from each processing element 132 located at the end of each row in the systolic array structure. In this case, the scale and accumulator 140 can apply a scaling factor and offset O to the partial sums to generate an output matrix corresponding to the output value, and the generated output matrix can be stored in the output buffer (Psum buffer) 150. That is, when binary coding quantization is applied, after performing matrix operations on the binary matrix and the input matrix, it is necessary to perform operations such as the Hadamard product of the scaling factor and the addition of the offset, and these operations can be performed using the scale and accumulator 140.

[0045] On the other hand, depending on the embodiment, the lookup table L can also be realized using multiple flip-flops (FFs). That is, the lookup table generator 131 can be configured so that each individual flip-flop FF outputs a calculation result value corresponding to the key value, and the processing element 132 can use a multiplexer M to selectively provide the calculation result value corresponding to the key value from among the calculation result values ​​input from the multiple flip-flop FFs. Depending on the embodiment, each read accumulator included in the processing element 132 can also be configured to selectively provide the calculation result value corresponding to the key value using the multiplexer M.

[0046] Specifically, referring to Figure 5(a), the lookup table generator 131 can generate key-value pairs to be included in each lookup table, assign key values ​​according to the position of each flip-flop FF, and input an enable signal EN to set the respective value values ​​-x0-x1, -x0+x1, x0-x1, and x0+x1 corresponding to the key values ​​into the flip-flop FF. In this case, since each value is stored in the flip-flop FF, it is possible to realize a lookup table via multiple flip-flop FFs.

[0047] Referring thereafter to Figure 5(b), the outputs of multiple flip-flops (FFs) can be provided to their respective connected processing elements 132, and to the read accumulators RAC0, RAC1, and RAC2 within those processing elements 132. Here, the read accumulators RAC0, RAC1, and RAC2 may include multiplexers, and when key values ​​key0, key1, and key2 are input to these multiplexers, the corresponding calculated values ​​can be provided.

[0048] In this way, multiple processing elements 132 and the read accumulators RAC0, RAC1, and RAC2 within each processing element 132 can receive lookup tables from multiple flip-flops FF. That is, since the calculation result values ​​are not extracted by accessing each lookup table stored in memory, it is possible to prevent memory bank collisions even when performing parallel processing.

[0049] Furthermore, the lookup table generator 131 can also generate a half-lookup table instead of a full lookup table. That is, when binary coding quantization is applied to have binary weights of -1 or 1, the symmetry of the lookup table can be utilized to generate and use a half-lookup table corresponding to half of the total number of possible combinations. In this case, the size of the existing lookup table can be reduced by half, thereby reducing the memory usage required for using the lookup table. Here, the calculation results for the remaining half can be restored and provided in real time from the half-lookup table.

[0050] Specifically, referring to Figure 6(a), when using a general lookup table L, inputting key value 0001 will output B as the calculation result, and inputting key value 1110 will output -B as the calculation result. In this case, calculation result values ​​corresponding to all key values ​​are stored in the lookup table L, but this may lead to problems such as the size of the lookup table L becoming large.

[0051] On the other hand, referring to Figure 6(b), a half-lookup table L can be used. In this case, the lookup table generator 131 can check the most significant bit (MSB) of the key value input by the processing element 132 and decide whether or not to restore the calculation result value. That is, as shown in Figure 6(b), if a key value 0001 with a most significant bit of 0 is input, the key value corresponding to the remaining key value 001 after removing the most significant bit is included in the half-lookup table HL, and the corresponding calculation result value B can be output. However, if the most significant bit is 1, such as in the key value 1110, the key value is not included in the half-lookup table, so it can be determined that restoration should proceed.

[0052] Specifically, by removing the most significant bit from the key value 1110, the resulting value 110 can be converted (for example, by performing a NOT operation) to generate the converted key value 001, and the calculation result value corresponding to the converted key value 001 can be extracted from the half-lookup table HL. At this time, the sign of the extracted calculation result value can be converted to output the final calculation result value -B. In other words, it can be confirmed that the same calculation result value -B as in Figure 6(b) can be output even when using the half-lookup table L. Thus, by using the half-lookup table L, it is possible to reduce memory usage by using only half of the existing lookup table L while achieving the same operation as the existing lookup table L.

[0053] In addition, the lookup table generator 131 can apply additional algorithms to reduce the number of operations repeated during lookup table generation, as shown in Figure 7.

[0054] Referring to Figure 7, if the hyperparameter is 4, 2 4While it is possible to generate a lookup table containing 16 calculation result values, by utilizing a half-lookup table, it is possible to omit half of them by leveraging symmetry. Furthermore, when generating calculation results for the remaining 8 cases, as shown in Figure 7, it can be confirmed that the calculation results for (x3+x4), (x3-x4), (-x3+x4), and (-x3-x4) are each repeated twice. Therefore, it is possible to generate the calculation results for (x3+x4), (x3-x4), (-x3+x4), and (-x-x4) once each and then reuse them. In the case of (x1+x2) and (x1-x2), it can be confirmed that they are each repeated four times, so after performing the calculation once, it is possible to construct a data pipeline to reuse the calculation result for the remaining three times.

[0055] The present invention is not limited by the embodiments described above and the accompanying drawings. It will be apparent to those who have ordinary skill in the art to which the present invention pertains that the components of the present invention can be substituted, modified, and changed without departing from the technical spirit of the invention.

[0056] The present invention further solves the following problems.

[0057] This invention aims to provide a hardware accelerator for lookup table-based general matrix operations that utilizes binary coding quantization (BCQ) to reduce computational complexity and support a variety of quantization accuracies.

[0058] The present invention aims to provide a hardware accelerator for lookup table-based general matrix operations that can implement a flip-flop-based lookup table and prevent memory bank collisions.

[0059] This invention aims to reduce memory storage space by utilizing half-lookup tables and to provide a hardware accelerator for lookup table-based general matrix operations.

[0060] The present invention aims to provide a hardware accelerator for general matrix operations based on lookup tables, which can replace existing MAC (Multiply-Accumulate) units using a read accumulator optimized for lookup table-based operations.

[0061] The present invention can further achieve the following effects.

[0062] According to one embodiment of the present invention, a hardware accelerator for lookup table-based general matrix operations can reduce computational complexity by utilizing binary coding quantization (BCQ) and support a variety of quantization accuracies.

[0063] According to a hardware accelerator for general matrix operations based on a lookup table, which is part of one embodiment of the present invention, memory bank collisions can be prevented because it utilizes a hardware structure that implements a flip-flop-based lookup table.

[0064] According to a hardware accelerator for general matrix operations based on a lookup table according to one embodiment of the present invention, it is possible to realize a half-lookup table, which reduces the size of the lookup table by half, thereby reducing the memory storage space for the lookup table and decreasing the required power consumption.

[0065] According to a hardware accelerator for general matrix operations based on lookup tables, which is part of one embodiment of the present invention, it is possible to reduce power consumption and computational latency by utilizing a read accumulator optimized for lookup table-based operations, thereby replacing existing MAC (Multiply-Accumulate) units.

[0066] However, the effects that can be achieved by the hardware accelerator for lookup table-based general matrix operations according to embodiments of the present invention are not limited to those described above, and other effects not mentioned above will be clearly understood by those with ordinary skill in the art to which the present invention pertains from the following description. [Explanation of Symbols]

[0067] 100 Hardware Accelerators 110 Weight Buffer 120 Input Buffers 130 Matrix Processing Units 140 Scale and Accumulator 150 output buffer

Claims

1. A hardware accelerator for lookup table-based general matrix operations, A weight buffer that stores at least a portion of a weight matrix containing integer quantum weights obtained by quantizing the weights of the target model, An input buffer in which at least a portion of an input matrix containing floating-point-based input activation values ​​for the target model is stored, The system includes a matrix processing unit that performs matrix operations between the weight matrix and the input matrix through communication with the weight buffer and the input buffer, The matrix processing unit is A lookup table generator that calculates the calculation result values ​​corresponding to all possible cases between the quantum weights and the input activation values ​​that can be generated during the matrix operation, and generates a lookup table that includes the calculation result values, A hardware accelerator characterized by including a plurality of processing elements that extract the calculation result values ​​corresponding to the quantum weights from the lookup table during the matrix operation and generate a partial sum.

2. The matrix processing unit is, The hardware accelerator according to claim 1, characterized in that it includes a systolic array structure in which the partial sum of the processing elements is input as other processing elements for data accumulation.

3. The aforementioned weight buffer is The hardware accelerator according to claim 1, characterized in that, when the weights of the target model are quantized based on binary coding quantization by a linear combination of q binary weights and the respective scaling factors and offsets applied to the binary weights, q binary matrices containing the binary weights are stored as the weight matrix.

4. The aforementioned lookup table generator, The hardware accelerator according to claim 3, characterized in that the binary matrix is ​​divided into small regions containing columns of hyperparameters, and the lookup tables corresponding to each of the small regions are generated.

5. The aforementioned lookup table generator, The hardware accelerator according to claim 4, characterized in that it assigns key values ​​to all possible combinations of the binary weights of the hyperparameters, generates value values ​​as the result of a matrix operation between the binary weights of the hyperparameters corresponding to the key values ​​and the input matrix, and generates the lookup table from the key value and value pair.

6. The aforementioned lookup table generator, The hardware accelerator according to claim 5, wherein each of the aforementioned sub-regions is provided, and when the input matrix is ​​input from the input buffer, it generates a lookup table corresponding to the sub-region.

7. The hardware accelerator according to claim 6, characterized in that when the lookup table is generated for each of the subregions, the processing elements assigned to the subregions in the systolic array structure share the same lookup table.

8. The aforementioned processing element is The hardware accelerator according to claim 5, characterized in that it generates a key value from the pattern of binary weights of the hyperparameters in the binary matrix, and uses the key value to extract a calculation result value, which is a corresponding value, from the lookup table.

9. The aforementioned processing element is The hardware accelerator according to claim 5, further comprising a plurality of read accumulators corresponding to each row assigned to the processing element from among the small regions.

10. The plurality of reading accumulators are, The hardware accelerator according to claim 9, characterized in that it can simultaneously access the lookup table assigned to the processing element, read the calculation result values ​​corresponding to the binary weights of the hyperparameters contained in the row from the lookup table, and generate the respective partial sums corresponding to the row in parallel.

11. The aforementioned lookup table generator, The hardware accelerator according to claim 5, characterized in that it provides the lookup table to the plurality of processing elements using a plurality of flip-flops.

12. The aforementioned lookup table generator, The individual flip-flops are configured to output the respective calculation result values ​​corresponding to the key values, The aforementioned processing element is The hardware accelerator according to claim 11, characterized in that a multiplexer is used to selectively provide the calculation result value corresponding to the key value from among the calculation result values ​​input from the plurality of flip-flops.

13. The aforementioned lookup table generator, The hardware accelerator according to claim 5, characterized in that, when the binary weight is -1 or 1, it uses symmetry to generate a half-lookup table corresponding to half of the total number of possible combinations of the binary weight, and the calculation result values ​​for the remaining half are restored in real time from the half-lookup table and provided.

14. The aforementioned lookup table generator, The hardware accelerator according to claim 13, characterized in that it checks the most significant bit of the key value input by the processing element and determines whether or not to restore it.

15. The aforementioned lookup table generator, The hardware accelerator according to claim 14, characterized in that, when restoring, it converts the key value to extract the corresponding calculation result value from the half-lookup table, and converts the sign of the extracted calculation result value to provide it.

16. The hardware accelerator according to claim 3, further comprising a scale and accumulator that, upon receiving the partial sum from each of the processing elements located at the end of each row in the systolic array structure, applies the scaling factor and the offset to the partial sum and stores it in an output buffer.

Images