Apparatus and method of processing memory computing by using weight matrix

Abstract

The present invention relates to a memory operation processing device and method using a weight matrix. According to one embodiment of the present invention, the device comprises: an input processing unit that processes inputs for a weight matrix and dense vectors to be memory operations; a permutation processing unit that selects at least one row from the rows constituting the input weight matrix by considering the number of negative values ​​among the positive and negative values ​​constituting each of the rows, and processes the selected at least one row into a permutation weight matrix by replacing the position of the negative value in the selected at least one row with the positive value of a row other than the selected at least one row; a sorting processing unit that searches for a row based on the number of negative values ​​included in the rows in the permutation weight matrix, and processes the sorted weight matrix by sorting the positions of the columns of the permutation weight matrix from one side according to whether a negative value exists on one side of the searched row; and the dense vectors and operation components that constitute the sorted weight matrix in the banks of the memory by considering the size of each of the banks. It may include a matrix operation processing unit that divides and maps, and processes operations on the components mapped to the banks and the dense vectors.

Description

Technology Field

[0001] The present invention relates to a memory operation processing device and method using a weight matrix, and more specifically to a memory operation acceleration technology that accelerates memory operations of a weight matrix and a vector by permuting and sorting a weight matrix based on an output aware matrix permutation (OMP) algorithm and a zero-aware matrix column sorting (ZMCS) algorithm, and mapping the sorted weight matrix and vector to a memory bank. Background Technology

[0002] Sparse matrix and vector operations are utilized as fundamental operations in numerous applications, such as deep neural networks, scientific computing, and graph analysis.

[0003] This operation is very important because it optimizes memory consumption and computation time by efficiently processing matrices where most elements are "0".

[0004] Nevertheless, despite the compressibility of sparsity matrices, the demand for processing is increasing with the emergence of new deep learning models, such as large-scale language models.

[0005] Accordingly, in conventional technology, memory constraints arise because the scarcity value matrix is ​​utilized only once per inference cycle.

[0006] Therefore, handling extensive memory operations in the existing von Neumann architecture can lead to serious problems related to the memory wall issue.

[0007] This problem occurs because the limited capacity of on-chip memory and the limited bandwidth of off-chip memory hinder efficient data processing and throughput.

[0008] To solve these memory wall problems, the Processing in Memory (PIM) architecture is attracting attention as a next-generation computing platform that goes beyond the Von Neumann architecture.

[0009] Among them, DRAM (dynamic random access memory)-based PIM is receiving attention compared to other memory technologies.

[0010] DRAM has a higher density than SRAM (static random access memory), so it can process matrix operations to match the memory size required by deep neural networks.

[0011] Many other PIMs are processed based on next-generation memory that exhibits high efficiency thanks to analog current sum MAC operations, while DRAM can be implemented using conventional technologies such as CMOS.

[0012] In addition, DRAM can handle complex computational functions, such as MAC operations, by performing bit logic operations within the DRAM.

[0013] This can be implemented by utilizing the large-scale parallelism inherent in DRAM banks.

[0014] Although it processes a single MAC operation, performance and energy efficiency in acceleration can be extremely low as numerous 3-row activation operations must be processed.

[0015] Conventional technology was developed to consider efficiency by performing design space search and mapping input data to DRAM banks to prevent performance and energy efficiency from decreasing, but it had limitations in improving performance and energy efficiency because it did not consider the sparsity of "0" in the weight matrix. Prior art literature

[0016] Korean Published Patent No. 10-2021-0086447, "Device, method, and system for instructions of a matrix operation accelerator" Korean Published Patent No. 10-2020-0070089, "Semiconductor circuit for a tensor computation dataflow accelerator" Korean Published Patent No. 10-2023-0005345, "Memory device for training a neural network" The problem to be solved

[0017] The present invention aims to provide a memory operation processing device and method that accelerate memory operations of weight matrices and vectors by permuting and sorting weight matrices based on an output aware matrix permutation (OMP) algorithm and a zero-aware matrix column sorting (ZMCS) algorithm, and mapping the sorted weight matrices and vectors to a memory bank.

[0018] The present invention aims to reduce the computational complexity of the components of a weight matrix and dense vectors by excluding rows composed only of negative values ​​from the mapping to a memory bank, by substituting the components of a row so that the bits constituting the row in the weight matrix are composed only of negative values ​​("0") based on an output recognition matrix permutation algorithm.

[0019] The present invention aims to reduce the computational complexity of components of a weight matrix and dense vectors by sorting the columns of a matrix in descending order according to the number of negative values ​​("0") in the rows based on a zero-recognition matrix column sorting algorithm.

[0020] The present invention aims to provide a memory operation processing device and method that prevents loss of accuracy by performing omitted in-bank multiplication operations on the upper bits of the surrounding area, while securing computational performance and energy efficiency by reducing the effective bit width required for multiplication operations within the memory bank by utilizing the general bit-level sparsity of matrix values. means of solving the problem

[0021] According to one embodiment of the present invention, a memory operation processing device comprises: an input processing unit that processes inputs for a weight matrix and dense vectors to be memory operations; a permutation processing unit that selects at least one row by considering the number of negative values ​​among the positive and negative values ​​constituting each of the rows constituting the input weight matrix, and processes the selected at least one row into a permutation weight matrix by replacing the position of the negative value in the selected at least one row with the positive value of a row other than the selected at least one row; and a sorting processing unit that searches for one row based on the number of negative values ​​included in the rows in the permutation weight matrix, and processes the permutation weight matrix into a sorting weight matrix by sorting the positions of the columns of the permutation weight matrix from one side according to whether a negative value exists on one side of the searched one row. It may include a matrix operation processing unit that divides and maps the operation components constituting the alignment weight matrix and the dense vectors to the banks of the memory, taking into account the size of each of the banks, and processes the operations of the components and the dense vectors mapped to the banks.

[0022] The permutation processing unit can select at least two rows containing negative values ​​among the rows, search for the number of negative values ​​constituting each of the selected at least two rows, calculate a matching score based on the number of negative values ​​searched, determine the row with the largest matching score as the permutation target row, and process the determined permutation target row by substituting the positive value from the row with the smallest matching score with the permutation weight matrix.

[0023] The permutation processing unit can increase the matching score when the number of searched negative values ​​increases for at least two selected rows, decrease the matching score when the number of searched negative values ​​decreases, and perform substitution with the positive value so that the matching score becomes "0" for the determined permutation target row.

[0024] The above permutation processing unit can repeatedly perform substitution with the positive value in the above selected at least two rows so that the matching score becomes "0".

[0025] The sorting processing unit can search for the location of a positive value from one side in any one of the searched rows, search for the location of a negative value located on one side from the location of the searched positive value, and process the sorting weight matrix such that, by swapping the column corresponding to the location of the searched negative value and the column corresponding to the location of the searched positive value, columns containing negative values ​​are sequentially located from one side, and then columns containing positive values ​​are located.

[0026] The above sorting processing unit changes the column order according to the change in the column position of the sorting weight matrix, and can also change the order of the density vectors corresponding to the column order according to the changed column order.

[0027] The above matrix operation processing unit reduces the effective bit width by skipping the matrix operation for the row in which the positive value exists in the column when a positive value exists in a column on one side of the alignment weight matrix, while delaying the matrix operation and supplementing the matrix operation for the skipped row by loading the bits for the row in which the matrix operation was skipped into a surrounding area.

[0028] A memory operation processing method according to an embodiment of the present invention comprises: a step of processing inputs for a weight matrix and dense vectors to be memory operations in an input processing unit; a step of, in a permutation processing unit, selecting at least one row by considering the number of negative values ​​among the positive and negative values ​​constituting each of the rows constituting the input weight matrix, and processing it into a permutation weight matrix by replacing the position of the negative value in the selected at least one row with the positive value of a row other than the selected at least one row; a step of, in a sorting processing unit, searching for a row based on the number of negative values ​​included in the rows in the permutation weight matrix, and processing it into a sorted weight matrix by sorting the positions of the columns of the permutation weight matrix from one side according to whether a negative value exists on one side of the searched row; and, in a matrix operation processing unit, operation components constituting the sorted weight matrix in the banks of the memory by considering the size of each of the banks and the It may include the step of dividing and mapping dense vectors, and processing operations on the components mapped to the banks and the dense vectors.

[0029] The step of processing with the above permutation weight matrix may include selecting at least two rows containing negative values ​​among the rows, searching for the number of negative values ​​constituting each of the selected at least two rows, calculating a matching score based on the number of negative values ​​searched, determining the row with the largest matching score as the row to be permuted, and processing with the above permutation weight matrix by substituting the positive value from the row with the smallest matching score for the determined row to be permuted.

[0030] The step of processing with the sorting weight matrix described above may include the step of searching for the location of a positive value from one side in any one of the searched rows, searching for the location of a negative value located on one side from the location of the searched positive value, and processing with the sorting weight matrix such that, by swapping the column corresponding to the location of the searched negative value and the column corresponding to the location of the searched positive value, columns containing negative values ​​are sequentially located from one side, and then columns containing positive values ​​are located. Effects of the invention

[0031] The present invention can provide a memory operation processing device and method that accelerate memory operations of weight matrices and vectors by permuting and sorting weight matrices based on an output aware matrix permutation (OMP) algorithm and a zero-aware matrix column sorting (ZMCS) algorithm, and mapping the sorted weight matrices and vectors to a memory bank.

[0032] The present invention can reduce the computational complexity of the components of the weight matrix and dense vectors by excluding rows composed only of negative values ​​from the mapping to the memory bank, by substituting the components of the rows so that the bits constituting the rows in the weight matrix are composed only of negative values ​​("0") based on an output recognition matrix permutation algorithm.

[0033] The present invention can reduce the computational complexity of components of a weight matrix and dense vectors by sorting the columns of a matrix in descending order according to the number of negative values ​​("0") in the rows based on a zero-recognition matrix column sorting algorithm.

[0034] The present invention can provide a memory operation processing device and method that prevents loss of accuracy by performing omitted in-bank multiplication operations on the upper bits of the surrounding area while securing computational performance and energy efficiency by utilizing the general bit-level sparsity of matrix values ​​to reduce the effective bit width required for multiplication operations within the memory bank. Brief explanation of the drawing

[0035] FIG. 1 is a diagram illustrating a memory operation processing device using a weight matrix according to an embodiment of the present invention. FIGS. 2 to 4 are drawings illustrating an output aware matrix permutation (OMP) algorithm performed in a memory operation processing device using a weight matrix according to an embodiment of the present invention. FIG. 5 is a diagram illustrating a zero-aware matrix column sorting (ZMCS) algorithm performed in a memory operation processing device using a weight matrix according to an embodiment of the present invention. FIG. 6 is a diagram illustrating the mapping result and computation effect for a bank of memory after an OMP algorithm in a memory computation processing device using a weight matrix according to an embodiment of the present invention. FIG. 7 is a diagram illustrating a deferring method performed in a memory operation processing device using a weight matrix according to an embodiment of the present invention. FIG. 8 is a diagram illustrating the application structure of a memory operation processing device using a weight matrix according to an embodiment of the present invention. FIG. 9 is a diagram illustrating the implementation of the MAC (multiple accumulate calculation) logic of a memory operation processing device using a weight matrix according to an embodiment of the present invention. FIG. 10 is a diagram illustrating the implementation of a multiplication delay in the MAC logic of a memory operation processing device using a weight matrix according to an embodiment of the present invention. FIG. 11 is a diagram illustrating the performance and energy efficiency of a memory computation processing device using a weight matrix according to an embodiment of the present invention. FIGS. 12 to 14 are drawings illustrating a memory operation processing method using a weight matrix according to an embodiment of the present invention. Specific details for implementing the invention

[0036] Hereinafter, various embodiments of this document are described with reference to the attached drawings.

[0037] The embodiments and the terms used therein are not intended to limit the technology described in this document to specific embodiments and should be understood to include various modifications, equivalents, and / or substitutions of said embodiments.

[0038] In describing various embodiments below, if it is determined that a detailed description of related known functions or configurations could unnecessarily obscure the essence of the invention, such detailed description will be omitted.

[0039] Furthermore, the terms described below are defined considering their functions in various embodiments, and these may vary depending on the intentions or practices of the user or operator. Therefore, their definitions should be based on the content throughout this specification.

[0040] In relation to the description of the drawings, similar reference numerals may be used for similar components.

[0041] A singular expression may include a plural expression unless the context clearly indicates otherwise.

[0042] In this document, expressions such as "A or B" or "at least one of A and / or B" may include all possible combinations of the items listed together.

[0043] Expressions such as "first," "second," "first," or "second" may modify the corresponding components regardless of order or importance, and are used merely to distinguish one component from another without limiting the components.

[0044] Where it is stated that a certain (e.g., first) component is "(functionally or telecommunicationally) connected" or "connected" to another (e.g., second) component, said certain component may be directly connected to said other component or connected through another component (e.g., third component).

[0045] In this specification, "configured to" may be used interchangeably with, depending on the context, for example, in hardware or software, "suitable for," "capable of," "modified to," "made to," "capable of," or "designed to."

[0046] In some situations, the expression "device configured to..." may mean that the device is "able to..." together with other devices or parts.

[0047] For example, the phrase “a processor configured (or set) to perform A, B, and C” may mean a dedicated processor for performing said operations (e.g., an embedded processor), or a general-purpose processor capable of performing said operations by executing one or more software programs stored in a memory device (e.g., a CPU or an application processor).

[0048] Also, the term 'or' means an inclusive or rather an exclusive or.

[0049] That is, unless otherwise noted or is not clear from the context, the expression 'x uses a or b' means any one of the natural inclusive permutations.

[0050] Terms such as '..bu', '..gi' used below refer to a unit that processes at least one function or operation, and this may be implemented in hardware or software, or a combination of hardware and software.

[0052] FIG. 1 is a diagram illustrating a memory operation processing device using a weight matrix according to an embodiment of the present invention.

[0053] FIG. 1 illustrates the components of a memory operation processing device using a weight matrix according to an embodiment of the present invention.

[0054] Referring to FIG. 1, a memory operation processing device (100) using a weight matrix according to an embodiment of the present invention may include an input processing unit (110), a permutation processing unit (120), a sorting processing unit (130), and a matrix operation processing unit (140).

[0055] A memory operation processing device (100) using a weight matrix according to an embodiment of the present invention can accelerate the memory-based operation speed for multiplying a large weight matrix and a vector by permuting and sorting the weight matrix based on an output aware matrix permutation (OMP) algorithm and a zero-aware matrix column sorting (ZMCS) algorithm.

[0056] For example, the input processing unit (110) can process inputs for weight matrices and dense vectors that are subject to memory operations.

[0057] For example, the input processing unit (110) processes inputs for a weight matrix that is mapped to a dense vector and a memory bank, which is the target of the MAC operation.

[0058] The weight matrix consists of multiple rows and columns, and includes bits as components at corresponding positions according to the rows and columns, which can be referred to as weight bits.

[0059] The weight bit consists of either a positive value corresponding to "1" or a negative value corresponding to "0", and the bit corresponding to the negative value can be considered as sparsity.

[0060] If all rows constituting the weight matrix consist of negative values, operations on those rows can be excluded.

[0061] According to one embodiment of the present invention, the permutation processing unit (120) can select at least one row by considering the number of negative values ​​among the positive values ​​and negative values ​​constituting each row in the rows constituting the input weight matrix.

[0062] Additionally, the permutation processing unit (120) can process the position of a negative value in at least one selected row into a positive value of another row different from the at least one selected row to form a permutation weight matrix.

[0063] According to one embodiment of the present invention, a permutation processing unit (120) selects at least two rows containing negative values ​​among the rows and searches for the number of negative values ​​constituting each of the selected at least two rows.

[0064] Additionally, the permutation processing unit (120) calculates a matching score based on the number of negative values ​​found and determines the row with the largest matching score as the row to be permuted.

[0065] Additionally, the permutation processing unit (120) can process the determined permutation target row into a permutation weight matrix by replacing it with the positive value from the row with the smallest matching score.

[0066] For example, the permutation processing unit (120) can increase the matching score when the number of negative values ​​searched for at least two selected rows increases, decrease the matching score when the number of negative values ​​searched decreases, and perform substitution with positive values ​​so that the matching score becomes "0" for the determined permutation target row.

[0067] According to one embodiment of the present invention, the permutation processing unit (120) can repeatedly perform substitution with the positive value so that the matching score becomes "0" in at least two selected rows.

[0068] For example, the permutation processing unit (120) can perform an output aware matrix permutation (OMP) algorithm.

[0069] The permutation processing unit (120) can reduce the number of rows to be processed by performing an output recognition matrix permutation algorithm to replace negative values ​​within the rows with positive values.

[0070] For example, the sorting processing unit (130) searches for one row based on the number of negative values ​​included in the rows in the permutation weight matrix.

[0071] According to one embodiment of the present invention, the sorting processing unit (130) can process the columns of the permutation weight matrix from one side by sorting the positions of the columns of the permutation weight matrix according to whether there is a negative value on one side in any one row searched, and process them into a sorting weight matrix.

[0072] For example, one side may be either the right or the left, and preferably, one side may be the left side where memory operation processing begins.

[0073] For example, the alignment processing unit (130) searches for the location of a positive value from one side of any one row that has been searched, and searches for the location of a negative value located on one side from the location of the searched positive value.

[0074] Additionally, the sorting processing unit (130) can process the column corresponding to the position of the searched negative value and the column corresponding to the position of the searched positive value into a sorting weight matrix such that columns containing negative values ​​are sequentially positioned from one side and columns containing positive values ​​are positioned thereafter.

[0075] According to one embodiment of the present invention, the alignment processing unit (130) changes the column order according to the change in the column position of the alignment weight matrix, and can also change the order of the dense vectors corresponding to the column order according to the changed column order.

[0076] For example, the sorting processing unit (130) can perform a zero-aware matrix column sorting (ZMCS) algorithm.

[0077] For example, the alignment processing unit (130) can increase computational performance and efficiency by aligning bits corresponding to negative values ​​that can be excluded from the operation target to one side using a zero-recognition matrix column alignment algorithm.

[0078] For example, the matrix operation processing unit (140) can divide and map the operation components and dense vectors that constitute an alignment weight matrix to the banks of memory, taking into account the size of each bank, and process the operation of the components and dense vectors mapped to the banks.

[0079] According to one embodiment of the present invention, the matrix operation processing unit (140) can reduce the effective bit width by skipping the matrix operation for the row in which a positive value exists in the column when a positive value exists in a column on one side of the alignment weight matrix, while delaying the matrix operation and compensating for the skipped row by loading the bits for the row in which the matrix operation was skipped into a surrounding area.

[0080] For example, the matrix operation processing unit (140) can prevent loss of accuracy by utilizing the general bit-level sparsity of the matrix value to reduce the effective bit width required for multiplication operations within the memory bank, thereby ensuring computational performance and energy efficiency, while performing omitted in-bank multiplication operations for the upper bits of the surrounding area.

[0081] Accordingly, the present invention can provide a memory operation processing device and method that accelerate memory operations of weight matrices and vectors by permuting and sorting weight matrices based on an output aware matrix permutation (OMP) algorithm and a zero-aware matrix column sorting (ZMCS) algorithm, and mapping the sorted weight matrices and vectors to a memory bank.

[0083] FIGS. 2 to 4 are drawings illustrating an output aware matrix permutation (OMP) algorithm performed in a memory operation processing device using a weight matrix according to an embodiment of the present invention.

[0084] FIG. 2 illustrates the procedure of an output recognition matrix permutation algorithm performed in a memory operation processing device using a weight matrix according to an embodiment of the present invention.

[0085] Referring to FIG. 2, in step (S201), a memory operation processing device using a weight matrix according to an embodiment of the present invention selects a fourth row (o4) that contains the fewest negative values ​​in the weight matrix, and replaces the position of the negative value of the fourth row (o4) with the positive value of the row (o8) that contains the most negative values.

[0086] In step (S202), a memory operation processing device using a weight matrix according to an embodiment of the present invention may select the next row, the third row (o3), and replace it with the positive value of the seventh row (o7).

[0087] In step (S203), a memory operation processing device using a weight matrix according to an embodiment of the present invention selects the next row, the 5th row (o5), and replaces it with the positive value of the 7th row (o7).

[0088] In step (S204), a memory operation processing device using a weight matrix according to an embodiment of the present invention selects the next row, the 5th row (o5), and replaces it with the positive value of the 1st row (o7).

[0089] In step (S205), the memory operation processing device using a weight matrix according to an embodiment of the present invention selects the next row, the 6th row (o6), and replaces it with the positive value of the 2nd row (o2).

[0090] In step (S206), a memory operation processing device using a weight matrix according to an embodiment of the present invention processes the weight matrix into a permutation weight matrix by permuting the weight matrix so that a row in which positive and negative values ​​are mixed is composed of a row composed only of positive values ​​without negative values.

[0091] In step (S207), a memory operation processing device using a weight matrix according to an embodiment of the present invention maps a permutation weight matrix to a bank of memory.

[0092] FIG. 3 illustrates a mapping of a permutation weight matrix to a memory bank based on an output recognition matrix permutation algorithm performed in a memory operation processing device using a weight matrix according to an embodiment of the present invention.

[0093] Referring to FIG. 3, a memory operation processing device using a weight matrix according to an embodiment of the present invention shows a configuration in which a permutation weight matrix (300) is divided and allocated to a plurality of banks constituting memory.

[0094] A plurality of banks may consist of a first bank (310), a second bank (320), a third bank (330), and a fourth bank (340).

[0095] The permutation weight matrix (300) can be divided into a first configuration (301), a second configuration (302), a third configuration (303), and a fourth configuration (304) depending on the size of the bank of memory.

[0096] A first configuration (301) is assigned to the first bank (310) and can be mapped with a density vector for the first configuration (301).

[0097] A second configuration (302) is assigned to the second bank (320) and can be mapped with a density vector for the second configuration (302).

[0098] A third configuration (303) is assigned to the third bank (330) and can be mapped with a density vector for the third configuration (303).

[0099] A fourth configuration (304) is assigned to the fourth bank (340) and can be mapped with a density vector for the fourth configuration (304).

[0100] More specifically, the permutation weight matrix (300) may indicate that null rows are assigned to the second bank (320) and the fourth bank (340) according to the size of each bank in case the entire matrix cannot be held in one bank in an actual application.

[0101] The permutation weight matrix (300) demonstrates its effect by reducing the eigenvalues ​​of the first partial sum row for the first bank (310), the eigenvalues ​​of the second partial sum row for the second bank (320), the eigenvalues ​​of the third partial sum row for the third bank (330), and the eigenvalues ​​of the fourth partial sum row for the fourth bank (340) as rows consisting of "0" in the final mapping are excluded from the operation.

[0102] FIG. 4 illustrates an output recognition matrix permutation algorithm performed in a memory operation processing device using a weight matrix according to an embodiment of the present invention.

[0103] Referring to FIG. 4, the configuration of an output recognition matrix permutation algorithm (400) performed in a memory operation processing device using a weight matrix according to an embodiment of the present invention is illustrated.

[0104] The output recognition matrix permutation algorithm (400) indicates that the score matching score of Rb and Rt is 0, and if Rb[i] is 0 or Rt[i] is not 0, the score is increased, and if Rb[i] is not 0 or Rt[i] is 0, the score is decreased to input the score.

[0105] The output recognition matrix permutation algorithm (400) is configured such that the best score for Rb is -1000, it is repeated up to i, and if the score is greater than the maximum score and not the i-th row, the score is equal to the best score and Rt is the i-th row, the best score is not positive, then Rb is updated to W and OWAR(W) is input, J is from 1 to the size of Rb, and if Rb is 0 and Rb[j], Rt[j], rk is 0, then the update W is input with the new Rb and Rt.

[0106] In summary, the output recognition matrix permutation algorithm (400) represents a configuration in which, for each bit of the row, the score is increased or decreased depending on the existence of a negative value of "0", thereby substituting the value of the bit constituting the row to process it into a row consisting only of negative values ​​and excluding the row from the operation target.

[0108] FIG. 5 is a diagram illustrating a zero-aware matrix column sorting (ZMCS) algorithm performed in a memory operation processing device using a weight matrix according to an embodiment of the present invention.

[0109] FIG. 5 shows an improved mapping in FIG. 4 of the weight matrix mapping by applying a zero-recognition matrix column alignment algorithm performed in a memory operation processing device using a weight matrix according to an embodiment of the present invention.

[0110] Referring to FIG. 5, a memory operation processing device using a weight matrix according to an embodiment of the present invention can process a permutation weight matrix (500) into a sorted weight matrix (510) by applying a zero-recognition matrix column sorting algorithm.

[0111] More specifically, the alignment weight matrix (510) is composed of a first configuration (511), a second configuration (512), a third configuration (513), and a fourth configuration (514).

[0112] A first configuration (511) is assigned to the first bank (520) and can be mapped with a density vector for the first configuration (511).

[0113] A second configuration (512) is assigned to the second bank (521) and can be mapped with a density vector for the second configuration (512).

[0114] A third configuration (313) is assigned to the third bank (522) and can be mapped with a density vector for the third configuration (313).

[0115] The fourth bank (523) is assigned the fourth configuration (514) and can be mapped with a density vector for the fourth configuration (514).

[0116] The zero-recognition matrix column sorting algorithm exemplifies a mapping of sorting weight matrices that excludes some operation targets from the actual mapping of the first bank (520) by generating another zero row (null row).

[0117] In other words, the first bank (520) can improve computational complexity, performance, etc. by processing the row with the lowest sparsity in relation to the first configuration (511) as a null row consisting only of "0".

[0119] FIG. 6 is a diagram illustrating the mapping result and computation effect for a bank of memory after an OMP algorithm in a memory computation processing device using a weight matrix according to an embodiment of the present invention.

[0120] Referring to FIG. 6, a mapping result (600) according to an embodiment of the present invention is shown, and a mapping result (610) according to the prior art is shown.

[0121] The mapping result (600) and the mapping result (610) represent the mapping results of a 4-bit matrix and a 4-bit vector.

[0122] Operations on the mapping result (600) and the mapping result (610) can be performed sequentially from MSB to LSB.

[0123] The mapping result (600) shows that the effective bit width of the matrix element for that specific bank is reduced to 3 bits by excluding from mapping and calculation as the bits constituting the row (601) are all negative values.

[0124] On the other hand, the mapping result (610) shows that the row (611) cannot be excluded from mapping and calculation because the upper bit (W2) of the row (611) contains a positive value, thereby reducing the effective bit width of the matrix element for that specific bank to 3 bits.

[0126] FIG. 7 is a diagram illustrating a deferring method performed in a memory operation processing device using a weight matrix according to an embodiment of the present invention.

[0127] FIG. 7 illustrates a deferring method performed in a memory operation processing device using a weight matrix according to an embodiment of the present invention.

[0128] As can be seen in the rows of the matrix, even if not all zero bits exist in the upper bit matrix, it is highly likely that only a few zero bits exist.

[0129] Based on these observations, the effective bit width of multiplication within the bank can be reduced by omitting calculations for upper bit rows with fewer zero bits.

[0130] Since multiplication operations within a bank involve a large amount of bit operations within DRAM, reducing the bit width can significantly improve performance and energy efficiency.

[0131] Subsequently, in the accumulation phase, skipped in-bank multiplication operations can be performed on the upper bits of the surrounding region to prevent loss of accuracy.

[0132] There is less additional overhead because the number of bits to be multiplied in the surrounding area is small.

[0133] The matrix (700) shows that even if there is a bit (701) corresponding to "1" instead of "0" in the first column of a row, that row is excluded from the operation and the bit width of the multiplication within the bank is reduced.

[0134] The operation of the corresponding row, such as matrix combination (710) on the box, is multiplied in the surrounding area.

[0135] Formula (720) shows that the formula (721) for the omitted bit and the formula (722) prepared by loading the input bit and weight sign bit into the surrounding area are combined to compensate for the missing multiplication.

[0136] Accordingly, the present invention can provide a memory operation processing device and method that prevents loss of accuracy by performing omitted in-bank multiplication operations on the upper bits of the surrounding area while securing computational performance and energy efficiency by utilizing the general bit-level sparsity of matrix values ​​to reduce the effective bit width required for multiplication operations within the memory bank.

[0138] FIG. 8 is a diagram illustrating the application structure of a memory operation processing device using a weight matrix according to an embodiment of the present invention.

[0139] FIG. 8 illustrates an application structure of a memory operation processing device using a weight matrix according to an embodiment of the present invention.

[0140] Referring to FIG. 8, the application structure (800) according to one embodiment of the present invention is configured to control and sense by dividing it into a chip controller, a bank controller, and a sense amplifier.

[0141] For example, the application structure (800) utilizes most of the standard DRAM configuration to minimize design overhead.

[0142] The application structure (800) can have a group added between the chip and the bank hierarchy so that the memory can be configured into multiple banks.

[0143] All groups are connected via a group bus and controlled by a chip controller, and all banks in each group are connected via a bank bus and controlled by a bank controller.

[0144] In addition, since they share the same MAC logic for delayed multiplication and accumulation, one group can handle a single SpMV operation.

[0145] The application structure (800) adopts a matrix fixed data flow for multiplication within the bank.

[0146] In other words, since the matrix is ​​not transmitted anywhere and remains fixed in the cell area of ​​the bank, energy consumption for open chip matrix data transmission can be completely eliminated, except in the case of a cold start.

[0147] Meanwhile, new vectors are streamed to the bank at each inference step, but in most applications, they consume only a small amount of data compared to the matrix.

[0149] FIG. 9 is a diagram illustrating the implementation of the MAC (multiple accumulate calculation) logic of a memory operation processing device using a weight matrix according to an embodiment of the present invention.

[0150] FIG. 9 illustrates the implementation process of the MAC (multiple accumulate calculation) logic of a memory operation processing device using a weight matrix according to an embodiment of the present invention.

[0151] Referring to FIG. 9, a memory operation processing device using a weight matrix according to an embodiment of the present invention can implement MAC logic operation through steps (S901), (S902), (S903), and (S904).

[0152] In other words, steps (S901) to (S904) exemplify the microarchitecture of peripheral MAC logic based on OS data flow and the operation in the partial sum accumulation step.

[0153] According to one embodiment of the present invention, a memory operation processing device using a weight matrix uses an output fixed data flow in which the accumulation of each output element is maintained in a fixed state in a MAC block.

[0154] Basically, peripheral MAC logic consists of an input buffer, a MAC block, and an output buffer.

[0155] In the partial sum accumulation step, the input buffer can initially be filled with partial sums and mask rows transmitted from the cell area of ​​the bank.

[0156] Since OMP can result in partial sums for multiple outputs in a single row, mask information is required to determine whether to add the partial sums entering the accumulator.

[0157] Then, at each time step, each MAC block reads one partial sum and its corresponding mask, and if the mask value is 1, it accumulates.

[0158] In steps (S901) to (S903), the left MAC block must generate the o3 output and the right MAC block must generate the o7 output; however, since the o3 mask of the left MAC block is 1, p3 is accumulated to generate p31 (p3 n ) This means that n partial sums of output o3 have been accumulated.

[0159] Meanwhile, in the right MAC block, no operation occurs because the o7 mask is 0.

[0160] Conversely, in step (S904), the right MAC block accumulates p7 and the left MAC is idle, so the o7 mask is 1 and the o3 mask is 0.

[0161] After sweeping all columns of the input buffer, the accumulated result of the MAC block is stored in the output buffer.

[0162] FIG. 10 is a diagram illustrating the implementation of a multiplication delay in the MAC logic of a memory operation processing device using a weight matrix according to an embodiment of the present invention.

[0163] FIG. 10 illustrates peripheral MAC logic in relation to the implementation of multiplication delay in the MAC logic of a memory operation processing device using a weight matrix according to an embodiment of the present invention.

[0164] Referring to FIG. 10, in step (S1001), a memory operation processing device using a weight matrix according to an embodiment of the present invention initializes an input buffer with an input vector and a weight upper bit row (W2) and a corresponding sign bit row (W3) that have a delayed bit.

[0165] In addition, the MAC block is the output partial sum (p3) to be added along with the delayed multiplication result. 6 Stores ) in the accumulator.

[0166] In the next step (S1002), the memory operation processing device using a weight matrix according to an embodiment of the present invention has a MAC block that reads weights from an input buffer, performs multiplication and accumulation, and stores the final output (o3).

[0168] FIG. 11 is a diagram illustrating the performance and energy efficiency of a memory computation processing device using a weight matrix according to an embodiment of the present invention.

[0169] FIG. 11 illustrates simulation results regarding the performance and energy efficiency of a memory computation processing device using a weight matrix according to an embodiment of the present invention.

[0170] Referring to FIG. 11, the graph (1100) shows the simulation results for the performance and energy efficiency of a memory computation processing device using a weight matrix according to an embodiment of the present invention.

[0171] Graph (1100) shows performance and energy efficiency normalized to space A values ​​for matrix 1, matrix 2, matrix 3, matrix 4 and matrix 5.

[0172] Matrix 1 to Matrix 3 are lacking mainly in terms of energy efficiency, but they are less efficient because the utilization of in-bank parallel operations is low due to the small matrix size or row invalidation.

[0173] On the other hand, it can be seen that the energy efficiency of matrices 4 and 5 is significantly improved.

[0174] Improved efficiency allows energy costs to be considered effective across a vast amount of computation because it makes full use of in-bank parallel processing.

[0175] The matrix fixed method performs multiplication within the bank and can accumulate partial sums in surrounding MACs according to the output fixed data flow.

[0176] Compared to DRAM-based PIM, the present invention achieves up to 205 times performance improvement and up to 24 times less energy consumption.

[0178] FIGS. 12 to 14 are drawings illustrating a memory operation processing method using a weight matrix according to an embodiment of the present invention.

[0179] FIG. 12 illustrates a procedure for a memory operation processing method using a weight matrix according to an embodiment of the present invention, which performs memory operations after processing a permutation weight matrix and a sort weight matrix.

[0180] Referring to FIG. 12, step (S1201) of the memory operation processing method using a weight matrix according to an embodiment of the present invention inputs a weight matrix to be mapped to memory.

[0181] That is, the memory operation processing method using a weight matrix according to one embodiment of the present invention can process inputs for a weight matrix and dense vectors that are targets of memory operation.

[0182] In step (S1202), the memory operation processing method using a weight matrix according to an embodiment of the present invention performs an output recognition matrix permutation by rearranging the rows of the weight matrix.

[0183] That is, a memory operation processing method using a weight matrix according to an embodiment of the present invention can be processed into a permutation weight matrix by considering the number of negative values ​​among the positive and negative values ​​constituting each row in the rows constituting the input weight matrix, and by replacing the position of the negative value in the selected at least one row with the positive value of another row with that of the selected at least one row.

[0184] In step (S1203), the memory operation processing method using a weight matrix according to an embodiment of the present invention sorts the columns according to the number of "0"s in the weight matrix.

[0185] That is, a memory operation processing method using a weight matrix according to an embodiment of the present invention can search for one row based on the number of negative values ​​included in the rows of a permutation weight matrix, and process the columns of the permutation weight matrix from one side by sorting the positions of the columns according to whether there is a negative value on one side of the searched row, thereby processing it into a sorted weight matrix.

[0186] In step (S1204), the memory operation processing method using a weight matrix according to an embodiment of the present invention performs matrix operations by mapping the weight matrix to a memory bank.

[0187] That is, a memory operation processing method using a weight matrix according to an embodiment of the present invention can divide and map operation components and dense vectors constituting an alignment weight matrix to banks of memory, taking into account the size of each of the banks, and process operations of the components and dense vectors mapped to the banks.

[0188] FIG. 13 illustrates a permutation weight matrix processing procedure in a memory operation processing method using a weight matrix according to an embodiment of the present invention.

[0189] Referring to FIG. 13, in step (S1301), a memory operation processing method using a weight matrix according to an embodiment of the present invention receives a weight matrix.

[0190] In step (S1302), the memory operation processing method using a weight matrix according to an embodiment of the present invention searches for the row with the fewest number of "0s" among the unprocessed rows in the weight matrix. Here, "0" may be referred to as a negative value.

[0191] That is, the memory operation processing method using a weight matrix according to one embodiment of the present invention calculates a matching score based on the number of "0"s in the row.

[0192] A memory operation processing method using a weight matrix according to one embodiment of the present invention increases the matching score if there is a "0" in the row, and decreases the matching score if there is no "0".

[0193] In step (S1303), the memory operation processing method using a weight matrix according to an embodiment of the present invention determines whether a search result exists, and if a search result exists, proceeds to step (S1304), and if not, terminates the procedure.

[0194] In step (S1303), the memory operation processing method using a weight matrix according to an embodiment of the present invention searches for the row with the highest matching score with the corresponding row and updates the score record.

[0195] In step (S1305), the memory operation processing method using a weight matrix according to an embodiment of the present invention determines whether the score record converges to 0.

[0196] A memory operation processing method using a weight matrix according to an embodiment of the present invention moves to step (S1307) when the score record converges to 0, marks the corresponding row as processed, and proceeds to step (S1308).

[0197] A memory operation processing method using a weight matrix according to one embodiment of the present invention proceeds to step (S1306) if the score record is greater than 0.

[0198] In step (S1306), the memory operation processing method using a weight matrix according to an embodiment of the present invention assigns the value of the target row to the "0" value position of the corresponding row.

[0199] That is, a memory operation processing method using a weight matrix according to one embodiment of the present invention replaces a positive value ("1") with a corresponding negative value ("0") in a row other than the current row.

[0200] In step (S1308), the memory operation processing method using a weight matrix according to an embodiment of the present invention performs an update by reflecting the processing result on the weight matrix.

[0201] That is, the memory operation processing method using a weight matrix according to one embodiment of the present invention terminates the permutation process for the weight matrix by updating the permutation result in which the positions of "0" and "1" are substituted and that there are no more targets for permutation processing.

[0202] According to one embodiment of the present invention, a memory operation processing method using a weight matrix can be performed by repeatedly carrying out steps (S1302) to (S1305) to process the weight matrix as in step (S1308).

[0203] Accordingly, the present invention can reduce the computational complexity of the components of the weight matrix and dense vectors by excluding rows composed only of negative values ​​from the mapping to the memory bank by substituting the components of the rows so that the bits constituting the rows in the weight matrix are composed only of negative values ​​("0") based on an output recognition matrix permutation algorithm.

[0204] FIG. 14 illustrates a procedure for processing a sorted weight matrix using a memory operation processing method according to an embodiment of the present invention.

[0205] Referring to FIG. 14, in step (S1401), the memory operation processing method using a weight matrix according to an embodiment of the present invention receives the weight matrix processed in FIG. 13.

[0206] In step (S1402), the memory operation processing method using a weight matrix according to an embodiment of the present invention searches for a non-zero value (A) on the far left of the matrix row.

[0207] That is, the memory operation processing method using a weight matrix according to one embodiment of the present invention sequentially searches for (A) to see if there is a "1" at the leftmost row of the matrix. Here, (A) may correspond to "1".

[0208] In step (S1403), the memory operation processing method using a weight matrix according to an embodiment of the present invention searches for the "0" value (B) at the far left of the matrix row.

[0209] That is, the memory operation processing method using a weight matrix according to one embodiment of the present invention searches for (B) sequentially starting from the leftmost row of the matrix.

[0210] In step (S1404), the memory operation processing method using a weight matrix according to an embodiment of the present invention proceeds to step (S1405) or step (S1406) depending on whether (B) exists.

[0211] In step (S1405), the memory operation processing method using a weight matrix according to an embodiment of the present invention swaps the positions of (B) and (A) and updates by recording the change.

[0212] Recognizing the "0" value and swapping the positions of (B) and (A) can be seen as column alignment.

[0213] According to one embodiment of the present invention, a memory operation processing method using a weight matrix according to one embodiment of the present invention can organize the columns of a weight matrix in descending order by repeatedly processing steps (S1402) to (S1405).

[0214] After step (S1405), the memory operation processing method using a weight matrix according to an embodiment of the present invention returns to step (S1402) and searches for the next "0".

[0215] In step (S1406), the memory operation processing method using a weight matrix according to an embodiment of the present invention checks whether the row to be sorted is the last one as (B) does not exist, and in step (S1407), proceeds to the next matrix row.

[0216] Accordingly, the present invention can reduce the computational complexity of the components of the weight matrix and dense vectors by sorting the columns of the matrix in descending order according to the number of negative values ​​("0") in the rows based on a zero-recognition matrix column sorting algorithm.

[0218] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0219] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0220] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0221] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below. Explanation of the symbols

[0222] 100: Memory processing unit 110: Input processing unit 120: Permutation processing unit 130: Sorting processing unit 140: Matrix operation processing unit

Claims

Claim 1 An input processing unit that processes inputs for a weight matrix and dense vectors to be subject to memory operations; a permutation processing unit that selects at least one row from the rows constituting the input weight matrix by considering the number of negative values ​​among the positive and negative values ​​constituting each of the rows, and processes it into a permutation weight matrix by replacing the position of the negative value in the selected at least one row with the positive value of a row other than the selected at least one row; and a sorting processing unit that searches for a row based on the number of negative values ​​included in the rows in the permutation weight matrix, and processes it into a sorted weight matrix by sorting the positions of the columns of the permutation weight matrix from one side according to whether a negative value exists on one side of the searched row. A memory operation processing device characterized by including a matrix operation processing unit that divides and maps the operation components constituting the alignment weight matrix and the dense vectors to the banks of the memory, taking into account the size of each of the banks, and processes the operations of the components and the dense vectors mapped to the banks. Claim 2 A memory operation processing device according to claim 1, wherein the permutation processing unit selects at least two rows containing negative values ​​among the rows, searches for the number of negative values ​​constituting each of the selected at least two rows, calculates a matching score based on the number of negative values ​​searched, determines the row with the largest matching score as the permutation target row, and processes the determined permutation target row by substituting it with the positive value from the row with the smallest matching score to form the permutation weight matrix. Claim 3 A memory operation processing device according to claim 2, wherein the permutation processing unit increases the matching score when the number of searched negative values ​​increases for at least two selected rows, decreases the matching score when the number of searched negative values ​​decreases, and performs substitution with the positive value so that the matching score becomes "0" for the determined permutation target row. Claim 4 A memory operation processing device according to claim 3, wherein the permutation processing unit repeatedly performs substitution with the positive value such that the matching score becomes "0" in at least two selected rows. Claim 5 A memory operation processing device according to claim 1, wherein the sorting processing unit searches for the position of a positive value from one side in any one of the searched rows, searches for the position of a negative value located on the one side from the position of the searched positive value, and processes with the sorting weight matrix such that, by swapping the column corresponding to the position of the searched negative value and the column corresponding to the position of the searched positive value, a column containing a negative value is sequentially located from the one side, and then a column containing a positive value is located. Claim 6 A memory operation processing device according to claim 5, wherein the sorting processing unit changes the column order according to the change in the column position of the sorting weight matrix, and also changes the order of the dense vectors corresponding to the column order according to the changed column order. Claim 7 A memory operation processing device according to claim 1, wherein the matrix operation processing unit reduces the effective bit width by skipping the matrix operation for the row in which the positive value exists in the column when a positive value exists in a column on one side of the alignment weight matrix, while compensating for the matrix operation for the skipped row by delaying the matrix operation as the bits for the row in which the matrix operation was skipped are loaded into a surrounding area. Claim 8 In an input processing unit, a step of processing inputs for a weight matrix and dense vectors to be subject to memory operations; in a permutation processing unit, a step of selecting at least one row from the rows constituting the input weight matrix by considering the number of negative values ​​among the positive and negative values ​​constituting each of the rows, and processing it into a permutation weight matrix by replacing the position of the negative value in the selected at least one row with the positive value of a row other than the selected at least one row; in a sorting processing unit, a step of searching for a row based on the number of negative values ​​included in the rows of the permutation weight matrix, and processing it into a sorted weight matrix by sorting the positions of the columns of the permutation weight matrix from one side according to whether a negative value exists on one side of the searched row. A memory operation processing method characterized by including the step of, in a matrix operation processing unit, dividing and mapping the operation components constituting the alignment weight matrix and the dense vectors to the banks of the memory, taking into account the size of each of the banks, and processing the operation of the components and the dense vectors mapped to the banks. Claim 9 A memory operation processing method according to claim 8, wherein the step of processing with the permutation weight matrix comprises: selecting at least two rows containing negative values ​​among the rows; searching for the number of negative values ​​constituting each of the selected at least two rows; calculating a matching score based on the number of negative values ​​searched; determining the row with the largest matching score as the permutation target row; and processing with the permutation weight matrix by substituting the positive value from the row with the smallest matching score with the determined permutation target row. Claim 10 In claim 8, the step of processing with the sorting weight matrix comprises the step of searching for the position of a positive value from one side in any one of the searched rows, searching for the position of a negative value located on the one side from the position of the searched positive value, and processing with the sorting weight matrix such that, by swapping the column corresponding to the position of the searched negative value and the column corresponding to the position of the searched positive value, a column containing a negative value is sequentially located from the one side, and then a column containing a positive value is located.

Images