Accelerator for deep neural network and its operation method

Abstract

A computation accelerator according to one aspect of the present invention comprises: a global buffer in which first data or second data is temporarily stored in the form of a dense matrix or a compressed sparse matrix; a first decompression unit that decompresses the first data output by the global buffer when the first data is in the form of a compressed sparse matrix; a second decompression unit that decompresses the second data output by the global buffer when the second data is in the form of a compressed sparse matrix; and a computation unit that performs a computation on the first data in the form of a dense matrix received from the global buffer or the first data in the form of a dense matrix decompressed through the first decompression unit, and on the second data in the form of a dense matrix received from the global buffer or the second data in the form of a dense matrix decompressed through the second decompression unit.

Description

Technology Field

[0001] The present invention relates to a computational accelerator for a deep neural network and a method of operation thereof. Background Technology

[0002] Recently, the scope of artificial intelligence applications has expanded and its accuracy has been continuously improving. The increase in the number of layers constituting AI and the parameters within each layer can be cited as the reason for this enhanced performance. However, this implies an increase in the amount of data required for the AI ​​computation process. Consequently, this entails not only increased energy consumption due to the rise in the number of computations but also increased time and energy consumption required to transfer and store data between data storage devices and computation accelerators.

[0003] Furthermore, if the size of the data required for artificial intelligence computation exceeds the capacity of the accelerator's internal storage, it is necessary to repeatedly exchange data with external storage devices such as DRAM. Communication with external storage devices incurs greater time and energy consumption than communication with the accelerator's internal storage, which causes a decrease in the AI ​​accelerator's efficiency.

[0004] Various methods have been proposed to mitigate the impact of increased parameters while maintaining the performance of artificial intelligence, and one well-known method is a data compression technique such as weight pruning. Weight pruning is a technique that sets weights to zero if they are small and close to zero, or if they are deemed unimportant. By skipping operations on zeros, the amount of computation itself can be reduced, and additionally, the burden of data communication and storage can also be reduced by using a data compression format.

[0005] Generally, compression formats designed to leverage the advantages of sparse data are implemented by storing only the non-zero elements from the existing data. One representative compression format, the Compressed Sparse Column (CSC) technique, stores non-zero elements in a column-by-column manner. The stored value consists of the non-zero value itself, an index indicating the row in the weight matrix where the value is located, and a column pointer representing the cumulative sum of the number of non-zero elements in each column. Since the CSC technique selectively stores only necessary information, it stores less data compared to methods that store all values, thereby reducing the complexity of sparse neural network processing.

[0006] Data compression can provide benefits in terms of storage and movement, but specialized accelerators are required to maximize these advantages. Conventional accelerators specialized for dense matrix multiplication are optimized for processing data stored in a continuous sequence. However, when using data compression formats, as mentioned above, data is stored in a non-continuous order, and its original coordinates are stored indirectly. Therefore, in the process of handling matrix multiplication, a fundamental operation in artificial intelligence, it becomes necessary to calculate the original coordinates of the compressed data and, based on this, identify the pairs of data to be multiplied.

[0007] Reflecting this need, various accelerators have been developed to efficiently process sparse neural networks.

[0008] Figure 1 illustrates the configuration of a commonly used artificial intelligence accelerator.

[0009] (a) illustrates the structure of a dense neural network accelerator performing dense matrix multiplication, and (b) illustrates the structure of a sparse neural network accelerator performing sparse matrix multiplication.

[0010] In the case of the dense neural network accelerator illustrated in (a), since it reads data in a fixed pattern and operates consistently, each processing element (PE) executing the operation includes only a minimal buffer that stores only the data necessary for the operation and an arithmetic unit. Therefore, the energy required for computation and the area required for design are relatively low. However, because it operates according to fixed rules, it is not suitable for processing compressed data with irregular characteristics, which presents a limitation in that it cannot utilize the advantages of sparse neural networks.

[0011] Furthermore, the PEs of the sparse neural network accelerators illustrated in (b) include an index matching device to find data pairs for actual computation using the original coordinates. Additionally, the number of computation executions in the PE is determined by the number of data pairs; since there is a high probability that the variation in the number of data pairs processed by each PE will be small when checking a large amount of data to find data pairs, a large buffer is used to store a large amount of data. Due to the influence of these additional devices, the PEs included in sparse neural network accelerators require a larger area compared to the PEs of dense neural network accelerators. Moreover, additional energy consumption beyond what is necessary occurs, such as storing a large amount of data in the buffer and performing multiplication and addition operations during the process of finding data pairs.

[0012] Since sparse neural network accelerators are specialized for processing irregular data, their performance varies depending on the sparsity of the network. When examining the sparsity distribution of existing sparse neural networks, data in some layers exhibits near-dense sparsity; consequently, using compression methods may require more storage capacity. This implies that there are inefficiencies in the process of handling the entire network, and to efficiently process the entire network, it is necessary to maintain consistent performance regardless of the characteristics of each individual layer constituting the network.

[0013] To solve these problems, the present invention proposes a computational accelerator capable of efficiently processing sparse neural networks stored in a compressed format while utilizing a conventional dense neural network accelerator with a simple structure. Prior art literature

[0014] Republic of Korea Registered Patent No. 10-2649482 (Title of Invention: Neural Processing Accelerator) The problem to be solved

[0015] To solve the aforementioned problems, the present invention aims to provide a computational accelerator and a method of operation thereof that can process sparse matrix data stored in a compressed format while utilizing a dedicated operator for dense matrix multiplication.

[0016] However, the technical problems that this embodiment aims to solve are not limited to the technical problems described above, and other technical problems may exist. means of solving the problem

[0017] As a technical means for achieving the technical problem described above, a computation accelerator according to one aspect of the present invention comprises: a global buffer in which first data or second data is temporarily stored in the form of a dense matrix or a compressed sparse matrix; a first decompression unit that decompresses the first data output by the global buffer when the first data is in the form of a compressed sparse matrix; a second decompression unit that decompresses the second data output by the global buffer when the second data is in the form of a compressed sparse matrix; and a computation unit that performs a computation on the first data in the form of a dense matrix received from the global buffer or the first data in the form of a dense matrix decompressed through the first decompression unit, and on the second data in the form of a dense matrix received from the global buffer or the second data in the form of a dense matrix decompressed through the second decompression unit.

[0018] Additionally, a method of operation of a computation accelerator according to another aspect of the present invention comprises: (a) a step of decompressing the first data output by a global buffer into a compressed sparse matrix form and transmitting it to a computation unit, or decompressing the first data into a dense matrix form and transmitting it to the computation unit, or decompressing the first data into a dense matrix form and transmitting it to the computation unit, or decompressing the second data output by the global buffer into a compressed sparse matrix form and transmitting it to the computation unit, or decompressing the second data into a dense matrix form and transmitting it to the computation unit, or decompressing the second data into a dense matrix form and transmitting it to the computation unit; and (c) a step of performing a computation on the first data in a dense matrix form received from the global buffer or the first data in a dense matrix form decompressed through the first decompressing unit, and on the second data in a dense matrix form received from the global buffer or the second data in a dense matrix form decompressed through the second decompressing unit. Effects of the invention

[0019] According to the means for solving the aforementioned problem, by using a processing unit with a simple structure for processing dense matrix data, it becomes possible to perform multiplication operations based on the dense matrix form even on compressed matrix data. In particular, since the global buffer only needs to store the necessary data in a compressed form, the energy consumed for data storage in the global buffer can be reduced. Furthermore, by configuring the operation unit with a processing unit with a simple structure for processing dense matrix data, the area required during accelerator design can also be reduced. Additionally, since data is processed in a specific order, a device for index matching is not required, thereby reducing the additional area required and the energy consumption required during operation execution. Brief explanation of the drawing

[0020] Figure 1 illustrates the configuration of a commonly used artificial intelligence accelerator. FIGS. 2 and FIGS. 3 illustrate the configuration of a computational accelerator according to one embodiment of the present invention. FIG. 4 illustrates the decompression process of a computational accelerator according to one embodiment of the present invention. FIG. 5 is a flowchart illustrating the operation method of a computational accelerator according to one embodiment of the present invention. FIG. 6 is a flowchart illustrating a decompression method of a computational accelerator according to one embodiment of the present invention. Specific details for implementing the invention

[0021] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0022] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other components interposed between them. Furthermore, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0023] In this specification, the term "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more pieces of hardware, and two or more units may be realized by one piece of hardware. Meanwhile, "part" is not limited to software or hardware, and "part" may be configured to reside in an addressable storage medium or configured to run on one or more processors. Accordingly, as an example, "part" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts." In addition, the components and '~parts' may be implemented to regenerate one or more CPUs within the device.

[0024] FIGS. 2 and FIGS. 3 illustrate the configuration of a computational accelerator according to one embodiment of the present invention, and FIG. 4 illustrates the decompression process of a computational accelerator according to one embodiment of the present invention.

[0025] The computation accelerator (10) includes a first decompression unit (100), a second decompression unit (200), a computation unit (300), and a global buffer (400).

[0026] First, the global buffer (400) stores input data and weight data received from an external device or external memory. The input data may be an activation value output from the immediate previous layer among the layers constituting the learning model and passed to the next layer. Additionally, the weight data may represent a weight multiplied by the activation value, or a weight in a perceptron simulating a neuron. At this time, the input data or weight data is temporarily stored in the global buffer (400) in the form of a dense matrix or a compressed sparse matrix. At this time, a CSC or CSR (Compressed Sparse Row) compression technique may be used as the compression technique.

[0027] And, the first decompression unit (100) performs the role of decompressing the input data output by the global buffer (400) when the input data is in the form of a compressed sparse matrix. Also, the second decompression unit (200) performs the role of decompressing the weight data output by the global buffer (400) when the weight data is in the form of a compressed sparse matrix. Each decompression unit (100, 200) decompresses the data compressed through CSC or CSR compression and transmits it to the operation unit (300).

[0028] Additionally, the computation accelerator (10) may further include a first multiplexer (150) that selectively transmits either the output of the global buffer (400) or the output of the first decompression unit (100) to the computation unit (300), and a second multiplexer (250) that selectively transmits either the output of the global buffer (400) or the output of the second decompression unit (200) to the computation unit (300).

[0029] The first mux (150) selects the output of the global buffer (400) and outputs it to the operation unit (300) when the input data is in the form of a dense matrix, and selects the output of the first decompression unit (100) and outputs it to the operation unit (300) when the input data is in the form of a compressed sparse matrix. Additionally, the second mux (250) selects the output of the global buffer (400) and outputs it to the operation unit (300) when the weight data is in the form of a dense matrix, and selects the output of the second decompression unit (200) and outputs it to the operation unit (300) when the weight data is in the form of a compressed sparse matrix.

[0030] For example, as illustrated in FIG. 3(b), when both the input data and the weight data are in the form of a dense matrix, the first multiplexer (150) and the second multiplexer (250) select the global buffer (400) and bypass the first decompression unit (100) and the second decompression unit (200) to directly transmit the input data and the weight data to the operation unit (300). Alternatively, when the input data is in the form of a dense matrix and the weight data is in the form of a compressed matrix, as illustrated in FIG. 3(a), the weight data is allowed to pass through the second decompression unit (200), and the input data is allowed to bypass the first decompression unit (100). Also, when the input data is in the form of a compressed matrix and the weight data is in the form of a dense matrix, the input data is allowed to pass through the first decompression unit (100), and the weight data is allowed to bypass the second decompression unit (200). Through this operation, input data in the form of a dense matrix and weight data in the form of a dense matrix are transmitted to the operation unit (300).

[0031] The operation unit (300) includes a plurality of processing units (310, PE) and an accumulation unit (320). The plurality of processing units (310) are included in an array form, which can be implemented as a systolic array. The operation unit (300) performs operations on dense matrix-shaped input data received from the global buffer (400) or dense matrix-shaped input data decompressed through the first decompression unit (100), and on dense matrix-shaped weight data received from the global buffer (400) or dense matrix-shaped weight data decompressed through the second decompression unit (200). Thus, the operation unit (300) of the present invention is composed of a processing unit (310) that performs operations on dense matrix-shaped data.

[0032] And, each processing unit (310) can perform multiplication on the input data and weight data received from the outside, and then add it with the partial sum output from the previous processing unit (310) and transmit it to the next processing unit. Then, the accumulation unit (320) receives the output of each processing unit, accumulates and sums it, and transmits it to the global buffer (400) so that it can be output to an external memory or external device.

[0033] Now, let us examine the detailed configuration of the first decompression unit (100) and the second decompression unit (200). Since the first decompression unit (100) and the second decompression unit (200) are substantially identical in specific configuration, differing only in the target data to be decompressed, the detailed configuration will be explained based on the first decompression unit (100).

[0034] The first decompression unit (100) may include a pointer buffer (110), a non-zero buffer (120), an element selection unit (130), a dense mapping unit (140), and a dense format buffer (150).

[0035] Referring to FIG. 2 and FIG. 4, we will examine the detailed configuration and operation method of the decompression unit (100).

[0036] First, let us examine the original matrix data before compression is performed and the compressed sparse matrix data. As illustrated, the original matrix data is a dense matrix data in which the values ​​of at least one non-zero element are recorded according to a general matrix.

[0037] As described, for the raw matrix data, it is assumed that a and b are stored in the first column, c, d, and e are stored in the second column, f is stored in the third column, and g is stored in the fourth column. When this raw matrix is ​​compressed according to the CSC format, a pointer and an index are determined for each value. The CSC format consists of three parts: the value of the non-zero element, an index representing the row number where each value is located, and a pointer representing the cumulative sum of the number of non-zero element values ​​in each column. Meanwhile, the CSR format may consist of the value of the non-zero element, an index representing the column number where each value is located, and a pointer representing the cumulative sum of the number of non-zero element values ​​in each row.

[0038] The pointer buffer (110) temporarily stores pointers among the compressed sparse matrix data. These pointers may be extracted from the compressed sparse matrix data stored in the global buffer (400). As previously explained, in the case of the CSC format, the pointer is the cumulative sum of the number of non-zero element values ​​for each column. In the case of the CSR format, the pointer is the cumulative sum of the number of non-zero element values ​​for each row.

[0039] The non-zero buffer (120) stores the value of a non-zero element in the compressed sparse matrix data and an index indicating the row or column location where the non-zero element value is located. In the case of the CSC format, the index indicates the row location where the non-zero element value is located, and in the case of the CSR format, it may indicate the column location where the non-zero element value is located.

[0040] The element selection unit (130) selects the necessary data from the non-zero buffer (120) based on the pointer stored in the pointer buffer (110), particularly using the relative index to be described later. Then, the dense mapping unit (140) decompresses the non-zero data by positioning it at a location corresponding to the index of the data where the data selected by the element selection unit (130) is stored, and stores it sequentially in the dense format buffer (150).

[0041] The dense format buffer (150) temporarily stores the decompressed data until the total number of data items is sufficient for one operation of the entire array of each processing unit (310) of the operation unit (300), and then transmits the data to the operation unit (300).

[0042] In the original matrix illustrated as an example, a, b, c, d, e, f, and g each represent the values ​​of non-zero elements. Then, the row numbers where each value is located are recorded from the first row (0) to the fourth row (3). That is, 0 is recorded for the indices of a, c, and g, 1 is recorded for the index of d, 2 is recorded for the indices of b and f, and 3 is recorded for the index of e. Next, the initial value of the pointer is set to 0, and the number of non-zero element values ​​for each column is accumulated and recorded. That is, after 0 is recorded as the initial value, the number of non-zero element values ​​in the first column (2) is accumulated and recorded as 2, the number of non-zero element values ​​in the second column (3) is accumulated and recorded as 5, the number of non-zero element values ​​in the third column (1) is accumulated and recorded as 6, and the number of non-zero element values ​​in the fourth column (1) is accumulated and recorded as 7. In this way, CSC format data can be generated for the raw matrix, and when this compressed data is transferred to the decompression unit (100) through the global buffer (400), decompression is performed on the compressed data.

[0043] First, the decompression unit (100) obtains a value corresponding to a pointer from the compressed data and stores it in the pointer buffer (110) (S110). It receives a value corresponding to a pointer from the CSC format data stored in the global buffer (400) and stores it in the pointer buffer (110).

[0044] Next, a relative index is calculated based on the values ​​stored in the pointer buffer (110). Among the values ​​stored in the pointer buffer (110), values ​​stored at adjacent positions are sequentially subtracted, and the difference in size between each value is set as the relative index (S120). Referring to the illustrated example, the difference in size (2) between the first pointer (0) and the second pointer (2) stored at the very front is calculated as the relative index, the difference in size (3) between the second pointer (2) and the third pointer (5) is calculated as the relative index, and the difference in size (1) between the third pointer (5) and the fourth pointer (6) is sequentially calculated as the relative index.

[0045] Then, the index and the value of a non-zero element are received from the CSC format data stored in the global buffer (400), and these are matched and stored in the non-zero buffer (120) respectively (S120). That is, the value of the non-zero element and the index representing the row number where the value is located are stored together in the individual buffers of the non-zero buffer (120). In this way, the inclusion of the value of the non-zero element and the index information where the value is located is defined as a non-zero element. That is, the non-zero element is stored in the non-zero buffer (120), and the value of the non-zero element and the index information where the value is located are stored in a matched state. As illustrated, among the values ​​stored in the global buffer (400), the non-zero element (0 / a) in which the first value (a) and its index (0) are matched, and the non-zero element (2 / b) in which the second value (b) and its index (2) are matched are each stored in the non-zero buffer (120). In this case, the index may represent the row number where the non-zero element value is located in the CSC format, and the column number where the non-zero element value is located in the CSR format.

[0046] Next, in accordance with the order of the previously calculated relative indices, a number of theoretical elements corresponding to the value of each relative index are selected from among the theoretical elements stored in the theoretical buffer (120) and then sequentially stored in the dense mapping unit (140) (S130). As illustrated, among the values ​​stored in the theoretical buffer (120), two theoretical elements (0 / a, 2 / b) are selected in order according to the first relative index of 2 and then stored in the dense mapping unit (140). Subsequently, three theoretical elements (0 / c, 1 / d, 3 / e) located next to them are selected in order according to the second relative index of 3 and then stored in the dense mapping unit (140).

[0047] Next, the dense mapping unit (140) stores the value of a non-zero element in the dense format buffer (150) based on the non-zero element received in the previous step (S140). First, the dense format buffer (150) is recorded with all values ​​initialized to 0. Then, based on the information of the non-zero element selected in the previous step, the value of a non-zero element is stored in the dense format buffer (150) with the index as the address. For example, the value of a non-zero element (a) among the information of the first non-zero element (0 / a) can be stored in the first buffer of the dense format buffer (150). Then, the value of a non-zero element (b) among the information of the second non-zero element (2 / b) can be stored in the third buffer of the dense format buffer (150) at an address separated by index (2). An example of the dense format buffer (150) in which the value of a non-zero element is recorded through this process can be seen in the drawing of the next step (S150). This is the same state as the data stored in the first column of the original matrix, confirming that the CSC format data has been decompressed and restored to be identical to the original matrix.

[0048] And, in this way, the values ​​recorded in the dense format buffer (150) are each transmitted to the PE array of the operation unit (300) for each cycle (S150). If the values ​​recorded in the dense format buffer (150) correspond to a preset amount, the corresponding data are output to each PE. At this time, as illustrated, the output values ​​(a, 0, b, 0) in the individual buffer units constituting the dense format buffer (150) can be transmitted to each processing unit (310).

[0049] FIG. 5 is a flowchart illustrating the operation method of a computational accelerator according to one embodiment of the present invention, and FIG. 6 is a flowchart illustrating the decompression method of a computational accelerator according to one embodiment of the present invention.

[0050] If the input data output by the global buffer (400) is in the form of a compressed sparse matrix, the compression is decompressed through the first decompression unit (100) and then transmitted to the operation unit (300), or if the input data is in the form of a dense matrix, the input data is directly transmitted to the operation unit (S210).

[0051] Additionally, if the weight data output by the global buffer (400) is in the form of a compressed sparse matrix, the compression is decompressed through the second decompression unit (200) and then transmitted to the operation unit (300), or if the weight data is in the form of a dense matrix, the corresponding weight data is transmitted directly to the operation unit (300) (S220). Meanwhile, step (S210) and step (S220) may proceed together, step (S220) may proceed first, or step (S210) may proceed first. For example, the weight data and input data may be transmitted together to the operation unit (330), the weight data may be transmitted to the input data after the weight data is transmitted, or the weight data may be transmitted after the input data is transmitted first.

[0052] Then, an operation is performed on the dense matrix type input data received from the global buffer (400) or the dense matrix type input data decompressed through the first decompression unit (100), and on the dense matrix type weight data received from the global buffer (400) or the dense matrix type weight data decompressed through the second decompression unit (200) (S230).

[0053] Next, we examine the process of decompressing each decompression unit (100, 200) shown in Fig. 6.

[0054] First, pointers among the compressed sparse matrix data are temporarily stored in the pointer buffer (110) (S211).

[0055] Next, the difference in magnitude of each pointer value, calculated by sequentially subtracting the values ​​stored at adjacent positions among the pointers stored in the pointer buffer (110), is set as a relative index (S213).

[0056] Next, the non-zero elements containing the values ​​and indices of non-zero elements among the compressed sparse matrix data are stored in the non-zero buffer (120) (S215). At this time, the process may be carried out in such a way that only some of the non-zero elements are stored in the non-zero buffer (120).

[0057] In accordance with the order of relative indices, a number of theoretical elements corresponding to the value of each relative index are sequentially selected from the theoretical elements stored in the theoretical buffer (120), and the value of a non-zero element included in each selected theoretical element is stored in the dense format buffer (150) according to the index stored by matching that value (S217). At this time, when a certain number of theoretical elements are recorded in the dense format buffer (150), the step (S217) of selecting the remaining theoretical elements from the theoretical buffer (120) and storing them in the dense format buffer (150) can be repeated as in the preceding step (S215).

[0058] If the uncompressed data stored in the dense format buffer (150) corresponds to a preset amount, the uncompressed data is transferred to the operation unit (300) (S219).

[0059] The present invention enables multiplication operations based on the dense matrix form to be performed even on compressed matrix data by utilizing a processing unit with a simple structure for processing dense matrix data. In particular, since the global buffer only needs to store the necessary data in a compressed form, the energy consumed for data storage in the global buffer can be reduced. That is, in the present invention, a decompression unit (100) is inserted between the global buffer (400) and the processing unit (310), and since the global buffer (400) only needs to store the necessary data in a compressed form, the energy consumed for storage in the global buffer (400) can be reduced. In addition, by configuring the operation unit with a processing unit with a simple structure for processing dense matrix data, the area required during accelerator design can also be reduced.

[0060] In addition, since data is processed in a fixed order, a device for index matching is not required, which reduces the additional area required and the energy consumption needed during operation execution.

[0061] In addition, the present invention enables the use of data of various compressed formats received from the outside by using a decompression unit. Accordingly, the required storage capacity is reduced, and a reduction in the energy required for data communication and data storage can be expected.

[0062] Furthermore, matrices used in the artificial intelligence processing are divided into small matrices called tiles for processing. By using the compression method of the present invention, it becomes possible to record more tiles into memory of the same capacity, thereby enabling a single tile to be processed with more tiles. If the amount of uncompressed data exceeds the storage capacity of the accelerator, the same data may be exchanged multiple times, significantly increasing the burden on data communication; however, this can be prevented by applying the compression method.

[0063] In addition, the present invention enables artificial intelligence computation processing regardless of whether data is compressed by changing the use of each device. Existing sparse neural networks have different sparsities depending on each layer and the given data. In this case, when the data is relatively dense, using a compressed format may occupy more capacity than storing it with zeros. Since sparse neural network accelerators process data by storing it in a compressed format even in such cases, there is unnecessary energy consumption in memory and index matching devices; however, the present invention can process data like a dense matrix multiplication accelerator by using a method of storing data in an uncompressed dense matrix form and bypassing the decompression unit.

[0064] Additionally, the present invention has the advantage of being capable of handling the processing of new neural network models. Conventional sparse neural networks typically require additional processing after the creation of a general dense neural network. Since the optimal methods and necessary variables for this additional processing vary by neural network, time is required to find the appropriate values. This implies that the latest neural network models cannot be processed directly in a sparse neural network accelerator. However, the present invention has the advantage of being able to handle the latest training models of artificial intelligence accelerators by operating as a general dense matrix multiplication accelerator when processing the latest neural network models for which sparse neural networks have not yet been developed.

[0065] A method according to one embodiment of the present invention may also be implemented in the form of a recording medium comprising computer-executable instructions, such as a program module executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media. Additionally, a computer-readable medium may include a computer storage medium. A computer storage medium includes both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data.

[0066] Although the method and system of the present invention have been described in relation to specific embodiments, some or all of their components or operations may be implemented using a computer system having a general-purpose hardware architecture.

[0067] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0068] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0069] 10: Computational Accelerator 100: First decompression section 110: Pointer buffer 120: Non-zero buffer 130: Element selection section 140: Dense mapping section 150: Dense format buffer 200: Second decompression section 300: Operation unit

Claims

Claim 1 In a computational accelerator, a global buffer in which first data or second data is temporarily stored in the form of a dense matrix or a compressed sparse matrix; a first decompression unit that decompresses the first data output by the global buffer when the first data is in the form of a compressed sparse matrix; and a second decompression unit that decompresses the second data output by the global buffer when the second data is in the form of a compressed sparse matrix. and includes an operation unit that performs operations on first data in the form of a dense matrix received from the global buffer or first data in the form of a dense matrix decompressed through the first decompression unit, and second data in the form of a dense matrix received from the global buffer or second data in the form of a dense matrix decompressed through the second decompression unit, wherein the operation unit is an operator dedicated to dense matrix multiplication, and the first decompression unit or the second decompression unit each include a pointer buffer, a non-zero buffer, and a dense format buffer, wherein the pointer buffer temporarily stores pointers among the compressed data in the form of a sparse matrix, and the non-zero buffer stores non-zero elements including the value and index of a non-zero element among the compressed data in the form of a sparse matrix, and the first decompression unit or the second decompression unit each sets the difference in magnitude of each pointer value calculated by sequentially subtracting values ​​stored at adjacent positions among the pointers stored in the pointer buffer as a relative index, and in accordance with the order of the relative index, each among the non-zero elements stored in the non-zero buffer A computation accelerator that sequentially selects elements for a number of topics corresponding to the value of a relative index, and stores the value of a non-zero element included in each selected topic in the dense format buffer according to the index stored by matching the value. Claim 2 A computational accelerator according to claim 1, wherein the first data is input data and the second data is weight data. Claim 3 A computation accelerator according to claim 1, further comprising a first multiplexer that selectively transmits either the output of the global buffer and the output of the first decompression unit to the computation unit, and a second multiplexer that selectively transmits either the output of the global buffer and the output of the second decompression unit to the computation unit. Claim 4 A computation accelerator according to claim 3, wherein the first multiplexer selects the output of the global buffer and outputs it to the computation unit when the first data is in the form of a dense matrix, and selects the output of the first decompression unit and outputs it to the computation unit when the first data is in the form of a compressed sparse matrix, and the second multiplexer selects the output of the global buffer and outputs it to the computation unit when the second data is in the form of a dense matrix, and selects the output of the second decompression unit and outputs it to the computation unit when the second data is in the form of a compressed sparse matrix. Claim 5 A computation accelerator according to claim 1, wherein the compressed sparse matrix data is compressed according to a CSC (Compressed Sparse Column) technique or a CSR (Compressed Sparse Row) technique, and when compressed according to the CSC technique, the compressed sparse matrix data includes a non-zero element value, an index indicating the row number where the non-zero element value is located, and a pointer that accumulates and sums the number of non-zero element values ​​in each column, and when compressed according to the CSR technique, the compressed sparse matrix data includes a non-zero element value, an index indicating the column number where the non-zero element value is located, and a pointer that accumulates and sums the number of non-zero element values ​​in each row. Claim 6 delete Claim 7 A computation accelerator according to claim 1, wherein the first decompression unit or the second decompression unit transmits the decompression data to the computation unit when the decompression data stored in the dense format buffer corresponds to a preset amount. Claim 8 In a method of operation of a computation accelerator, (a) a step of decompressing the first data output by the global buffer into a compressed sparse matrix form and transmitting it to the computation unit, or, if the first data is in a dense matrix form, transmitting the first data to the computation unit; (b) a step of decompressing the second data output by the global buffer into a compressed sparse matrix form and transmitting it to the computation unit, or, if the second data is in a dense matrix form, transmitting the second data to the computation unit; and (c) a step of performing an operation on the first data in the form of a dense matrix received from the global buffer or the first data in the form of a dense matrix decompressed through the first decompression unit, and on the second data in the form of a dense matrix received from the global buffer or the second data in the form of a dense matrix decompressed through the second decompression unit, wherein the operation unit is an operator dedicated to dense matrix multiplication, and the step of the first decompression unit decompressing in step (a) comprises: a step of temporarily storing pointers among the compressed data in the form of a sparse matrix in a pointer buffer; a step of setting the difference in magnitude of each pointer value calculated by sequentially subtracting values ​​stored at adjacent positions among the pointers stored in the pointer buffer as a relative index; a step of storing non-zero elements including the value and index of a non-zero element among the compressed data in the form of a sparse matrix in a non-zero buffer; and sequentially selecting a number of non-zero elements corresponding to the value of each relative index among the non-zero elements stored in the non-zero buffer according to the order of the relative index, and for each selected non-zero element A step of storing the value of an included non-zero element in a dense format buffer according to the index stored by matching that value;A method of operation of a computation accelerator, comprising the step of transferring the decompressed data to the computation unit when the decompressed data stored in the dense format buffer corresponds to a preset amount. Claim 9 A method of operation of a computational accelerator according to claim 8, wherein the first data is input data and the second data is weight data. Claim 10 A method of operation of a computational accelerator according to claim 8, wherein step (a) is a first multiplexer connected to the global buffer and the first decompression unit, which selects and outputs the output of the global buffer when the first data is in the form of a dense matrix and selects and outputs the output of the first decompression unit when the first data is in the form of a compressed sparse matrix, and step (b) is a second multiplexer connected to the global buffer and the second decompression unit, which selects and outputs the output of the global buffer when the second data is in the form of a dense matrix and selects and outputs the output of the second decompression unit when the second data is in the form of a compressed sparse matrix. Claim 11 A method of operation of a computation accelerator, wherein, in claim 8, the compressed sparse matrix data is compressed according to a CSC (Compressed Sparse Column) technique or a CSR (Compressed Sparse Row) technique, and when compressed according to the CSC technique, the compressed sparse matrix data includes a value of a non-zero element, an index indicating the row number where the value of the non-zero element is located, and a pointer that accumulates and sums the number of non-zero element values ​​in each column, and when compressed according to the CSR technique, the compressed sparse matrix data includes a value of a non-zero element, an index indicating the column number where the value of the non-zero element is located, and a pointer that accumulates and sums the number of non-zero element values ​​in each row. Claim 12 delete Claim 13 In claim 8, the step of the second decompression unit decompressing in step (b) comprises: temporarily storing the pointer among the compressed sparse matrix data in a pointer buffer; storing a non-zero element including the value of a non-zero element among the compressed sparse matrix data and the index in the non-zero buffer; setting the difference in magnitude of each pointer value calculated by sequentially subtracting values ​​stored at adjacent positions among the pointers stored in the pointer buffer as a relative index; sequentially selecting a number of non-zero elements corresponding to the value of each relative index among the non-zero elements stored in the non-zero buffer according to the order of the relative index, and storing the value of a non-zero element included in each selected non-zero element in a dense format buffer according to the index stored that matches the value; and, if the decompressed data stored in the dense format buffer corresponds to a preset amount, transferring the decompressed data to the computation unit, wherein the method of operation of a computation accelerator. Claim 14 A non-transient recording medium on which a computer program is recorded for operating a computational accelerator according to any one of claims 8 through 11 and 13.

Images