Accelerator configured to perform accumulation operation on floating-point data and operation method thereof

Abstract

To provide an accelerator performing accumulation operation on floating-point data, and an operation method.SOLUTION: A method includes: loading two or more pieces of first data; finding a first exponent, which is the maximum value among exponents of the two or more pieces of first data; generating aligned first fractions by performing a bit shift on first fractions of the first data on the basis of the first exponent, and generating a first accumulated value by accumulation operation on the aligned first fractions; loading two or more pieces of second data; finding a second exponent, which is the maximum value among exponents of the two or more pieces of second data, and generating a first aligned accumulated value by a bit shift on the first accumulated value, on the basis of a difference between the second exponent and the first exponent, the second exponent being larger than the first exponent; generating aligned second fractions by a bit shift on second fractions of the second data, on the basis of the second exponent; and generating a second accumulated value by accumulation operation on the aligned second fractions and the first aligned accumulated value.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to an artificial intelligence system, and more particularly to an accelerator configured to perform accumulation operations on floating-point data and a method of operating the same. [Background technology]

[0002] Artificial intelligence (AI) is a field of computer science that aims to artificially realize various human capabilities such as learning, reasoning, and perception. Recently, AI has been widely used in diverse fields such as natural language understanding, natural language translation, robotics, artificial vision, problem solving, learning, knowledge acquisition, and cognitive science.

[0003] Artificial intelligence is realized based on various algorithms. For example, a neural network is composed of a complex network in which nodes and synapses are repeatedly connected. In the process of data moving from a current node to a next node, various signal processing may occur according to the corresponding synapse, and such a signal processing process is called a layer. That is, a neural network may include various layers that are intricately interconnected. Since the various layers included in a neural network require a large amount of calculation, various methods for optimizing this are being researched. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] US Patent Application Publication No. 2021 / 0182024 [Patent Document 2] U.S. Pat. No. 5,021,986 [Patent Document 3] US Patent Application Publication No. 2023 / 0008856 [Patent Document 4] Published Patent Application No. 2001-043067 [Patent Document 5] US Patent Application Publication No. 2021 / 0312012 [Patent Document 6] U.S. Patent No. 6,363,476 [Patent Document 7] U.S. Patent No. 9,274,750 [Non-patent literature]

[0005] [Non-Patent Document 1] “WINNING BOTH THE ACCURACY OF FLOATING POINT ACTIVATION AND THE SIMPLICITY OF INTEGER ARITHMETIC” Yulhwa Kim Et al., Published as a conference paper at ICLR 2023 Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention has been made in consideration of the above-mentioned conventional techniques, and an object of the present invention is to provide an accelerator configured to perform accumulation operations on floating-point data, and an operating method thereof, which have improved reliability and improved performance. [Means for solving the problem]

[0007] According to an embodiment of the present invention, a method of operating an accelerator configured to perform an accumulation operation on a plurality of data includes loading at least two first data of the plurality of data, searching for a first exponent that is a maximum value among exponents of the at least two first data, performing a bit shift on first mantissas of the at least two first data based on the first exponent to generate an aligned first mantissa, performing an accumulation operation on the aligned first mantissa to generate a first accumulation value, and loading at least two second data of the plurality of data. searching for a second exponent that is a maximum value among the exponents of the at least two second data, the second exponent being greater than the first exponent; performing a bit shift on the first accumulation value based on a difference between the second exponent and the first exponent to generate a first aligned accumulation value, performing a bit shift on second mantissas of the at least two second data based on the second exponent to generate an aligned second mantissa, and performing an accumulation operation on the aligned second mantissas and the first aligned accumulation value to generate a second accumulation value, wherein each of the plurality of data has a floating-point type.

[0008] According to an embodiment of the present invention, an accelerator configured to perform an accumulation operation on a plurality of data includes: a unified buffer unit configured to store the plurality of data; a pre-sorting unit configured to load at least two first data among the plurality of data, search for a first maximum exponent that is a maximum value among exponents of the at least two first data, and perform a bit shift on a mantissa of the at least two first data based on the first maximum exponent and a previous maximum exponent to generate a first sorted mantissa; a plurality of processing units configured to perform a bit shift on a previous accumulated value based on a previous maximum exponent and the first maximum exponent to generate an sorted accumulated value, and perform an accumulation operation on the sorted accumulated value and the first mantissa; and a normalization unit configured to normalize an operation result of the plurality of processing units based on the first maximum exponent to generate an output value, wherein each of the plurality of data is a floating-point type.

[0009] According to an embodiment of the present invention, a method for operating an accelerator configured to perform an accumulation operation on a plurality of data includes: performing an accumulation operation on at least two data among a plurality of data to generate a 0th maximum exponent and a 0th accumulation value; when a first exponent of a first data among the plurality of data is greater than the 0th maximum exponent, performing a bit shift on the 0th accumulation value based on the first exponent and the 0th maximum exponent to generate a 0th aligned accumulation value; and performing an accumulation operation on a first mantissa of the first data and the 0th aligned accumulation value to generate a first accumulation value, wherein each of the plurality of data has a floating-point type. Effect of the Invention

[0010] SUMMARY OF THE PRESENT DISCLOSURE In accordance with the present invention, an accelerator configured to perform accumulation operations on floating-point data and a method of operating the same are provided, with improved reliability and improved performance. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram illustrating a system according to one embodiment of the present invention. [Figure 2A] 2 is a diagram for explaining a MAC calculation performed in the accelerator of FIG. 1; [Figure 2B] 2 is a diagram for explaining a MAC calculation performed in the accelerator of FIG. 1; [Diagram 3] 2 is a flowchart illustrating the accelerator of FIG. 1. [Figure 4] 4 is a flowchart showing the operation of the accelerator of FIG. 3. [Diagram 5] FIG. 4 is a block diagram showing the pre-alignment unit of FIG. 3. [Figure 6] FIG. 4 is a timing diagram for explaining artificial intelligence calculations of the accelerator of FIG. [Figure 7] FIG. 2 is a block diagram illustrating the accelerator of FIG. 1. [Figure 8] 8 is a flowchart showing the operation of the accelerator of FIG. 7. [Figure 9] FIG. 8 is a block diagram showing the pre-alignment unit of FIG. 7. [Figure 10] 10 is a diagram for explaining the configuration of a maximum exponent searcher in FIG. 9; FIG. [Figure 11] FIG. 8 is a block diagram showing one of the processing units in FIG. 7. [Figure 12] FIG. 8 is a timing diagram illustrating the operation of the accelerator of FIG. [Figure 13] FIG. 4 is a diagram for explaining the operation of the accelerator in FIG. [Figure 14A] FIG. 8 is a diagram for explaining the operation of the accelerator in FIG. [Figure 14B] FIG. 8 is a diagram for explaining the operation of the accelerator in FIG. [Figure 14C] FIG. 8 is a diagram for explaining the operation of the accelerator in FIG. [Figure 15] 2 is a flowchart showing the operation of the accelerator of FIG. 1. [Figure 16] 1 is a block diagram illustrating a system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following, embodiments of the present invention are described clearly and in detail so that those skilled in the art can easily practice the present invention.

[0013] Terms such as "unit," "module," and the like used in the detailed description or drawings, or functional blocks shown in the drawings, are implemented in the form of hardware, software, or a combination thereof configured to perform specific functions.

[0014] 1 is a block diagram illustrating a system according to an embodiment of the present invention. Referring to FIG. 1, system 100 may include memory 110, accelerator 120, and controller 130. In one embodiment, system 100 may be dedicated hardware configured to perform processing of artificial intelligence models. For example, system 100 may be a graphics processing unit (GPU), a neural processing unit (NPU), or a separate dedicated hardware.

[0015] In one embodiment, the artificial intelligence models driven by system 100 are generated through machine learning, which may include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, although the scope of the invention is not limited in this respect.

[0016] In one embodiment, the artificial intelligence model is generated or trained through one or a combination of at least two of various neural networks such as a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDN), a deep Q-network, etc. The artificial intelligence model may include multiple neural network layers, each of which may be configured to perform an artificial intelligence operation based on a trained model or weights.

[0017] In the following, the term "artificial intelligence operations" is used to facilitate the description of embodiments of the present invention. "Artificial intelligence operations" may refer to various operations performed within the system 100 for the system 100 to train an artificial intelligence model or infer a result. As an example, the artificial intelligence operations may include multiply and accumulate (MAC) operations performed at various layers of the artificial intelligence model.

[0018] Memory 110 is configured to store various data, weights, parameters, etc. necessary for the artificial intelligence calculations of system 100. In one embodiment, memory 110 may be dynamic random access memory (DRAM). However, the scope of the invention is not limited in this respect, and memory 110 may include various forms of memory, such as SRAM, PRAM, MRAM, RRAM, FRAM, flash memory, etc.

[0019] The accelerator 120 is configured to perform an artificial intelligence operation using the data, weights, or parameters stored in the memory 110. In one embodiment, the accelerator 120 may include multiple processing elements (PEs) for the artificial intelligence operation. Each of the multiple processing elements may be configured to perform a multiply and accumulate (MAC) operation on the data, weights, or parameters stored in the memory 110. An artificial intelligence model is trained or a specific result is inferred based on the operation results of the multiple processing elements.

[0020] The controller 130 is configured to control the memory 110 and the accelerator 120. In one embodiment, the controller 130 may be a central processing unit (CPU) configured to control various operations of the system 100.

[0021] Figures 2A and 2B are diagrams for explaining a MAC operation performed in the accelerator of Figure 1. Referring to Figures 2A and 2B, the MAC operation may include one multiplication operation and one accumulation operation on two data.

[0022] For example, as shown in FIG. 2A, multiplication is performed on an N-bit weight and an N-bit activation. In this case, the multiplication result has a size of 2N bits. Then, as the multiplication results are accumulated, the accumulation result may have a size of (2N + M) bits. If the size of the accumulation result increases, the memory capacity for defining the accumulation result increases, so the size of the accumulation result needs to be reduced. Thus, the accumulation result may be normalized to N-bit output data. The normalized output data may be stored again in the memory 110. The normalized output data may be transmitted to an external device.

[0023] In one embodiment, the data of the MAC operation may have various data types. For example, the data of the MAC operation may be an integer type. Alternatively, the data of the MAC operation may be a floating point type. The floating point type is a method of expressing data in the form of a sign, a fraction, and an exponent. There are floating point types such as 32-bit single precision and 64-bit double precision.

[0024] The accuracy of the calculation result, the area of ​​the hardware structure, and the power consumption may vary depending on the type of data, the size of the data, etc. Therefore, the type of data and the size of the data may be determined in various ways depending on the purpose of the system 100.

[0025] In one embodiment, when data is of floating-point type, exponent alignment, addition, and normalization processes are required to perform data addition, and losses due to hardware resource limitations may occur during the exponent alignment, addition, and normalization processes.

[0026] For example, as shown in Figure 2B, +1.001×2 -1 and -1.001 x 2 -3 Let's assume that two floating-point data are added. The maximum exponent of the two data is -1, so the exponents of the two data are aligned to -1. In this case, both data are +1.001×2 -1 and -0.01001 x 2 -1 In this case, due to the size limit of the register for storing two pieces of data, it is expressed as -0.01001 × 2 -1 Some significant values ​​(e.g., lower bits, 01) may be lost from the mantissa (i.e., 0.01001). That is, the actual addition of the mantissas of the two data is performed for +1.001 and -0.010.

[0027] The result of this addition is +0.111 x 2 -1 Since the most significant bit of the mantissa of the sum is 0, the sum is normalized to fit the floating-point format. As a result, the normalized sum is +1.110×2 -2 On the other hand, +1.001×2 -1 and -1.001 x 2 -3 The actual result of the addition is +1.1011×2 -2 That is, a loss due to a hardware resource limitation may occur during the exponent sorting process, which may cause an error in the final addition result. In one embodiment, although not shown in the example of FIG. 2B, a similar error may occur in the normalization process for the addition result.

[0028] That is, as described above, an error may occur due to exponent alignment or normalization during the addition process of floating-point data. In the embodiment of the present invention, an accelerator structure that minimizes the above-mentioned error is proposed.

[0029] Fig. 3 is a flowchart showing the accelerator of Fig. 1. With reference to Fig. 1 and Fig. 3, the accelerator 120 may include a unified buffer unit 121, a pre-alignment unit 122, a normalization unit 123, and a plurality of processing units PE11 to PE44.

[0030] The unified buffer unit 121 may be configured to store various data, weights, or parameters for artificial intelligence calculations performed by the accelerator 120. In one embodiment, the information stored in the unified buffer unit 121 may be provided from the memory 110 or obtained from calculation results of the multiple processing units PE11 to PE44 of the accelerator 120.

[0031] The pre-alignment unit 122 is configured to perform pre-alignment on the data provided from the unified buffer unit 121. For example, the data provided from the unified buffer unit 121 may have a floating-point type. As a more detailed example, an artificial intelligence model processed by the system 100 may operate activations as floating-point types and learn weights as integer types. In this case, floating-point accumulation operations on multiple data are required. In this case, as described with reference to FIG. 2B, exponent alignment on the data is required for addition on floating-point type data.

[0032] For example, pre-sorting is necessary to add floating-point data. This allows the pre-sorting unit 122 to sort the exponents of the number of pieces of data that can be calculated in parallel by the accelerator 120. The exponent sorting is performed by sorting the maximum exponent e of the multiple pieces of data. max Search for the maximum exponent e maxIn this case, the number of data that can be operated simultaneously or in parallel is the same maximum exponent e. max Therefore, the pre-alignment unit 122 must have the maximum exponent e for all the data that can be calculated simultaneously or in parallel. max Search for the maximum exponent e max A bit shift operation is performed on the mantissas of the plurality of data based on the bit shift operation.

[0033] The processing units PE11 to PE44 may perform operations on the data input from the pre-alignment unit 122. At this time, each of the processing units PE11 to PE44 may perform a multiplication operation or an accumulation operation on the input data to generate a partial sum PSUM. The generated partial sum PSUM may be provided to the normalization unit 123.

[0034] For example, each of the multiple processing units PE11 to PE44 may receive activations from the pre-alignment unit 122 and weights from the unified buffer unit 121. Each of the multiple processing units PE11 to PE44 may perform a MAC operation on the received activations and weights to generate or accumulate partial sums PSUM. Hereinafter, to easily explain the embodiments of the present invention, the multiple processing units PE11 to PE44 will be described as performing an add operation or an accumulate operation on data or activations to generate partial sums PSUM.

[0035] In one embodiment, the processing units PE11 to PE44 perform an addition operation on the sorted mantissas (i.e., the mantissas sorted by the same exponent) generated from the pre-sorting unit 122. At this time, since the exponents corresponding to the sorted mantissas are equal to each other, the addition operation is realized through a simple integer operation, an integer addition, or an integer accumulation. That is, each of the processing units PE11 to PE44 may have the structure of an integer adder.

[0036] The normalization unit 123 receives the partial sums PSUM from each of the processing units PE11 to PE44, calculates an accumulated value of the received partial sums PSUM, and performs normalization on the calculated accumulated value to generate an output value. For example, the normalization unit 123 may max The normalization unit 123 can receive information about the received maximum exponent e max The accumulated values ​​may be normalized based on information about the generated output values. The generated output values ​​may be stored in the unified buffer unit 121. The output values ​​stored in the unified buffer unit 121 may be used as input values ​​for other artificial intelligence operations, including generating outputs for a user.

[0037] Fig. 4 is a flowchart showing the operation of the accelerator of Fig. 3. In the following, for convenience of explanation, it is assumed that the multiple processing units PE11 to PE44 of the accelerator 120 perform accumulation operations on data.

[0038] In stage S10, the accelerator 120 may load N pieces of data (N data units). For example, the pre-alignment unit 122 of the accelerator 120 may load the N pieces of data from the unified buffer unit 121. Alternatively, the accelerator 120 may load the N pieces of data from the memory 110. In one embodiment, each of the N pieces of data may have a floating-point type.

[0039] In one embodiment, N may be a natural number corresponding to the number of processing units PE11 to PE44 of the accelerator 120. For example, the accelerator 120 may perform operations on a plurality of data simultaneously or in parallel to improve the operation speed. At this time, the number of data items on which the operation is performed simultaneously corresponds to the array size of the processing units PE11 to PE44 included in the accelerator 120. That is, N may indicate the number or size of data items that can be operated on simultaneously or in parallel by the accelerator 120.

[0040] In step S20, the accelerator 120 calculates the maximum index e of the N pieces of data. max For example, each of the N pieces of data may have a floating-point type. That is, each of the N pieces of data may include a sign, a mantissa, and an exponent. The pre-alignment unit 122 of the accelerator 120 compares the exponents of the N pieces of data and finds the maximum exponent e of the largest value. max It is possible to determine the following.

[0041] In step S30, the accelerator 120 calculates the maximum exponent e max For example, as shown in FIG. 2B, when data is +1.001×2, the exponent alignment is performed on the N data to generate an aligned mantissa. -1 and -1.001 x 2 -3 In this case, the maximum exponent e max is −1, so each mantissa of the data should be aligned to +1.001 and −0.01001. The aligned mantissas may be provided to the multiple processing units PE11 to PE44.

[0042] In step S40, the accelerator 120 may generate a partial sum by accumulating the aligned mantissas using the multiple processing units PE11 to PE44. For example, each of the multiple processing units PE11 to PE44 may receive aligned mantissas from the pre-alignment unit 122. Each of the multiple processing units PE11 to PE44 may be configured to perform an accumulation operation on the aligned mantissas. In one embodiment, since the aligned mantissas are aligned based on the maximum exponent through the operation of step S30, each of the multiple processing units PE11 to PE44 may perform an accumulation operation on the aligned mantissas.

[0043] In step S50, the accelerator 120 accumulates the partial sums PSUM of each of the multiple processing units PE11 to PE44, and can normalize the accumulated result. For example, the normalization unit 123 of the accelerator 120 can receive the partial sums PSUM from each of the multiple processing units PE11 to PE44. The exponents corresponding to the multiple partial sums PSUM are the maximum exponent e max The normalization unit 123 is the same as the maximum exponent e max In one embodiment, the normalized result may be stored in the unified buffer unit 121 or memory 110, or output based on a received command or user input.

[0044] Fig. 5 is a block diagram showing the pre-alignment unit of Fig. 3. Referring to Figs. 1, 3, 4 and 5, the pre-alignment unit 122 may include a maximum exponent searcher MEF and a plurality of converters CVT1 to CVTn.

[0045] The maximum exponent finder (MEF) finds the maximum exponent e for N data DT_1 to DT_N. max For example, each of the N pieces of data DT_1 to DT_N may be a floating-point type. The maximum exponent searcher MEF compares the exponents of each of the N pieces of data DT_1 to DT_N to find the maximum exponent e max The maximum exponent e that can be found is max may be provided to each of the multiple converters CVT1 to CVTn.

[0046] Multiple converters CVT1 to CVTn have a maximum index e max For example, the first converter CVT1 may perform exponent sorting for the plurality of data DT_1 to DT_N based on the first data DT_1 and the maximum exponent e maxThe first data DT_1 may include a first mantissa M1 and a first exponent e1. The first converter CVT1 may calculate a first exponent difference Δe1, which is a difference between the first exponent e1 of the first data DT_1 and the maximum exponent emax. The first converter CVT1 may bit-shift the first mantissa M1 of the first data DT_1 by the first exponent difference Δe1 to generate a first aligned mantissa sM1. For example, if the first exponent e1 is "-5" and the maximum exponent emax is "-5", the first converter CVT1 may calculate a first exponent difference Δe1, which is a difference between the first exponent e1 of the first data DT_1 and the maximum exponent emax. max Assume that the first mantissa M1 is "-1" and the first mantissa M1 is "1.0100111". In this case, the first converter CVT1 can output the first aligned mantissa sM1 (i.e., 0.00010100111) by bit-shifting the first mantissa M1 by -4.

[0047] The second converter CVT2 receives the second data DT_2 and the maximum exponent e max The second converter CVT2 receives the second exponent e2 and the maximum exponent e of the second data DT_2. max The n-th converter CVTn may perform a bit shift on the second mantissa M2 of the second data DT_N based on a second exponent difference Δe2, which is the difference between the n-th data DT_N and the maximum exponent eM2, to output a second aligned mantissa sM2. max The n-th converter CVTn receives the n-th index eN and the maximum index e of the n-th data DT_N. max Based on the n-th exponent difference Δen, which is the difference between the n-th data DT_N and the n-th mantissa MN, a bit shift is performed on the n-th data DT_N to output the n-th aligned mantissa sMN.

[0048] In one embodiment, the aligned mantissas sM1 to sMN generated by the converters CVT1 to CVTn may be provided to the processing units PE11 to PE44. A cumulative operation may be performed on the processing units PE11 to PE44 to generate partial sums.

[0049] FIG. 6 is a timing diagram for explaining an artificial intelligence calculation of the accelerator of FIG. 3. Referring to FIG. 1, FIG. 3 and FIG. 6, the accelerator 120 may perform an artificial intelligence calculation on the first to second N-th data (DT_1 to DT_2N). The accelerator 120 may perform an artificial intelligence calculation on N pieces of data simultaneously or in parallel. In this case, the accelerator 120 performs an artificial intelligence calculation on the first to N-th data (DT_1 to DT_N) and then performs an artificial intelligence calculation on the remaining data (DT_N+1 to DT_2N).

[0050] For example, during a period t0 to t1 from the 0th time point to the 1st time point, the pre-alignment unit 122 of the accelerator 120 can load the first to Nth data (DT_1 to DT_N) from the unified buffer unit 121. After all of the first to Nth data (DT_1 to DT_N) are loaded, during a period t1 to t2 from the 1st time point to the 2nd time point, the pre-alignment unit 122 of the accelerator 120 loads the maximum exponent e among the exponents of the first to Nth data (DT_1 to DT_N). max1 Then, during a period t2 to t5 from the second time point to the fifth time point, the pre-ordering unit 122 of the accelerator 120 may load the first to Nth data (DT_1 to DT_N) again. During a period t3 to t6 from the third time point to the sixth time point, the pre-ordering unit 122 of the accelerator 120 may search for the maximum exponent e max1 Based on this, a bit shift operation may be performed on each mantissa of the first to Nth data (DT_1 to DT_N) to generate an aligned mantissa. During a period t4 to t7 from a fourth time point to a seventh time point, the processing units PE11 to PE44 of the accelerator 120 may receive the aligned mantissas from the pre-alignment unit 122 and perform an accumulation operation on the aligned mantissas. After the accumulation operation by the processing units PE11 to PE44 is completed, during a period t7 to t8 from a seventh time point to an eighth time point, the normalization unit 123 may receive partial sums from the processing units PE11 to PE44, accumulate the received partial sums, and perform normalization on the accumulated partial sums.

[0051] During a period t8 to t9 from the eighth time point to the ninth time point, the pre-alignment unit 122 of the accelerator 120 can load the N+1th to 2Nth data (DT_N+1 to DT_2N) from the unified buffer unit 121. After all of the N+1th to 2Nth data (DT_N+1 to DT_2N) are loaded, during a period t9 to t10 from the ninth time point to the tenth time point, the pre-alignment unit 122 of the accelerator 120 loads the maximum exponent e max2 Then, during a section t10 to t11 from the 10th time point to the 11th time point, the pre-ordering unit 122 of the accelerator 120 can load the (N+1)th to 2Nth data (DT_N+1 to DT_2N) again. During a section t11 to t14 from the 11th time point to the 14th time point, the pre-ordering unit 122 of the accelerator 120 can search for the maximum exponent e max2 Based on this, a bit shift operation may be performed on each mantissa of the (N+1)th to 2Nth data (DT_N+1 to DT_2N) to generate an aligned mantissa. During a period t12 to t15 from a 12th time point to a 15th time point, the processing units PE11 to PE44 of the accelerator 120 may receive the aligned mantissas from the pre-alignment unit 122 and perform an accumulation operation on the aligned mantissas. After the accumulation operation by the processing units PE11 to PE44 is completed, the normalization unit 123 may receive partial sums from the processing units PE11 to PE44, accumulate the received partial sums, and perform normalization on the accumulated partial sums during a period t15 to t16 from a 15th time point to a 16th time point.

[0052] As described above, the accelerator 120 can perform artificial intelligence operations on N pieces of data simultaneously or in parallel. Therefore, exponent sorting for N pieces of data is performed collectively based on the maximum exponent of the N pieces of data. In this case, the bit size corresponding to the mantissa of the N pieces of data may increase due to the exponent sorting, which may cause a loss of valid values ​​of the mantissa due to hardware or resource limitations. As a result, the artificial intelligence operations are performed in a state where operation errors are reflected, which may reduce the overall reliability of the operation.

[0053] In addition, when the number of data items required for the AI ​​calculation is N or more, the accelerator 120 performs the AI ​​calculation for each N data items. In this case, the number of normalizations for the output data increases, which may reduce the reliability of the calculation.

[0054] The accelerator 120 also calculates the maximum exponent e max Therefore, the pre-alignment unit 122 of the accelerator 120 must search for the maximum exponent e max For searching, all N pieces of data must be loaded. This may cause a performance degradation due to the loading of data. As described above, when the hardware structure of the accelerator 120 performs artificial intelligence calculations in a predetermined unit, there is a possibility that the overall calculation reliability and performance may be degraded.

[0055] Fig. 7 is a block diagram showing the accelerator of Fig. 1. For convenience of explanation, detailed explanations of overlapping components are omitted. With reference to Fig. 1 and Fig. 7, the accelerator 120-1 may include a unified buffer unit 121-1, a pre-alignment unit 122-1, a normalization unit 123-1, and a plurality of processing units PE11 to PE44.

[0056] The unified buffer unit 121-1 may be configured to store various data, weights, or parameters for artificial intelligence calculations performed by the accelerator 120-1. In one embodiment, the information stored in the unified buffer unit 121-1 may be provided from the memory 110 or may be obtained from the results calculated by the multiple processing units PE11 to PE44 of the accelerator 120-1.

[0057] The pre-aligner 122-1 may be configured to perform exponential alignment on the data provided from the unified buffer unit 121-1.

[0058] For example, in the embodiment described with reference to FIGS. 3 to 6, the pre-alignment unit 122 calculates the maximum exponent e for N pieces of data that can be calculated simultaneously or in parallel. max Search for the maximum exponent e max A bit shift operation is performed on N pieces of data based on the above.

[0059] On the other hand, the pre-sorting unit 122-1 in FIG. 7 searches for the maximum exponent for K pieces of data (K pieces of data units) and finds the maximum exponent e max_new and the previous maximum exponent e max_old and perform a bit shift operation on the K pieces of data based on the comparison result. Here, K is a natural number smaller than N. In an embodiment, K may be 1 / 2 of N, 1 / 3 of N, 2 / 3 of N, or another fractional form of N, but the scope of the present invention is not limited thereto. That is, the pre-alignment unit 122-1 can search for the maximum exponent regardless of the number of data and perform the bit shift operation, thereby reducing the time required to search for the maximum exponent. As a result, the power consumption of the device can be reduced.

[0060] The processing units PE11 to PE44 may perform operations on the data input from the pre-alignment unit 122. At this time, each of the processing units PE11 to PE44 may perform a multiplication operation or an accumulation operation on the input data to generate a partial sum PSUM. The generated partial sum PSUM may be provided to the normalization unit 123.

[0061] For example, according to an embodiment of the present invention, the speed, accuracy or power efficiency of the operation and communication of the device can be improved based on the above-mentioned method. Thus, the improved device and method can eliminate the shortcomings of the conventional device and method using floating-point data by neural networks, artificial intelligence, and can reduce resource consumption, improve data accuracy, and improve data clarity. For example, by using an embodiment of the present invention described, the repetitive calculation of the operation is reduced, thereby reducing the error of the calculation, improving the reliability of the cumulative calculation, and improving the speed, accuracy and power efficiency.

[0062] In one embodiment, each of the processing units PE11 to PE44 receives the maximum exponent difference Δe max The maximum exponent difference Δe max is the maximum exponent of the currently accumulated data (i.e., e max_new ) and the maximum exponent of the previously accumulated data (i.e., e max_old Each of the processing units PE11 to PE44 may include an accumulation register configured to store the result of the accumulation operation. The accumulated value stored in the accumulation register is the difference between the previous maximum exponent e max_old Therefore, the value corresponding to the currently input data (i.e., the current maximum exponent e max_new To add the accumulated value stored in the accumulation register to the current maximum exponent e (the mantissa aligned based on max_new Therefore, each of the processing units PE11 to PE44 must be aligned to the maximum exponent difference Δe max The structures and operations of the processing units PE11 to PE44 will be described in more detail with reference to the following drawings.

[0063] The normalization unit 123-1 can receive the partial sums PSUM from each of the multiple processing units PE11 to PE44, calculate an accumulated value of the received partial sums PSUM, and perform normalization on the calculated accumulated value to generate an output value.

[0064] For example, the normalization unit 123-1 normalizes the maximum exponent e max The normalization unit 123-1 is configured to convert the partial sums PSUM received from each of the multiple processing units PE11 to PE44 based on the above. The normalization unit 123-1 can accumulate or combine the converted partial sums to generate a final output value. The output value is stored in the unified buffer unit 121-1 and is used as an input value for other artificial intelligence calculations.

[0065] In one embodiment, normalizer 123-1 may include a converter configured to convert the partial sums to a floating-point type and an accumulator configured to accumulate the partial sums converted to floating-point type, although the scope of the present invention is not limited in this respect.

[0066] Fig. 8 is a flowchart showing the operation of the accelerator of Fig. 7. Referring to Fig. 7 and Fig. 8, in step S110, the accelerator 120-1 may load K pieces of data. For example, as described above, the accelerator 120 may perform artificial intelligence calculations on N pieces of data simultaneously or in parallel. Meanwhile, the accelerator 120-1 may load K pieces of data, which is smaller than N, from the unified buffer unit 121-1 or the memory 110.

[0067] In step S120, the accelerator 120-1 calculates the maximum index e for the K data. max_new For convenience of explanation, the maximum exponent for K data items is called the new maximum exponent e max_new It is called.

[0068] In step S130, the accelerator 120-1 calculates a new maximum exponent e max_new The previous maximum exponent e max_oldFor example, the pre-ordering unit 122-1 of the accelerator 120-1 may determine whether the maximum exponent of data used or converted for a previous artificial intelligence operation (hereinafter, the previous maximum exponent e max_old That is, the data used in the previous artificial intelligence calculation may include information about the previous maximum exponent e max_old In this case, the values ​​accumulated in each of the processing units PE11 to PE44 are also sorted based on the previous maximum exponent e max_old Corresponds to.

[0069] Therefore, the new maximum exponent e max_new is the previous maximum exponent e max_old If not greater than the previous maximum exponent e max_old Even if the exponent sorting for the K data is performed based on the new maximum exponent e, the accumulation operation is normally performed in the multiple processing units PE11 to PE44. max_new is the previous maximum exponent e max_old If not, in step S140, the pre-alignment unit 122-1 of the accelerator 120-1 determines the previous maximum exponent e max_old Based on this, an exponent alignment can be performed on the K data to generate an aligned mantissa.

[0070] On the other hand, the new maximum exponent e max_new is the previous maximum exponent e max_old If it is greater than 0, a bit shift is required for the accumulated value in each of the processing units PE1 to PE44. max_new is the previous maximum exponent e max_old If so, in step S150, the accelerator 120-1 calculates the maximum exponent difference Δe max In this case, the maximum exponent difference Δe max is the new maximum exponent e max_new and the previous maximum exponent e max_old It can refer to the difference between

[0071] Then, in step S160, the accelerator 120-1 calculates a new maximum exponent e max_new Based on the previous maximum exponent e max_old Then, the accelerator 120-1 performs the operation of step S140. In this case, the updated previous maximum exponent (i.e., the new maximum exponent e max_new ) a bit shift operation is performed based on

[0072] In operation S170, the accelerator 120-1 may generate partial sums by performing an accumulation operation on the aligned mantissa using a plurality of processing units PE11 to PE44. In one embodiment, the accumulation operation on the aligned mantissa may be performed through an addition operation on the aligned mantissa and a previous accumulation value. In one embodiment, since the aligned mantissa and the previous accumulation value (or the aligned accumulation value) correspond to the same maximum exponent, the addition operation thereon may be realized through integer-type addition.

[0073] In step S180, the accelerator 120-1 may determine whether the AI ​​calculation for all the data is completed. If the AI ​​calculation for all the data is not completed (i.e., if there is data remaining on which the AI ​​calculation is performed), the accelerator 120-1 may perform the operations of steps S110 to S170 again.

[0074] When the artificial intelligence calculation for all data is completed, in step S190, the accelerator 120-1 may accumulate the partial sums PSUM of each of the multiple processing units PE11 to PE44 and normalize the accumulated result.

[0075] As described above, the accelerator 120-1 according to the embodiment of the present invention may perform an accumulation operation on a plurality of floating-point type data. In this case, the accelerator 120-1 may obtain a new maximum exponent e max_new and the previous maximum exponent e max_oldBased on the bit shift operation, a bit shift operation can be performed on a plurality of data and an accumulation value. In this case, a loss of a valid value due to a bit shift can be reduced, and thus an error in an operation result can be reduced. In addition, since it is not necessary to load all data that can be operated simultaneously or in parallel, a maximum exponent search and a bit shift are performed on data that is input sequentially, and therefore an operation speed can be improved. In addition, since normalization is performed only once during an accumulation operation process, an error due to normalization can be reduced. Therefore, an accelerator having improved reliability and performance is provided.

[0076] Fig. 9 is a block diagram showing the pre-alignment unit of Fig. 7. Fig. 10 is a diagram for explaining the configuration of a maximum exponent searcher of Fig. 9. For convenience of explanation, detailed explanation of the above-mentioned components will be omitted. Referring to Figs. 7 and 9, the pre-alignment unit 122-1 may include a maximum exponent manager MEM and a plurality of converters CVT1 to CVTk.

[0077] The maximum exponent manager MEM calculates a new maximum exponent e for the K data items (K DT) currently being calculated. max_new Search for the new maximum exponent e max_new and the previous maximum exponent e max_old and based on the comparison result, the maximum exponent e max and the maximum exponential difference Δe max For example, the maximum exponent manager MEM may include a maximum exponent searcher FD, a maximum exponent discriminator DET, a previous maximum exponent storage STR, and a maximum exponent subtractor SUB.

[0078] The maximum exponent searcher FD compares the exponents of K data and finds a new maximum exponent e based on the comparison result. max_new For example, as shown in FIG. 10, the maximum exponent searcher FD may include multiple comparators. The multiple comparators are connected in a tree structure, and multiple exponents are compared with each other to find a new maximum exponent e max_new may be configured to output

[0079] The maximum exponent discriminator DET obtains a new maximum exponent e from the maximum exponent searcher FD. max_new Receive the previous maximum exponent e from the previous maximum exponent storage STR. max_old The maximum exponent discriminator DET receives the new maximum exponent e max_new is the previous maximum exponent e max_old It is possible to determine whether the new maximum exponent e is greater than or equal to max_new is the previous maximum exponent e max_old If it is greater than e, the maximum exponent discriminator DET determines the new maximum exponent e max_new Maximum exponent e max The new maximum exponent e max_new is the previous maximum exponent e max_old If not, the maximum exponent discriminator DET determines the previous maximum exponent e max_old Maximum exponent e max The maximum exponent e can be output as max are provided to each of the multiple converters CVT1 to CVTk.

[0080] The maximum exponent subtractor SUB subtracts a new maximum exponent e from the maximum exponent searcher FD. max_new Receive the previous maximum exponent e from the previous maximum exponent storage STR. max_old Receive the new maximum exponent e max_new and the previous maximum exponent e max_old Based on the maximum exponential difference Δe max For example, we can output the new maximum exponent e max_new is the previous maximum exponent e max_old If it is greater than e, the maximum exponent subtractor SUB calculates the new maximum exponent e max_new and the previous maximum exponent e max_old The maximum exponential difference △e max The new maximum exponent e max_new is the previous maximum exponent e max_old If the maximum exponent subtractor SUB is not greater than the maximum exponent difference Δe max can be output as 0 (zero).

[0081] The multiple converters CVT1 to CVTk have a maximum index e maxBased on the above, a bit shift operation is performed on each of the plurality of data DT_1 to DT_K to generate aligned mantissas sM1 to sMK. The operations of the plurality of converters CVT1 to CVTk are similar to those described above, and therefore detailed description thereof will be omitted.

[0082] Fig. 11 is a block diagram showing one of the processing units in Fig. 7. In order to easily explain the embodiment of the present invention, only a part of the configuration of the processing unit PE is shown in Fig. 11. However, the scope of the present invention is not limited thereto, and the processing unit PE may further include various other components.

[0083] 7 and 11, the processing unit PE may include an adder ADD, an accumulation register AR, a bit shifter BSHF, and a stall control circuit STC. The adder ADD may receive the aligned mantissa sM from the pre-alignment unit 122-1 and may receive an accumulation value from the accumulation register AR. The adder ADD may perform an accumulation operation on the aligned mantissa sM and the accumulation value to generate a partial sum PSUM. The partial sum PSUM is stored in the accumulation register AR and used as an accumulation value for the next accumulation operation. Alternatively, the partial sum PSUM may be provided to another processing unit. Alternatively, the partial sum PSUM may be provided to the normalization unit 123-1.

[0084] The bit shifter BSHF receives the maximum exponent difference Δe max The bit shifter BSHF can receive the maximum exponent difference Δe max For example, the accumulated value stored in the accumulation register AR may be shifted by a bit from the previous maximum exponent e max_old The maximum exponent e max If is changed (i.e., the new maximum exponent e max_new is the previous maximum exponent e max_old (if greater), the aligned mantissa sM is the new maximum exponent e max_newIn this case, for the adder ADD to operate normally, the cumulative value stored in the accumulation register AR must be updated to the new maximum exponent e max_new Therefore, the bit shifter BSHF must be aligned to fit the new maximum exponent e max_new and the previous maximum exponent e max_old The maximum exponential difference Δe max In this case, the aligned accumulated values ​​(i.e., bit-shifted accumulated values) of the accumulation register AR are shifted to the new maximum exponent e max_new and the adder ADD can normally perform accumulation operations on the aligned mantissa (sM) and accumulation value. In one embodiment, the adder ADD may be an integer type adder or an integer adder.

[0085] In one embodiment, the bit shifter BSHF can perform a bit shift operation on the accumulation register AR bit by bit in synchronization with a clock signal. For example, the maximum exponent difference Δe max is 3, the bit shift operation on the accumulation register AR can be performed during three periods of the operating clock. Therefore, the stall control circuit STC can control the input of the aligned mantissa sM to the adder ADD so that the aligned mantissa sM is not input to the adder ADD until the bit shift operation on the accumulation register AR is completed.

[0086] Fig. 12 is a timing diagram showing the operation of the accelerator of Fig. 7. Referring to Fig. 7 and Fig. 12, the accelerator 120-1 can perform artificial intelligence calculations on first to second N data (DT_1 to DT_2N).

[0087] The pre-alignment unit 122-1 of the accelerator 120-1 sequentially loads the first to second N data (DT_1 to DT_2N) and calculates the maximum exponent e max Search for the maximum exponent e maxFor example, the pre-alignment unit 122-1 loads the first and second data (DT_1, DT_2) and performs a bit shift operation on the first to second N data (DT_1 to DT_2N) based on the maximum exponent e max Search for the maximum exponent e max Then, the pre-alignment unit 122-1 loads the third data DT_3 and performs a bit shift operation on the first and second data DT_1 and DT_2 based on the maximum exponent e max , and performs a bit shift operation on the third data DT_3. The result of the bit shift operation (i.e., the aligned mantissa) is provided to the processing unit PE. The processing unit PE receives the aligned mantissa from the pre-alignment unit 122-1, and can perform an addition operation on the aligned mantissa.

[0088] At this time, the maximum exponent e of the pre-sorting unit 122-1 max In the search process, the new maximum exponent e max_new is the previous maximum exponent e max_old In this case, the processing unit PE determines the maximum exponent difference Δe max Based on this, a bit shift operation can be performed on the accumulated value of the accumulation register AR. While the bit shift operation is being performed in the processing unit PE, data input can be stopped by the stall control circuit STC.

[0089] As the above operations are sequentially performed, the accelerator 120-1 may complete the accumulation operation on the first through second N data (DT_1 through DT_2N). When the accumulation operation on the first through second N data (DT_1 through DT_2N) is all completed, the normalization unit 123-1 of the accelerator 120-1 may perform normalization NML on the partial sums of the processing units PE.

[0090] FIG. 13 is a diagram for explaining the operation of the accelerator of FIG. 3. FIGS. 14A to 14C are diagrams for explaining the operation of the accelerator of FIG. 7. For convenience of explanation, it is assumed that the accelerator (120 or 120-1) performs an accumulation operation on the first to eighth data. In this case, it is assumed that the first to eighth data (DT1 to DT8) may include the first and eighth mantissas (M1 to M8), respectively, and that the exponent of the first data DT1 is −5, the exponent of the second data DT2 is −4, the exponent of the third data DT3 is −3, the exponent of the fourth data DT4 is −2, the exponent of the fifth data DT5 is −4, the exponent of the sixth data DT6 is −2, the exponent of the seventh data DT7 is −1, and the exponent of the eighth data DT8 is −1.

[0091] In one embodiment, a bit shift register configured to store the mantissa of data to be operated on is assumed to correspond to the size of the mantissa, i.e., when a bit shift is performed on the mantissa, a loss of a valid value corresponding to the bit-shifted size may occur.

[0092] 3 and 13, the accelerator 120 may perform an accumulation operation on the first to eighth data (DT_1 to DT_8). For example, the pre-alignment unit 122 of the accelerator 120 may perform an accumulation operation on the first to eighth data (DT_1 to DT_8) by accumulating the maximum exponent e max In this case, we can search for the maximum exponent e max is -1.

[0093] The pre-alignment unit 122 of the accelerator 120 finds the maximum exponent e max (i.e., -1), a bit shift operation can be performed on the mantissas (M1 to M8) of the first to eighth data (DT_1 to DT_8). As an example, since the exponent of the first data DT_1 is -5, the first mantissa M1 can be bit-shifted by 4. Similarly, since the exponents of the second to sixth data (DT_2 to DT_6) are -4, -3, -2, -4 and -1, respectively, the second to sixth mantissas (M2 to M6) can be bit-shifted by 3, 2, 1, 3 and 1, respectively.

[0094] Due to the bit shift for the first to sixth mantissas (M1 to M6) described above, a loss of validity for each mantissa (shown by hatching in FIG. 13) may occur. Such a loss of validity may cause a decrease in reliability of the result of the accumulation operation.

[0095] 7, 14A, 14B, and 14C, the accelerator 120-1 may perform an accumulation operation on the first to eighth data (DT_1 to DT_8). For example, as shown in FIG. 14A, the pre-alignment unit 122-1 of the accelerator 120-1 may perform a bit shift operation on the first and second data (DT_1, DT_2). The maximum exponent e of the first and second data (DT_1, DT_2) may be max_new Since the first data DT_1 is −4, the first mantissa M1 of the first data DT_1 is bit-shifted by 1. The accelerator 120-1 performs an accumulation operation on the bit-shifted first mantissa M1 and second mantissa M2 to generate a first accumulation value sum_1. The first accumulation value sum_1 may be stored in an accumulation register AR of at least one of the processing units PE11 to PE44.

[0096] 14B, the accelerator 120-1 may further perform an accumulation operation on the third and fourth data (DT_3, DT_4). For example, the accelerator 120-1 may further perform an accumulation operation on the third and fourth data (DT_3, DT_4) by accumulating the first and second data (DT_1, DT_2) with a first accumulation value sum_1 and a previous maximum exponent (e max_old Accelerator 120-1 can store information about the previous maximum exponent (e max_old That is, the maximum exponent e of the third and fourth data (DT_3, DT_4) can be used to perform a bit shift operation on the third and fourth data (DT_3, DT_4) based on the maximum exponent e max_new is -2, and the previous maximum exponent e max_old Since the exponent difference Δe is −4, the third mantissa M3 of the third data DT_3 is bit-shifted by 1. maxSince M is 2, the first accumulation value sum_1 is bit-shifted by 2. The accelerator 120-1 performs an accumulation operation on the bit-shifted first accumulation value sum_1, the bit-shifted third mantissa M3, and the bit-shifted fourth mantissa M4 to generate a second accumulation value sum_2. The second accumulation value sum_2 may be stored in an accumulation register AR of at least one of the processing units PE11 to PE44.

[0097] Thereafter, as shown in FIG. 14C, the accelerator 120-1 may perform an accumulation operation on the fifth to eighth data (DT_5 to DT_8). For example, the accelerator 120-1 may perform an accumulation operation on the first to fourth data (DT_1 to DT_4), and may perform an accumulation operation on the fifth to eighth data (DT_5 to DT_8). For example, the accelerator 120-1 may perform an accumulation operation on the first to fourth data (DT_1 to DT_4), and may perform an accumulation operation on the fifth to eighth data (DT_5 to DT_8). max_old =-2) can be stored.

[0098] Accelerator 120-1 is the previous maximum exponent (e max_old In other words, the maximum exponent e of the fifth to eighth data (DT_5 to DT_8) can be used to perform a bit shift operation on the fifth to eighth data (DT_5 to DT_8) based on the maximum exponent e max_new is -1, and the previous maximum exponent e max_old Since the exponent difference Δe is −2, the fifth mantissa M5 of the fifth data DT_5 is bit-shifted by 4, and the sixth mantissa M6 of the sixth data DT_6 is bit-shifted by 1. max Since M is 1, the second accumulated value sum_2 is bit-shifted by 1. The accelerator 120-1 performs an accumulation operation on the bit-shifted second accumulated value sum_2, the bit-shifted fifth mantissa M5, the bit-shifted sixth mantissa M6, the seventh mantissa M7, and the eighth mantissa M8, to generate a third accumulated value sum_3. The third accumulated value sum_3 may be stored in an accumulation register AR of at least one of the multiple processing units PE11 to PE44.

[0099] In one embodiment, since the accumulation calculations for the first to eighth data (DT_1 to DT_8) have all been completed, the accelerator 120-1 outputs the results of the accumulation calculations, that is, the third accumulation value sum_3 and the maximum exponent (e max =-1) to generate the final results (i.e., SUM' and e_p).

[0100] In one embodiment, in FIG. 14A, a bit shift of 1 is performed on the first mantissa M1, resulting in a loss of 1 bit of effective value for the first mantissa M1. In FIG. 14B, a bit shift of 1 is performed on the third mantissa M3, resulting in a loss of 1 bit of effective value for the third mantissa M3. The loss of the first mantissa M1 and the third mantissa M3 is relatively small compared to the embodiment of FIG. 13. Although a bit shift is performed on the accumulated value, the loss of the accumulated value has a relatively smaller error compared to the direct loss on the operand, so the embodiment of FIG. 14A to FIG. 14C reduces the error of the accumulated operation result compared to the embodiment of FIG. 13.

[0101] As described above, according to an embodiment of the present invention, the accelerator may be configured to perform various operations to generate, learn, or infer an artificial intelligence model. In this case, the accelerator may perform an accumulation operation on a plurality of data having a floating-point type. The accelerator may search for a new maximum exponent for the plurality of data, and perform a bit shift operation on the plurality of data and the accumulated value accumulated up to now based on the new maximum exponent and the previous maximum exponent. In this case, loss due to the bit shift operation of the operand may be reduced, and since only one normalization is performed during an accumulation operation on a plurality of data, the reliability of the operation result may be improved.

[0102] Fig. 15 is a flowchart showing the operation of the accelerator of Fig. 7. For the sake of convenience, detailed description of overlapping contents will be omitted.

[0103] 7 and 15, the accelerator 120-1 may perform an operation of step S210. The operation of step S210 is the same as step S110 of FIG 8, and therefore a detailed description thereof will be omitted.

[0104] In step S220, the accelerator 120-1 calculates the minimum index e of the K data. min For example, in the embodiment described with reference to FIG. 7 to FIG. 14C, the pre-ordering unit 122-1 of the accelerator 120-1 can search for the maximum exponent e max However, the scope of the present invention is not limited to this, and the pre-ordering unit 122-1 of the accelerator 120-1 searches for the K smallest exponents e min can be explored.

[0105] In step S230, the accelerator 120-1 calculates a new minimum index e min_new is the previous minimum exponent e min_old It is possible to determine whether the new minimum exponent e is smaller than the min_new is the previous minimum exponent e min_old If not, in step S240, the accelerator 120-1 calculates the previous minimum exponent e min_old , an exponent alignment can be performed on the K data to generate an aligned mantissa.

[0106] New minimum exponent e min_new is the previous minimum exponent e min_old If it is smaller, in step S250, the accelerator 120-1 determines whether the minimum exponent difference Δe min In this case, the minimum exponent difference Δe min is the new minimum exponent e min_new and the previous smallest exponent e min_old It can refer to the difference between

[0107] Then, in step S260, the accelerator 120-1 calculates the new minimum index e min_newBased on the previous minimum exponent e min_old Then, the accelerator 120-1 performs the operation of step S240. In this case, the updated previous minimum exponent (i.e., the new maximum exponent e min_new ) a bit shift operation is performed based on

[0108] Thereafter, the accelerator 120-1 performs operations from steps S270 to S290, which are similar to the operations from steps S170 to S190 of FIG. 8, and therefore a detailed description thereof will be omitted.

[0109] As described above, the accelerator 120-1 may perform exponent sorting on K pieces of data in order to perform an accumulation operation on a plurality of pieces of data. In this case, the accelerator 120-1 may perform exponent sorting based on a maximum or minimum exponent. However, the scope of the present invention is not limited thereto. For example, the accelerator 120-1 may perform exponent sorting based on a maximum or minimum value among absolute values ​​of exponents of the plurality of pieces of data or K pieces of data.

[0110] Fig. 16 is a block diagram showing a system according to an embodiment of the present invention. Referring to Fig. 16, a system 1000 may include a memory 1100, a first accelerator 1210, a second accelerator 1220, and a controller 1300. The memory 1100 and the controller 1300 are similar to those described with reference to Fig. 1, and therefore detailed description thereof will be omitted.

[0111] The system 1000 may be a dedicated hardware configured to perform processing of an artificial intelligence model. In one embodiment, the first accelerator 1210 and the second accelerator 1220 of the system 1000 may perform operations in parallel to process a large artificial intelligence model. For example, the first accelerator 1210 and the second accelerator 1220 may process a large artificial intelligence model through data parallelism, model parallelism, or tensor parallelism. In one embodiment, the first accelerator 1210 and the second accelerator 1220 may operate according to the operating method or the computing method described with reference to FIG. 1 to FIG. 15.

[0112] As described above, electronic devices according to embodiments of the present invention and portions thereof may be implemented by, included in, or include at least one or more processing circuits, such as hardware including logic circuits, a hardware / software combination, such as a processor executing software, or a combination thereof. For example, the processing circuits may include, in more detail, a central processing unit (CPU), an arithmetic logic unit (ALU), a graphics processing unit (GPU), an application processor (AP), a digital signal processor (DSP), a microcomputer, a field programmable gate array (FPGA), a programmable logic unit, a microprocessor, an application-specific integrated circuit (ASIC), a neural network processing unit (NPU), an electronic control unit (ECU), an image signal processor (ISP), etc. In one embodiment, the processing circuitry may include a non-transitory computer-readable storage medium (e.g., memory), such as a DRAM device that stores a program of instructions, a processor (e.g., a CPU) configured to execute the program of instructions by a device, system, module, unit, controller, circuit, structure and / or combinations thereof, and / or portions thereof, according to an embodiment of the invention.

[0113] The above is a specific embodiment for carrying out the present invention. The present invention includes not only the above-mentioned embodiment, but also an embodiment that can be simply modified or easily modified. The present invention also includes a technique that can be easily modified and carried out using the embodiment. Therefore, the scope of the present invention should not be limited to the above-mentioned embodiment, but should be determined not only by the claims described below, but also by equivalents to the claims of the present invention. [Explanation of symbols]

[0114] 110 Memory 120 Accelerator 130 Controller 121 Unified Buffer Section 122 Pre-arrangement Section 123 Normalization section 1100 Memory 1210 First Accelerator 1220 Second Accelerator 1300 Controller

Claims

1. 1. A method of operating an accelerator configured to perform an accumulation operation on a plurality of data, comprising: loading at least two first data of the plurality of data; searching for a first index that is a maximum value among the at least two first data indexes; performing a bit shift on first mantissas of the at least two first data based on the first exponent to generate an aligned first mantissa, and performing an accumulation operation on the aligned first mantissas to generate a first accumulation value; loading at least two second data of the plurality of data; searching for a second index that is a maximum value among the at least two second data indexes, the second index being greater than the first index; performing a bit shift on the first accumulation value based on a difference between the second exponent and the first exponent to generate a first aligned accumulation value, performing a bit shift on second mantissas of the at least two second data based on the second exponent to generate aligned second mantissas, and performing an accumulation operation on the aligned second mantissas and the first aligned accumulation value to generate a second accumulation value, A method of operating wherein each of the plurality of data has a floating point type.

2. The method of claim 1 further comprising: storing information about the first exponent as a maximum exponent.

3. 3. The method of claim 2, further comprising: if the second exponent is greater than the first exponent, updating the maximum exponent to the second exponent.

4. loading a third data set of at least two of the plurality of data sets; searching for a third index that is a maximum value among the at least two third data indexes, the third index being not greater than the second index; 4. The method of claim 1, further comprising: performing a bit shift on a third mantissa of the at least two third data based on the third exponent to generate an aligned second mantissa; and performing an accumulation operation on the aligned second mantissa and the second accumulation value to generate a third accumulation value.

5. performing normalization based on the third accumulated value and the second exponent to generate an output value; The method of claim 4 , wherein the output value has the floating point type.

6. The method of claim 1 , wherein the accumulation operation on the aligned first mantissa and the accumulation operation on the aligned second mantissa and the first aligned accumulation value are performed by integer addition.

7. The method of claim 1, wherein the bit shift for the first accumulation value is performed bit by bit in synchronization with a period of a clock signal.

8. 8. The method of claim 7, wherein input of the aligned second mantissa is stopped while a bit shift is being performed on the first accumulation value.

9. The accelerator is configured to perform an accumulation operation on N pieces of data in parallel; The method according to claim 1 , wherein the number of the at least two first data and the number of the at least two second data are each smaller than N.

10. 10. The method of claim 9, wherein the number of the plurality of data is greater than N, and after all accumulation operations on the plurality of data are completed, the accelerator performs normalization on a result of the accumulation operations.

Images