Information processing device, information processing method, and computer-readable non-transitory storage medium
Patent Information
- Application Number
- JP2025527333
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Filing Date
- 2023-06-22
- Publication Date
- 2024-12-26
AI Technical Summary
Existing technologies face challenges in performing product-sum calculations with high accuracy due to the large scale of circuit implementations and register limitations, making it difficult to improve processing efficiency when implementing multiple accumulators or accumulators as computer instructions.
The proposed solution involves dividing integer data into partial data, performing product-sum operations on these partial data, and converting cumulative sums between integer and floating-point types to efficiently calculate and store results without increasing circuit scale, using an input data path, product-sum calculator, converter, and inverse transformer.
This approach allows for high-accuracy product-sum calculations with reduced circuit scale and processing efficiency, enabling efficient handling of a wide range of values with fewer bits, comparable to 32-bit integer accuracy using 16-bit floating-point accumulators.
Abstract
Description
Information processing device, information processing method, and computer-readable non-transitory storage medium
[0001] The present invention relates to an information processing device, an information processing method, and a computer-readable non-transitory storage medium.
[0002] Various companies have released computer and accelerator technologies for performing DNN inference, etc. In all prior art, integer quantization uses integer types of around 32 bits as the data type of the accumulator. The problem with these technologies is that the number of bits is large, which increases the size of registers and adders in circuit implementation.
[0003] Japanese Patent Application Laid-Open No. 2021-077347
[0004] In particular, when implementing multiple multiply-accumulators or accumulators on a circuit to improve processing efficiency, the large size of each unit can be a barrier. Furthermore, when implementing it as a computer instruction, the accumulator is placed in the computer's general-purpose register, but since there is a limit to the number of registers, it becomes difficult to increase the processing unit for efficiency or to realize sequential parallel processing using a pipeline.
[0005] Therefore, the present disclosure proposes an information processing device, an information processing method, and a computer-readable non-transitory storage medium that are capable of performing product-sum operations with high accuracy without increasing the circuit scale.
[0006] According to the present disclosure, there is provided an information processing device having: an input data path that divides integer data to be subjected to a multiply-and-accumulate operation into a plurality of partial data and sequentially inputs each of the partial data obtained by the division; a multiply-and-accumulate calculator that performs a multiply-and-accumulate operation on the partial data to calculate a partial sum and adds the partial sum to an integer-type addition input value to calculate an integer-type cumulative sum; a converter that converts the cumulative sum to a floating-point number and outputs it to an accumulator; and an inverse converter that inversely converts the data held in the accumulator back to the cumulative sum and outputs it as the addition input value to be added to the partial sum calculated for the next partial data. Also according to the present disclosure, there is provided an information processing method in which information processing of the information processing device is executed by a computer, and a computer-readable non-transitory storage medium that stores a program that causes a computer to realize the information processing of the information processing device.
[0007] FIG. 1 is a diagram illustrating an example of a system configuration for performing a sum-of-products operation process according to the present disclosure; FIG. 2 is a diagram illustrating an example of a processing flow relating to sum-of-products operation; FIG. 3 is a diagram illustrating an example of a processing flow relating to sum-of-products operation; FIG. 4 is a diagram illustrating an example of a processing flow relating to sum-of-products operation; FIG. 5 is a diagram illustrating a comparison with a conventional example; FIG. 6 is a diagram illustrating an example of a hardware configuration of an information processing device.
[0008] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are designated by the same reference numerals, and redundant description will be omitted.
[0009] The description will be given in the following order: [1. Overview of product-sum operation processing] [2. System configuration example] [3. Processing flow] [4. Comparison with conventional examples] [5. Hardware configuration example] [6. Effects]
[0010] [1. Overview of the product-sum operation process] The product-sum operation process of the present disclosure will be described below. First, the content of the conventional product-sum operation process is shown in equation (1). Note that Z represents a set of integers. Z Ｐ means a set of integer sequences containing P integers as elements.
[0011]
[0012] The sum-of-products operation is also called the dot product or inner product. A pair of integer value sequences (Input1, Input2) with the same number of elements P is given as integer data that serves as input to the sum-of-products operation. An integer value sequence refers to a sequence consisting of multiple elements expressed as integer values. In the sum-of-products operation, two elements with the same element number are set as an element pair, and the product of the two elements included in the element pair is calculated as the augend. The sum (cumulative sum) of the calculated P augend components is the output of the sum-of-products operation.
[0013] In the convolutional layer and fully connected layer of a DNN, a set of features and weights is input and a sum-of-products operation is performed, and as a result, the feature output from the layer can be obtained. In addition, the sum-of-products operation has an input called a "bias term" (Input Ａｄｄ ) and provide this bias term as an additional input value to calculate a cumulative sum including the bias term.
[0014] Next, the sum-of-products operation process proposed in this disclosure is shown in equations (2) to (7). The difference from equation (1) is that partial sums are calculated before calculating the cumulative sum, and that the partial sums are temporarily converted to floating-point numbers. Note that R represents a set of floating-point numbers, and N represents a set of natural numbers.
[0015]
[0016]
[0017]
[0018]
[0019]
[0020]
[0021] The "cast" in the formulas (2) and (3) is a function that converts the data type of the argument x between an integer type and a floating-point type. For example, ＜浮動小数点数へ＞ " outputs the input integer as a real number of the same size. ＜整数へ＞" outputs the input real number as an integer of the same magnitude.
[0022] "SCALE" is a coefficient for converting the scale of data. "ToFP" is a function that outputs the value obtained by multiplying the argument x by the constant SCALE as floating-point data. "ToINT" is a function that outputs the value obtained by dividing the argument x by the constant SCALE as integer data. The functions ToFP and ToINT are inversely related to each other in that they perform conversions. For example, when SCALE=10, －３ Then, ToFP(150)=0.15 and ToINT(0.15)=150.
[0023] In this process, the input integer data is first divided into a plurality of partial data. For example, let P elements in an integer sequence Input1 be (Input1[1], Input1[2], Input1[3], ..., Input1[P]), and let P elements in Input2 be (Input2[1], Input2[2], Input2[3], ..., Input2[P]).
[0024] If the two elements to be multiplied are considered as an element pair, the integer data is divided into P element pairs: (Input1[1], Input2[1]), (Input1[2], Input2[2]), (Input1[3], Input2[3]), ..., (Input1[P], Input2[P]).
[0025] The P element pairs are grouped so that one group contains G element pairs. When P element pairs are grouped into groups of G, K groups are obtained, where K = ceil(P / G). Each group obtained by grouping becomes partial data.
[0026] Note that "ceil" is a ceiling function, and "ceil(P / G)" means the smallest integer equal to or greater than P / G. If, after grouping, the number of element pairs in the last group is less than G (if a remainder occurs when P is divided by G), the missing data is filled with dummy data (for example, data indicating 0).
[0027] As shown in equation (4), the product of two elements included in the same element pair is calculated as the augend component. The sum of the augend components of G element pairs included in the same group is calculated as the partial sum (SumGrp(g)), where "g" indicates the group number. The augend component is calculated as a product-sum operation of integer data.
[0028] "Accum(g)" in equation (6) refers to the floating-point data held in the accumulator immediately before the partial sum of the gth group (SumGrp(g)) is calculated. Accum(g) corresponds to the most recent cumulative sum obtained by adding up the partial sums up to the (g-1)th. Accum(g) is obtained by converting the most recent cumulative sum with the function ToFP. Accum(g) is converted back to an integer cumulative sum with the function ToINT, and then added to the partial sum of the gth group. Hereinafter, the floating-point data held in the accumulator will be referred to as the accumulator value.
[0029] "Accum(0)" in equation (5) is the data to be added to the partial sum (SumGrp(0)) of the 0th group converted by the function ToFP. Specifically, Ａｄｄ Accum(0) is input from the input data path and is not held in the accumulator.
[0030] In the conventional method shown in equation (1), the bias term (Input Ａｄｄ ) is input as integer data. This is because the bias term (Input Ａｄｄ On the other hand, in the method of the present disclosure, the data type handled by the accumulator is a floating-point number type, so the bias term (Input Ａｄｄ ') is also input as floating-point data.
[0031] As shown in equation (6), the partial sum (SumGrp(g)) is added to the accumulator value (Accum(g)) converted to integer type by the function ToINT. The data obtained by the addition is converted to floating-point type data by the function ToFP and stored in the accumulator. This calculation process is performed on the partial sums of all groups, and the final cumulative sum is calculated. The final cumulative sum is converted to floating-point type by the function ToFP and stored in the accumulator. As shown in equation (7), the finally obtained accumulator value (Accum(ceil(P / G))) is converted to integer type by the function ToINT and output as the final cumulative sum data.
[0032] The constant SCALE used for scale conversion is set to a value that ensures a sufficient range of values that can be expressed when integers are expressed as floating-point numbers.
[0033] For example, in floating-point numbers, the exponent is often set wide toward the negative side to ensure precision after the decimal point. For example, IEEE 754 binary 16 (16-bit floating-point number) can express real numbers with absolute values up to approximately 2 to the power of 15 and as low as 2 to the power of -14 within the range of normalized numbers (floating-point numbers that can express values with the same number of significant digits as the number of mantissa bits). Therefore, when an integer is treated as a real number and converted to a floating-point number, information is allocated after the decimal point, making it impossible to express large integer values and prone to overflow.
[0034] On the other hand, if the constant SCALE is set to, for example, the smallest positive normalized number, 2 to the power of -14, integers up to approximately 2 to the power of 29 can be expressed, making it easier to avoid overflow. Such multipliers can also be applied to other floating-point formats. For example, the smallest absolute value of a normalized number can be specified as the constant SCALE.
[0035] Regarding the number of pairs G when grouping element pairs, there is a trade-off between accuracy and processing efficiency.
[0036] For example, when adding two floating-point numbers with significantly different absolute values, the number with the smaller absolute value is less likely to be taken into account in the result. In equation (6), each time the partial sum (SumGrp(g)) is added to the most recent cumulative sum (ToINT(Accum(g))), the calculation result is converted to a floating-point number. If the absolute value of the cumulative sum (ToINT(Accum(g))) is large and the absolute value of the partial sum (SumGrp(g)) is small, the addition result is less likely to be affected by the partial sum, and an underflow, which loses information in the lower bits, is more likely to occur.
[0037] On the other hand, the partial sum (SumGrp(g)) in the sum-of-products operation is added with integer precision, so information in the lower bits is not lost. The smaller the number of pairs G, the more floating-point numbers are added, and the greater the information loss in the lower digits. However, the larger the number of pairs G, the more integer precision is added, and the less information loss there is.
[0038] The processing efficiency related to the number of pairs G differs due to the occurrence of remainders when the number of elements P is divided by the number of pairs G. Increasing the number of pairs G makes it easier for remainders to occur. When remainders occur, unnecessary multiplications and additions are performed, reducing processing efficiency. In general, the inference results of a DNN are less affected by the loss of information in the lower bits, so it is conceivable to prioritize processing efficiency and set a small number of pairs G.
[0039] cast of ToFP(x) ＜浮動小数点数へ＞ Regarding (x), depending on the floating-point number format used, an overflow may occur when converting the input x from an integer to a floating-point number. For example, when using binary16, the maximum value of a floating-point number is 65504, so if x is greater than 65504, the calculation will not be correct. In this case, the overflow problem can be avoided by using a floating-point number format with a larger maximum value within ToFP(x) and converting the output to a different floating-point number format.
[0040] 2. System Configuration Example FIG. 1 is a diagram showing an example of a system configuration for performing the product-sum operation processing of the present disclosure.
[0041] The product-sum operation of the present disclosure is realized by an information processing device 1. For example, the information processing device 1 includes an input data path 10, a selector 20, an inverse transformer 30, a product-sum calculator 40, a transformer 50, an accumulator 60, an output unit 70, and an output data path 80.
[0042] The input data path 10 receives the input value and bias term (Input Ａｄｄ The input value is integer type data (integer data) including a set of integer value sequences (Input1, Input2), and the bias term (Input Ａｄｄ ') is floating-point data. For example, the product-sum operation of the present disclosure is used for the calculation of each layer of a DNN. The integer data includes, for example, an integer value sequence having input feature amounts as elements and an integer value sequence having weights as elements.
[0043] The input data path 10 divides integer data into a plurality of partial data and sequentially inputs each partial data (a group of data consisting of G element pairs) obtained by the division. For example, the input data path 10 extracts one element from each integer sequence to obtain one element pair. The input data path 10 groups the element pairs obtained from a set of integer sequences to generate a plurality of partial data having the same number of element pairs.
[0044] If a remainder occurs when grouping a plurality of element pairs included in the integer data, the input data path 10 fills the remaining data with dummy data (for example, data indicating 0). The input data path 10 can vary the number of element pairs included in one piece of partial data depending on the number of element pairs included in the integer data.
[0045] When the method of the present disclosure is applied to the sum-of-products operation of each layer of a DNN, the number of element pairs included in one piece of partial data can be set for each layer of the DNN to which the sum-of-products operation is applied, in order to optimize the processing efficiency for each layer. Assuming a DNN of a normal size currently used, the number of element pairs included in the partial data can be set to 8 or more and 25 or less, more preferably 8 or 9. This is because if the number of element pairs is too large or too small, it becomes difficult to achieve both calculation accuracy and processing efficiency.
[0046] The number of element pairs included in one piece of partial data can be set as a divisor of the number of element pairs included in the integer data, which allows the integer data to be divided evenly and eliminates the need for dummy data.
[0047] The sum-of-products calculator 40 calculates a partial sum (SumGrp(g)) by performing a sum-of-products operation on the partial data, and then adds the partial sum to an integer-type addition input value to calculate an integer-type cumulative sum. The cumulative sum indicates the total sum (the result of the sum-of-products operation) of all partial sums calculated up to the present time.
[0048] For example, the product-sum calculator 40 performs a product-sum calculation for each partial data and calculates an integer value as the calculation result as the partial sum (SumGrp(g)) for the partial data. The product-sum calculator 40 acquires the most recently calculated cumulative sum as the addition input value (ToINT(Accum(g))), and adds the addition input value (ToINT(Accum(g))) to the calculated partial sum (SumGrp(g)) to calculate a new cumulative sum.
[0049] Note that the addition process of the partial sum and the addition input value, and the addition process of the augend components to obtain the partial sum, are both performed as addition processes of integer data. Therefore, there is no loss of accuracy due to the addition process. Therefore, the order of addition is arbitrary, and the addition of the addition input value and each augend component can be performed in any order. It is not necessary to first obtain the partial sum and then perform the addition process of the partial sum and the addition input value.
[0050] Converter 50 converts the cumulative sum calculated by product-sum calculator 40 into a floating-point number and outputs it to accumulator 60. Converter 50 scales the cumulative sum so that the value of the data obtained by converting the cumulative sum is equal to or less than 1. For example, the coefficient SCALE by which the cumulative sum is multiplied in the scale conversion is the smallest absolute value among the normalized numbers used in the product-sum calculation.
[0051] The accumulator 60 stores the cumulative sum data (Accum(g)) converted to floating-point format by the converter 50. The most recent cumulative sum data (Accum(g)) held in the accumulator 60 is overwritten by the new cumulative sum data (Accum(g+1)) output from the converter 50.
[0052] In the present disclosure, since a high-precision product-sum operation is possible with a small number of bits, the bit width of the accumulator 60 may be narrow. In the present disclosure, for example, the bit width of the accumulator 60 may be 16 bits. The accumulator 60 holds 16-bit floating-point number data as the accumulator value.
[0053] The inverse transformer 30 inversely transforms the accumulator value (Accum(g)) to return it to a cumulative sum, and outputs it as an addition input value (ToINT(Accum(g))) to be added to the partial sum (SumGrp(g)) calculated for the next partial data. The inverse transform means a transform (ToINT) that is the reverse of the transform (ToFP) performed by the transformer 50.
[0054] The selector 20 switches the input source of data to be input to the inverse transformer 30 between the input data path 10 and the accumulator 60. The selector 20 outputs floating-point data acquired from the input source to the inverse transformer 30. The inverse transformer 30 converts the data input from the selector 20 using the function ToINT, and outputs the integer data obtained by the conversion to the multiply-accumulate calculator 40 as an addition input value.
[0055] For example, the first calculated partial sum (SumGrp(1)) is added with a bias term (Input ＡｄｄWhen adding the bias term (Input Ａｄｄ The floating-point data input as the sum (ToINT(Input Ａｄｄ ')).
[0056] When adding a newly calculated partial sum to the most recent cumulative sum, the selector 20 switches the data input source to the accumulator 60. The selector 20 converts the accumulator value (Accum(g)) into an integer type using the function ToINT, and outputs it as the addition input value (ToINT(Accum(g))).
[0057] The output unit 70 converts the accumulator value into an appropriate output format and outputs the data obtained by the conversion as an output value. If the output format is a floating-point number, the accumulator value may be used as the output value as is. If the output format is an integer, the output unit 70 may perform the same processing as the inverse converter 30, and perform appropriate arithmetic shifting and multiplication operations to obtain the output value.
[0058] The timing of the output can be set arbitrarily. The output may be performed each time the product-sum calculator 40 operates once, or may be performed when the product-sum calculator 40 operates multiple times. The output unit 70 converts the accumulator value (Accum(ceil(P / G)) after the addition process of all partial sums has been performed using the function ToINT, and can output the final cumulative sum obtained by the conversion as the output value.
[0059] The output data path 80 is a block for transferring the output value to the next process. If the output value is to be used in the next sum-of-products operation, the output value may be transferred to the input data path 10. If the output value is to be input to another accelerator or computer instruction, the output value may be transferred to a storage area other than the accumulator 60 for the sum-of-products operation.
[0060] This system can be implemented in a variety of ways. One example is an accelerator that performs multiply-and-accumulate operations independently under external control. In this case, each element is implemented independently on a dedicated circuit and driven by external control.
[0061] As another application example, it is also conceivable that the present invention may be implemented as a computer instruction. For example, by providing a single computer instruction having the functions of the multiply-accumulator 40 and the converter 50, and storing and specifying the input and output values in the general-purpose registers of the computer, the memory area of the computer can be regarded as an accumulator.
[0062] In either implementation, a mechanism can be realized in which the result of an integer multiplication and addition operation is converted to a floating-point number and stored, and the implementation form is not the essence of this proposal.
[0063] 2 to 6 are diagrams showing an example of a processing flow relating to a product-sum operation.
[0064] It is assumed that all input data, that is, integer data and bias terms, flow in via the input data path 10. The input data is controlled so that it can be read at least when the block connected to the input data path 10 needs the data. This can be done by placing all input data in a storage area such as memory or a general-purpose register and reading it based on the address or number designation, or by supplying input data sequentially in a stream by a hardware control circuit or software; the implementation method is not the essence of this system.
[0065] The output value is calculated by an output unit 70 based on the value stored in the accumulator 60. The output value is output via an output data path 80. The data type of the output value is not particularly limited, and may be an integer type or a floating-point number type.
[0066] First, the selector 20 switches the input source of the data. The selector 20 sets the input source of the data to the input data path 10, and the output of the selector 20 is set to the bias term (Input Ａｄｄ') (step S1).
[0067] The inverse transformer 30 receives a bias term (Input Ａｄｄ The inverse converter 30 receives the bias term (ToINT(Input Ａｄｄ The sum of products is output to the sum of products calculator 40 as an addition input value (step S4).
[0068] The sum-of-products calculator 40 receives a part of the integer data as partial input data from the input data path 10. The sum-of-products calculator 40 receives an addition input value (ToINT(Input Ａｄｄ The sum-of-products calculator 40 receives the sum-of-products operation on the partial data to calculate the partial sum (SumGrp(1)) (step S6). The sum-of-products calculator 40 calculates the partial sum (SumGrp(1)) and the addition input value (ToINT(Input Ａｄｄ The sum of the sums ...
[0069] The converter 50 receives the cumulative sum from the multiply-and-accumulate calculator 40 (step S9). The converter 50 converts the cumulative sum from integer data to floating-point data using the function ToFP (step S10). The converter 50 outputs the converted cumulative sum data (Accum(1)) to the accumulator 60 (step S11).
[0070] The accumulator 60 receives the converted cumulative sum data (Accum(1)) from the converter 50 and stores it as an accumulator value (step S12). The input data path 10 determines whether all partial data have been processed (step S13).
[0071] If there is unprocessed partial data (step S13: No), the selector 20 sets the data input source to the accumulator 60, and the output of the selector 20 becomes the accumulator value (step S17). Then, with the accumulator value as the addition input value, the processes from step S2 to step S13 are repeated until all the partial data are processed.
[0072] If all partial data have been processed (step S13: Yes), the output unit 70 reads the accumulator value from the accumulator 60 (step S14). The output unit 70 converts the accumulator value into a preset output format (step S15). The output unit 70 outputs the accumulator value whose output format has been converted as an output value to the output data path 80 (step S16).
[0073] 7 is a diagram showing a comparison with a conventional example. In FIG. 7, "Comparative Example 1" and "Comparative Example 2" are shown as conventional examples. The method of the present disclosure is shown as "Example 1" and "Example 2."
[0074] In "Comparative Example 1," a large-bit accumulator, such as a 32-bit integer, is used for the multiply-and-accumulate operation. The problem with using a large-bit accumulator is that the size of the registers and adders increases when implemented in a circuit. In particular, when implementing multiple multiply-and-accumulators and accumulators on a circuit to improve processing efficiency, the large size of each unit can be a barrier. Furthermore, when implemented as a computer instruction, the accumulator is placed in the computer's general-purpose register, but the number of registers is limited, making it difficult to increase the processing unit for efficiency or to achieve sequential parallel processing in a pipelined manner.
[0075] As a small-scale improvement, cases such as "Comparative Example 2" have been seen in which an integer data type with a small number of bits is used for the accumulator. The problem with using an integer type with a small number of bits is that errors can occur in the values accumulated in the accumulator. As the number of elements P increases, the range of values that the accumulator can take becomes larger, making it difficult to store the accumulated value in an integer type with a small number of bits. The number of elements P is determined by the number of channels and kernel size of the feature map, but as the scale of machine learning models increases year by year, it is becoming necessary to be able to take on a larger range of values.
[0076] In the technique of the present disclosure, floating-point numbers can be handled by the accumulator as in "Example 1." This allows a wide range of values to be handled with a small number of bits. For example, using IEEE 754 binary16 (16-bit floating-point numbers) allows values in the same range as signed 32-bit integers to be stored. Furthermore, by accumulating partial sums as integers rather than directly as floating-point numbers as in "Example 2," underflow due to floating-point numbers can be avoided.
[0077] 5. Example of Hardware Configuration FIG. 8 is a diagram showing an example of the hardware configuration of the information processing device 1. As shown in FIG.
[0078] The information processing of the information processing device 1 is realized by, for example, a computer 1000. The computer 1000 has a CPU (Central Processing Unit) 1100, a RAM (Random Access Memory) 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The components of the computer 1000 are connected by a bus 1050.
[0079] The CPU 1100 operates and controls each component based on a program (program data 1450) stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the program stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs.
[0080] The ROM 1300 stores boot programs such as a Basic Input Output System (BIOS) that is executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the hardware of the computer 1000 .
[0081] The HDD 1400 is a non-transitory computer-readable recording medium that non-temporarily records programs executed by the CPU 1100 and data used by such programs. Specifically, the HDD 1400 is a recording medium that records an information processing program according to an embodiment as an example of program data 1450.
[0082] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.
[0083] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to an output device such as a display device, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs recorded on a predetermined recording medium. Examples of media include optical recording media such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), magneto-optical recording media such as an MO (Magneto-Optical Disk), tape media, magnetic recording media, and semiconductor memories.
[0084] For example, when the computer 1000 functions as the information processing device 1 according to the embodiment, the CPU 1100 of the computer 1000 executes an information processing program loaded onto the RAM 1200 to realize the functions of the aforementioned components. The information processing program, various models, and various data according to the present disclosure are stored in the HDD 1400. The CPU 1100 reads and executes program data 1450 from the HDD 1400. Alternatively, the CPU 1100 may acquire these programs from another device via an external network 1550.
[0085] [6. Effects] The information processing device 1 has an input data path 10, a product-sum calculator 40, a converter 50, and an inverse converter 30. The input data path 10 divides integer data to be subjected to a product-sum calculation into a plurality of partial data and sequentially inputs the individual partial data obtained by the division. The product-sum calculator 40 performs a product-sum calculation on the partial data to calculate a partial sum. The product-sum calculator 40 adds the partial sum to an integer-type addition input value to calculate an integer-type cumulative sum. The converter 50 converts the cumulative sum to a floating-point number and outputs it to the accumulator 60. The inverse converter 30 inversely converts the data held in the accumulator 60 back to a cumulative sum and outputs it as an addition input value to be added to the partial sum calculated for the next partial data. In the information processing method disclosed herein, the processing of the information processing device 1 is executed by a computer 1000. A computer-readable non-transitory storage medium disclosed herein stores a program that causes the computer 1000 to implement the processing of the information processing device 1.
[0086] With this configuration, the cumulative sum calculated in integer format is written to the accumulator 60 as floating-point data. When a new partial sum is calculated and added to the cumulative sum, the accumulator value is inversely converted back to the cumulative sum in integer format. Because floating-point numbers are used for the accumulator value, a wide range of values can be handled with a small number of bits. This allows for accurate product-sum calculations without increasing the circuit size.
[0087] The converter 50 scales the cumulative sum so that the value of the data obtained by converting the cumulative sum is 1 or less.
[0088] This configuration allows a wide range of values to be expressed.
[0089] The coefficient by which the cumulative sum is multiplied in the scale conversion is the smallest absolute value of the normalized numbers used in the product-sum operation.
[0090] This configuration makes it possible to maximize the range of values that can be represented.
[0091] The accumulator 60 holds 16-bit floating-point data.
[0092] This configuration reduces the circuit size compared to a typical 32-bit configuration, and reduces implementation costs. The inventor's investigation has revealed that the inference accuracy using the accumulator 60 for 16-bit floating-point numbers is comparable to that when using 32-bit integers.
[0093] The integer data includes a set of integer strings each having the same number of elements. Each integer string includes multiple integer-type elements. The input data path 10 extracts one element from each integer string to obtain one element pair. The input data path 10 groups the multiple element pairs obtained from the set of integer strings to generate multiple data portions each having the same number of element pairs.
[0094] According to this configuration, since the number of element pairs is the same for all partial data, it is possible to perform the multiply-and-accumulate operation on all partial data in the same process.
[0095] If a remainder occurs when grouping a plurality of element pairs contained in integer data, the input data path 10 fills in the missing data with dummy data.
[0096] According to this configuration, the number of element pairs in all partial data can be made equal regardless of the number of element pairs included in the integer data.
[0097] The input data path 10 varies the number of element pairs included in one piece of partial data depending on the number of element pairs included in the integer data.
[0098] This configuration makes it easier to divide integer data evenly. If a remainder occurs during grouping, dummy data must be added to generate one partial data. This requires unnecessary multiply-and-accumulate operations, reducing processing efficiency. If integer data is divided roughly evenly, less dummy data needs to be added, making it less likely to reduce processing efficiency. On the other hand, as the number of divisions increases, the addition result becomes less susceptible to the influence of the partial sum, making it more likely that underflow, in which information in lower-order bits is lost, will occur. Adjusting the number of element pairs included in partial data according to the size of the integer data makes it easier to achieve both calculation accuracy and processing efficiency.
[0099] The number of element pairs included in one piece of partial data is set for each layer of the DNN to which the product-sum operation is applied.
[0100] This configuration makes it easier to improve the processing efficiency of the DNN.
[0101] The number of element pairs included in one piece of partial data is set as a divisor of the number of element pairs included in the integer data.
[0102] With this configuration, the integer data is divided evenly, and dummy data is not required.
[0103] The number of element pairs included in the partial data is 8 to 25.
[0104] This configuration makes it easier to improve the calculation accuracy and processing efficiency in currently used DNNs of normal size.
[0105] The inverse converter 30 converts the floating-point data input as the bias term into an integer type and outputs it as an addition input value.
[0106] According to this configuration, it is possible to perform the product-sum operation and the addition of the bias term simultaneously.
[0107] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.
[0108] [Additional Notes] The present technology may also be configured as follows. (1) An information processing device comprising: an input data path that divides integer data to be subjected to a multiply-and-accumulate operation into a plurality of partial data and sequentially inputs each of the partial data obtained by the division; a multiply-and-accumulate calculator that calculates a partial sum by performing a multiply-and-accumulate operation on the partial data and adds the partial sum to an integer-type addition input value to calculate an integer-type cumulative sum; a converter that converts the cumulative sum into a floating-point number and outputs it to an accumulator; and an inverse converter that inversely converts the data held in the accumulator back to the cumulative sum and outputs it as the addition input value to be added to a partial sum calculated for the next partial data. (2) The information processing device according to (1), wherein the converter scales the cumulative sum so that the value of the data obtained by converting the cumulative sum is 1 or less. (3) The information processing device according to (2), wherein a coefficient by which the cumulative sum is multiplied in the scale conversion is the smallest absolute value among normalized numbers used in the multiply-and-accumulate operation. (4) The information processing device according to (2) or (3), wherein the accumulator holds 16-bit floating-point number data. (5) The information processing device according to any one of (1) to (4), wherein the integer data includes a set of integer value sequences having the same number of elements, each integer value sequence consisting of a plurality of elements represented by integer values, and the input data path extracts elements one by one from each integer value sequence to obtain one element pair, and groups the plurality of element pairs obtained from the set of integer value sequences to generate the plurality of partial data having the same number of element pairs. (6) The information processing device according to (5), wherein, if a remainder occurs when grouping the plurality of element pairs included in the integer data, the input data path fills the missing data with dummy data. (7) The information processing device according to (6), wherein the input data path varies the number of element pairs included in one partial data depending on the number of element pairs included in the integer data. (8) The information processing device according to (7), wherein the number of the element pairs included in one piece of the partial data is set for each layer of the DNN to which the product-sum operation is applied.(9) The information processing device according to (7) or (8), wherein the number of element pairs included in one piece of partial data is set as a divisor of the number of element pairs included in the integer data. (10) The information processing device according to any one of (7) to (9), wherein the number of element pairs included in one piece of partial data is equal to or greater than 8 and equal to or less than 25. (11) The information processing device according to any one of (1) to (10), wherein the inverse transformer converts floating-point number data input as a bias term into integer type and outputs the converted data as the addition input value. (12) An information processing method executed by a computer, comprising: dividing integer data to be subjected to a multiply-and-accumulate operation into a plurality of partial data; sequentially inputting each partial data obtained by the division; performing a multiply-and-accumulate operation on the partial data to calculate a partial sum; adding the partial sum to an integer-type addition input value to calculate an integer-type cumulative sum; converting the cumulative sum to a floating-point number and outputting it to an accumulator; and inversely converting the data held in the accumulator back to the cumulative sum, which is output as the addition input value to be added to the partial sum calculated for the next partial data. (13) A computer-readable non-transitory storage medium storing a program that causes a computer to perform the following operations: divide integer data to be subjected to a multiply-and-accumulate operation into a plurality of partial data; sequentially input each partial data obtained by the division; calculate a partial sum by performing a multiply-and-accumulate operation on the partial data; add the partial sum to an integer-type addition input value to calculate an integer-type cumulative sum; convert the cumulative sum into a floating-point number and output it to an accumulator; and inversely convert the data held in the accumulator back to the cumulative sum, which is output as the addition input value to be added to the partial sum calculated for the next partial data.
[0109] REFERENCE SIGNS LIST 1 Information processing device 10 Input data path 30 Inverse converter 40 Multiply-accumulator 50 Converter 60 Accumulator
Claims
1. An information processing device having: an input data path which divides integer data to be subjected to a multiply-and-accumulate operation into a plurality of partial data, and sequentially inputs each of the partial data obtained by the division; a multiply-and-accumulate calculator which performs a multiply-and-accumulate operation on the partial data to calculate a partial sum, and adds the partial sum to an integer-type addition input value to calculate an integer-type cumulative sum; a converter which converts the cumulative sum to a floating-point number type and outputs it to an accumulator; and an inverse converter which inversely converts the data held in the accumulator back to the cumulative sum, and outputs it as the addition input value to be added to the partial sum calculated for the next partial data.
2. The information processing device according to claim 1, wherein the converter scales the cumulative sum so that the value of the data obtained by converting the cumulative sum is 1 or less.
3. The information processing device according to claim 2, wherein a coefficient by which the cumulative sum is multiplied in the scale conversion is a minimum absolute value among normalized numbers handled in the product-sum operation.
4. The information processing device according to claim 2, wherein the accumulator holds 16-bit floating-point data.
5. The information processing device of claim 1, wherein the integer data includes a set of integer value sequences having an equal number of elements, each integer value sequence consisting of a plurality of elements expressed by integer values, and the input data path extracts one element from each integer value sequence to obtain one element pair, and groups the plurality of element pairs obtained from the set of integer value sequences to generate the plurality of partial data having the same number of element pairs.
6. The information processing device according to claim 5, wherein, if a remainder occurs when grouping the plurality of element pairs included in the integer data, the input data path fills in the missing data with dummy data.
7. The information processing device according to claim 6, wherein the input data path varies the number of element pairs included in one piece of partial data according to the number of element pairs included in the integer data.
8. The information processing device according to claim 7, wherein the number of element pairs included in one piece of partial data is set for each layer of a DNN to which the product-sum operation is applied.
9. The information processing device according to claim 7, wherein the number of element pairs included in one piece of partial data is set as a divisor of the number of element pairs included in the integer data.
10. The information processing device according to claim 7, wherein the number of element pairs included in the partial data is 8 or more and 25 or less.
11. The information processing device according to claim 1, wherein the inverse transformer converts floating-point data input as a bias term into an integer type and outputs the integer type as the addition input value.
12. An information processing method executed by a computer, comprising: dividing integer data to be subjected to a multiply-and-accumulate operation into a plurality of partial data; sequentially inputting each partial data obtained by the division; performing a multiply-and-accumulate operation on the partial data to calculate a partial sum; adding the partial sum to an integer-type addition input value to calculate an integer-type cumulative sum; converting the cumulative sum to a floating-point number and outputting it to an accumulator; and inversely converting the data held in the accumulator back to the cumulative sum, which is output as the addition input value to be added to the partial sum calculated for the next partial data.
13. A computer-readable non-transitory storage medium storing a program that causes a computer to perform the following operations: divide integer data to be subjected to a multiply-and-accumulate operation into a plurality of partial data, sequentially input each partial data obtained by the division, perform a multiply-and-accumulate operation on the partial data to calculate a partial sum, add the partial sum to an integer-type addition input value to calculate an integer-type cumulative sum, convert the cumulative sum to a floating-point number type and output it to an accumulator, inversely convert the data held in the accumulator back to the cumulative sum, and output it as the addition input value to be added to the partial sum calculated for the next partial data.