Floating-point weighted sum operation method using random number-based partitioning

Abstract

The present invention relates to a method for performing a weighted sum operation of floating-point numbers using random number-based partitioning to prevent side-channel analysis. A method for performing a weighted sum operation of floating-point numbers according to one embodiment of the present invention is a method for performing a weighted sum of real numbers expressed as floating-point numbers and weights, comprising the steps of: partitioning the sign, exponent, and mantissa of a first and second real number and a first and second weight, respectively, based on random numbers; calculating a first multiplication value obtained by multiplying a first real number and a first weight and a second multiplication value obtained by multiplying a second real number and a second weight using the partitioned sign, exponent, and mantissa and a multiplication random number; and shifting at least one mantissa among the first and second multiplication values ​​so that the exponents of the first and second multiplication values ​​become the same, and then summing the first and second multiplication values.

Description

Technology Field

[0001] The present invention relates to a method for performing a weighted sum operation of floating-point numbers using random number-based partitioning to prevent side-channel analysis. Background Technology

[0003] Side-channel analysis refers to a technology that obtains internal information about a device by statistically analyzing physical auxiliary signals generated by the device (e.g., computation time, temperature changes, power consumption, electromagnetic waves, etc.).

[0004] With the advancement of the deep learning field, neural network models are being installed in various devices. However, as the risk of side-channel analysis attacks on neural network models is increasingly being raised, there is a need for various side-channel countermeasures for neural network models.

[0005] In particular, if secret information such as the weights and biases of the nodes constituting the neural network model is analyzed, the neural network model itself can be reconstructed, which can expose it to attacks such as authentication bypass or adversarial example generation.

[0006] In neural network models, a 'weighted sum operation' is performed in which the products of input values ​​to multiple nodes in the same layer and the weights set for each node are all summed. Since most recent neural network models apply floating-point representation for this weighted sum operation, there is a need for a new weighted sum method that can prevent side-channel analysis under floating-point representation. The problem to be solved

[0008] The present invention aims to perform a weighted sum operation of floating-point numbers using random number-based partitioning.

[0009] The objects of the present invention are not limited to those mentioned above, and other unmentioned objects and advantages of the present invention may be understood from the following description and will be more clearly understood by the embodiments of the present invention. Furthermore, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof set forth in the claims. means of solving the problem

[0011] A floating-point weighted sum operation method according to an embodiment of the present invention for achieving the aforementioned purpose is a method for weighted summing of real numbers expressed as floating-point numbers and weights, comprising: a step of dividing the sign, exponent, and mantissa of a first and second real number and a first and second weight, respectively, based on random numbers; a step of calculating a first multiplication value obtained by multiplying a first real number and a first weight and a second multiplication value obtained by multiplying a second real number and a second weight using the divided sign, exponent, and mantissa and a multiplication random number; and a step of shifting at least one mantissa among the first and second multiplication values ​​so that the exponents of the first and second multiplication values ​​become the same, and then summing the first and second multiplication values. Effects of the invention

[0013] The present invention has the advantage of enabling floating-point operations to be performed using only integer operations, even in hardware environments without a floating-point arithmetic unit for real-valued operations, by performing a weighted sum operation of floating-point numbers using random number-based partitioning.

[0014] In addition, the present invention has the advantage of preventing side-channel analysis of an artificial neural network performing a weighted sum operation by performing a weighted sum operation of floating-point numbers using random number-based partitioning.

[0015] In addition, the present invention has the advantage of being able to simulate an activation function based on the result of floating-point operations using random number-based partitioning, thereby reducing resources associated with the application of a separate activation function.

[0016] In addition to the effects described above, the specific effects of the present invention are described together with the specific details for implementing the invention below. Brief explanation of the drawing

[0018] FIG. 1 is a flowchart illustrating a floating-point weighted sum operation method according to one embodiment of the present invention. FIG. 2 is a diagram illustrating the structure of an artificial neural network requiring a weighted sum operation. FIG. 3 is a diagram illustrating a floating-point representation method according to the IEEE754 standard. Figure 4 is a diagram illustrating the process of multiplying real numbers divided based on random numbers by weights. FIG. 5 is a diagram for explaining the process of adding by shifting the mantissa of one of two multiplicative values ​​based on a comparison of the exponents of the two multiplicative values. FIG. 6 is a flowchart illustrating a method for implementing a ReLU activation function according to an embodiment of the present invention. FIG. 7 is a diagram illustrating an example of implementing a ReLU activation function based on weighted sum values ​​divided based on random numbers. Specific details for implementing the invention

[0019] The aforementioned objectives, features, and advantages are described in detail below with reference to the attached drawings, thereby enabling those skilled in the art to easily implement the technical concept of the present invention. In describing the present invention, detailed descriptions of known technologies related to the present invention are omitted if it is determined that such descriptions would unnecessarily obscure the essence of the invention. Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings. In the drawings, the same reference numerals are used to indicate the same or similar components.

[0020] In this specification, terms such as "first," "second," etc. are used to describe various components, but these components are not limited by these terms. These terms are used merely to distinguish one component from another, and unless specifically stated otherwise, the first component may be the second component.

[0021] Additionally, in this specification, the statement that any configuration is disposed on the "upper (or lower)" or "upper (or lower)" of a component may mean not only that any configuration is disposed in contact with the upper (or lower) surface of said component, but also that another configuration may be interposed between said component and any configuration disposed on (or below) said component.

[0022] Furthermore, where it is stated in this specification that one component is "connected," "coupled," or "connected" to another component, it should be understood that while the components may be directly connected or connected to each other, another component may be "interposed" between each component, or each component may be "connected," "coupled," or "connected" through another component.

[0023] Additionally, singular expressions used in this specification include plural expressions unless the context clearly indicates otherwise. In this application, terms such as "composed of" or "comprising" should not be interpreted as necessarily including all of the various components or steps described in the specification, and should be interpreted as meaning that some of the components or steps may not be included, or that additional components or steps may be included.

[0024] Additionally, in this specification, "A and / or B" means A, B, or A and B unless specifically stated otherwise, and "C to D" means C or more and D or less, unless specifically stated otherwise.

[0025] The present invention relates to a method for performing a weighted sum operation of floating-point numbers using random number-based partitioning to prevent side-channel analysis. Hereinafter, with reference to FIGS. 1 to 7, a method for performing a weighted sum operation according to an embodiment of the present invention and a method for implementing a ReLU activation function based thereon will be described in detail.

[0026] FIG. 1 is a flowchart illustrating a floating-point weighted sum operation method according to one embodiment of the present invention.

[0027] Figure 2 is a diagram illustrating the structure of an artificial neural network that requires a weighted sum operation.

[0028] Figure 3 is a diagram illustrating a floating-point representation method according to the IEEE754 standard.

[0029] Figure 4 is a diagram illustrating the process of multiplying a real number divided based on a random number and a weight, and Figure 5 is a diagram illustrating the process of adding by shifting the mantissa of one of the multiplication values ​​according to the comparison of the exponents of the two multiplication values.

[0030] FIG. 6 is a flowchart illustrating a method for implementing a ReLU activation function according to an embodiment of the present invention, and FIG. 7 is a diagram illustrating an example of implementing a ReLU activation function based on weighted sum values ​​divided based on random numbers.

[0031] First, a floating-point weighted sum operation method will be explained with reference to FIGS. 1 to 5.

[0032] Referring to FIG. 1, a floating-point weighted sum operation method according to one embodiment of the present invention may include the step of dividing the sign, exponent, and mantissa of the first and second real numbers and the first and second weights, respectively, based on random numbers (S10); the step of calculating a first multiplication value obtained by multiplying the first real number and the first weight and a second multiplication value obtained by multiplying the second real number and the second weight using the divided sign, exponent, and mantissa and a multiplication random number (S20); the step of calculating a weighted sum value by shifting at least one mantissa among the first and second multiplication values ​​so that the exponents of the first and second multiplication values ​​become the same, and then adding the first and second multiplication values ​​(S30); and the step of normalizing the weighted sum value (S40).

[0033] However, the floating-point weighted sum operation method illustrated in FIG. 1 is according to one embodiment, and the steps constituting the invention are not limited to the embodiment illustrated in FIG. 1, and some steps may be added, changed, or deleted as necessary.

[0034] Meanwhile, each step illustrated in FIG. 1 can be performed by a processor implemented as a CPU (central processing unit), GPU (graphic processing unit), etc., and in addition, the processor may be implemented by including at least one physical element among ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, and microcontrollers for each operation described below.

[0035] The present invention is a method for performing a weighted sum of real numbers expressed as floating-point numbers and weights using only integer operations, and can be utilized, for example, in artificial neural networks.

[0036] Referring to FIG. 2, an artificial neural network (ANN) has a plurality of inputs (I1 to I N It may include an input layer composed of multiple nodes to receive a weighted sum of inputs, a hidden layer that receives the result of an activation function for the weighted sum of inputs, and an output layer (not shown) that outputs a classification result or a regression result for the input based on the hidden state output from the hidden layer.

[0037] In this structure, the processor utilizes hardware resources to input each node (I1 to I N ) and weights (w1 to w NA 'weighted sum operation' is performed by multiplying ) for all nodes and finally adding them all together. However, if secret information such as the weights and biases of the nodes shown in Fig. 2 is identified through side-channel analysis, there is a problem in that an attacker can replicate the artificial neural network itself.

[0038] The present invention was developed with the motivation to prevent side-channel analysis of such artificial neural networks. Hereinafter, the present invention will be explained through a weighted sum operation process for two inputs, but it is obvious that the operation method described below can be applied equally to all inputs of three or more.

[0039] Each step illustrated in Fig. 1 will be explained in detail below.

[0040] The processor can divide the sign, exponent, and mantissa of the first and second real numbers and the first and second weights, respectively, based on random numbers (S10). Here, it is assumed that both real numbers and two weights are represented as floating-point numbers.

[0041] Referring to FIG. 3, in one embodiment, the real numbers and weights used in the calculation of the present invention can all be expressed in a single-precision manner according to the IEEE754 standard. In a specific example, if the first real number is 13.625 in decimal, converting it to binary gives (1101.101)2, and the corresponding binary number is according to the IEEE754 standard It can be expressed as.

[0042] When expressed in single precision, the 1 bit representing the sign is 0, and the 8 bits representing the exponent are 0000 0011(2 ) considering the bias. 3The exponent of 3) + 0111 1111 (deflection 127) is 10000010, and the 23 bits representing the mantissa can be represented as 1011 0100 0000 0000 0000 000. Meanwhile, if this is simply expressed in hexadecimal, it can be 0x415a0000, and in describing the operation of the present invention below, a hexadecimal representation may be used when it is difficult to write all binary bits.

[0043] The processor can partition the sign, exponent, and mantissa based on random numbers according to the single-precision floating-point representation method of the aforementioned IEEE754 standard. Although various methods can be applied for random-based data partitioning, for the sake of convenience of explanation, the following description assumes that the processor partitions the sign, exponent, and mantissa through random-based 2-sharing.

[0044] Referring to FIG. 4, the processor can divide the sign, exponent, and mantissa of the first real number (I1) based on a random number, and for this purpose, a random number (r a ) is the first random number (r) for sign separation a1 ), second random number for exponential separation (r a2 ), third random number for singer separation (r a3 It can be composed of ). Meanwhile, the random number may be an integer randomly selected from a preset range.

[0045] First, the processor [determines] the sign of the first real number (I1) )cast and It can be divided into. At this time, the divided codes ( ) is a random number operation (logical AND operation ( as shown in [Mathematical Formula 1] below ) and XOR operation( It can be defined including )), and since the bit determining the sign is 1 bit, it is a random number (r a1 ) can be selected as any value between 0 and 1.

[0046] [Mathematical Formula 1]

[0047] ,

[0048] If the first real number (I1) is In cases where it is expressed as, the sign Since is 0, it is a random number (r a1 If ) is selected as 1, Is It can be divided into.

[0049] Next, the processor [determines] the exponent of the first real number (I1) )cast and It can be divided into. At this time, the divided indices ( , ) can be defined including random number operations (addition operations) as shown in [Mathematical Formula 2] below, and since the bit determining the sign is 8 bits, the random number (r a2 ) can be selected as any value from 0 to 127.

[0050] [Mathematical Formula 2]

[0051] ,

[0052] (Meanwhile, here (This may account for a bias (-127) according to the IEEE 754 standard.)

[0053] If the first real number (I1) is In the case where it is expressed as, the exponent Since is 3, it is a random number (r a2 If ) is selected as 43, Is It can be divided into.

[0054] Finally, the processor [determines] the mantissa of the first real number (I1) )cast and It can be divided into. At this time, the divided indices ( , ) can be defined including random number operations (addition operations) as shown in [Mathematical Equation 3] below, and since the bit determining the sign is 32 bits, the random number (r a3 ) is 0 to 2 32 It can be selected as any value from -1.

[0055] [Mathematical Formula 3]

[0056] ,

[0057] (Here (Used to match the number of digits when the separated mantissa is restored in the future)

[0058] If the first real number (I1) is When expressed as, the singer go Since it is a random number (r a3 )go If selected as, silver It can be divided into.

[0059] Referring again to Fig. 4, in the example described above, the first real number (I1) can be separated into a random number-based quadratic sharing method as shown in [Equation 4] below.

[0060] [Mathematical Formula 4]

[0061] ,

[0062]

[0063] Referring again to FIG. 4, the processor [represents] the sign, exponent, and mantissa of the first weight (w1) as a random number (r b It can be partitioned based on ), and here too, random number (r b) may consist of three random numbers for sign separation, exponent separation, and mantissa separation, and the division process using random numbers may be the same as the division process of the first real number (I1) described above. In addition, the processor may also divide the sign, exponent, and mantissa of the second real number (I2) and the second weight (w2) based on random numbers, and the process may also be the same as the division process of the first real number (I1) described above.

[0064] As described above, when the sign, exponent, and mantissa of the first and second real numbers (I1, I2) and the first and second weights (w1, w2) are each divided based on random numbers, the processor uses the divided sign, exponent, and mantissa and a random number for multiplication to obtain a first multiplication value (O) obtained by multiplying the first real number (I1) and the first weight (w1). 1_1 , O 1_2 ) and the second multiplication value (O) obtained by multiplying the second real number (I2) and the second weight (w2) 2_1 , O 2_2 ) can be calculated (S20).

[0065] Referring again to FIG. 4, the processor performs a multiplication operation on the first real number (I1) and the first weight (w1), wherein a random number (r) for multiplication is multiplied by the divided sign, exponent, and mantissa. c ...can be applied. In other words, the processor can apply random number-based operations even when multiplying divided real numbers and weights.

[0066] Specifically, the processor has the sign of the first real number (I1) and the first weight (w1) and the first multiplication random number (r c1 A first code part can be produced by performing an XOR operation on ), and a second multiplication operation random number (r) with the exponent of the first real number (I1) and the first weight (w1). c2 The first exponent part can be calculated by adding ) and the third multiplication operation random number (r) to the cross-product between the mantissas of the first real number (I1) and the first weight (w1). c3 The first part of the mantissa can be calculated by combining ).

[0067] First multiplication value is the first code part as shown in [Mathematical Formula 5] below. , 1st index part , 1st Singer Division It can be composed of, and the first real number is divided as in [Equation 4], and the first weight is assumed to be divided as in [Equation 6] below.

[0068] [Mathematical Formula 5]

[0069]

[0070] [Mathematical Formula 6]

[0071]

[0072] At this time, the processor [represents] the first to third multiplication random numbers (r c1 , r c2 , r c3 By utilizing ) in the operation according to [Mathematical Formula 7] below, the first code part , 1st index part , 1st Singer Division It can calculate, and to aid in understanding the invention, the first to third multiplication random numbers (r c1 , r c2 , r c3 ) are assumed to be arbitrarily selected as 1, 56, and 0x6ebad6, respectively.

[0073] [Mathematical Formula 7]

[0074]

[0075]

[0076]

[0077]

[0078]

[0079]

[0080]

[0081]

[0082] Consequently, the first multiplication value can be expressed as shown in [Equation 8] below.

[0083] [Mathematical Formula 8]

[0084]

[0085]

[0086] The processor uses a multiplicative random number in the same manner as the aforementioned process to obtain a second multiplicative value (O) obtained by multiplying the second real number (I2) and the second weight (w2). 2_1 , O 2_2 ) can be produced.

[0087] Next, the processor can shift at least one of the mantissas of the first and second multiplication values ​​so that the exponents of the first and second multiplication values ​​become the same, and then sum the first and second multiplication values ​​(S30).

[0088] Since it is necessary to unify the exponents for the addition of two real numbers represented as floating-point numbers, the processor can match the exponents of the first and second multiplication values ​​and shift the digits of the mantissa according to the matched exponents.

[0089] Meanwhile, to reduce memory waste caused by the increase in the mantissa and the increase in the number of shifts during the exponent matching process, the processor can change the exponent and mantissa of one multiplication value to match the exponent of the one multiplication value having the larger exponent among the first and second multiplication values.

[0090] To explain using Fig. 5 as an example, the first multiplication value (O 1_1 , O 1_2 The first index part of ) and the second multiplication value (O 2_1 , O 2_2 The second index part of ) If is divided as shown in [Equation 9] below, the first multiplication value (O 1_1 , O 1_2 The exponent of ) is 5 and the second multiplication value (O 2_1 , O 2_2The exponent of ) can be 3. In this case, the processor the second multiplication value (O 2_1 , O 2_2 The exponent of ) is the first multiplication value (O 1_1 , O 1_2 Change to 5, the exponent of ), and the second multiplication value (O 2_1 , O 2_2 The singer of ) can be shifted by two digits, which is the difference in exponents.

[0091] [Mathematical Formula 9]

[0092]

[0093]

[0094] Next, the processor [calculates] the first multiplication value (O 1_1 , O 1_2 ) and the shifted second multiplication value (O 2_1 ', O 2_2 The final weighted sum value (H) can be calculated by adding ').

[0095] Meanwhile, according to the method described above, the processor must determine the exponent of the two multiplication values ​​by summing the divided exponent parts (e.g., (126, -121)) and then compare the magnitudes of the two exponents. However, to prevent side-channel analysis even during the process of determining the exponents by summing the exponent parts, the processor does not sum either of the exponent parts of the first and second multiplication values, and can identify the larger exponent of the two multiplication values ​​based on the difference between the two exponent parts.

[0096] To this end, first, the processor [describes] the first exponent of the first multiplication value and the second exponent of the second multiplication value The difference can be XORed. In the example represented by [Equation 9], the processor [calculates] the difference between the first and second exponent parts ( ) can be calculated as shown in [Mathematical Formula 10] below.

[0097] [Mathematical Formula 10]

[0098]

[0099] Next, the processor is the difference ( The partition elements constituting ) can be XORed sequentially starting from the least significant bit, excluding the sign. The first partition element, 87, can be represented as (1010111)2, and the second partition element, 89, can be represented as (1011001)2. Accordingly, the result of the XOR operation between the two elements can be represented as 0001110.

[0100] The processor can identify the most significant bit, which is 1 as the result of the XOR operation, as the fourth bit, and can identify any partition element in which the bit corresponding to the most significant bit among the partition elements is 1, i.e., (1011001)2=89 in the example above.

[0101] At this time, if the sign of the identified partitioning element is positive, the processor [represents] the first exponent A 2nd index part Judging to be larger (1st index part) In the second index part Since the result of subtracting is consequently positive) the first exponent part can be identified as the maximum exponent. Conversely, if the sign of the previously identified division element is negative, the first exponent A 2nd index part Judging as smaller (1st exponent part) In the second index part Since the result of subtracting is ultimately negative) the second exponent It can be identified as the maximum exponent part.

[0102] Next, the processor must shift the digits of the mantissa to correspond with the exponents of the first and second multiplicative values ​​while matching them to the maximum exponent part. However, when calculating the difference between the exponents of the two multiplicative values ​​to determine the digits to be shifted, side-channel analysis may be performed during the calculation process. To prevent such side-channel analysis, the processor may match the exponents of the first and second multiplicative values ​​to the maximum exponent part, shift the mantissas of the first and second multiplicative values ​​by the maximum exponent part, and then shift them in the opposite direction by the first and second exponent parts.

[0103] In the example represented by [Mathematical Equation 9], the first exponent If is identified as the maximum exponent, the number of digits of the mantissa of the first multiplication value is the maximum exponent (first exponent Shifted by ) and then back to the first exponent The original value can be maintained by shifting in the opposite direction by that amount.

[0104] On the other hand, the number of digits of the mantissa of the second multiplication value is the maximum exponent (first exponent) Shifted by ) and then back to the second exponent By shifting in the opposite direction by that amount, the mantissa of the second multiplication value can consequently be shifted by the difference in exponents between the two multiplication values, that is, by two digits.

[0105] Next, the processor [determines] the first multiplication value (O) after shifting is complete 1_1 , O 1_2 ) and the shifted second multiplication value (O 2_1 ', O 2_2 The final weighted sum value (H) can be calculated by adding '), and the weighted sum value can be normalized according to the IEEE754 standard. In other words, if the number of digits in the mantissa changes due to the sum of the two sum values, the processor can perform normalization by changing the exponent and mantissa in accordance with the IEEE754 standard.

[0106] As described above, the present invention has the advantage of being able to perform floating-point operations using only integer operations even in a hardware environment without a floating-point arithmetic unit for performing real-point operations by performing a weighted sum operation of floating-point numbers using random number-based partitioning.

[0107] In addition, the present invention has the advantage of preventing side-channel analysis of an artificial neural network performing a weighted sum operation by performing a weighted sum operation of floating-point numbers using random number-based partitioning.

[0108] Hereinafter, with reference to FIGS. 6 and FIGS. 7, a method for implementing an activation function according to the aforementioned floating-point weighted sum operation method will be explained.

[0109] Referring to FIG. 6, the activation function implementation method may include the step (S50) of dividing the sign, exponent, and mantissa of the weighted sum value based on random numbers, and the step (S60) of outputting 0 or the weighted sum value according to the sign part corresponding to the larger mantissa part among the divided mantissa parts.

[0110] However, the method of implementing the activation function illustrated in FIG. 6 is according to one embodiment, and the steps constituting the invention are not limited to the embodiment illustrated in FIG. 6, and some steps may be added, changed, or deleted as necessary.

[0111] The following describes how the processor applies random-based partitioning to the weighted sum value and then simulates the ReLU activation function.

[0112] Referring to FIG. 7, the processor can divide the sign, exponent, and mantissa of a weighted sum value (H) expressed in a single-precision manner according to the IEEE754 standard, respectively, based on random numbers (S50). For convenience of explanation, it will be assumed below that the sign, exponent, and mantissa of the weighted sum value (H) are divided in a two-sharing manner.

[0113] In one example, the sign, exponent, and mantissa of the weighted sum value (H) can be divided as shown in [Equation 11] below.

[0114] [Mathematical Formula 11]

[0115] ,

[0116]

[0117] The processor divides the first and second mantissas of the weighted sum value (H). It can be XORed sequentially starting from the least significant bit while excluding the sign. The first mantissa ( 135, which is ), can be expressed as (10000111)2, and the second mantissa( 133, which is ), can be expressed as (10000101)2. Accordingly, the two elements The result of the XOR operation between them can be expressed as 00000010.

[0118] The processor can identify the most significant bit, where the result of the XOR operation is 1, as the 7th bit, and the first and second mantissas Any singer whose beat corresponding to the most significant beat is 1, that is, in the aforementioned example, (10000111)2=135, i.e., the first singer ( ) can be identified as the maximum singer.

[0119] In this case, the sign corresponding to the maximum mantissa, that is, the first mantissa with an absolute value of 135 ( If the sign of ) is positive, it is determined that the weighted sum value (H) is positive (since the sign of the value with the larger absolute value among the two values ​​separated by the addition operation is positive), and the weighted sum value (H) itself can be output.

[0120] In another example, the sign, exponent, and mantissa of the weighted sum value (H) can be divided as shown in [Equation 12] below.

[0121] [Mathematical Formula 12]

[0122] ,

[0123]

[0124] The processor is the first and second singer It can be XORed sequentially starting from the least significant bit while excluding the sign. The first mantissa ( 129, which is ), can be expressed as (10000001)2, and the second mantissa( 133, which is ), can be expressed as (10000101)2. Accordingly, the two elements The result of the XOR operation between them can be expressed as 00000100.

[0125] The processor can identify the most significant bit, where the result of the XOR operation is 1, as the 6th bit, and the first and second mantissas Any singer whose beat corresponding to the most significant beat is 1, that is, in the aforementioned example, (10000101)2=133, i.e., the second singer ( ) can be identified as the maximum singer.

[0126] In this case, the sign corresponding to the maximum mantissa, that is, the second mantissa with an absolute value of 133 ( If the sign of ) is negative, the weighted sum value (H) is determined to be negative (since the sign of the value with the larger absolute value among the two values ​​separated by the addition operation is negative), and 0 can be output.

[0127] That is, the processor can determine whether the weighted sum value (H) is negative or positive without restoring it even when the weighted sum value (H) is divided, and can mimic the function of the ReLU activation function by outputting 0 if the weighted sum value (H) is negative and outputting the weighted sum value (H) itself if it is positive.

[0128] As described above, the present invention has the advantage of being able to simulate an activation function based on the result of floating-point operations using random number-based partitioning, thereby reducing resources associated with the application of a separate activation function.

[0129] Although the present invention has been described above with reference to the illustrated drawings, the present invention is not limited by the embodiments and drawings disclosed in this specification, and it is obvious that various modifications can be made by a person skilled in the art within the scope of the technical concept of the present invention. Furthermore, even if the effects of the configuration of the present invention were not explicitly described while explaining the embodiments of the present invention above, it is natural to acknowledge that the effects predictable by said configuration should also be recognized.

Claims

Claim 1 A method for performing a weighted sum of real numbers and weights expressed as floating-point numbers, comprising: a step in which a processor divides the sign, exponent, and mantissa of a first and second real number and a first and second weight, respectively, based on random numbers; a step in which the processor calculates a first multiply value obtained by multiplying a first real number and a first weight and a second multiply value obtained by multiplying a second real number and a second weight using the divided sign, exponent, and mantissa and a multiplying random number; and a step in which the processor shifts at least one mantissa of the first and second multiply values ​​so that the exponents of the first and second multiply values ​​become the same, and then sums the first and second multiply values. Claim 2 A floating-point weighted sum operation method according to claim 1, wherein the dividing step comprises the step of dividing the sign, exponent, and mantissa according to a single-precision floating-point representation method. Claim 3 A floating-point weighted sum operation method according to claim 1, wherein the dividing step comprises dividing each of the sign, exponent, and mantissa through a random number-based 2-sharing. Claim 4 A floating-point weighted sum operation method according to claim 1, wherein the dividing step comprises the step of dividing the sign, exponent, and mantissa according to the IEEE754 standard. Claim 5 A floating-point weighted sum operation method according to claim 1, wherein the first multiplication value is composed of a first code part, a first exponent part, and a first mantissa part, and the step of calculating the first multiplication value comprises: a step of calculating the first code part by performing an XOR operation between the sign of the first real number and the first weight and a first multiplication random number; a step of calculating the first exponent part by adding the exponent of the first real number and the second weight and a second multiplication random number; and a step of calculating the first mantissa part by adding a third multiplication random number to the cross-product between the first real number and the first weight. Claim 6 A floating-point weighted sum operation method according to claim 1, wherein the step of adding the first and second multiplication values ​​comprises the step of changing the exponent and mantissa of the other multiplication value to match the exponent of the one multiplication value having the larger exponent among the first and second multiplication values, and then adding the two multiplication values. Claim 7 A floating-point weighted sum operation method according to claim 1, wherein the step of adding the first and second multiplication values ​​comprises: a step of performing an XOR operation on the difference between the first exponent of the first multiplication value and the second exponent of the second multiplication value; a step of identifying the maximum exponent among the first and second exponents based on the result of the XOR operation; a step of matching the exponents of the first and second multiplication values ​​with the maximum exponent, shifting the mantissas of the first and second multiplication values ​​by the maximum exponent and then shifting them in the opposite direction by the first and second exponents; and a step of adding the first and second multiplication values ​​after the shifting is completed. Claim 8 A floating-point weighted sum operation method according to claim 7, wherein the step of identifying the maximum exponent part comprises identifying the most significant bit for which the result of the XOR operation is 1, and identifying the first exponent part as the maximum exponent part if the sign of any one of the division elements constituting the difference, for which the bit corresponding to the most significant bit is 1, and the second exponent part as the maximum exponent part if the sign is positive. Claim 9 A floating-point weighted sum operation method according to claim 1, wherein the processor further comprises the step of normalizing the sum of the first and second multiplication values ​​according to the IEEE754 standard. Claim 10 A floating-point weighted sum operation method according to claim 1, wherein the processor further comprises the step of dividing the sign, exponent, and mantissa of the weighted sum value obtained by adding the first and second multiplication values, respectively, based on random numbers. Claim 11 A floating-point weighted sum operation method according to claim 10, further comprising: a step in which the processor performs an XOR operation on the first and second mantissas from which the mantissas of the weighted sum value are divided; a step in which the processor identifies the maximum mantissa among the first and second mantissas based on the result of the XOR operation; and a step in which the processor outputs 0 or the weighted sum value according to a sign corresponding to the maximum mantissa. Claim 12 A floating-point weighted sum operation method according to claim 11, wherein the step of identifying the maximum mantissa comprises identifying the most significant bit for which the result of the XOR operation is 1, and identifying as the maximum mantissa any one of the first and second mantissas for which the bit corresponding to the most significant bit is 1.

Images