Neural network processing device

Abstract

A neural network processing device includes a convolution section that receives an input signal and a learning-completed weight parameter, performs a convolution operation on the input signal and the learning-completed weight parameter, and outputs a convolution signal; a configuration adjustment section that receives the convolution signal and a learning-completed normalization parameter, and outputs an adjustment signal in which an output bias of the convolution signal is adjusted; and an activation section that transforms the adjustment signal into an activation function to output an output signal.

Description

Technical Field

[0001] The present invention relates to a neural network processing apparatus, and more particularly to a neural network processing apparatus for processing image signals. Background Technology

[0002] In recent years, the performance of Convolutional Neural Networks (CNNs) in image recognition has improved, and much research is underway to further enhance their performance and apply them to various problems. As a result, adequate detection performance has been achieved, and research related to small-scale network construction and hardware installation for commercialization is increasing in recent years. Summary of the Invention

[0003] The purpose of this invention is to provide a neural network processing device that is easy to install in hardware.

[0004] According to a feature of the present invention for achieving the objectives described above, a neural network processing apparatus may include: a convolution unit that receives an input signal and learned weight parameters, performs a convolution operation on the input signal and the learned weight parameters, and outputs a convolution signal; a configuration adjustment unit that receives the convolution signal and learned normalization parameters, and outputs an adjustment signal that adjusts the output deviation of the convolution signal; and an activation unit that transforms the adjustment signal into an activation function to output an output signal.

[0005] In one embodiment, the activation unit may normalize the output signal to a value between 0 and 1.

[0006] In one embodiment, the learned weight parameters may be expressed by a mathematical formula. We can calculate that w is the learning parameter.

[0007] In one embodiment, the learning completion normalization parameter may be normalized to a value between -1 and 1.

[0008] In one embodiment, the convolutional unit may include: a multiplier that performs a multiplication operation on the input signal and the learned weight parameters; and a quantizer that quantizes the output of the multiplier and outputs the convolutional signal.

[0009] In one embodiment, the quantizer may perform operations based on a mathematical formula. The quantization operation is defined as follows: z is the output of the multiplier, t is the bit width of the convolution signal, and M is the maximum value in the range -a to b. It is the convolutional signal.

[0010] In one embodiment, the normalized parameters after learning may include a first parameter and a second parameter, and the configuration adjustment unit may include: a multiplier that performs a multiplication operation on the convolution signal and the first parameter; a first quantizer that quantizes the output of the multiplier; an adder that performs an addition operation on the output of the first quantizer and the second parameter; and a second quantizer that quantizes the output of the adder.

[0011] In one embodiment, the first parameter may be based on a mathematical formula. The calculated values, where w is the learning parameter and y is the output signal. It is the maximum value of the absolute value of the learning parameter w. It is the maximum value of the absolute value of the output signal y. , The learning parameters are determined at the moment the learning is completed. It is the standard deviation of the input signal.

[0012] In one embodiment, the second parameter may be based on a mathematical formula. The calculated value, y, is the output signal. It is the maximum value of the absolute value of the output signal y. , and These are the learning parameters determined at the moment the learning is completed. It is the average of the input signal. It is the standard deviation of the input signal.

[0013] In one embodiment, the activation unit may transform the adjustment signal into the output signal between 0 and 1 according to the activation function.

[0014] In one embodiment, the input signal may be an image signal.

[0015] The neural network processing device includes: a convolution unit comprising an input layer, an intermediate layer, and an output layer, wherein the intermediate layer receives an input signal and learned weight parameters from the input layer, performs a convolution operation on the input signal and the learned weight parameters, and outputs a convolution signal; an adjustment unit receiving the convolution signal and learned normalization parameters, and outputs an adjustment signal that adjusts the output deviation of the convolution signal; and an activation unit transforming the adjustment signal into an activation function to output an output signal to the output layer.

[0016] In one embodiment, the learning completion weight parameter may be normalized to a value between -1 and 1.

[0017] In one embodiment, the learning completion weight parameters may be based on a mathematical formula. We can calculate that w is the learning parameter.

[0018] In one embodiment, the learning completion normalization parameter may be normalized to a value between -1 and 1.

[0019] In one embodiment, the convolutional unit may include: a multiplier that performs a multiplication operation on the input signal and the learned weight parameters; and a quantizer that quantizes the output of the multiplier and outputs the convolutional signal.

[0020] In one embodiment, the quantizer may perform operations based on a mathematical formula. The quantization operation is defined as follows: z is the output of the multiplier, t is the bit width of the convolution signal, and M is the maximum value in the range -a to b. It is the convolutional signal.

[0021] In one embodiment, the normalized parameters after learning may include a first parameter and a second parameter, and the configuration adjustment unit may include: a multiplier that performs a multiplication operation on the convolution signal and the first parameter; a first quantizer that quantizes the output of the multiplier; an adder that performs an addition operation on the output of the first quantizer and the second parameter; and a second quantizer that quantizes the output of the adder.

[0022] In one embodiment, the first parameter may be based on a mathematical formula. The calculated values, where w is the learning parameter and y is the output signal. It is the maximum value of the absolute value of the learning parameter w. It is the maximum value of the absolute value of the output signal y. , The learning parameters are determined at the moment the learning is completed. It is the standard deviation of the input signal.

[0023] In one embodiment, the second parameter may be based on a mathematical formula. The calculated value, y, is the output signal. It is the maximum value of the absolute value of the output signal y. , and These are the learning parameters determined at the moment the learning is completed. It is the average of the input signal. It is the standard deviation of the input signal.

[0024] (Invention effect) In the neural network processing device configured as described above, the learning parameters are values ​​between -1 and 1, and the output signal is transformed into values ​​between 0 and 1, thus quantization is easily achieved. Therefore, efficient hardware installation of the neural network processing device can be realized. Attached Figure Description

[0025] Figure 1 This is a block diagram of a neural network processing device according to an embodiment of the present invention.

[0026] Figure 2 This is a diagram used to illustrate the operation of a neural network processing device.

[0027] Figure 3 This is an explanation Figure 2 The diagram shows the functionalities of the intermediate layer.

[0028] Figure 4 It is used for explanation Figure 3 The diagram shows the operation of the quantizer.

[0029] Symbol explanation: 100: Neural network processing unit; 105: Bus; 110: Input memory; 120: Parameter memory; 130: Convolution unit; 140: Adder; 150: Quantization unit; 160: Temporary memory; 170: Output memory; 210: Input layer; 220: Intermediate layer; 230: Output layer; 221: Symbolization unit; 222: Convolution unit; 223: Configuration adjustment unit; 224: Activation unit. Detailed Implementation

[0030] In this specification, when it is mentioned that a certain constituent element (or region, layer, part, etc.) is located on, connected to or combined with other constituent elements, it means that it can be directly configured / connected / combined with other constituent elements, or a third constituent element can be configured therein.

[0031] The same symbols refer to the same constituent elements. Furthermore, in the accompanying drawings, the thickness, proportions, and dimensions of the constituent elements are exaggerated for the purpose of effectively illustrating the technical content. "And / or" includes all combinations that can define the related constituent elements.

[0032] The terms "first," "second," etc., can be used to describe various constituent elements, but the constituent elements described should not be limited to these terms. These terms are used only to distinguish one constituent element from others. For example, without departing from the scope of this invention, a first constituent element can be named a second constituent element, and similarly, a second constituent element can be named a first constituent element. Singular expressions include plural expressions unless explicitly stated otherwise in the text.

[0033] Furthermore, terms such as "below," "on the lower side," "above," and "on the upper side" are used to explain the connection relationships between the components in the diagram. These terms are relative concepts and are explained based on the direction shown in the diagram.

[0034] Terms such as “including” or “having” should be understood as referring to the presence of features, figures, steps, operations, constituent elements, components, or combinations thereof as recorded in the instruction manual, and do not preclude the existence or additional possibilities of one or more other features, figures, steps, operations, constituent elements, components, or combinations thereof.

[0035] Unless otherwise defined, all terms used in this specification (including technical and scientific terms) shall have the same meaning as commonly understood by those skilled in the art. Furthermore, terms defined in commonly used dictionaries shall be interpreted as having a meaning consistent with the relevant technical context, and shall not be interpreted as having an idealized or overly formal meaning unless explicitly defined in this application.

[0036] Hereinafter, various embodiments of the present invention will be described with reference to the accompanying drawings.

[0037] Figure 1 This is a block diagram of a neural network processing device according to an embodiment of the present invention.

[0038] Reference Figure 1 The neural network processing device 100 includes an input memory 110, a parameter memory 120, a convolution operator 130, an adder 140, a quantization operator 150, a temporary memory 160, and an output memory 170. The input memory 110, parameter memory 120, convolution operator 130, adder 140, quantization operator 150, temporary memory 160, and output memory 170 can transmit and receive signals via a bus 105.

[0039] Input memory 110, parameter memory 120, temporary memory 160, and output memory 170 can be buffers. Figure 1 The diagram illustrates the case where the input memory 110, parameter memory 120, temporary memory 160, and output memory 170 are independent components, but the invention is not limited thereto. For example, the input memory 110, parameter memory 120, temporary memory 160, and output memory 170 can be implemented using a single memory.

[0040] exist Figure 1The diagram illustrates a scenario where the convolution operator 130, adder 140, and quantizer 150 are independent components, but the invention is not limited thereto. For example, the convolution operator 130, adder 140, and quantizer 150 can be implemented using a single processor.

[0041] Input memory 110 stores input signals from external sources. The input signals may be image signals. In one embodiment, the input signals provided to input memory 110 may be image signals that have undergone preprocessing. For example, the input signals may be image signals that have undergone image processing such as black-and-white conversion, contrast adjustment, and brightness adjustment.

[0042] The parameter memory 120 stores one or more parameters. Parameters may include learning parameters. Parameters may be preset values.

[0043] Temporary memory 160 may be a memory for temporarily storing the outputs of convolution unit 130, adder 140 and quantization unit 150.

[0044] The output memory 170 can store the final output signal of the neural network processing device 100.

[0045] The convolution unit 130 can perform convolution operations on the input signal stored in the input memory 110 and the parameters stored in the parameter memory 120. In addition, the convolution unit 130 can perform convolution operations on the intermediate operation results stored in the temporary memory 160 and the parameters stored in the parameter memory 120.

[0046] The quantizer 150 performs quantization operations on the signals output from either the convolutional unit 130 or the adder 140.

[0047] Figure 2 This is a diagram used to illustrate the operation of a neural network processing device.

[0048] Reference Figure 2 The neural network processing device 100 may include an input layer 210, an intermediate layer 220, and an output layer 230.

[0049] Input layer 210, intermediate layer 220 and output layer 230 can respectively represent the input layer 210, intermediate layer 220 and output layer 230 respectively. Figure 1 The hardware shown illustrates the operational steps to be performed.

[0050] Input layer 210 may include operations for storing input signals in input memory 110 and operations for storing parameters (or learned parameters) in parameter memory 120. The signal IN provided to input layer 210 may include... Figure 3 The input signals X[i]~X[i+9] and parameters w are shown. [i]~w [i+9].

[0051] The intermediate layer 220 may include operations of convolution operator 130, adder 140 and quantizer 150. That is, the intermediate layer 220 may be executed by at least one of the convolution operator 130, adder 140 and quantizer 150.

[0052] The output layer 230 may include the operation of outputting the output signal OUT stored in the output memory 170.

[0053] Figure 3 This is an explanation Figure 2 The diagram shows the functionalities of the intermediate layer.

[0054] Figure 3 The various functional modules shown can be Figure 1 The neural network processing device 100 shown is in operation.

[0055] Reference Figure 1 and Figure 3 The intermediate layer 220 of the neural network processing device 100 receives the input signals X[i]~X[i+9] and the parameters w. [i]~w [i+9]、A B The system receives input signals X[i] to X[i+9] from input memory 110 and parameters w from parameter memory 120. [i]~w [i+9]、A B The parameters w'[i] to w'[i+9] can be referred to as the learned weight parameters, and parameter A B These can be referred to as the learned normalized parameters.

[0056] The intermediate layer 220 of the neural network processing device 100 includes a symbolization unit 221, a convolution unit 222, a configuration adjustment unit 223, and an activation unit 224.

[0057] The input signals X[i] to X[i+9] are unsigned 11-bit signals. The parameter w [i]~w [i+9]、A B These are parameters obtained through learning, and they are 8-bit signals.

[0058] The symbolization unit 221 includes symbolizers S1 to S9. Symbolizers S1 to S9 correspond to input signals X[i] to X[i+9], respectively. Symbolizers S1 to S9 add a sign bit to the input signals X[i] to X[i+9] in units of 1 bit. Therefore, the input signal X output from the symbolization unit 221... [i]~X [i+9] are 12 bits each.

[0059] The convolution unit 222 processes the input signal X output from the symbolization unit 221. [i]~X [i+9] and parameter w [i]~w [i+9] performs a convolution operation and outputs a convolution signal Ci. The convolution signal Ci can be stored in temporary memory 160.

[0060] The convolution unit 222 includes multipliers M11~M19, quantizers Q11~Q19, Q21, and adder A11.

[0061] Multipliers M11~M19 correspond to the input signal X respectively. [i]~X [i+9]. The number of multipliers M11~M19 can be determined based on the input signal X. [i]~X The number of [i+9] is used to determine this.

[0062] Multipliers M11~M19 respectively process the input signal X [i]~X The corresponding input signal and parameter w in [i+9] [i]~w The multiplication operation is performed on the corresponding parameters in [i+9]. This is achieved using a 12-bit input signal X. [i]~X [i+9] and the 8-bit parameter w [i]~w Multiplying each of [i+9] will output a 20-bit multiplication result.

[0063] Quantizers Q11 to Q19 correspond to multipliers M11 to M19, respectively. The number of quantizers Q11 to Q19 can be determined based on the number of multipliers M11 to M19.

[0064] Quantizers Q11 to Q19 convert the 20-bit multiplication results output from the corresponding multipliers M11 to M19 into 16-bit signals, respectively.

[0065] The adder A11 adds signals from the outputs of quantizers Q11 to Q19. The signal output from adder A11 can be 22 bits.

[0066] The quantizer Q21 transforms the 22-bit signal output from the adder A11 into a 12-bit convolution signal Ci.

[0067] Configuration adjustment unit 223 receives convolution signal Ci and parameter A B The configuration adjustment unit 223 outputs an adjustment signal Di that adjusts the output deviation of the convolution signal Ci.

[0068] The configuration adjustment unit 223 includes a multiplier M21, an adder A21, and quantizers Q31 and Q41.

[0069] Multiplier M21 processes the convolution signal Ci and parameter A from convolution unit 222. Perform multiplication. The convolution signal Ci has a bit width of 12 bits, and the parameter A... The bit width is 8 bits, so the bit width of the signal output from the multiplier M21 can be 20 bits.

[0070] The quantizer Q31 converts the 20-bit signal output from the multiplier M21 into a 16-bit signal.

[0071] The output and parameter B of adder A21 and quantizer Q31 The signal output from quantizer Q31 has a bit width of 16 bits, and parameter B... The bit width is 8 bits, so the bit width of the signal output from adder A21 can be 17 bits.

[0072] The output of quantizer Q41 transforms the 17-bit signal from adder A21 into a 12-bit adjustment signal Di.

[0073] The activation unit 224 outputs an output signal yi between 0 and 1 according to the activation function (reLU). When the adjustment signal Di is negative, the activation unit 224 transforms the output signal yi to 0. Therefore, the bit width of the adjustment signal Di output from the activation unit 224 is 11 bits.

[0074] If the operation of the convolution part 222 is expressed mathematically, it is as shown in the following mathematical expression 1.

[0075] [Mathematical Expression 1]

[0076] In mathematical formula 1, x[i+d] is the input signal and w[d] is the parameter. Figure 3 The example represents the case where d=9.

[0077] If we normalize mathematical expression 1, it becomes like mathematical expression 2.

[0078] [Mathematical Expression 2]

[0079] In mathematical formula 2, the average The standard deviation can be expressed by mathematical formula 3. It can be represented by mathematical formula 4.

[0080] [Mathematical Expression 3]

[0081] [Mathematical Expression 4]

[0082] In mathematical formula 2, and These are the learning parameters.

[0083] Typically, the input signal, learning parameters, and output signal are 32-bit floating decimal numbers. Therefore, in order to implement neural networks in hardware, they need to be converted to fixed decimal numbers and miniaturized by reducing the number of bits.

[0084] At the moment when learning is completed, and The learning parameters are determined, so mathematical expression 2 can be transformed into mathematical expression 5.

[0085] [Mathematical Expression 5]

[0086] In mathematical expression 5, A can be represented by mathematical expression 6, and B can be represented by mathematical expression 7.

[0087] [Mathematical Expression 6]

[0088] [Mathematical Expression 7]

[0089] If we substitute mathematical expression 5 into mathematical expression 1, it becomes mathematical expression 8.

[0090] [Mathematical Expression 8]

[0091] The output signal yi of mathematical formula 8 can be calculated by performing a multiplication operation on the learning parameter A and an addition operation on the learning parameter B.

[0092] However, in mathematical formula 8, the operation d multiplication operations and d addition operations should be performed. Figure 3 In mathematical formulas 1 to 8, for ease of explanation, the number of input signals is represented as d. However, in reality, neural networks require three-dimensional convolution operations such as (number of vertical input signals × number of horizontal input signals × number of nodes).

[0093] For example, when the (number of vertical input signals, number of horizontal input signals, number of nodes) is (5, 5, 16), the operation of mathematical formula 8... It should perform 400 multiplication operations and 399 addition operations.

[0094] Here, if the range of the input signal x and the parameter w is normalized to -1 to 1, then the output range of the multiplier can be -1 to 1, and the output range of the adder can be -400 to 400.

[0095] Mathematical expression 9 can be used to modify mathematical expression 8.

[0096] [Mathematical Expression 9]

[0097] In mathematical formula 9, the learned weight parameter w' can be represented by mathematical formula 10, the learned normalized parameter A' can be represented by mathematical formula 11, and the learned normalized parameter B' can be represented by mathematical formula 12.

[0098] [Mathematical Expression 10]

[0099] In mathematical formula 10, This represents the maximum absolute value of the learning parameter w.

[0100] [Mathematical Expression 11]

[0101] [Mathematical Expression 12]

[0102] In mathematical expressions 11 and 12, This represents the maximum absolute value of the output signal y. In equations 11 and 12, the output signal y can be a value predetermined during the learning process.

[0103] The number of multiplication and addition operations in mathematical expression 9 is the same as in mathematical expression 8. However, the parameter w in mathematical expression 9... The range of the parameter w is normalized to -1 to 1, and the range of the output signal y is normalized to 0 to 1. The correct range can be determined after the learning is completed.

[0104] Figure 4 It is used for explanation Figure 3 The diagram shows the operation of the quantizer. In the following description, quantizer Q11 is illustrated as an example, but other quantizers Q12~Q19, Q21, Q31, and Q41 can also operate in a similar manner to quantizer Q11.

[0105] exist Figure 4 In the diagram, curve L1 illustratively represents the distribution (generation frequency) of the 20-bit signal output from multiplier M11.

[0106] Reference Figure 4 The quantizer Q11 transforms the 20-bit signal output from the multiplier M11 into a 16-bit signal. With the quantizer Q11 truncating the least significant bit (LSB) of the 20-bit signal output from the multiplier M11, it can include most values ​​that can be represented by a 20-bit signal, but as with fixed-decimal-point C, it may exceed the maximum range of the actual signal represented by curve L1.

[0107] If the quantizer Q11 truncates the most significant bit (MSB) of the 20-bit signal output from the multiplier M11, it is difficult to include the full range of the actual signal represented by curve L1, as is the case with fixed decimal point type A.

[0108] Therefore, similar to the fixed decimal type B, it is necessary to set the range of the signal output from quantizer Q11 so that it corresponds to the range of the actual signal represented by curve L1. For this setting, it is appropriate to set quantizer Q11 based on actual measurement data.

[0109] In addition, overflow may occur when the vectorizer Q11 is input with data other than the actual measurement data, so overflow prevention clamping is also required.

[0110] The quantization setting method for quantizer Q11 is as follows.

[0111] Due to parameter w The range is normalized to -1 to 1. Therefore, when the bit width of the quantizer Q11's output signal is a symbolic 2 bits, the output signal of the quantizer Q11 can be quantized to [-1, 0, 1] by rounding to the nearest integer. Furthermore, when the bit width of the quantizer Q11's output signal is greater than 2 bits, the output signal of the quantizer Q11... It can be calculated based on the following mathematical formula 13.

[0112] [Mathematical Expression 13]

[0113] In mathematical formula 13, z is the input value of vectorizer Q11 (i.e., the output signal of multiplier M11), and b is the bit width of the output signal of quantizer Q11.

[0114] Figure 4 The output signal of quantizer Q11 for fixed decimal point type B It can be calculated based on mathematical formula 14.

[0115] [Mathematical Expression 14]

[0116] In mathematical formula 14, M = max(a, b), where a and b are... Figure 4 The range of the fixed decimal point type B is from -a to b.

[0117] Equation 14 is one example of a quantization method, but the invention is not limited thereto. In other embodiments, fine quantization can achieve good precision without using a maximum value. The quantization performance of quantizer Q11 can be verified while changing the value of M in Equation 14.

[0118] According to the present invention as described above, the numbers used in the hundreds of multiplication and addition operations of the neural network fall within the range of -1 to 1. The parameters A' and B', which are normalization parameters after learning, can change their range based on the learning results, but each change occurs only once. Even if changes occur to parameters A' and B', the impact is limited, thus facilitating hardware redesign.

[0119] The preferred embodiments of the present invention have been described above with reference to the present invention. However, those skilled in the art or those of ordinary skill in the art should understand that the present invention can be modified and altered in various ways without departing from the scope of the concept and technical field of the present invention as set forth in the claims. Therefore, the technical scope of the present invention should not be limited by the content set forth in the detailed description of the specification, but should be determined solely by the claims.

Claims

1. A neural network processing device, comprising: The convolutional unit receives the input signal and the learned weight parameters, performs a convolution operation on the input signal and the learned weight parameters, and outputs a convolutional signal. The configuration adjustment unit receives the convolution signal and the learning completion normalization parameters, and outputs an adjustment signal that adjusts the output deviation of the convolution signal; as well as The activation unit transforms the adjustment signal into an activation function to output a signal. The convolutional part includes: The multiplier performs a multiplication operation on the input signal and the learned weight parameters; and A quantizer quantizes the output of the multiplier and outputs the convolution signal; The quantizer is executed based on a mathematical formula. Quantitative operations, z is the output of the multiplier, t is the bit width of the convolution signal, and M is the maximum value in the range -a to b. It is the convolutional signal, The input signal is an image signal.

2. The neural network processing apparatus according to claim 1, wherein, The activation unit normalizes the output signal to a value between 0 and 1.

3. The neural network processing apparatus according to claim 1, wherein, The learning completion weight parameters are expressed by mathematical formulas. We can calculate that w is the learning parameter.

4. The neural network processing apparatus according to claim 1, wherein, The learning process completes by normalizing the parameters to values ​​between -1 and 1.

5. The neural network processing apparatus according to claim 1, wherein, The normalization parameters for the learning process include a first parameter and a second parameter. The configuration adjustment unit includes: The multiplier performs a multiplication operation on the convolution signal and the first parameter; The first quantizer quantizes the output of the multiplier; An adder that performs an addition operation on the output of the first quantizer and the second parameter; and The second quantizer quantizes the output of the adder.

6. The neural network processing apparatus according to claim 5, wherein, The first parameter is based on the mathematical formula The calculated values, where w is the learning parameter and y is the output signal. It is the maximum value of the absolute value of the learning parameter w. It is the maximum value of the absolute value of the output signal y. , The learning parameters are determined at the moment the learning is completed. It is the standard deviation of the input signal.

7. The neural network processing apparatus according to claim 5, wherein, The second parameter is based on the mathematical formula The calculated value, y is the output signal. It is the maximum value of the absolute value of the output signal y. , and These are the learning parameters determined at the moment the learning is completed. It is the average of the input signal. It is the standard deviation of the input signal.

8. The neural network processing apparatus according to claim 1, wherein, The activation unit transforms the adjustment signal into the output signal between 0 and 1 according to the activation function.

9. A neural network processing device, comprising: The convolutional unit includes an input layer, an intermediate layer, and an output layer. The intermediate layer receives an input signal and learned weight parameters from the input layer, performs a convolution operation on the input signal and the learned weight parameters, and outputs a convolutional signal. The configuration adjustment unit receives the convolution signal and the learning completion normalization parameters, and outputs an adjustment signal that adjusts the output deviation of the convolution signal; as well as The activation unit transforms the adjustment signal into an activation function to output the output signal to the output layer. The convolutional part includes: The multiplier performs a multiplication operation on the input signal and the learned weight parameters; and A quantizer quantizes the output of the multiplier and outputs the convolution signal. The quantizer is executed based on mathematical formulas. Quantitative operations, z is the output of the multiplier, t is the bit width of the convolution signal, and M is the maximum value in the range -a to b. It is the convolutional signal, The input signal is an image signal.

10. The neural network processing apparatus according to claim 9, wherein, The activation unit normalizes the output signal to a value between 0 and 1.

11. The neural network processing apparatus according to claim 9, wherein, The learning completion weight parameter is based on the mathematical formula. We can calculate that w is the learning parameter.

12. The neural network processing apparatus according to claim 9, wherein, The learning process completes by normalizing the parameters to values ​​between -1 and 1.

13. The neural network processing apparatus according to claim 9, wherein, The normalization parameters for the learning process include a first parameter and a second parameter. The configuration adjustment unit includes: The multiplier performs a multiplication operation on the convolution signal and the first parameter; The first quantizer quantizes the output of the multiplier; An adder that performs an addition operation on the output of the first quantizer and the second parameter; and The second quantizer quantizes the output of the adder.

14. The neural network processing apparatus according to claim 13, wherein, The first parameter is based on the mathematical formula The calculated values, where w is the learning parameter and y is the output signal. It is the maximum value of the absolute value of the learning parameter w. It is the maximum value of the absolute value of the output signal y. , The learning parameters are determined at the moment the learning is completed. It is the standard deviation of the input signal.

15. The neural network processing apparatus according to claim 13, wherein, The second parameter is based on the mathematical formula The calculated value, y is the output signal. It is the maximum value of the absolute value of the output signal y. , and These are the learning parameters determined at the moment the learning is completed. It is the average of the input signal. It is the standard deviation of the input signal.

Images