Processor for processing softmax function and method for operating processor

Abstract

A method performed by at least one processor configured to implement an accelerator for processing a softmax function may be provided. The method may comprise the steps of: obtaining input data including a plurality of quantized input values; generating input data distribution information indicating the frequency of the plurality of quantized input values included in the input data; identifying the largest value from among the plurality of quantized input values as a first maximum value on the basis of the input data distribution information; determining the difference between the first maximum value and a second maximum value as an offset value; determining a plurality of index values by applying an offset value to each of the plurality of quantized input values; and outputting a value of a softmax function corresponding to each of the plurality of quantized input values on the basis of the plurality of index values.

Description

Processor and Processor Operation Method for Processing Softmax Functions

[0001] The present disclosure relates to a processor and a method of operating the processor. More specifically, the present disclosure relates to a processor for processing artificial intelligence operations and a method of operating the processor.

[0002] Electronic devices such as TVs, portable terminals, and home appliances may include processors for processing or implementing artificial intelligence (AI). A processor for implementing AI may refer to a central processing unit (CPU) or a separate processor for accelerating the processing of AI operations, including neural networks (NN). A processor for processing AI operations may include a graphics processing unit (GPU) or a neural processing unit (NPU).

[0003] The AI ​​operations processed by the processor may include deep learning neural network operations. Deep learning neural networks may include convolution operations, matrix-to-matrix (or matrix-to-vector) multiplication operations, and operations involving nonlinear activation functions. Unlike servers with sufficient computational power, electronic devices may not have sufficient computational power for AI processing. Depending on the processing efficiency of each AI operation involved, the performance of on-device AI processes (e.g., inference using AI technology) on electronic devices may vary significantly.

[0004] Non-linear activation functions can include the sigmoid function, the hyperbolic tangent (Tanh) function, the Rectified Linear Unit (ReLU) function, the Exponential Linear Unit (ELU) function, the Gaussian Error Linear Unit (GELU) function, and the Softmax function.

[0005] The softmax function, a nonlinear activation function, involves exponential and division operations, making its processing complexity high. Processors used for artificial neural networks (or artificial intelligence operations) can utilize a separate processor to process the softmax function. However, using a separate processor to process the softmax function consumes time and resources due to data transfer. Therefore, a method may be required for a processor used for artificial neural networks (or artificial intelligence operations) to process the softmax function.

[0006] In one embodiment of the present disclosure, a method is provided, which is performed by at least one processor configured to implement an accelerator for processing a softmax function. The method may include obtaining input data including a plurality of quantized input values ​​using an input module of the accelerator. The method may include generating input data distribution information indicating frequencies of the plurality of quantized input values ​​included in the input data using a distribution information processing module of the accelerator. The method may include identifying a largest value among the plurality of quantized input values ​​as a first maximum value based on the input data distribution information using the distribution information processing module. The method may include determining a difference between a second maximum value and the first maximum value, which indicates a maximum quantized value that can be expressed as an input value of the input data, as an offset value using the distribution information processing module. The method may include determining a plurality of index values ​​by applying an offset value to each of the plurality of quantized input values ​​using a look-up table (LUT) module of the accelerator. The method may include outputting a value of a softmax function corresponding to each of the plurality of quantized input values ​​based on the plurality of index values ​​using the look-up table module.

[0007] In one embodiment of the present disclosure, a processor for processing a softmax function is provided. The processor may be configured to implement an accelerator including an input module, a distribution information processing module, and a lookup table module. The input module may be configured to obtain input data including a plurality of quantized input values. The distribution information processing module may generate input data distribution information indicating frequencies of the plurality of quantized input values ​​included in the input data. The distribution information processing module may identify a largest value among the plurality of quantized input values ​​as a first maximum value based on the input data distribution information. The distribution information processing module may be configured to determine a difference between a second maximum value indicating a maximum quantized value that can be expressed as an input value of the input data and the first maximum value as an offset value. The lookup table module may determine a plurality of index values ​​by applying the offset value to each of the plurality of quantized input values. The lookup table module may be configured to output values ​​corresponding to the plurality of index values ​​as values ​​of a softmax function corresponding to each of the plurality of quantized input values.

[0008] In one embodiment of the present disclosure, an electronic device may be provided. The electronic device may include a first processor, a second processor, and a memory for processing a softmax function. The second processor may control the first processor to process an operation related to artificial intelligence including the softmax function by executing instructions contained in the memory. The first processor may be configured to implement an accelerator. The accelerator may include an input module, a distribution information processing module, and a lookup table module. The input module may be configured to obtain input data including a plurality of quantized input values. The distribution information processing module may generate input data distribution information indicating frequencies of the plurality of quantized input values ​​included in the input data. Based on the input data distribution information, a largest value among the plurality of quantized input values ​​may be identified as a first maximum value. The electronic device may be configured to determine a difference between the first maximum value and a second maximum value indicating a maximum quantized value that can be expressed as an input value of the input data as an offset value. The lookup table module may determine a plurality of index values ​​by applying the offset value to each of the plurality of quantized input values. The values ​​corresponding to multiple index values ​​can be output as the values ​​of the softmax function corresponding to each of multiple quantized input values.

[0009] FIG. 1 is a block diagram illustrating a processor for processing an artificial neural network (or artificial intelligence operation) according to one embodiment of the present disclosure.

[0010] FIG. 2 is a block diagram illustrating a processor for processing an artificial neural network (or artificial intelligence operation) according to one embodiment of the present disclosure.

[0011] FIG. 3 is a flowchart illustrating an operation method of a processor including an accelerator for processing a softmax function according to one embodiment of the present disclosure.

[0012] FIG. 4a is a diagram for explaining quantization of input values ​​according to one embodiment of the present disclosure.

[0013] FIG. 4b is a diagram for explaining quantization of exponential function values ​​according to one embodiment of the present disclosure.

[0014] FIG. 4c is a diagram for explaining quantization of input values ​​and exponential function values ​​according to one embodiment of the present disclosure.

[0015] FIG. 5 is a diagram for explaining an operation of modifying input data distribution information according to one embodiment of the present disclosure.

[0016] FIG. 6 is a diagram for explaining an operation of calculating a denominator value of a softmax function according to one embodiment of the present disclosure.

[0017] FIG. 7 is a flowchart illustrating the operation of a distribution information processing module and an input module of an accelerator according to one embodiment of the present disclosure.

[0018] FIG. 8a is a flowchart illustrating the operation of a parameter module of an accelerator according to one embodiment of the present disclosure.

[0019] FIG. 8b is a flowchart illustrating the operation of a parameter module of an accelerator according to one embodiment of the present disclosure.

[0020] FIG. 9 is a flowchart for explaining the operation of a post-processing module of an accelerator according to one embodiment of the present disclosure.

[0021] FIG. 10 is a block diagram illustrating the operation of an accelerator according to one embodiment of the present disclosure.

[0022] FIG. 11 is a flowchart of a processor operation method according to input data according to one embodiment of the present disclosure.

[0023] FIG. 12A is a diagram illustrating a process in which a processor performs artificial intelligence operations according to one embodiment of the present disclosure.

[0024] FIG. 12b is a diagram for explaining a process in which a processor performs artificial intelligence operations according to one embodiment of the present disclosure.

[0025] FIG. 13 is a block diagram illustrating an electronic device according to one embodiment of the present disclosure.

[0026] FIG. 14 is an exemplary drawing for explaining an electronic device including a processor according to one embodiment of the present invention.

[0027] In this disclosure, the expression “at least one of a, b or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “all of a, b and c”, or variations thereof.

[0028] The terms used in this disclosure are selected from widely used, common terms, taking into account the functions of the disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings can be understood through the relevant description. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of the disclosure.

[0029] In this disclosure, singular expressions may include plural expressions unless the context clearly dictates otherwise. Terms containing ordinal numbers, such as "first" or "second," used in this disclosure may be used to describe various components, but the components should not be limited by these terms. These terms are used solely to distinguish one component from another.

[0030] When a part of this disclosure is said to "include" a component, this does not exclude other components, but rather may include other components, unless otherwise specifically stated. In this disclosure, terms such as "part" and "module" may refer to a unit that processes at least one function or operation.

[0031] The expression "configured to" as used herein can be used interchangeably with, for example, "suitable for", "having the capacity to", "designed to", "adapted to", "made to", or "capable of", depending on the context. The term "configured to" does not necessarily mean something that is "specifically designed to" in terms of hardware. In some contexts, the expression "a system configured to" can mean that the system is "capable of" together with other devices or components. The expression "a module does" as used herein can be used interchangeably with "a module is configured to", depending on the context.

[0032] When a component is referred to as being "connected" or "connected" to another component in this disclosure, it should be understood that the component may be directly connected or connected to the other component, but may also be connected or connected via another component in between, unless otherwise specifically stated.

[0033] In this disclosure, "transmitting" or "forwarding" data by a module may mean providing data to another module. A module may be connected to another module via wires or wirelessly. A module may provide data directly to another module, or may provide data through another module.

[0034] In describing the present disclosure, descriptions of technical contents that are well known in the technical field to which the present disclosure pertains and are not directly related to the present disclosure may be omitted. This is to convey the gist of the present disclosure more clearly without obscuring the gist of the present disclosure by omitting some descriptions. In the drawings, parts that are irrelevant to the description are omitted for clear description of the present disclosure, and similar parts are designated with similar reference numerals. In addition, the reference numerals used in each drawing are only for describing each drawing, and different reference numerals used in different drawings do not indicate different elements. The size of each component does not entirely reflect the actual size. The same or corresponding components in each drawing are given the same reference numerals.

[0035] The advantages and features of the present disclosure, and methods for achieving them, will become clearer with reference to the embodiments described below in detail with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. The disclosed embodiments are provided to ensure that the disclosure is complete and to fully inform those skilled in the art of the disclosure of the scope of the disclosure. An embodiment of the present disclosure may be defined according to the claims.

[0036] In the present disclosure, each block of the flowchart drawings and combinations of the flowchart drawings can be performed by computer program instructions. The computer program instructions can be installed on a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, and the instructions executed by the processor of the computer or other programmable data processing equipment can create means for performing the functions described in the flowchart block(s). The computer program instructions can also be stored in a computer-available or computer-readable memory that can direct a computer or other programmable data processing equipment to implement functions in a specific manner, and the instructions stored in the computer-available or computer-readable memory can also produce an article of manufacture that includes instruction means for performing the functions described in the flowchart block(s). The computer program instructions can also be installed on a computer or other programmable data processing equipment.

[0037] In the present disclosure, each block in the flowchart diagram may represent a module, segment, or portion of code that includes one or more executable instructions for performing a specified logical function(s). In one embodiment, the functions described in the blocks may occur out of order. For example, two blocks depicted in succession may be executed substantially simultaneously or, depending on the function, may be executed in reverse order.

[0038] As is customary in the art, the embodiments may be described in terms of functional blocks, units, and / or modules, and are illustrated in the drawings. Those skilled in the art will appreciate that such blocks, units, and / or modules may be physically implemented by electronic (or optical) circuits, such as logic circuits, discrete components, microprocessors, hard-wired circuits, memory elements, wiring connections, and the like, and may be formed using semiconductor-based manufacturing techniques or other manufacturing techniques. For blocks, units, and / or modules implemented by microprocessors or the like, they may be programmed using software (e.g., microcode) to perform the various functions discussed herein and may optionally be driven by firmware and / or software. Alternatively, each block, unit, and / or module may be implemented as dedicated hardware, or a combination of dedicated hardware for performing some functions and processors (e.g., one or more programmed microprocessors and associated circuitry) for performing other functions. Furthermore, each block, unit, and / or module of the embodiments may be physically separated into two or more interacting individual blocks, units, and / or modules without departing from the scope of the present invention. Additionally, the blocks, units and / or modules of the embodiments may be physically combined into more complex blocks, units and / or modules without departing from the scope of the present invention.

[0039] The term '~ unit' used in one embodiment of the present disclosure may represent a hardware component such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit), and the '~ unit' may perform a specific role. Meanwhile, the '~ unit' may not be limited to hardware. The '~ unit' may be configured to be on an addressable storage medium and may be configured to play one or more processors. In one embodiment, the '~ unit' may include components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided through a specific component or a specific '~ unit' may be combined to reduce the number thereof or separated into additional components. In addition, in one embodiment, the '~ unit' may include one or more processors.

[0040] Below, with reference to the attached drawings, embodiments of the present disclosure are described in detail to facilitate their implementation by those skilled in the art. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. The present disclosure will be described in detail below with reference to the attached drawings.

[0041] In the present disclosure, the input module may include a module configured to acquire data for the accelerator. The input module of the present disclosure may be replaced with terms such as an input queue or a data acquisition module. Alternatively, the input module may include or be replaced with an input buffer.

[0042] In the present disclosure, quantization may include converting data that can be expressed continuously into discontinuous data or discrete data. Quantization may mean converting a value within a predetermined range into a reference value. For example, quantization may mean expressing a value greater than or equal to x0 and less than x1 as 0, and expressing a value greater than or equal to x1 and less than x2 as 1. Quantization in the present disclosure may be replaced with terms such as “sampling,” “discretization,” or “integerization.” Quantized data according to one embodiment of the present disclosure may be understood with reference to FIGS. 4A, 4B, and 4C.

[0043] In the present disclosure, the maximum quantization value may mean the largest value (e.g., largest value, or greatest value) that quantized data can represent. For example, when data is quantized to an integer value greater than or equal to “0” and less than or equal to “255,” the maximum quantization value may mean “255.” According to one embodiment, the maximum quantization value of the output data of the accelerator may be the same as the maximum quantization value of the input data of the accelerator, but is not limited thereto. The maximum quantization value of the present disclosure may be replaced with terms such as “quantization upper limit,” “maximum representable value,” or “quantization range.” The maximum quantization value according to one embodiment of the present disclosure may be understood with reference to FIGS. 4A, 4B, and 4C.

[0044] In the present disclosure, the frequency of an input value may include the number of times the input value is included in the input data. For example, if the input data is [0, 2, 2, 1, 0], the input values ​​are “0”, “2”, “2”, “1”, “0”. Here, the frequencies for the input values ​​“0”, “1”, and “2” may be 2, 1, and 2, respectively. The frequency of a quantized value not included in the input data may be 0. In the above example, the frequency for the input value “3” may be 0. In one embodiment, the distribution information processing module of the accelerator may generate input data distribution information including the frequency of input values ​​of the input data. The input data distribution information may refer to information related to the distribution of input values ​​included in the input data. For example, the input data distribution information may refer to information indicating the frequency of input values ​​(e.g., the number of times an input value is included in the input data). The input data distribution information of the present disclosure may be replaced with terms such as "input data frequency information" or "input data statistical information." The frequency of input values ​​may be determined for all quantization values. For example, if the maximum quantization value is "255," the frequency of input values ​​may be determined for each integer greater than or equal to "0" and less than or equal to "255." For example, the input data distribution information may include or represent the number of times each integer greater than or equal to "0" and less than or equal to "255" is included in the input data.

[0045] In the present disclosure, the distribution information processing module may include a module configured to process input data distribution information. For example, the distribution information processing module may generate input data distribution information. The distribution information processing module in the present disclosure may be replaced with terms such as "distribution analysis module" or "data processing module."

[0046] In the present disclosure, the lookup table module may include a module configured to output the value of a softmax function. The lookup table module may output the value of the softmax function corresponding to input data. The lookup table module of the present disclosure may be replaced with terms such as "reference table module," "data dictionary module," or "data mapping module."

[0047] In the present disclosure, an index value may mean a value representing one of a plurality of values ​​stored in a lookup table module. In one embodiment, the lookup table module may store information including a plurality of index values ​​and a value corresponding to each index value. The lookup table module may obtain an index value and provide a stored value corresponding to the index value. The index value in the present disclosure may be replaced with terms such as "index," "key," "key value," or "lookup key."

[0048] In the present disclosure, the exponential function information may include information regarding a quantized exponential function value. The exponential function information may include an exponential function value for a value that quantized input data can express. For example, if the input data is quantized to one of the integer values ​​greater than or equal to “0” and less than or equal to “255,” the exponential function information may include the integer values ​​greater than or equal to “0” and less than or equal to “255” and an exponential function value corresponding to each integer value. In one embodiment, the exponential function value may be a quantized value. According to one embodiment, the exponential function information may include data expressed in a table format, in which a plurality of values ​​that the quantized input data can express and the exponential function value corresponding to each value are expressed. The exponential function information in the present disclosure may be replaced with terms such as “exponential function table,” “exponential function LUT information,” “exponential function data,” or “exponential value information.”

[0049] In the present disclosure, shifting may mean an operation of modifying input data distribution information. According to one embodiment, shifting the input data distribution information by an offset value may mean that the frequency of input values ​​of the input data distribution information is shifted by the offset value. When the input data distribution information is shifted by the offset value, the offset value may be added to each of the input values ​​included in the input data distribution information. For example, if the frequency of the input value “x” is “f1” and the input data is shifted by the offset value “a”, the frequency of the input value (“x+a”) may become “f1”. For example, when the frequencies of the input values ​​“0”, “1”, “2”, and “3” are 5, 1, 3, and 0, respectively, shifting by 1 by the accelerator may mean modifying the input data distribution information so that the frequencies of the input values ​​“0”, “1”, “2”, and “3” become 0, 5, 1, and 3, respectively. The modified input data distribution information may mean the result of shifting the input data distribution information by an offset value.

[0050] In the present disclosure, a shift operation may mean an arithmetic shift. The shift operation may include a left shift operation or a right shift operation. A left shift operation may be expressed as "<<", and a right shift operation may be expressed as ">>". A left shift operation may be understood to mean shifting binary bits of a value to be operated on to the left. Shifting to the left by N bits means shifting the value to be operated on by 2. N (2 to the power of N) can be understood as multiplying. For example, 3<<2 can mean 12. In the present disclosure, performing a left shift operation by N means multiplying the target value by 2. N (2 to the Nth power) can mean multiplying. The right shift operation can be understood as moving the binary bits of the value to be operated on to the right. Shifting to the right by N bits means multiplying the value to be operated on by 2. N (2 to the power of N) can be understood as dividing by 2. In the present disclosure, performing a right shift operation by N means dividing the target value by 2. N It can mean dividing (2 to the power of N). For example, 12>>2 can mean 3.

[0051] FIG. 1 is a block diagram illustrating a processor for processing an artificial neural network (or artificial intelligence operation) according to one embodiment of the present disclosure.

[0052] Referring to FIG. 1, a processor (100) may include an accelerator (110). The accelerator (110) may include an input module (120), a distribution information processing module (130), and a look-up table (LUT) module (140). The input module (120), the distribution information processing module (130), and the look-up table module (140) may be implemented in the form of hardware (HW) in the accelerator (110).

[0053] In one embodiment, the processor (100) may receive an activation function as input. The activation function may be a non-linear function that outputs a non-linear value for the input. The activation function may include a softmax function. The softmax function may be expressed as in mathematical equation 1.

[0054]

[0055] The value of the softmax function for the ith input data is determined based on the sum of the exponents of all input data and the value of the exponent of the ith input data.

[0056] The softmax function can produce the same result value even if a reference value is added to or subtracted from the input data. For example, the value of the softmax function for the case where the input data is [x1, x2, x3] is the same as the value of the softmax function for the case where the input data is [x1+a, x2+a, x3+a]. The processor (100) can calculate the value of the softmax function by adjusting the input data. For example, the processor (100) can calculate the value of the softmax function for the case where the input data is [4000, 3999, 3998] by changing it to the case where the input data is [2, 1, 0]. Similarly, the processor (100) can calculate the value of the softmax function by changing the case where the input data is [2, 1, 0] to [4000, 3999, 3998]. Therefore, the accelerator (110) according to one embodiment can determine the value of the softmax function by adjusting the input data. The accelerator (110) can generate modified input data distribution information and determine the value of the softmax function using the modified input data distribution information. An exemplary operation of the accelerator (110) determining the value of the softmax function using the modified input data distribution information according to one embodiment will be described in detail with reference to FIGS. 1 to 11.

[0057] In one embodiment, the processor (100) may process an artificial intelligence operation, such as a softmax function. The processor (100) may process the softmax function to produce at least one result value related to the operation of an electronic device including the processor (100). The processor (100) may include an accelerator (110).

[0058] In one embodiment, the accelerator (110) may be an auxiliary computational device included in or associated with an electronic device that processes or is configured to process artificial intelligence computations. For example, the accelerator (110) may be an auxiliary computational device used in a processor (100) that performs operations using machine learning or deep learning. The accelerator (110) may increase the efficiency of computations performed by the processor (100). For example, the accelerator (110) may reduce the time required for repeated computations to process a function input to the processor (100). In one embodiment, the processor (100) may process a softmax function using the accelerator (110).

[0059] The input module (120) can obtain input data including a plurality of quantized input values. The input module (120) can include an input buffer configured to store or store the input data. For example, the input buffer can sequentially obtain input values ​​of the input data from at least one of a memory or another accelerator. In one embodiment, the input module (120) can obtain the input data from a memory included in the accelerator (110) (e.g., 270 of FIG. 2) or another accelerator within the processor (100).

[0060] In one embodiment, the input data can be represented in the form of a vector, a matrix, or a tensor. For example, the input data can be vector type data having a size of 1 x 1 x D. Or, for example, the input data can be matrix type data having a size of 1 x W x H, tensor type data having a size of C x W x H, or a tensor having a dimension of 4 or more.

[0061] In one embodiment, the input value of the input data may be a quantized value. The input value may be an integer within a predetermined range. The input value may be an integer between “0” and the maximum quantized value. For example, if the maximum quantized value is “255,” the input value may be one of the integers between “0” and “255.” An example of quantized data according to one embodiment of the present disclosure is described in detail with reference to FIGS. 4A, 4B, and 4C.

[0062] The input module (120) can transmit input data to the distribution information processing module (130). In one embodiment, the input module (120) can sequentially transmit input values ​​of the input data to the distribution information processing module (130). The input module (120) can transmit the input data to the lookup table module (140). The lookup table module (140) can use the input data obtained from the input module (120) to output the value of the softmax function.

[0063] The distribution information processing module (130) can generate input data distribution information indicating the frequency of a plurality of quantized input values ​​included in the input data. For example, the distribution information processing module (130) can generate information including the number of times an input value appears in the input data. For example, if the input values ​​of the input data are “0”, “3”, “2”, “2”, and “0”, the input data distribution information can include information that the frequency of “0” is 2, the frequency of “1” is 0, the frequency of “2” is 2, and the frequency of “3” is 1. An example of input data distribution information according to an embodiment of the present disclosure is described in detail with reference to FIG. 5.

[0064] The distribution information processing module (130) can identify the largest value among a plurality of quantized input values ​​as the first maximum value based on the input data distribution information. The distribution information processing module (130) can identify the largest value among the quantized values ​​whose frequency is not 0 among the input data distribution information as the first maximum value. For example, if the input values ​​of the input data are “0”, “3”, “2”, “2”, and “0”, the quantized values ​​whose frequency is not 0 among the input data distribution information are “0”, “2”, and “3”. The distribution information processing module (130) can identify “3”, which is the largest value among “0”, “2”, and “3”, as the first maximum value.

[0065] The distribution information processing module (130) can determine the difference between the second maximum value, which represents the maximum quantization value that can be expressed as the input value of the input data, and the first maximum value as an offset value. For example, if the maximum quantization value of the input data is “7” and the input values ​​are “0”, “3”, “2”, “2”, and “0”, the first maximum value is “3” and the second maximum value is “7”. Accordingly, the distribution information processing module (130) can determine the offset value as “4” (because = 7 - 3). The distribution information processing module (130) can transmit the offset value to the lookup table module (140).

[0066] The lookup table module (140) can determine index values ​​by applying an offset value to each of a plurality of quantized input values. For example, if the input values ​​of the input data are “0”, “3”, “2”, “2”, and “0”, and the offset value is 4, the lookup table module (140) can determine the index values ​​for each input value as “4”, “7”, “6”, “6”, and “4”. The lookup table module (140) can identify a plurality of quantized input values ​​based on input data obtained from the input module (120).

[0067] The lookup table module (140) can output a value corresponding to an index value as a value of a softmax function for an input value. The lookup table module (140) can store a plurality of values ​​and output a value corresponding to an index value from among the plurality of stored values. In one embodiment, the plurality of values ​​stored in the lookup table module (140) can be values ​​of a softmax function (or an approximate value of a softmax function) corresponding to each input value. The lookup table module (140) can store a lookup table including values ​​of a softmax function corresponding to a plurality of index values.

[0068] As shown in mathematical expression 1, the value of the softmax function includes an exponential function operation and a division operation. The accelerator (110) can output the value of the softmax function without performing at least some of the exponential function operation and the division operation by using the input module (120), the distribution information processing module (130), and the lookup table module (140).

[0069] FIG. 2 is a block diagram illustrating a processor for processing artificial intelligence operations according to one embodiment of the present disclosure.

[0070] Referring to FIG. 2, the processor (100) may further include a control module (210), a direct memory access (DMA) module (220), and a digital signal processing (DSP) module (230).

[0071] The control module (210) may be configured to control the operation of other components within the processor (100). In one embodiment, the control module (210) may control the operation of at least one of the accelerator (110) and the digital signal processing module (230). The control module (210) may control the accelerator (110) to output output data including the value of the softmax function based on the input data. For example, the control module (210) may control the accelerator (110) to perform at least one of a convolution operation and a softmax operation. According to one embodiment, the control module (210) may control the accelerator (110) based on a control command obtained from a central processing unit of an electronic device including the processor (100). For example, in response to a command to perform a neural network operation for the processor (100) of the central processing unit, the control module (210) can control at least one of the accelerator (110) or the digital signal processing module (230) to perform at least a part of the neural network operation.

[0072] The direct memory access module (220) may be configured to communicate with other components of an electronic device including the processor (100). The direct memory access module (220) may communicate with other components without going through the central processing unit of the electronic device. For example, the processor (100) may communicate with a graphics processing unit or a neural network processing unit.

[0073] The digital signal processing module (230) may be configured to process digital signals. The control module (210) may perform complex operations, excluding multiplication and addition operations, through the digital signal processing module (230). However, during the process of processing artificial intelligence operations, data exchange between the accelerator (110) and the digital signal processing module (230) may increase processing time. Therefore, when the processor (100) performs operations by maximizing the use of the accelerator (110), the operation execution time may be reduced.

[0074] The accelerator (110) may additionally include an input module (120), a distribution information processing module (130), and a lookup table module (140) of FIG. 1, as well as an operation module (240), a parameter module (250), a post-processing module (260), a memory (270), and an output module (280).

[0075] In one embodiment, the distribution information processing module (130) can transmit modified input data distribution information generated by shifting the input data distribution information by an offset value to the calculation module (240).

[0076] The calculation module (240) may be configured to perform calculations utilized in the accelerator (110). In one embodiment, the calculation module (240) may be configured to perform multiplication and addition operations. For example, the calculation module (240) may be configured to perform a MAC (Multipliy-Accumulate) operation. Since convolution operations and matrix multiplication, which account for a large proportion of artificial intelligence operations, are operations expressed as multiplication and addition, the accelerator (110) may accelerate artificial intelligence operations using the calculation module (240).

[0077] In one embodiment, the calculation module (240) may calculate the denominator value of the softmax function using the modified input data distribution information and the exponential function information. In one embodiment of the present disclosure, the value corresponding to the denominator of the softmax function may be referred to as the denominator value of the softmax function. The exponential function information may include information about the quantized exponential function value. The exponential function information according to one embodiment may be obtained from the memory (270). The denominator value of the softmax function may be calculated by performing a multiplication and addition operation between the modified input data distribution information and the exponential function information. In one embodiment, the value corresponding to the denominator of the softmax function may be referred to as the denominator value of the softmax function or the sum (SUM) of the exponential function. An example of an operation of the calculation module (240) according to one embodiment of the present disclosure to calculate the denominator value of the softmax function is described in detail with reference to FIG. 6. According to one embodiment, the calculation module (240) may transmit the denominator value of the softmax function to the parameter module (250).

[0078] The parameter module (250) may be configured to determine parameters for calculating the value of the softmax function. In one embodiment, the parameter module (250) may determine at least one parameter for representing an approximation to a denominator value of the softmax function. In one embodiment, the at least one parameter includes a first parameter and a second parameter, and a value obtained by dividing a second maximum value (e.g., a maximum quantized value of input data) by the denominator value of the softmax function may be determined based on a value obtained by performing a right shift operation on the first parameter by the second parameter. The parameter module (250) according to one embodiment may determine the first parameter and the second parameter for representing the value of the quantized softmax function. An example of an operation of determining a parameter according to one embodiment of the present disclosure will be described in detail with reference to FIGS. 8A and 8B. The parameter module (250) according to one embodiment may transmit at least one parameter to the post-processing module (260).

[0079] The post-processing module (260) may be configured to determine the value of the softmax function. In one embodiment, the post-processing module (260) may determine the value of the softmax function based on at least one parameter and exponential function information. For example, the post-processing module (260) may determine the value of the softmax function by applying at least one parameter to the exponential function value of the exponential function information.

[0080] In one embodiment, the post-processing module (260) may store the value of the generated softmax function in the lookup table module (140). The lookup table stored or included in the lookup table module (140) may include the values ​​of the softmax function corresponding to a plurality of index values. The value of the softmax function in the lookup table may be dynamically determined according to input data. The value of the softmax function may be dynamically determined according to input data (e.g., input data distribution information) because the accelerator (110) uses some approximate values ​​in the process of calculating the value of the softmax function.

[0081] The memory (270) may be configured to store information used in the accelerator (110). The memory (270) may store exponential function information. The exponential function information may include a quantized exponential function value. For example, the exponential function information may include a quantized exponential function value corresponding to the value "1" that a quantized input value can express. The accelerator (110) may determine the value of a softmax function using the exponential function information. In one embodiment, the memory (270) may store a plurality of pieces of exponential function information according to a maximum quantization value. The memory (270) may store output data obtained from the output module (280).

[0082] The output module (280) can output output data including the value of the softmax function. The output module (280) can include an output buffer. The output module (280) can obtain the value of the softmax function from the lookup table module (140). In one embodiment, the output module (280) can sequentially store the values ​​of the softmax function output from the lookup table module (140) in the memory (270).

[0083] FIG. 2 shows the structure of a processor (100) and an accelerator (110) according to one embodiment of the present disclosure, but is not limited thereto, and the processor (100) and the accelerator (110) may be configured by additionally including some components or by omitting some components.

[0084] In one embodiment, the accelerator (110) may additionally include a quantization module. The quantization module may be configured to perform quantization when the input data is not quantized. The maximum quantization value used by the accelerator (110) may be predetermined or transmitted along with the input data.

[0085] FIG. 3 is a flowchart illustrating the operation of a processor including an accelerator for processing a softmax function according to one embodiment of the present disclosure.

[0086] In operation 310, an input module (120) according to one embodiment can obtain input data including a plurality of quantized input values.

[0087] In operation 320, the distribution information processing module (130) according to one embodiment can generate input data distribution information indicating the frequency of a plurality of quantized input values ​​included in the input data.

[0088] In operation 330, the distribution information processing module (130) according to one embodiment can identify the largest value among a plurality of quantized input values ​​as the first maximum value based on input data distribution information.

[0089] In operation 340, the distribution information processing module (130) according to one embodiment may determine the difference between the second maximum value and the first maximum value, which represents the maximum quantization value that can be expressed as an input value of the input data, as an offset value.

[0090] In operation 350, the lookup table module (140) according to one embodiment can determine index values ​​by applying an offset value to each of a plurality of quantized input values.

[0091] In operation 360, the lookup table module (140) according to one embodiment can output a value corresponding to each of the index values ​​as a value of a softmax function for each of the plurality of quantized input values.

[0092] The operation of a processor (100) including an accelerator (110) for processing a softmax function does not necessarily include operations 310 to 360, and may omit some operations or include more operations.

[0093] FIG. 4a is a diagram for explaining quantization of input values ​​according to one embodiment of the present disclosure.

[0094] Referring to Fig. 4a, the unquantized input value (input) may be a value included in a predetermined range. For example, the unquantized input value may be a lower bound of the input value (x min ) is greater than or equal to the upper limit of the input value (x max ) can be a rational number less than (or less than or equal to) .

[0095] The quantized input value can be an integer within a given range. For example, the quantized input data can represent one of the integers greater than or equal to “0” and less than or equal to the maximum quantized value (Qmax). The maximum quantized value is “2 N -1” (where N is an integer). For example, the maximum quantization value can be one of “63”, “127”, “255”, “511”, or “1023”.

[0096] Values ​​within a given range can be quantized to the same value. For example, x min Greater than or equal to x minValues ​​less than +scale_x are quantized to “0”, and x min +scale_x is greater than or equal to x min Values ​​less than +2*scale_x can be quantized to “1”. The quantization interval scale_x can be determined based on the maximum quantized value. For example, the quantization interval is x min Greater than or equal to x max A smaller range can be divided into equal parts by the maximum quantization value. For example, scale_x = (x max - x min ) / can be determined by Qmax.

[0097] In one embodiment, input data x i is m = (x i -x min ) / scale_x can be quantized into an integer m satisfying the following equation. For example, the unquantized input data (x i ) can be expressed as in mathematical expression 2 using quantized input data (m).

[0098]

[0099] However, without limitation thereto, the quantization interval scale_x may be determined differently for each section by a design change of a typical technician.

[0100] In one embodiment, the accelerator (110) can quantize at least one of input data and output data. The accelerator (110) according to one embodiment can obtain quantized input data. The input data and the output data can include at least one of vector type, matrix type, or tensor type data including a plurality of quantized values. The input data can have the same vector, matrix, or tensor type data size as the output data. The input data and the output data can be quantized so that the maximum quantization value is the same.

[0101] When the input data of the accelerator (110) of FIG. 1 according to one embodiment of the present disclosure is quantized, the memory capacity required to store the data is reduced. Furthermore, calculating integers has the advantage of lower computational complexity than calculating prime numbers.

[0102] FIG. 4b is a diagram for explaining quantization of exponential function values ​​according to one embodiment of the present disclosure.

[0103] Referring to FIG. 4b, the unquantized exponential function value (EXP) may be a value included in a predetermined range. For example, the unquantized exponential function value may be a lower bound value of the exponential function value (e min ) is greater than or equal to the upper limit of the exponential function value (e max ) can be a rational number less than (or less than or equal to) the input value. The lower limit of the exponential function can be the exponential function value for the lower limit of the input value, and the upper limit of the exponential function can be the exponential function value for the upper limit of the input value.

[0104] The quantized exponential function values ​​(denoted as “Quantized EXP”) can be integers within a given range. The exponential function values ​​can be quantized based on the quantization interval (scale_EXP) of the exponential function. The quantization interval (scale_EXP) of the exponential function can be determined based on the upper bound value of the exponential function and the lower bound value of the exponential function. For example, scale_EXP = (e max - e min ) / Emax can be determined. The maximum quantization value (Emax) can be a predetermined value. For example, Emax can be “2^64”.

[0105] According to one embodiment, the exponential function value is a lower bound value (e min) can be quantized to be “0”. Referring to the first graph (410b), the quantized exponential function value can represent one of the integers greater than or equal to “0” and less than or equal to “Emax”. For example, the exponential function value (e i ) can be quantized into an integer n satisfying mathematical expression 3.

[0106]

[0107] According to one embodiment, the exponential function value is a lower bound value (e min ) can be quantized to become a predetermined constant. Referring to the second graph (420b), the quantized exponential function value can represent one of the integers greater than or equal to the constant "ZP" and less than or equal to "Emax+ZP". The second graph (420b) can be the first graph (410b) translated in the y-axis direction by ZP. In one embodiment, the predetermined constant can be determined based on the lower bound value and the quantization interval. For example, the constant (ZP) can be ZP = e min / can be determined by scale_EXP. In this example, mathematical expression 3 can be expressed as mathematical expression 4.

[0108]

[0109] The exponential function information according to one embodiment may include a quantized exponential function value described with reference to the first graph (410b) or may include a quantized exponential function value described with reference to the second graph (420b).

[0110] FIG. 4c is a diagram for explaining quantization of input values ​​and exponential function values ​​according to one embodiment of the present disclosure.

[0111] Referring to FIG. 4c, a third graph (410c) using non-quantized data and a fourth graph (420c) using quantized data are illustrated. The third graph (410c) is a graph between an input value (illustrated as “input”) and an exponential function value (EXP). In one embodiment, the input value (input) and the exponential function value (EXP) may each be quantized. The fourth graph (420c) is a graph between a quantized exponential function value (illustrated as “Quantized EXP”) and a quantized input value (illustrated as “Quantized input”).

[0112] According to one embodiment, the input value may be quantized based on a maximum quantization value Qmax for the input value, and the exponential function value may be quantized based on a maximum quantization value Emax for the exponential function value. For example, the input value may be quantized as in FIG. 4a, and the exponential function value may be quantized as in the first graph (410b) or the second graph (420b) of FIG. 4b.

[0113] The fourth graph (420c) shows the values ​​E0, E1, ..., E of the exponential function for the quantized input values ​​“0”, “1”, ..., Qmax. Qmax can be represented. According to one embodiment, the value of the exponential function may be set to “0” for the lower limit value, as in the first graph (410b) of FIG. 4b, or the value of the exponential function for the lower limit value, as in the second graph (420b), may be set to a predetermined constant (e.g., constant ZP).

[0114] FIG. 5 is a diagram for explaining an operation of modifying input data distribution information according to one embodiment of the present disclosure.

[0115] Referring to FIG. 5, the input data is data quantized to a maximum quantization value of “255” according to one embodiment. The input data distribution information may be represented in the form of a first graph (510) or a first table (520). However, the input data distribution information may be various forms of information indicating frequency information of input values. In one embodiment, at least one of the first graph (510) and the first table (520) may be generated by the distribution information processing module (130) of FIG. 1.

[0116] The distribution information processing module (130) can use the input data distribution information to identify the largest value among the input values ​​as the first maximum value. For example, since the largest value with a non-zero frequency in the first graph (510) and the first table (520) is “161,” the distribution information processing module (130) can identify “161” as the first maximum value.

[0117] The distribution information processing module (130) can determine an offset value based on the first maximum value and the maximum quantization value. The offset value can be determined as a value obtained by subtracting the first maximum value from the maximum quantization value. The distribution information processing module (130) can determine the offset value obtained by subtracting the first maximum value, “161,” from the maximum quantization value, “255,” as “94.”

[0118] The distribution information processing module (130) can generate modified input data distribution information based on the input data distribution information, based on the offset value. The modified input data distribution information may be updated information in the input data distribution information or newly generated information separately from the input data distribution information. The distribution information processing module (130) can modify the input data distribution information indicating the first graph (510) or the first table (520) into modified input data distribution information indicating the second graph (530) or the second table (540) by shifting the input data distribution information by the offset value (e.g., the value “94”). The modified input data distribution information may be represented in the form of the second graph (530) or the second table (540). However, the modified input data distribution information, like the input data distribution information, may also be various forms of information corresponding to frequency information of input values.

[0119] The distribution information processing module (130) can generate modified input data distribution information by shifting the input data distribution information by an offset value. The distribution information processing module (130) can generate modified input data distribution information by modifying the frequency of input values ​​of the input data distribution information to shift by the offset value. For example, the distribution of the second graph (530) can be the same as the result of shifting the distribution of the first graph (510) to the right by the offset value. For example, the distribution of the second table (540) can be the same as the result of shifting the distribution of the first table (520) by a value that adds the offset value. As illustrated in FIG. 5, the frequency Y corresponding to the quantization value “i” of the first table (520) i may mean that the input value has the same frequency as “i”. The frequency Z corresponding to the quantized value “i” of the second table (540) i is Y i+offset can be the same as (where “offset” represents the offset value). Frequency Y i+offsetThe frequency of the input value may be equal to "i" shifted by the offset value.

[0120] The distribution information processing module (130) can obtain data distribution information having the same distribution as the input data but with a larger value by generating modified input data distribution information. By generating modified input data distribution information, the distribution information processing module (130) can increase the accuracy of the value of the softmax function.

[0121] FIG. 6 is a diagram for explaining an operation of calculating a denominator value of a softmax function according to one embodiment of the present disclosure.

[0122] Referring to FIG. 6, the exponential function information (620) may include quantized exponential function values ​​(illustrated as “output”) corresponding to quantized input values ​​(illustrated as “input”). The exponential function information (620) may include quantized input values ​​i (e.g., “0”, “1”, ..., Qmax) and exponential function values ​​corresponding to each input value (e.g., E0, E1, ..., E Qmax ) may be included. According to one embodiment, the input value may be quantized based on the maximum quantization value Qmax for the input value, and the exponential function value may be quantized based on the maximum quantization value Emax for the exponential function value. For example, the input value may be quantized as in FIG. 4a, and the exponential function value may be quantized as in the first graph (410b) or the second graph (420b) of FIG. 4b.

[0123] As in mathematical expression 1, an exponential function summation can be used to calculate the value of the softmax function. The processor (100) can determine the exponential function summation using the input data distribution information (610) and the exponential function information (620). For example, the processor (100) can calculate the exponential function summation by multiplying corresponding values ​​between the input data distribution information (610) and the exponential function information (620). According to one embodiment, the processor (100) can calculate the exponential function summation by multiplying the result of adding a predetermined constant (e.g., ZP) to the exponential function information (620) by the input data distribution information (610). The input data distribution information (610) can be one of the input data distribution information or the modified input data distribution information of FIG. 5.

[0124] The value of the softmax function can be expressed as in mathematical equation 1. According to one embodiment, the processor (100) can determine the value of the softmax function based on the quantized input data and the value of the quantized exponential function. For example, the processor (100) can determine the value of the softmax function as in mathematical equation 5.

[0125]

[0126] By calculating the value of the softmax function using quantized input data and quantized exponential function values, we can obtain the same value as the value of the softmax function using the non-quantized exponential function.

[0127] In one embodiment, when the exponential function information (620) includes quantized exponential function values, data storage space can be reduced and computational speed can be improved. For example, by quantizing the exponential function information, the values ​​of each exponential function can be reduced, and by representing integer values, multiplication or addition operations can be performed quickly.

[0128] However, when quantizing an exponential function, an error due to quantization may occur. The exponential function can be quantized based on the largest and smallest values ​​of the exponential function. Since the exponential function increases exponentially, the smaller the value of the exponential function, the larger the error may be. For example, exp(0) = 1, exp(1) = 2.71, exp(7) = 1073.46. For example, if the value of the exponential function is quantized to [0, 255], scale_EXP may be approximately “4.2” (or, for example, the integer “4”), and E0 = 0, E1 = 0, E7 = 255. Here, E i can mean the quantized value of the exponential function of i. exp(0) and exp(1) have different values, but they can both represent “0” in the quantized value. Therefore, when the input value of the input data is small, the error of the quantized exponential function is large, and when the input value of the input data is large, the error of the quantized exponential function is small. Similarly, the error of the value of the softmax function of the input data can be smaller as the input data is large. For example, the value of the softmax function can be more similar to the actual value when the input data is [253, 254, 255] than when the input data is [0, 1, 2].

[0129] The processor (100) can generate corrected input distribution information by shifting the input distribution information generated by the distribution information processing module (130) using an offset value. The processor (100) can determine the value of a softmax function with a small error by using the corrected input distribution information.

[0130] In one embodiment, the exponential function information (620) may be stored in the memory (270). In one embodiment, the accelerator (110) may calculate the denominator value of the softmax function based on the input data distribution information (610) and the exponential function information (620).

[0131] FIG. 7 is a flowchart illustrating the operation of a distribution information processing module and an input module of an accelerator according to one embodiment of the present disclosure.

[0132] In operation 710, the distribution information processing module (130) according to one embodiment can generate modified input data distribution information by shifting the input data distribution information by an offset value.

[0133] In operation 720, the operation module (240) according to one embodiment may calculate the denominator value (or the sum of the exponential function) of the softmax function using the modified input data distribution information and the exponential function information. The exponential function information may include an exponential function value corresponding to a quantized value. The operation module (240) according to one embodiment may calculate the denominator value of the softmax function by performing a multiplication and addition operation between the modified input data distribution information and the exponential function information.

[0134] In one embodiment, the plurality of values ​​stored in the lookup table module (140) may be values ​​determined based on the denominator value of the softmax function. For example, the plurality of values ​​stored in the lookup table module (140) may be values ​​of the softmax function determined using the denominator value of the softmax function calculated by the calculation module (240).

[0135] The operation of the distribution information processing module (130) and the input module (120) of the accelerator (110) according to one embodiment does not necessarily include operations 710 and 720, and may omit or include some operations.

[0136] FIG. 8a is a flowchart illustrating the operation of a parameter module of an accelerator according to one embodiment of the present disclosure.

[0137] In operation 810a, the parameter module (250) according to one embodiment may determine at least one parameter for representing an approximate value that approximates the denominator value of the softmax function. In one embodiment, the at least one parameter includes a first parameter and a second parameter, and a value obtained by dividing the second maximum value by the denominator of the softmax function may be determined based on a value obtained by performing a right shift operation on the first parameter by the amount of the second parameter.

[0138] In operation 820a, the post-processing module (260) according to one embodiment may determine the value of a softmax function corresponding to a quantized value based on at least one parameter and exponential function information. In one embodiment, the lookup table module (140) may store the value of the softmax function generated by the post-processing module (260).

[0139] The operation of the parameter module (250) of the accelerator (110) according to one embodiment does not necessarily include operations 810a and 820a, and may omit or include some operations.

[0140] FIG. 8b is a flowchart illustrating the operation of a parameter module of an accelerator according to one embodiment of the present disclosure.

[0141] A method for determining a parameter according to one embodiment of the present disclosure may be performed by the parameter module (250) of FIG. 2. The parameter module (250) may determine the parameter based on a maximum quantized value (QMAX) and a sum of quantized exponential functions (SUM). The parameter may include a first parameter (MULT) and a second parameter (RSHAMT). The parameter may be a value for approximating a value obtained by dividing the maximum quantized value by the sum of quantized exponential functions. In one embodiment, the value obtained by dividing the maximum quantized value by the sum of quantized exponential functions may be approximated as a value obtained by performing a right shift operation on the first parameter by the amount of the second parameter.

[0142] At operation 810b, the parameter module (250) may determine the values ​​of variables for determining parameters. The variable DIVIDEND may be initialized to the maximum quantization value. The variable DIVISOR may be initialized to the sum of the exponential function. The variable RSHMAT may be initialized to “0.” The parameter module (250) may proceed to operation 820b.

[0143] At operation 820b, the parameter module (250) may compare the sizes of the variables DIVISOR and DIVIDEND. For example, the parameter module (250) may determine whether the variables DIVISOR and DIVIDEND are greater than each other. If the variable DIVISOR is greater than the variable DIVIDEND, the parameter module (250) may proceed to operation 830b. If the variable DIVISOR is less than or equal to the variable DIVIDEND, the parameter module (250) may proceed to operation 840b.

[0144] At operation 830b, the parameter module (250) may update the value of the variable DIVIDEND and the value of the variable RSHMAT. In one embodiment, the parameter module (250) may perform a left shift operation on the value of the variable DIVIDEND. For example, the parameter module (250) may perform a left shift operation on the value of the variable DIVIDEND by 1. For example, the parameter module (250) may update the value of the variable DIVIDEND to a value that is doubled. In one embodiment, the parameter module (250) may update the value of the variable RSHMAT to a value that is increased by 1. For example, if the value of the variable RSHMAT is 0, the parameter module (250) may change the value of the variable RSHMAT to 1. The parameter module (250) may proceed to operation 832b.

[0145] In operation 832b, the parameter module (250) may compare the sizes of the variables DIVISOR and DIVIDEND. For example, the parameter module (250) may determine whether the variables DIVISOR and DIVIDEND are greater than each other. If the variable DIVISOR is greater than the variable DIVIDEND, the parameter module (250) may proceed to operation 830b. If the variable DIVISOR is less than or equal to the variable DIVIDEND, the parameter module (250) may proceed to operation 850b.

[0146] At operation 840b, the parameter module (250) may update the value of the variable DIVISOR and the value of the variable RSHMAT. In one embodiment, the parameter module (250) may perform a left shift operation on the value of the variable DIVISOR. For example, the parameter module (250) may perform a left shift operation on the value of the variable DIVISOR by 1. That is, the parameter module (250) may update the value of the variable DIVISOR to a value that is doubled. In one embodiment, the parameter module (250) may update the value of the variable RSHMAT to a value that is decreased by 1. For example, if the value of the variable RSHMAT is 0, the parameter module (250) may change the value of the variable RSHMAT to -1. The parameter module (250) may proceed to operation 842b.

[0147] In operation 842b, the parameter module (250) may compare the sizes of the variables DIVISOR and DIVIDEND. For example, the parameter module (250) may determine whether the variables DIVISOR and DIVIDEND are less than or equal to each other. If the variable DIVISOR is less than or equal to the variable DIVIDEND, the parameter module (250) may proceed to operation 840b. If the variable DIVISOR is greater than the variable DIVIDEND, the parameter module (250) may proceed to operation 844b.

[0148] At operation 844b, the parameter module (250) may update the value of the variable DIVISOR and the value of the variable RSHMAT. In one embodiment, the parameter module (250) may perform a right shift operation on the value of the variable DIVISOR. For example, the parameter module (250) may perform a right shift by 1 on the value of the variable DIVISOR. That is, the parameter module (250) may update the value of the variable DIVISOR to a value that is decreased by a factor of 2. In one embodiment, the parameter module (250) may update the value of the variable RSHMAT to a value that is increased by 1. For example, if the value of the variable RSHMAT is -1, the parameter module (250) may change the value of the variable RSHMAT to 0. The parameter module (250) may proceed to operation 850b.

[0149] At operation 850b, the parameter module (250) can determine the values ​​of variables for determining the parameters. The parameter module (250) can update the value of the variable DIVIDEND to the difference value between the value of the variable DIVIDEND and the value of the variable DIVISOR. The parameter module (250) can determine the variable MULT to 1. The parameter module (250) can determine the variable MULT_BITS to 1. In one embodiment, the parameter module (250) can determine the variables MULT and MULT_BITS before operation 850b. For example, the parameter module (250) can determine the variables MULT and MULT_BITS at operation 810b. The parameter module (250) can proceed to operation 860b.

[0150] At operation 860b, the parameter module (250) may compare the sizes of the variables DIVISOR and DIVIDEND. For example, the parameter module (250) may determine whether the variables DIVISOR and DIVIDEND are greater than each other. If the variable DIVISOR is greater than the variable DIVIDEND, the parameter module (250) may proceed to operation 870b. If the variable DIVISOR is less than or equal to the variable DIVIDEND, the parameter module (250) may proceed to operation 875b.

[0151] In operation 870b, the parameter module (250) can update the value of the variable DIVIDEND, the value of the variable MULT, the value of the variable MULT_BITS, and the value of the variable RSHMAT. In one embodiment, the parameter module (250) can perform a left shift operation on the value of the variable DIVIDEND. For example, the parameter module (250) can perform a left shift by 1 on the value of the variable DIVIDEND. That is, the parameter module (250) can update the value of the variable DIVIDEND to a value that has been increased by two times. In one embodiment, the parameter module (250) can perform a left shift operation on the value of the variable MULT. For example, the parameter module (250) can perform a left shift by 1 on the value of the variable MULT. In one embodiment, the parameter module (250) can update the value of the variable MULT_BITS to a value that has been increased by 1. For example, if the value of the variable MULT_BITS is 1, the parameter module (250) may change the value of the variable MULT_BITS to 2. In one embodiment, the parameter module (250) may update the value of the variable RSHMAT to a value that is increased by 1. For example, if the value of the variable RSHMAT is 1, the parameter module (250) may change the value of the variable RSHMAT to 2. The parameter module (250) may proceed to operation 880b.

[0152] At operation 875b, the parameter module (250) may update the value of the variable DIVIDEND to the difference between the value of the variable DIVIDEND and the value of the variable DIVISOR. The parameter module (250) may update the variable MULT to a value increased by 1. The parameter module (250) may proceed to operation 880b.

[0153] At operation 880b, the parameter module (250) can determine whether the variable BITS_MAX is greater than the variable MULT_BITS. If the variable BITS_MAX is greater than the variable MULT_BITS, the parameter module (250) can proceed to operation 860b. If the variable BITS_MAX is less than or equal to the variable MULT_BITS, the parameter module (250) can proceed to operation 890b.

[0154] In operation 890b, the parameter module (250) can pass the variable MULT and the variable RSHAMT to the post-processing module (260).

[0155] FIG. 9 is a flowchart for explaining the operation of a post-processing module of an accelerator according to one embodiment of the present disclosure.

[0156] In operation 910, a post-processing module (260) according to one embodiment can determine a value of a softmax function corresponding to a quantized value based on at least one parameter and exponential function information.

[0157] In operation 920, the post-processing module (260) according to one embodiment can store the value of the softmax function in the lookup table module (140).

[0158] The operation of the post-processing module (260) does not necessarily include operations 910 and 920, and may omit some operations or include more operations.

[0159] FIG. 10 is a block diagram illustrating the operation of an accelerator according to one embodiment of the present disclosure.

[0160] Referring to FIG. 10, an accelerator (110) according to one embodiment of the present disclosure can generate output data representing the value of a softmax function of input data based on input data.

[0161] According to one embodiment, the accelerator (110) may obtain input data using the input module (120). The input data may be obtained from the memory (270) or may be obtained externally to the accelerator (110). According to one embodiment, the input module (120) may obtain the input data sequentially using an input buffer. For example, the input data may be vector type data having a size of 1 x 1 x D and may include quantized input values. The input module (120) may transmit the input data to the distribution information processing module (130) and the lookup table module (140). According to one embodiment, the accelerator (110) may include the input module (120) configured to directly transmit the input data to the distribution information processing module (130) and the lookup table module (140).

[0162] According to one embodiment, the distribution information processing module (130) may generate input data distribution information based on input data obtained from the input module (120). The input data distribution information may indicate the frequency of input values ​​of the input data.

[0163] The distribution information processing module (130) can determine an offset value based on the input data distribution information. The offset value can mean the difference between a second maximum value indicating a maximum quantization value and a first maximum value which is the largest value included in the input data (e.g., the largest value among the input values). For example, if the maximum quantization value is 255 and the largest value among the input values ​​is 164, the distribution information processing module (130) can determine the offset value as 91. The distribution information processing module (130) can transmit the offset value to the lookup table module (140).

[0164] The distribution information processing module (130) can generate corrected input data distribution information based on the input data distribution information and the offset value. The distribution information processing module (130) can generate corrected input data distribution information by shifting the input data distribution information by the offset value. The distribution information processing module (130) can transmit the corrected input data distribution information to the calculation module (240). According to one embodiment, the distribution information processing module (130) can transmit the input data distribution information to the calculation module (240) without determining the offset value based on the result of comparing the first maximum value and the reference value.

[0165] According to one embodiment, the calculation module (240) can calculate the sum of the exponential function (or the denominator value of the softmax function) based on the modified input data distribution information (or input data distribution information) obtained from the distribution information processing module (130).

[0166] According to one embodiment, the memory (270) may store exponential function information including the value of the exponential function corresponding to the quantized value. The operation module (240) may obtain the exponential function information from the memory (270).

[0167] The calculation module (240) may calculate the exponential function sum based on the modified input data distribution information obtained from the distribution information processing module (130) and the exponential function information obtained from the memory (270). According to one embodiment, the calculation module (240) may be a MAC (Multiplicate-Accumulate) calculation module configured to perform multiplication and addition operations. The calculation module (240) may calculate the exponential function sum by calculating the sum of the product of the frequency of the input value of the input data distribution information and the exponential function value of the exponential function information. For example, if the frequency of the input value i is f i , and if the value of the exponential function is Yi, the operation module (240) calculates f for all i iWow Y i The sum of the products of can be used to calculate the exponential function sum. Since the operation module (240) is configured to perform multiplication and addition operations, it can also be utilized in artificial intelligence operations including convolution operations. The operation module (240) can transmit the exponential function sum to the parameter module (250).

[0168] According to one embodiment, the parameter module (250) may determine at least one parameter for a softmax operation based on the sum of the exponential functions obtained from the operation module (240). If the accelerator (110) outputs a value of a quantized softmax function, the parameter module (250) may determine at least one parameter for approximating the maximum quantization value and the sum of the exponential functions of the output data. For example, the parameter module (250) may determine the parameters so that a value obtained by dividing the maximum quantization value of the output data by the sum of the exponential functions is approximated using the first parameter and the second parameter. For example, a value obtained by dividing the maximum quantization value of the output data by the sum of the exponential functions may be approximated by a value obtained by right-shifting the first parameter by the second parameter. In this example, the parameter module (250) may determine the first parameter and the second parameter based on the maximum quantization value and the sum of the exponential functions of the output data. The parameter module (250) may determine the parameter using at least one of a shift operation, a multiplication operation, and an addition operation. The parameter module (250) can transmit at least one determined parameter to the post-processing module (260).

[0169] According to one embodiment, the post-processing module (260) may obtain exponential function information from the memory (270). The post-processing module (260) may generate softmax function information including a value of a softmax function based on at least one parameter obtained from the parameter module (250) and the exponential function information obtained from the memory (270). The at least one parameter may be for approximating the sum of the exponential functions, and the exponential function information may include a value of the exponential function corresponding to each quantization value, so the post-processing module (260) may determine a value of the softmax function obtained by dividing the value of the exponential function corresponding to each quantization value by the sum of the exponential functions. The post-processing module (260) may transmit the softmax function information to the lookup table module (140).

[0170] According to one embodiment, the lookup table module (140) may generate output data including a value of a softmax function corresponding to the input data based on input data obtained from the input module (120), an offset value obtained from the distribution information processing module (130), and softmax function information obtained from the post-processing module (260). The lookup table module (140) may include a lookup table that stores a plurality of values ​​and outputs a value corresponding to each of the index values ​​among the plurality of values. The lookup table module (140) may store softmax function information in the lookup table.

[0171] The lookup table module (140) can generate an index value corresponding to each input value by adding an offset value to the input value of the input data. The lookup table module (140) can sequentially identify the input values ​​of the input data and generate an index value by adding an offset value to the identified value.

[0172] The lookup table module (140) can generate output data including the value of the softmax function based on the lookup table in which softmax function information is stored and the generated index. The lookup table module (140) can obtain (or identify) the value of the softmax function corresponding to the generated index by referring to the lookup table in which softmax function information is stored. The lookup table module (140) can transmit the output data to the output module (280). The size of the output data can be the same as the size of the input data. For example, the size of the output data can be vector type data having a size of 1 x 1 x D, which is the same as the size of the input data. The output data can include the value of the quantized softmax function as the output value. The output value of the output data can be quantized in the same maximum quantization value range as the input value.

[0173] According to one embodiment, the output module (280) may transmit output data obtained from the lookup table module (140) to the outside of the accelerator (110) or store it in the memory (270).

[0174] FIG. 10 illustrates an embodiment in which the input module (120) and the output module (280) of the accelerator (110) may be simply lines for transmitting information. If the input module (120) of the accelerator (110) is a line for simply transmitting information, the input data may be the same as that directly transmitted to the distribution information processing module (130) and the lookup table module (140). If the output module (280) of the accelerator (110) is a line for simply transmitting information, the output data may be the same as that directly transmitted from the lookup table module (140) to the outside of the accelerator (110) or to a memory.

[0175] FIG. 11 is a flowchart of a processor operation method according to input data according to one embodiment of the present disclosure.

[0176] In one embodiment, the accelerator (110) may compare the largest value (the first maximum value) among the input values ​​of the input data with a predetermined reference value, and apply an offset only if the first maximum value is smaller than the reference value. If the first maximum value is larger than the reference value, it may mean that the accuracy of the softmax function can be maintained even without applying the offset, and thus the accelerator (110) may process the softmax function without applying the offset. The reference value may be determined to an appropriate value depending on the required accuracy or the degree of available resources. For example, the higher the reference value is set, the higher the likelihood that the offset will be applied, and thus the accelerator (110) may increase the accuracy even if it uses a little more resources. In operation 1110, the accelerator (110) according to one embodiment may compare the largest value among the input values ​​of the input data with the reference value. In one embodiment, the reference value may be a predetermined value. In one embodiment, if the maximum quantization value is “255” and the reference value is set to “128”, which is an intermediate value of the maximum quantization value “255”, the accelerator (110) can determine whether the maximum value among the input values ​​is less than the reference value “128”. For example, the reference value may be a value determined in advance according to the range of input data values.

[0177] The accelerator (110) proceeds to operation 1120 when the first maximum value, which is the largest value among the input values, is less than or equal to the reference value, and proceeds to operation 1140 when the first maximum value is greater than the reference value. Operation 1110 may be performed in a separate module of the accelerator (110) or in the distribution information processing module (130).

[0178] In operation 1120, the distribution information processing module (130) according to one embodiment may determine an offset value. Operation 1120 according to one embodiment may correspond to operation 340 of FIG. 3. In operation 1130, the distribution information processing module (130) according to one embodiment may generate modified input data distribution information. Operation 1130 according to one embodiment may correspond to operation 710 of FIG. 7. In operation 1140, the lookup table module (140) according to one embodiment may generate an index value using an input value and an offset value. Operation 1140 according to one embodiment may correspond to operation 350 of FIG. 3.

[0179] In operation 1150, the lookup table module (140) according to one embodiment may generate an index value using an input value. The lookup table module (140) may determine the input value as an index value. In one embodiment, the accelerator (110) may not determine an offset value if the input value is smaller than a reference value. Additionally, the accelerator (110) may not generate modified input data distribution information.

[0180] The accelerator (110) can obtain the value of the softmax function without performing the operation of applying the offset value when the maximum value among the input values ​​is greater than the reference value. The accelerator (110) can reduce the operating time by not performing the operation of applying the offset value.

[0181] FIG. 12A is a diagram illustrating a process in which a processor performs artificial intelligence operations according to one embodiment of the present disclosure.

[0182] Referring to FIG. 12a, the artificial intelligence operations according to one embodiment may include a convolution operation, a Rectified Linear Unit (ReLU) operation, a pooling operation, a Fully Connected (FC) operation, and a Softmax operation. An artificial intelligence model having a structure like that of FIG. 12a may be referred to as a Convolutional Neural Network (CNN) model. Convolutional neural networks can be utilized in image processing fields such as object recognition.

[0183] The processor (100) can perform at least one of the convolutional neural network operations using the accelerator (110). For example, the processor (100) can perform a convolution operation, a pooling operation, an FC operation, and a softmax operation using the accelerator (110). The convolution operation, the pooling operation, and the FC operation can be implemented only with addition operations, multiplication operations, and shift operations. However, in order to directly perform the softmax operation, an exponential function operation and a division operation are required. However, the processor (100) can process the softmax function using the input module (120), the distribution information processing module (130), and the lookup table module (140) of the accelerator (110). Therefore, the electronic device can process both other operations of the artificial intelligence operation and the softmax operation using the accelerator (110) of the processor (100).

[0184] FIG. 12b is a diagram for explaining a process in which a processor performs artificial intelligence operations according to one embodiment of the present disclosure.

[0185] Referring to Figure 12B, an LSTM model operation and a softmax operation according to one embodiment may be included. An artificial intelligence model with a structure like Figure 12B can be utilized in the field of natural language processing. For natural language processing, an electronic device can provide weights using a softmax function to data (e.g., an embedding corresponding to an input word as shown in Figure 12B).

[0186] As shown in FIG. 12A, a processor (100) according to one embodiment can perform a softmax operation via an accelerator (110). An electronic device can process a softmax operation required for natural language processing using the accelerator (110) of the processor (100). Similarly, according to one embodiment, the processor (100) can be used to process a softmax operation for generative artificial intelligence (Generative AI) or a large language model (LLM).

[0187] FIG. 13 is a block diagram illustrating an electronic device according to one embodiment of the present disclosure.

[0188] Referring to FIG. 13, an electronic device (1300) may be a device configured to process or process input information and perform a function corresponding to the input. For example, the electronic device (1300) may include a video display device, a mobile terminal, and a home appliance. For example, the electronic device (1300) may be at least one of a TV, a smart phone, a tablet, an air conditioner, a washing machine, and a vacuum cleaner. According to an embodiment, the electronic device (1300) may include a first processor (1310), a second processor (1320), and a memory (1330). In an embodiment, the electronic device (1300) may further include a communication module and a display. The first processor (1310) may include either the processor (100) of FIG. 1 or the processor (100) of FIG. 2, and may be a processor (100) with some components added or deleted.

[0189] In one embodiment, the second processor (1320) may control the overall operation of the electronic device (1300). The second processor (1320) may load at least one instruction stored in the memory (1330) to control the operation of the electronic device (1300). The second processor (1320) may perform various data processing or calculations to control the communication module and the display. The second processor (1320) may include at least one of a central processing unit (CPU), an application processor (AP), and an image signal processor (ISP).

[0190] In one embodiment, the first processor (1310) may be a processor for processing artificial intelligence operations or a processor configured to process artificial intelligence operations. For example, the processing speed of the artificial intelligence operations of the first processor (1310) may be faster than the processing speed of the artificial intelligence operations of the second processor (1320). The first processor (1310) may include at least one of a graphics processing unit (GPU) or a neural processing unit (NPU).

[0191] In one embodiment, the first processor (1310) can train a neural network. The first processor (1310) can perform operations to train the neural network based on input information. The first processor (1310) can perform iterative training in a manner similar to machine learning (ML).

[0192] In one embodiment, the first processor (1310) may analyze input data using a trained neural network. The first processor (1310) may then allow the second processor (1320) to utilize the results of analyzing the input data. The second processor (1320) may then implement artificial intelligence (AI) using the results of the analysis by the first processor (1310).

[0193] In one embodiment, the communication module may enable the electronic device (1300) to perform a communication connection with an external electronic device or server. The communication module may include at least one communication processor (CP) to support wireless communication. The communication module may establish a wired or wireless communication channel between the electronic device (1300) and at least one of the external electronic device or server. The communication module may transmit a signal notifying the second processor (1320) of the establishment of the wired or wireless communication channel. The communication module may receive a signal from the second processor (1320) allowing the establishment of the wired or wireless communication channel. The second processor (1320) may determine whether to allow the establishment of the wired or wireless communication channel based on the results analyzed by the first processor (1310).

[0194] In one embodiment, the communication module may support the electronic device (1300) to transmit and receive signals or data from an external electronic device or server through an established wired or wireless communication channel. For example, if the electronic device (1300) is a video display device, the communication module may establish a wired or wireless communication channel with the external electronic device or server to receive video data from the external electronic device or server. For example, if the electronic device (1300) is a mobile terminal, the communication module may establish a wireless communication channel between the electronic device (1300) and the external electronic device, and may connect calls or send and receive messages through the established wireless communication channel. For example, if the electronic device (1300) is a home appliance, the communication module may connect to the external electronic device through short-range wireless communication such as Bluetooth Low Energy (BLE) communication, and may connect to the server through long-range wireless communication such as Wi-Fi communication. The communication module can receive information for artificial intelligence operations from at least one of an external electronic device or a server, or transmit results for artificial intelligence operations.

[0195] In one embodiment, the memory (1330) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., Secure Digital (SD) memory, eXtreme Digital (XD) memory), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read-Only Memory (ROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Programmable Read-Only Memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. The memory (1330) may receive a command or data from the second processor (1320). The memory (1330) may store a command or data received from the second processor (1320). The command or data stored in the memory (1330) may include a program for the second processor (1320) to process the input data and control the operation of the electronic device (1300). The second processor (1320) may load the command or data stored in the memory (1330). The second processor (1320) may analyze or process the command or data loaded from the memory (1330) using the first processor (1310).

[0196] In one embodiment, the display can provide visual information to the outside of the electronic device (1300). For example, the display can display a screen regarding information regarding the status of the electronic device (1300). For example, the display can display a screen regarding an operation or function performed by the electronic device (1300). The second processor (1320) can transmit image data related to the result of the artificial intelligence operation of the first processor (1310) to the display. For example, the display can display image data including object information in an image generated by the first processor (1310). The display can include a display panel for displaying a screen and a display driver integrated circuit (DDI) for driving the display panel. For example, the display can be at least one of an organic light emitting display (OLED), a quantum dot display (QD display), and a micro light emitting diode (Micro LED) display.

[0197] FIG. 14 is an exemplary drawing for explaining an electronic device including a processor according to one embodiment of the present invention.

[0198] Referring to FIG. 14, an electronic device including a processor according to an embodiment of the present disclosure may include, but is not limited to, a plurality of electronic devices (1400) including a TV (1410), a head mounted display (HMD) (1420), a mobile device (1430) (e.g., a smartphone), a wearable device (1440), a home appliance (1450), and a server (1460). The plurality of electronic devices (1400) may include a processor including an accelerator (110) for processing a softmax function.

[0199] A plurality of electronic devices (1400) including a processor according to one embodiment of the present disclosure can perform on-device image processing. For example, the plurality of electronic devices (1400) can perform image processing to recognize objects included in an image using artificial intelligence through the accelerator (110). In one embodiment, the processor (100) can obtain information about an image, perform artificial intelligence image processing through the accelerator (110), and provide the resulting information to another processor, display, or memory.

[0200] A plurality of electronic devices (1400) including a processor according to one embodiment of the present disclosure can perform on-device voice recognition. For example, the plurality of electronic devices (1400) can perform an operation of identifying a user voice and a user command using artificial intelligence through the accelerator (110). In one embodiment, the processor (100) can obtain voice information, perform artificial intelligence voice recognition through the accelerator (110), and provide the resulting information to another processor, display, speaker, or memory.

[0201] Each of a plurality of electronic devices (1400) including a processor (100) according to one embodiment of the present disclosure can perform on-device natural language processing (e.g., translation). For example, the plurality of electronic devices (1400) can perform an operation of identifying the meaning and context of text or generating text using artificial intelligence through the accelerator (110). In one embodiment, the processor (100) can obtain text information, perform artificial intelligence natural language processing through the accelerator (110), and provide the resulting information to another processor, speaker, display, or memory.

[0202] According to one embodiment, the server (1460) may use the processor (100) to process artificial intelligence operations on behalf of another electronic device, or perform procedures requiring large-scale operations, such as large-scale language model processing.

[0203] According to an embodiment of the present disclosure, a softmax operation can be processed quickly and accurately through an accelerator (110). Accordingly, the inference speed of the entire neural network can be increased. (For example, latency can be reduced.) In addition, an accelerator (110) for processing a softmax function can be produced by adding a simple configuration of a processor (100) including a MAC (Multiplication-Accumulation) accelerator (110). Accordingly, a separate processor related to the softmax function is not required, and the softmax function and MAC operation can be processed using only the processor (100). In this case, bandwidth and power consumption can be reduced due to a reduction in data movement between the accelerator (110) and the separate processor.

[0204] In one embodiment of the present disclosure, a method is provided, which is performed by at least one processor configured to implement an accelerator for processing a softmax function. The method may include obtaining input data including a plurality of quantized input values ​​using an input module of the accelerator. The method may include generating input data distribution information indicating frequencies of the plurality of quantized input values ​​included in the input data using a distribution information processing module of the accelerator. The method may include identifying a largest value among the plurality of quantized input values ​​as a first maximum value based on the input data distribution information using the distribution information processing module. The method may include determining a difference between a second maximum value and the first maximum value, which indicates a maximum quantized value that can be expressed as an input value of the input data, as an offset value using the distribution information processing module. The method may include determining a plurality of index values ​​by applying an offset value to each of the plurality of quantized input values ​​using a look-up table (LUT) module of the accelerator. The method may include outputting a value of a softmax function corresponding to each of the plurality of quantized input values ​​based on the plurality of index values ​​using the look-up table module.

[0205] In one embodiment of the present disclosure, the method may include a step of generating corrected input data distribution information by shifting input data distribution information by an offset value using a distribution information processing module. The method may further include a step of calculating a denominator value of a softmax function using the corrected input data distribution information and exponential function information using a calculation module of an accelerator, wherein the exponential function information includes an exponential function value corresponding to a quantized value. The plurality of values ​​stored in the lookup table module may be determined based on the denominator value of the softmax function.

[0206] In one embodiment of the present disclosure, the denominator value of the softmax function can be calculated by performing multiplication and addition operations between the modified input data distribution information and the exponential function information by the calculation module.

[0207] In one embodiment of the present disclosure, the method may further include a step of determining at least one parameter for representing an approximation corresponding to a denominator value of a softmax function using a parameter module of the accelerator.

[0208] In one embodiment of the present disclosure, at least one parameter may include a first parameter and a second parameter. The value obtained by dividing the second maximum value by the denominator value of the softmax function may be determined based on a value obtained by performing a right shift operation on the first parameter by the amount of the second parameter.

[0209] In one embodiment of the present disclosure, the method may further include a step of determining a value of a softmax function corresponding to a quantized value based on at least one parameter and exponential function information using a post-processing module of the accelerator. In one embodiment of the present disclosure, a plurality of values ​​stored in the lookup table module may include values ​​of the softmax function.

[0210] In one embodiment of the present disclosure, the method may determine a plurality of index values ​​by applying an offset value to each of the plurality of quantized input values ​​based on whether the first maximum value is less than or equal to a reference value, using a lookup table module. The method may further include a step of determining a plurality of index values ​​to be equal to the plurality of quantized input values ​​based on whether the first maximum value is greater than the reference value, using the lookup table module.

[0211] In one embodiment of the present disclosure, the method may include a step of sequentially storing the values ​​of a softmax function output from a lookup table module into the memory of the accelerator using an output buffer of an output module included in the accelerator. The method may include a step of sequentially obtaining input data from the memory using an input buffer of an input module.

[0212] In one embodiment of the present disclosure, the method may further include a step of causing the accelerator to output output data including a value of a softmax function based on input data, using a control module of the processor.

[0213] In one embodiment of the present disclosure, the input data and the output data may include at least one of vector type data, matrix type data, or tensor type data including a plurality of quantized values. The size of the input data may be the same as the size of the output data. Each of the plurality of quantized input values ​​may represent at least one value from 0 to 2 to the Nth power (where N is an integer) - 1. The value of the output data may represent at least one value from 0 to 2 to the Nth power.

[0214] In one embodiment of the present disclosure, a processor for processing a softmax function is provided. The processor may be configured to implement an accelerator including an input module, a distribution information processing module, and a lookup table module. The input module may be configured to obtain input data including a plurality of quantized input values. The distribution information processing module may generate input data distribution information indicating frequencies of the plurality of quantized input values ​​included in the input data. The distribution information processing module may identify a largest value among the plurality of quantized input values ​​as a first maximum value based on the input data distribution information. The distribution information processing module may be configured to determine a difference between a second maximum value indicating a maximum quantized value that can be expressed as an input value of the input data and the first maximum value as an offset value. The lookup table module may determine a plurality of index values ​​by applying the offset value to each of the plurality of quantized input values. The lookup table module may be configured to output values ​​corresponding to the plurality of index values ​​as values ​​of a softmax function corresponding to each of the plurality of quantized input values.

[0215] In one embodiment of the present disclosure, the distribution information processing module can generate corrected input data distribution information by shifting the input data distribution information by an offset value. The accelerator may further include a calculation module. The calculation module calculates a denominator value of a softmax function using the corrected input data distribution information and exponential function information, and the exponential function information may include an exponential function value corresponding to a quantized value. A plurality of values ​​stored in the lookup table module may be determined based on the denominator value of the softmax function.

[0216] In one embodiment of the present disclosure, the calculation module can process multiplication and addition operations between matrices or perform product and sum operations between vectors. The denominator value of the softmax function can be calculated by performing multiplication and addition operations between the modified input data distribution information and the exponential function information by the calculation module.

[0217] In one embodiment of the present disclosure, the accelerator may further include a parameter module. The parameter module may determine at least one parameter representing an approximate value corresponding to the denominator value of the softmax function.

[0218] In one embodiment of the present disclosure, at least one parameter may include a first parameter and a second parameter. The value obtained by dividing the second maximum value by the denominator value of the softmax function may be determined based on a value obtained by performing a right shift operation on the first parameter by the amount of the second parameter.

[0219] In one embodiment of the present disclosure, the accelerator may further include a post-processing module. The post-processing module may determine the value of a softmax function corresponding to a quantized value based on at least one parameter and exponential function information. The plurality of values ​​stored in the lookup table module may include the value of the softmax function.

[0220] In one embodiment of the present disclosure, the lookup table module may determine a plurality of index values ​​by applying an offset value to each of the plurality of quantized input values ​​based on whether the first maximum value is less than or equal to a reference value. The lookup table module may determine a plurality of index values ​​to be equal to each of the plurality of quantized input values ​​based on whether the first maximum value is greater than the reference value.

[0221] In one embodiment of the present disclosure, the accelerator may further include a memory and an output module. The output module may include an output buffer. The output buffer may sequentially store the values ​​of the softmax function output from the lookup table module in memory. The input module may include an input buffer. The input buffer may sequentially obtain input data from the memory.

[0222] In one embodiment of the present disclosure, the device further includes a control module, and the control module is operable to cause the accelerator to output output data including a value of a softmax function based on input data.

[0223] In one embodiment of the present disclosure, the input data and the output data may include at least one of vector type data, matrix type data, or tensor type data including a plurality of quantized values. The size of the input data may be the same as the size of the output data. Each of the plurality of quantized input values ​​may represent at least one value from 0 to 2 to the Nth power (where N is an integer) - 1. The value of the output data may represent at least one value from 0 to 2 to the Nth power.

[0224] In one embodiment of the present disclosure, an electronic device may be provided. The electronic device may include a first processor, a second processor, and a memory for processing a softmax function. The second processor may control the first processor to process an operation related to artificial intelligence including the softmax function by executing instructions contained in the memory. The first processor may be configured to implement an accelerator. The accelerator may include an input module, a distribution information processing module, and a lookup table module. The input module may be configured to obtain input data including a plurality of quantized input values. The distribution information processing module may generate input data distribution information indicating frequencies of the plurality of quantized input values ​​included in the input data. Based on the input data distribution information, a largest value among the plurality of quantized input values ​​may be identified as a first maximum value. The electronic device may be configured to determine a difference between the first maximum value and a second maximum value indicating a maximum quantized value that can be expressed as an input value of the input data as an offset value. The lookup table module may determine a plurality of index values ​​by applying the offset value to each of the plurality of quantized input values. The values ​​corresponding to multiple index values ​​can be output as the values ​​of the softmax function corresponding to each of multiple quantized input values.

[0225] According to one embodiment of the present disclosure, a recording medium storing at least one instruction for controlling a processor including an accelerator for processing a softmax function may be provided. The at least one instruction may control the processor to obtain input data including a plurality of quantized input values ​​using an input module of the accelerator. The at least one instruction may control the processor to generate input data distribution information indicating frequencies of the plurality of quantized input values ​​included in the input data using a distribution information processing module of the accelerator. The at least one instruction may control the processor to identify a largest value among the plurality of quantized input values ​​as a first maximum value based on the input data distribution information using the distribution information processing module. The at least one instruction may control the processor to determine a difference between a second maximum value indicating a maximum quantized value that can be expressed as an input value of the input data and the first maximum value as an offset value using the distribution information processing module. The at least one instruction may control the processor to determine a plurality of index values ​​by applying an offset value to each of the plurality of quantized input values ​​using a look-up table (LUT) module of the accelerator. At least one instruction can control the processor to output a value of a softmax function corresponding to each of a plurality of quantized input values ​​based on a plurality of index values, using a lookup table module.

[0226] The processor, the method of operating the processor, and the electronic device including the processor according to the present disclosure can reduce the size and complexity of the processor or shorten the time required for calculation by using an accelerator that processes a nonlinear activation function.

[0227] According to one embodiment of the present disclosure, the computer-executable instructions, such as program modules executed by a computer, may also be implemented in the form of a recording medium. Computer-readable media may be any available media that can be accessed by a computer, and include both volatile and nonvolatile media, removable and non-removable media. Computer-readable media may include computer storage media and communication media. Computer storage media include both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Communication media may typically include computer-readable instructions, data structures, or other data in a modulated data signal, such as program modules.

[0228] A computer-readable storage medium according to one embodiment of the present disclosure may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is stored semi-permanently in the storage medium and cases where data is stored temporarily. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0229] A method according to one embodiment of the present disclosure may be provided as a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0230] The above description of the present disclosure is provided for illustrative purposes only, and those skilled in the art will readily appreciate that modifications to other specific forms can be made without altering the technical spirit or essential features of the present disclosure. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, components described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined manner.

[0231] The scope of the present disclosure is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present disclosure.

Claims

1. A method performed by at least one processor configured to implement an accelerator for processing a softmax function, A step (310) of obtaining input data including a plurality of quantized input values ​​using an input module of the above accelerator; A step (320) of generating input data distribution information representing the frequency of the plurality of quantized input values ​​included in the input data by using the distribution information processing module of the accelerator; A step (330) of identifying the largest value among the plurality of quantized input values ​​as the first maximum value based on the input data distribution information using the distribution information processing module; A step (340) of determining the difference between the second maximum value, which represents the maximum quantization value that can be expressed as an input value of the input data, and the first maximum value as an offset value using the above distribution information processing module; A step (350) of determining a plurality of index values ​​by applying the offset value to each of the plurality of quantized input values ​​using the look-up table module (LUT) of the accelerator; and A method comprising a step (360) of outputting a value of a softmax function corresponding to each of the plurality of quantized input values ​​based on the plurality of index values ​​using the lookup table module.

2. In paragraph 1, A step of generating modified input data distribution information by shifting the input data distribution information by the offset value using the above distribution information processing module; Using the operation module of the accelerator, the method further includes calculating the denominator value of the softmax function using the modified input data distribution information and the exponential function information, and the exponential function information includes an exponential function value corresponding to a quantized value. A method wherein the plurality of values ​​stored in the above lookup table module are determined based on the denominator value of the softmax function.

3. In paragraph 2, A method in which the denominator value of the above softmax function is calculated by performing multiplication and addition operations between the modified input data distribution information and the exponential function information by the above operation module.

4. In any one of paragraphs 2 to 3, A method further comprising the step of determining at least one parameter for representing an approximate value corresponding to a denominator value of the softmax function using a parameter module of the accelerator.

5. In paragraph 4, wherein said at least one parameter comprises a first parameter and a second parameter, A method in which the value obtained by dividing the second maximum value by the denominator value of the softmax function is determined based on the value obtained by performing a right shift operation on the first parameter by the amount of the second parameter.

6. In any one of paragraphs 4 to 5, Further comprising a step of determining a value of a softmax function corresponding to a quantized value based on the at least one parameter and the exponential function information using a post-processing module of the accelerator, A processor wherein the plurality of values ​​stored in the lookup table module include the value of the softmax function.

7. In any one of paragraphs 2 to 6, Using the lookup table module, a step of determining the plurality of index values ​​by applying the offset value to each of the plurality of quantized input values ​​based on whether the first maximum value is less than or equal to a reference value; and A method further comprising the step of determining the plurality of index values ​​to be equal to the plurality of quantized input values ​​based on the first maximum value being greater than the reference value, using the lookup table module.

8. In any one of paragraphs 1 to 7, A step of sequentially storing the values ​​of the softmax function output from the lookup table module in the memory of the accelerator using the output buffer of the output module included in the accelerator; A method comprising a step of sequentially obtaining input data from the memory using the input buffer of the input module.

9. In any one of paragraphs 1 to 8, A method further comprising the step of causing the accelerator to output output data including a value of the softmax function based on the input data, using a control module of the processor.

10. In any one of paragraphs 1 to 9, The above input data and output data include at least one of vector type data, matrix type data, or tensor type data including a plurality of quantized values, The size of the above input data is the same as the size of the above output data, Each of the above multiple quantized input values ​​represents at least one value between 0 and 2 to the Nth power (N is an integer)-1, A method in which the value of the above output data represents at least one value among values ​​between 0 and 2 to the power of N.

11. In a processor (100) for processing a softmax function, An accelerator (110) including an input module (120); a distribution information processing module (130); and a lookup table module (140), The above input module (120) is configured to obtain input data including a plurality of quantized input values, The above distribution information processing module (130) generates input data distribution information indicating the frequency of the plurality of quantized input values ​​included in the input data, Based on the above input data distribution information, the largest value among the plurality of quantized input values ​​is identified as the first maximum value, It is configured to determine the difference between the first maximum value and the second maximum value representing the maximum quantization value that can be expressed as the input value of the input data as an offset value, The above lookup table module (140) determines a plurality of index values ​​by applying the offset value to each of the plurality of quantized input values, A processor configured to output a value corresponding to the plurality of index values ​​as a value of a softmax function corresponding to each of the plurality of quantized input values.

12. In paragraph 11, The above distribution information processing module (130) is configured to generate modified input data distribution information by shifting the input data distribution information by the offset value. The above accelerator (110) further includes a computational module, The above operation module calculates the denominator value of the softmax function by using the modified input data distribution information and the exponential function information, and the exponential function information includes an exponential function value corresponding to a quantized value. A processor wherein the plurality of values ​​stored in the above lookup table module are determined based on the denominator value of the softmax function.

13. In paragraph 12, The above operation module It is configured to process multiplication and addition operations between matrices or to perform product and sum operations between vectors, A processor in which the denominator value of the above softmax function is calculated by performing multiplication and addition operations between the modified input data distribution information and the exponential function information by the above operation module.

14. In any one of paragraphs 12 to 13, The above accelerator (110) further includes a parameter module, The above parameter module is, A processor configured to determine at least one parameter representing an approximate value corresponding to a denominator value of the softmax function.

15. In an electronic device (1300), A first processor (1310) for processing a softmax function; a second processor (1320); and Contains memory (1330), The second processor (1320) controls the first processor (1310) to process an operation related to artificial intelligence including a softmax function by executing instructions included in the memory (1330). The above first processor (1310) is configured to implement an accelerator (110), The above accelerator (110) includes an input module (120); a distribution information processing module (130); and a look-up table (LUT) module (140). The above input module is configured to obtain input data including a plurality of quantized input values, The above distribution information processing module (130) generates input data distribution information indicating the frequency of the plurality of quantized input values ​​included in the input data, Based on the above input data distribution information, the largest value among the plurality of quantized input values ​​is identified as the first maximum value, It is configured to determine the difference between the first maximum value and the second maximum value representing the maximum quantization value that can be expressed as the input value of the input data as an offset value, The above lookup table module (140) determines a plurality of index values ​​by applying the offset value to each of the plurality of quantized input values, An electronic device configured to output a value corresponding to the plurality of index values ​​as a value of a softmax function corresponding to each of the plurality of quantized input values.

Images