Processor for processing softmax function and operation method of processor

Abstract

A processor comprising an accelerator for processing a softmax function and an operation method thereof may be provided. Provided may be a method comprising the operations of: obtaining, by an exponential function processing module, input data including a plurality of input values; obtaining, by the exponential function processing module, a value of an exponential function corresponding to the input values included in the input data by using a lookup table module; determining, by the exponential function processing module, a scale factor corresponding to the input values; and determining, by a softmax function processing module, a value of a softmax function corresponding to the input values on the basis of the value of the exponential function and the scale factor.

Description

Processor and Processor Operation Method for Processing Softmax Functions

[0001] The present disclosure relates to a processor and a method of operating the processor. More specifically, the present disclosure relates to a processor for processing artificial intelligence operations and a method of operating the processor.

[0002] Electronic devices such as TVs, portable terminals, and home appliances may include processors for processing or implementing artificial intelligence (AI). A processor for implementing AI may refer to a central processing unit (CPU) or a separate processor for accelerating the processing of AI operations, including neural networks (NN). A processor for processing AI operations may include a graphics processing unit (GPU) or a neural processing unit (NPU).

[0003] The AI ​​operations processed by the processor may include deep learning neural network operations. Deep learning neural networks may include convolution operations, matrix-to-matrix (or matrix-to-vector) multiplication operations, and operations involving nonlinear activation functions. Unlike servers with sufficient computational power, electronic devices may not have sufficient computational power for AI processing. Depending on the processing efficiency of each AI operation involved, the performance of on-device AI processes (e.g., inference using AI technology) on electronic devices may vary significantly.

[0004] Non-linear activation functions can include the sigmoid function, the hyperbolic tangent (Tanh) function, the Rectified Linear Unit (ReLU) function, the Exponential Linear Unit (ELU) function, the Gaussian Error Linear Unit (GELU) function, and the Softmax function.

[0005] The softmax function, a nonlinear activation function, involves exponential and division operations, making its processing complexity high. Processors used for artificial neural networks (or artificial intelligence operations) can utilize a separate processor to process the softmax function. However, using a separate processor to process the softmax function consumes time and resources due to data transfer. Therefore, a method may be required for a processor used for artificial neural networks (or artificial intelligence operations) to process the softmax function.

[0006] In one embodiment of the present disclosure, a method is provided, which is performed by a processor configured to implement an accelerator for processing a softmax function. The method may include an operation in which an exponential function processing module obtains input data including a plurality of input values. The method may include an operation in which the exponential function processing module obtains a value of an exponential function corresponding to an input value included in the input data using a lookup table module. The method may include an operation in which the exponential function processing module determines a scale factor corresponding to the input value. The method may include an operation in which the softmax function processing module determines a value of the softmax function corresponding to the input value based on the value of the exponential function and the scale factor. The lookup table module may store values ​​relating to a plurality of exponential functions. The values ​​relating to the plurality of exponential functions may be classified into a plurality of groups according to corresponding scale factors.

[0007] In one embodiment of the present disclosure, a processor for processing a softmax function is provided. The processor may include an accelerator. The accelerator may include an exponential function processing module and a softmax function processing module. The exponential function processing module may obtain input data including a plurality of input values. The exponential function processing module may obtain a value of an exponential function corresponding to an input value included in the input data using a lookup table module. The exponential function processing module may determine a scale factor corresponding to the input value. The softmax function processing module may determine a value of the softmax function corresponding to the input value based on the value of the exponential function and the scale factor. The lookup table module may store values ​​for a plurality of exponential functions. The values ​​for the plurality of exponential functions may be classified into a plurality of groups according to corresponding scale factors.

[0008] In one embodiment of the present disclosure, an electronic device is provided. The electronic device may include a first processor, a second processor, and a memory for processing a softmax function. The second processor may control the first processor to process an artificial intelligence operation including a softmax function by executing instructions contained in the memory. The first processor may include an accelerator. The accelerator may implement an exponential function processing module and a softmax function processing module. The exponential function processing module may obtain input data including a plurality of input values. The exponential function processing module may obtain a value of an exponential function corresponding to an input value included in the input data using a lookup table module. The exponential function processing module may determine a scale factor corresponding to the input value. The softmax function processing module may determine a value of the softmax function corresponding to the input value based on the value of the exponential function and the scale factor. The lookup table module may store values ​​for a plurality of exponential functions. The values ​​for the plurality of exponential functions may be classified into a plurality of groups according to corresponding scale factors.

[0009] FIG. 1 is a block diagram illustrating a processor for processing an artificial neural network (or artificial intelligence operation) according to one embodiment of the present disclosure.

[0010] FIG. 2 is a block diagram illustrating a processor for processing an artificial neural network (or artificial intelligence operation) according to one embodiment of the present disclosure.

[0011] FIG. 3 is a flowchart illustrating an operation method of a processor including an accelerator for processing a softmax function according to one embodiment of the present disclosure.

[0012] FIG. 4a is a diagram for explaining quantization of input values ​​according to one embodiment of the present disclosure.

[0013] FIG. 4b is a diagram for explaining quantization of exponential function values ​​according to one embodiment of the present disclosure.

[0014] FIG. 4c is a diagram for explaining quantization of input values ​​and exponential function values ​​according to one embodiment of the present disclosure.

[0015] FIG. 5A is a diagram illustrating groups and scale factors of data stored in a lookup table module according to one embodiment of the present disclosure.

[0016] FIG. 5b is a diagram illustrating groups and scale factors of data stored in a lookup table module according to one embodiment of the present disclosure.

[0017] FIG. 6 is a block diagram illustrating the operation of an exponential function processing module according to one embodiment of the present disclosure.

[0018] FIG. 7A is a flowchart illustrating a method by which a processor including an accelerator according to one embodiment of the present disclosure determines a value of a softmax function.

[0019] FIG. 7b is a flowchart illustrating a method by which a processor including an accelerator according to one embodiment of the present disclosure determines a sum of exponential functions.

[0020] FIG. 8A is a flowchart illustrating a method by which a processor including an accelerator according to one embodiment of the present disclosure determines a value of a softmax function using parameters.

[0021] FIG. 8b is a flowchart illustrating the operation of a parameter module of an accelerator according to one embodiment of the present disclosure.

[0022] FIG. 8c is a flowchart illustrating a method by which a processor including an accelerator according to one embodiment of the present disclosure determines a value of a softmax function.

[0023] FIG. 9 is a flowchart illustrating an operation of a processor including an accelerator according to one embodiment of the present disclosure to initialize settings.

[0024] FIG. 10 is a block diagram illustrating the operation of a processor including an accelerator according to one embodiment of the present disclosure.

[0025] FIG. 11 is a flowchart of a processor operation method according to input data according to one embodiment of the present disclosure.

[0026] FIG. 12A is a diagram illustrating a process in which a processor performs artificial intelligence operations according to one embodiment of the present disclosure.

[0027] FIG. 12b is a diagram for explaining a process in which a processor performs artificial intelligence operations according to one embodiment of the present disclosure.

[0028] FIG. 13 is a block diagram illustrating an electronic device according to one embodiment of the present disclosure.

[0029] FIG. 14 is an exemplary drawing for explaining an electronic device including a processor according to one embodiment of the present invention.

[0030] In this disclosure, the expression “at least one of a, b or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “all of a, b and c”, or variations thereof.

[0031] The terms used in this disclosure are selected from widely used, common terms, taking into account the functions of the disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings can be understood through the relevant description. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of the disclosure.

[0032] In this disclosure, singular expressions may include plural expressions unless the context clearly dictates otherwise. Terms containing ordinal numbers, such as "first" or "second," used in this disclosure may be used to describe various components, but the components should not be limited by these terms. These terms are used solely to distinguish one component from another.

[0033] When a part in this disclosure is said to "include" a component, this may mean that other components may be included, rather than excluding other components, unless specifically stated otherwise. In this disclosure, terms such as "part" and "module" may refer to a unit that processes or is configured to process at least one function or operation.

[0034] The expression "configured to" as used herein can be used interchangeably with, for example, "suitable for", "having the capacity to", "designed to", "adapted to", "made to", or "capable of", depending on the context. The term "configured to" does not necessarily mean something that is "specifically designed to" in terms of hardware. In some contexts, the expression "a system configured to" can mean that the system is "capable of" together with other devices or components. The expression "a module does" as used herein can be used interchangeably with "a module is configured to", depending on the context.

[0035] When a component is referred to as being "connected" or "connected" to another component in this disclosure, it should be understood that the component may be directly connected or connected to the other component, but unless there is a specific description to the contrary, it should also be understood that the component may be connected or connected via another component in between.

[0036] In this disclosure, "transmitting" or "forwarding" data by a module may mean providing data to another module. A module may be connected to another module via wires or wirelessly. A module may provide data directly to another module, or may provide data through another module.

[0037] In describing the present disclosure, descriptions of technical contents that are well known in the technical field to which the present disclosure pertains and are not directly related to the present disclosure may be omitted. This is to convey the gist of the present disclosure more clearly without obscuring the gist of the present disclosure by omitting some descriptions. In the drawings, parts that are irrelevant to the description may be omitted for clear description of the present disclosure, and similar parts are designated with similar reference numerals. In addition, the reference numerals used in each drawing are only for describing each drawing, and different reference numerals used in different drawings do not indicate different elements. The size of each component does not entirely reflect the actual size. The same or corresponding components in each drawing are given the same reference numerals.

[0038] The advantages and features of the present disclosure, and methods for achieving them, will become clearer with reference to the embodiments described below in detail with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. The disclosed embodiments are provided to ensure that the disclosure is complete and to fully inform those skilled in the art of the disclosure of the scope of the disclosure. An embodiment of the present disclosure may be defined according to the claims.

[0039] In the present disclosure, each block of the flowchart drawings and combinations of the flowchart drawings can be performed by computer program instructions. The computer program instructions can be installed on a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, and the instructions executed by the processor of the computer or other programmable data processing equipment can create means for performing the functions described in the flowchart block(s). The computer program instructions can also be stored in a computer-available or computer-readable memory that can direct a computer or other programmable data processing equipment to implement a function in a specific manner, and the instructions stored in the computer-available or computer-readable memory can also produce an article of manufacture that includes instruction means for performing the functions described in the flowchart block(s). The computer program instructions can also be installed on a computer or other programmable data processing equipment.

[0040] In the present disclosure, each block in the flowchart diagram may represent a module, segment, or portion of code that includes one or more executable instructions for performing a specified logical function(s). In one embodiment, the functions described in the blocks may occur out of order. For example, two blocks depicted in succession may be executed substantially simultaneously or, depending on the function, may be executed in reverse order.

[0041] As is customary in the art, the embodiments are described and illustrated in the drawings in terms of functional blocks, units, and / or modules. Those skilled in the art will appreciate that these blocks, units, and / or modules are physically implemented by electronic (or optical) circuits such as logic circuits, discrete components, microprocessors, hard-wired circuits, memory elements, wiring connections, etc., which may be formed using semiconductor-based manufacturing techniques or other manufacturing techniques. When the blocks, units, and / or modules are implemented by a microprocessor or similar device, they may be programmed using software (e.g., microcode) to perform the various functions described herein and may optionally be driven by firmware and / or software. Alternatively, each block, unit, and / or module may be implemented by dedicated hardware, or by a combination of dedicated hardware performing some functions and processors (e.g., one or more programmed microprocessors and associated circuitry) performing other functions. Furthermore, each block, unit, and / or module of the embodiments may be physically separated into two or more interacting individual blocks, units, and / or modules without departing from the scope of the present invention. Additionally, the blocks, units and / or modules of the embodiments may be physically combined into more complex blocks, units and / or modules without departing from the scope of the present invention.

[0042] In one embodiment of the present disclosure, a component ending with “~bu(“~er”, “or”)” (which may be referred to as a “~bu(“~er / or”) component) may represent a hardware component such as a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC), and the “~bu” component may perform a specific role. Meanwhile, the “~bu” component may not be limited to hardware. The “~bu” component may be configured to be in an addressable storage medium and may be configured to play one or more processors. In one embodiment, the “~bu” component may include components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functionality provided by a particular component or a particular '~sub' configuration may be combined to reduce the number of components or separated into additional components. Furthermore, in one embodiment, a '~sub' configuration may include one or more processors.

[0043] Below, with reference to the attached drawings, embodiments of the present disclosure are described in detail to facilitate their implementation by those skilled in the art. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. The present disclosure will be described in detail below with reference to the attached drawings.

[0044] In the present disclosure, the input module may include a module configured to acquire data for the accelerator. The input module in the present disclosure may be replaced with terms such as an input queue or a data acquisition module. In one embodiment, the input module may include or be replaced with an input buffer.

[0045] In the present disclosure, quantization may include converting data that can be expressed continuously into discontinuous or discrete data. Quantization may mean converting a value within a predetermined range into a reference value. For example, quantization may mean representing a value greater than or equal to x0 and less than x1 as 0, and representing a value greater than or equal to x1 and less than x2 as 1. Quantization in the present disclosure may be replaced with terms such as “sampling,” “discretization,” or “integerization.” Examples of quantized data according to one embodiment of the present disclosure may be understood with reference to FIGS. 4A, 4B, and 4C.

[0046] In the disclosure, the maximum quantization value may mean the largest value that the quantized data can express. For example, if the data is quantized to an integer value greater than or equal to 0 and less than or equal to 255, the maximum quantization value may mean 255. According to one embodiment, the maximum quantization value of the output data of the accelerator may be the same as the maximum quantization value of the input data of the accelerator, but is not limited thereto. The maximum quantization value of the present disclosure may be replaced with terms such as "quantization upper limit", "maximum representable value", or "quantization range". The maximum quantization value according to one embodiment of the present disclosure may be understood with reference to FIGS. 4A, 4B, and 4C.

[0047] In the present disclosure, the lookup table module may include a module configured to output the value of an exponential function. The lookup table module may output the value of the exponential function corresponding to an input value. The lookup table module of the present disclosure may be replaced with terms such as a reference table module, a data dictionary module, or a data mapping module.

[0048] In the present disclosure, an index value may refer to a value used to determine one of a plurality of values ​​stored in a lookup table module. In one embodiment, the lookup table module may store information including a plurality of index values ​​and a value corresponding to each index value. The lookup table module may obtain an index value and provide a stored value corresponding to the index value. The index value in the present disclosure may be replaced with terms such as "index," "key," "key value," or "lookup key."

[0049] In the present disclosure, the exponential function information may include information regarding a quantized exponential function value. The exponential function information may include an exponential function value for a value that quantized input data can express. For example, if the input data is quantized to one of integer values ​​greater than or equal to 0 and less than or equal to 255, the exponential function information may include an integer value greater than or equal to 0 and less than or equal to 255 and an exponential function value corresponding to each integer value. In one embodiment, the exponential function value may be a quantized value. According to one embodiment, the exponential function information may include data expressed in a table format, wherein a plurality of values ​​that the quantized input data can express and the exponential function value corresponding to each value are represented. The exponential function information in the present disclosure may be replaced with terms such as "exponential function table", "exponential function LUT data", "exponential function data", or "exponential value data".

[0050] In the present disclosure, a shift operation may include an arithmetic shift operation. The shift operation may include a left shift operation or a right shift operation. A left shift operation may be expressed as '<<', and a right shift operation may be expressed as '>>'. A left shift operation may be understood to mean shifting the binary bits of a value to be operated on to the left. Shifting to the left by N bits may be understood to mean multiplying the value to be operated on by 2N (e.g., 2 to the Nth power), where N is an integer. For example, 3<<2 may mean 12. In the present disclosure, performing a left shift operation by N bits may mean multiplying the value to be operated on by 2 to the Nth power. A right shift operation may be understood to mean shifting the binary bits of a value to be operated on to the right. Shifting to the right by N bits may be understood to mean dividing the value to be operated on by 2 to the Nth power. In the present disclosure, performing a right shift operation by N may mean dividing the target value by 2 to the power of N. For example, 12 >> 2 may mean 3.

[0051] In the present disclosure, a scale factor may include a scaling variable for a value of an exponential function stored in a lookup table module (e.g., lookup table module 230 of FIG. 2). In one embodiment, the value of the exponential function stored in the lookup table module may be a value scaled using the scale factor. In one embodiment, the value of the exponential function stored in the lookup table module may be a value scaled by a left shift operation by the scale factor. The scale factor in the present disclosure may be replaced with terms such as "log scale multiple", "scale parameter", or "ratio". The scale factor in the present disclosure may mean a scale factor corresponding to an input value rather than a base scale factor.

[0052] In the present disclosure, interpolation may include an operation of obtaining a result value based on a plurality of reference values. In one embodiment, interpolation may include an operation of determining a value of an exponential function corresponding to an input value based on a first reference value and a second reference value. In one embodiment, interpolation may include an operation of performing a weighted sum using weights on the plurality of reference values. In the present disclosure, interpolation may be replaced with terms such as "prediction," "weighted sum," or "generate."

[0053] FIG. 1 is a block diagram illustrating a processor for processing an artificial neural network (or artificial intelligence operation) according to one embodiment of the present disclosure.

[0054] Referring to FIG. 1, the processor (100) may include an accelerator (110). The accelerator (110) may include an exponential function processing module (120) and a softmax function processing module (130). The exponential function processing module (120) and the softmax function processing module (130) may be implemented in the form of hardware (HW) in the accelerator (110).

[0055] In one embodiment, the processor (100) can process an artificial intelligence operation, such as a softmax function. The processor (100) can process the softmax function to produce at least one result value related to the operation of an electronic device including the processor (100). In one embodiment, the processor (100) can determine the value of the softmax function for input data. The softmax function can be expressed as in Mathematical Formula 1.

[0056]

[0057] The value of the softmax function for the ith input data is determined based on the sum of the exponents of all input data and the value of the exponent of the ith input data.

[0058] In one embodiment, the exponential function processing module (120) can obtain input data including a plurality of input values. The exponential function processing module (120) can obtain a plurality of input values ​​sequentially. In one embodiment, the input data can be expressed in the form of a vector, a matrix, or a tensor. For example, the input data can be vector type data having a size of 1 x 1 x D. Or, for example, the input data can be matrix type data having a size of 1 x W x H, tensor type data having a size of C x W x H, or a tensor having a dimension of 4 or more.

[0059] In one embodiment, the input value of the input data may be a quantized value. The input value may be an integer within a predetermined range. The input value may be an integer between 0 and a maximum quantized value. For example, if the maximum quantized value is 255, the input value may be one of the integers between 0 and 255. An example of quantized data according to one embodiment of the present disclosure is described in detail with reference to FIGS. 4A, 4B, and 4C.

[0060] In one embodiment, the exponential function processing module (120) can obtain the value of the exponential function based on the input data. The exponential function processing module (120) can obtain the value of the exponential function corresponding to the input value of the input data. In one embodiment, the exponential function processing module (120) can obtain the value of the exponential function corresponding to the input value using a lookup table module. The exponential function processing module (120) can extract the value of the exponential function corresponding to the input value by referring to the lookup table module. The exponential function processing module (120) can provide the value of the exponential function corresponding to the input value to the softmax function processing module (130).

[0061] In one embodiment, the lookup table module may store exponential function information. The lookup table module may store values ​​for a plurality of exponential functions. For example, the lookup table module may include values ​​for exponential functions corresponding to a plurality of quantized values. The values ​​of the exponential functions stored in the lookup table module may be scaled according to a scale factor. The scale factor may be determined for each value of the exponential function. The values ​​for the plurality of exponential functions stored in the lookup table module may be determined to fall into one of a plurality of groups based on the input values ​​corresponding to each of the values. For example, the values ​​for the plurality of exponential functions stored in the lookup table module may be classified into a plurality of groups based on the upper bits of the input values. Examples of scale factors and data stored in the lookup table module according to one embodiment of the present disclosure are described in detail with reference to FIGS. 5A and 5B .

[0062] In one embodiment, the exponential function processing module (120) can determine a scale factor corresponding to an input value. In one embodiment, the exponential function processing module (120) can determine the scale factor based on at least a portion of the input value. For example, the exponential function processing module (120) can determine the scale factor using the upper bits of the input value. An example of a process for determining a scale factor by the exponential function processing module (120) according to one embodiment of the present disclosure is described in detail with reference to FIG. 6.

[0063] In one embodiment, the softmax function processing module (130) can determine the value of the softmax function for the input value based on the value of the exponential function and the scale factor. The softmax function processing module (130) can determine the value of the exponential function for the input value based on the value of the exponential function and the scale factor. In one embodiment, the softmax function processing module (130) can determine the value of the softmax function corresponding to the input value based on the sum of the determined exponential functions. An example of a method for the softmax function processing module (130) according to one embodiment of the present disclosure to determine the value of the softmax function is described in detail with reference to FIGS. 7a, 7b, 8a, 8b, and 8c.

[0064] In one embodiment, the accelerator (110) may be an auxiliary computational device configured to process artificial intelligence operations, or included in or associated with an electronic device that processes the same. For example, the accelerator (110) may be an auxiliary computational device used in a processor (100) that performs operations using machine learning or deep learning. The accelerator (110) may increase the efficiency of operations performed by the processor (100). For example, the accelerator (110) may reduce the time required for repeated operations to process a function input to the processor (100). In one embodiment, the processor (100) may process a softmax function using the accelerator (110).

[0065] As shown in mathematical expression 1, the value of the softmax function includes an exponential function operation and a division operation. The accelerator (110) can output the value of the softmax function without performing at least some of the exponential function operation and the division operation by using the exponential function processing module (120) and the softmax function processing module (130).

[0066] FIG. 2 is a block diagram illustrating a processor for processing an artificial neural network (or artificial intelligence operation) according to one embodiment of the present disclosure.

[0067] Referring to FIG. 2, a processor (100) according to one embodiment of the present disclosure may further include a control module (210), a direct memory access (DMA) module (215), and a digital signal processing (DSP) module (220).

[0068] The control module (210) may be configured to control the operation of other components within the processor (100). In one embodiment, the control module (210) may control the operation of at least one of the accelerator (110) and the digital signal processing module (220). The control module (210) may control the accelerator (110) to output output data including the value of the softmax function based on the input data. For example, the control module (210) may control the accelerator (110) to perform at least one of a convolution operation and a softmax operation.

[0069] According to one embodiment, the control module (210) may control the accelerator (110) based on a control command obtained from a central processing unit of an electronic device including a processor (100). For example, in response to a command to perform a neural network operation for the processor (100) of the central processing unit, the control module (210) may control at least one of the accelerator (110) or the digital signal processing module (220) to perform at least a part of the neural network operation.

[0070] According to one embodiment, the control module (210) can initialize the settings of the accelerator (110). For example, the control module (210) can determine at least one of a group index, a shift unit of the group, a parameter expression bit, or a zero point value of the accelerator (110) based on setting data according to at least one of a type or a range of input data. The control module (210) can control the lookup table module of the accelerator (110) to store exponential function information based on exponential function information included in the setting data. A process of initializing the settings of the accelerator (110) according to one embodiment of the present disclosure is described in detail with reference to FIG. 9.

[0071] The direct memory access module (215) may be configured to communicate with other components of the electronic device including the processor (100). The direct memory access module (215) may communicate with other components without going through the central processing unit of the electronic device. For example, the processor (100) may communicate with a graphics processing unit or a neural network processing unit.

[0072] The digital signal processing module (220) may be configured to process digital signals. The one-control module (210) may perform complex operations, excluding multiplication and addition operations, through the digital signal processing module (220). However, during the process of processing artificial intelligence operations, data exchange between the accelerator (110) and the digital signal processing module (220) may increase processing time. Therefore, when the processor (100) performs operations by maximizing the use of the accelerator (110), the operation execution time may be reduced.

[0073] In one embodiment, the exponential function processing module (120) may include a lookup table module (230) and an interpolation module (240).

[0074] The lookup table module (230) can store multiple values ​​and output a value corresponding to an index value from among the multiple stored values. In one embodiment, the lookup table module mentioned in the description of FIG. 1 above may correspond to the lookup table module (230) of FIG. 2.

[0075] The lookup table module (230) can store exponential function information. In one embodiment, the plurality of values ​​stored in the lookup table module (230) can be values ​​of an exponential function for a quantized value. The quantized value can include a value that can be expressed as an input value. For example, if the input data is quantized to an integer value from 0 to 255, the quantized value can include 256 values ​​including integer values ​​from 0 to 255. For example, the lookup table module (230) can output an exponential function value corresponding to an index (e.g., "0xDF") determined based on an input value (e.g., "0xDFB4").

[0076] In one embodiment, the lookup table module (230) may store the value of a scaled exponential function. For example, if the value of the exponential function for an input value is A and the scale factor is S, the lookup table module (230) may store the value of A shifted left by S (e.g., the value of A << S).

[0077] In one embodiment, the lookup table module (230) may store a smaller number of exponential function information than the number of values ​​that the input value can express. The number of data stored in the exponential function may be smaller than the maximum quantization value. For example, the input data may be quantized to be expressed in 16 bits to express 216 values, and the exponential function information may include exponential function values ​​for indices expressed in 8 bits. For example, the input value may be expressed as one value from 0x0000 to 0xFFFF, and the exponential function information may include an exponential function value corresponding to an index expressed as 0xYZ00 (Y, Z are one of the values ​​from 0 to F) from 0x0000 to 0xFFFF. According to one embodiment, since the lookup table module (230) stores a smaller number of exponential function information than the expression range of the input data, the area (region) implemented in hardware may be reduced.

[0078] The interpolation module (240) can determine the value of an exponential function corresponding to an input value based on the value stored in the lookup table module (230). In one embodiment, the interpolation module (240) can obtain values ​​of a plurality of exponential functions corresponding to the input value by referring to the lookup table module (230). The interpolation module (240) can determine the value of the exponential function corresponding to the input value using the values ​​of the plurality of exponential functions. In one embodiment, the interpolation module (240) can obtain a first reference value and a second reference value corresponding to the input value. The interpolation module (240) can determine the value of the exponential function corresponding to the input value by performing interpolation on the first reference value and the second reference value. For example, in order to determine the value of the exponential function for the input value 0xDF16, the interpolation module (240) may obtain the value of the exponential function for 0xDF00 stored in the lookup table module (230) as a first reference value, and may obtain the value of the exponential function for 0xE000 as a second reference value. The interpolation module (240) may perform interpolation using an interpolation weight on the first reference value and the second reference value. The value on which interpolation is performed by the interpolation module (240) may be the value of the exponential function corresponding to the input value. The process of determining the value of the exponential function corresponding to the input value by the interpolation module (240) according to one embodiment of the present disclosure is described in detail with reference to FIG. 6.

[0079] In one embodiment, the softmax function processing module (130) may include a calculation module (250), a parameter module (260), and a post-processing module (270). The calculation module (250) may be configured to perform calculations required by the accelerator (110). In one embodiment, the calculation module (250) may be configured to perform multiplication and addition operations. For example, the calculation module (250) may be configured to perform a MAC (Multiplicate-Accumulate) operation. Since convolution operations and matrix multiplication, which account for a large proportion of artificial intelligence operations, are operations expressed as multiplication and addition, the accelerator (110) may accelerate artificial intelligence operations using the calculation module (250).

[0080] The calculation module (250) can calculate the sum of the exponential function (e.g., a value corresponding to the denominator of the softmax function) based on at least one of the value of the exponential function and the scale factor. In one embodiment, the calculation module (250) can scale the value of the exponential function based on at least one of the value of the exponential function and the scale factor. The calculation module (250) can calculate the sum of the exponential function by accumulating and adding the values ​​of the scaled exponential function.

[0081] In one embodiment, the calculation module (250) can scale the value of the exponential function based on a scale factor. In one embodiment, the calculation module (250) can scale the value of the exponential function by performing a right shift operation by the scale factor. For example, the calculation module (250) can determine the value “4”, which is the result of performing a right shift operation by the scale factor 1 on the value 8 of the exponential function, as the value of the scaled exponential function.

[0082] In one embodiment, the calculation module (250) can scale the value of the exponential function based on the base scale factor. The calculation module (250) can scale the value of the exponential function by performing a left shift operation by the base scale factor. For example, the calculation module (250) can determine 16, which is the result of performing a left shift operation by the base scale factor 1 on the value of the exponential function 8, as the value of the scaled exponential function. In one embodiment, the base scale factor can be a value predetermined according to the settings of the accelerator (110).

[0083] In one embodiment, the calculation module (250) can scale the value of the exponential function based on the scale factor and the base scale factor. The calculation module (250) can scale the value of the exponential function by performing a left shift operation by a value obtained by subtracting a scale factor corresponding to the input value from the base scale factor. For example, the calculation module (250) can determine 16, which is a result of performing a left shift operation by 1, which is obtained by subtracting a scale factor corresponding to the input value from the base scale factor 2, on the value of the exponential function 8, as the value of the scaled exponential function.

[0084] An example of an operation of the calculation module (250) according to one embodiment of the present disclosure to determine the sum of exponential functions is described in detail with reference to FIGS. 7A and 7B . According to one embodiment, the calculation module (250) may transmit the sum of exponential functions to the parameter module (260).

[0085] The parameter module (260) may be configured to determine parameters for calculating the value of the softmax function. In one embodiment, the parameter module (260) may determine at least one parameter for representing an approximation to the sum of an exponential function. In one embodiment, the at least one parameter may include a first parameter and a second parameter. A value obtained by dividing the maximum quantized value of the input data by the denominator of the softmax function may be approximated as a value obtained by performing a right shift operation on the first parameter by the second parameter. The parameter module (260) according to one embodiment may determine the first parameter and the second parameter for representing the value of the quantized softmax function. An example of an operation of determining a parameter according to one embodiment of the present disclosure will be described in detail with reference to FIGS. 8A, 8B, and 8C. The parameter module (260) according to one embodiment may transmit at least one parameter to the post-processing module (270).

[0086] The post-processing module (270) may be configured to determine the value of the softmax function. In one embodiment, the post-processing module (270) may determine the value of the softmax function based on at least one parameter, the value of the exponential function, and a scale factor corresponding to the input value. For example, the post-processing module (270) may determine the value of the softmax function by applying the scale factor and at least one parameter to the value of the exponential function.

[0087] In one embodiment, the post-processing module (270) may determine the value of the softmax function based on the sum of the exponential function of the operation module (250), the value of the exponential function, and the scale factor. For example, the post-processing module (270) may determine the value of the softmax function by dividing the value scaled using the scale factor by the sum of the exponential function.

[0088] The accelerator (110) may additionally include an input module (280), a memory (285), and an output module (290) in addition to the exponential function processing module (120) and the softmax function processing module (130) of FIG. 1.

[0089] The input module (280) can obtain input data including a plurality of input values. In one embodiment, the input values ​​may be quantized values. The input module (280) may be configured to store the input data or may include an input buffer that stores the input data. For example, the input buffer may sequentially obtain input values ​​of the input data from at least one of the memory (285) or another accelerator. In one embodiment, the input module (280) may obtain the input data from the memory (285) included in the accelerator (110) or another accelerator.

[0090] The input module (280) can transmit input data to the exponential function processing module (120). In one embodiment, the input module (280) can sequentially transmit input values ​​of the input data to the exponential function processing module (120). The input module (280) can transmit the input data to the softmax function processing module (130). The softmax function processing module (130) can use the input data obtained from the input module (280) to output the value of the softmax function.

[0091] The memory (285) may be configured to store information used in the accelerator (110). The memory (285) may store exponential function information. The exponential function information may include quantized exponential function values. For example, the exponential function information may include quantized exponential function values ​​corresponding to values ​​that the quantized input values ​​can represent. The accelerator (110) may use the exponential function information to determine the value of the softmax function. In one embodiment, the memory (285) may store a portion of a plurality of exponential function information according to the maximum quantization value. The memory (285) may store output data obtained from the output module (290).

[0092] The output module (290) can output output data including the value of the softmax function. The output module (290) can include an output buffer. The output module (290) can obtain the value of the softmax function from the softmax function processing module (130). In one embodiment, the output module (290) can sequentially store the value of the softmax function output from the softmax function processing module (130) in the memory (285) or transmit it as input to another accelerator.

[0093] FIG. 2 illustrates an exemplary structure of a processor (100) and an accelerator (110) according to one embodiment of the present disclosure, but is not limited thereto, and the processor (100) and the accelerator (110) may be configured by additionally including some components or by omitting some components.

[0094] In one embodiment, the accelerator (110) may further include a quantization module. The quantization module may be configured to perform quantization when the input data is not quantized. The maximum quantization value used by the accelerator (110) may be predetermined or transmitted along with the input data.

[0095] FIG. 3 is a flowchart illustrating an operation method of a processor including an accelerator for processing a softmax function according to one embodiment of the present disclosure.

[0096] In operation 310, a processor (100) according to one embodiment may obtain input data including a plurality of input values. In one embodiment, operation 310 may be performed by an exponential function processing module (120) included in an accelerator (110) of the processor (100).

[0097] In operation 320, the processor (100) according to one embodiment may obtain the value of an exponential function corresponding to an input value included in input data using a lookup table module. In one embodiment, operation 320 may be performed by an exponential function processing module (120) included in an accelerator (110) of the processor (100).

[0098] In operation 330, the processor (100) according to one embodiment may determine a scale factor corresponding to an input value. In one embodiment, operation 330 may be performed by an exponential function processing module (120) included in an accelerator (110) of the processor (100).

[0099] In operation 340, the processor (100) according to one embodiment may determine the value of a softmax function for an input value based on the value of an exponential function and a scale factor. In one embodiment, operation 340 may be performed by a softmax function processing module (130) included in an accelerator (110) of the processor (100).

[0100] The operation of the processor (100) including the accelerator (110) for processing the softmax function may not include one or more of the operations 310 to 340, and may omit or include some of the operations.

[0101] FIG. 4a is a diagram for explaining quantization of input values ​​according to one embodiment of the present disclosure.

[0102] Referring to Fig. 4a, the unquantized input value may be a value within a predetermined range. For example, the unquantized input value may be a rational number that is greater than or equal to the lower limit value (xmin) of the input value and less than (or less than or equal to) the upper limit value (xmax) of the input value.

[0103] The quantized input value can be an integer within a given range. For example, the quantized input data can represent one of the integers greater than or equal to 0 and less than or equal to the maximum quantized value (Qmax). The maximum quantized value can be a value of 2N-1, where N is an integer. For example, the maximum quantized value can be one of 63, 127, 255, 511, or 1023.

[0104] Values ​​within a given range can be quantized to the same value. For example, a value greater than or equal to xmin and less than xmin+scale_x can be quantized to 0, and a value greater than or equal to xmin +scale_x and less than xmin +2*scale_x can be quantized to 1. The quantization interval scale_x can be determined based on the maximum quantization value. For example, the quantization interval can divide the range greater than or equal to xmin and less than xmax into equal parts by the maximum quantization value. For example, it can be determined as scale_x = (xmax - xmin) / Qmax.

[0105] In one embodiment, input data xi can be quantized into an integer m satisfying m = (xi-xmin) / scale_x. For example, unquantized input data (xi) can be expressed using quantized input data (m) as in mathematical expression 2.

[0106]

[0107] However, without limitation thereto, the quantization interval scale_x may be determined differently for each section by a design change of a typical technician.

[0108] In one embodiment, the accelerator (110) can quantize at least one of input data and output data. The accelerator (110) according to one embodiment can obtain quantized input data. The input data and the output data can include at least one of vector type, matrix type, or tensor type data including a plurality of quantized values. The input data can have the same data size as the output data in the vector type, matrix type, or tensor type. The input data and the output data can be quantized so that the maximum quantization value is the same.

[0109] When the input data of the accelerator (110) of FIG. 1 according to one embodiment of the present disclosure is quantized, the memory capacity required to store the data is reduced. Furthermore, calculating integers has the advantage of lower computational complexity than calculating prime numbers.

[0110] FIG. 4b is a diagram for explaining quantization of exponential function values ​​according to one embodiment of the present disclosure.

[0111] Referring to FIG. 4b, the unquantized exponential function value (EXP) may be a value within a predetermined range. For example, the unquantized exponential function value may be a rational number that is greater than or equal to a lower bound value (emin) of the exponential function value and less than (or less than or equal to) an upper bound value (emax) of the exponential function value. The lower bound value of the exponential function may be an exponential function value for a lower bound value of the input value, and the upper bound value of the exponential function may be an exponential function value for an upper bound value of the input value.

[0112] The quantized exponential function values ​​(denoted as “Quantized EXP”) can be integers within a given range. The exponential function values ​​can be quantized based on a quantization interval (scale_EXP) of the exponential function. The quantization interval (scale_EXP) of the exponential function can be determined based on an upper bound value of the exponential function and a lower bound value of the exponential function. For example, scale_EXP can be determined as (emax - emin) / Em. The maximum quantized value (Em) can be a predetermined value. For example, Em can be 65,535 (e.g., 216-1).

[0113] According to one embodiment, the exponential function value (EXP) may be quantized such that the quantization value for the lower bound value (emin) becomes 0. Referring to the first graph (410b), the quantized exponential function value may represent one of the integers greater than or equal to "0" and less than or equal to "Em". For example, the exponential function value (ei) may be quantized to an integer n that satisfies Equation 3.

[0114]

[0115] In one embodiment, the exponential function value may be quantized such that the quantization value for the lower bound value (emin) becomes a predetermined constant. Referring to the second graph (420b), the quantized exponential function value may represent one of integers greater than or equal to the constant "ZP" and less than or equal to "Em+ZP". The second graph (420b) may be the first graph (410b) translated in the y-axis direction by the constant ZP. In one embodiment, the predetermined constant may be determined based on the lower bound value and the quantization interval. For example, the predetermined constant (ZP) may be determined as ZP = emin / scale_EXP. The predetermined constant may be expressed as a zero point. In this example, mathematical expression 3 may be expressed as mathematical expression 4.

[0116]

[0117] The exponential function information according to one embodiment may include a quantized exponential function value described with reference to the first graph (410b) or may include a quantized exponential function value described with reference to the second graph (420b).

[0118] FIG. 4c is a diagram for explaining quantization of input values ​​and exponential function values ​​according to one embodiment of the present disclosure.

[0119] Referring to FIG. 4c, a third graph (410c) using non-quantized data and a fourth graph (420c) using quantized data are illustrated. The third graph (410c) is a graph between an input value (illustrated as “input”) and an exponential function value (EXP). In one embodiment, the input value and the exponential function value (EXP) may each be quantized. The fourth graph (420c) is a graph between a quantized exponential function value (illustrated as Quantized EXP) and a quantized input value (illustrated as Quantized input).

[0120] According to one embodiment, the input value may be quantized based on a maximum quantization value Qmax for the input value, and the exponential function value may be quantized based on a maximum quantization value Em for the exponential function value. For example, the input value may be quantized as in FIG. 4a, and the exponential function value may be quantized as in the first graph (410b) or the second graph (420b) of FIG. 4b.

[0121] The fourth graph (420c) may represent the values ​​(E0, E1, ..., EQmax) of an exponential function for quantized input values ​​(0, 1, ..., Qmax). According to one embodiment, the value of the exponential function may be set to 0 for a lower bound value, as in the first graph (410b) of FIG. 4b, or the value of the exponential function for a lower bound value may be set to a predetermined constant (e.g., constant ZP), as in the second graph (420b).

[0122] FIG. 5A is a diagram illustrating groups and scale factors of data stored in a lookup table module according to one embodiment of the present disclosure. FIG. 5B is a diagram illustrating groups and scale factors of data stored in a lookup table module according to one embodiment of the present disclosure.

[0123] Referring to FIG. 5A, original data and scaled data of the original data are illustrated, relating to values ​​of an exponential function according to one embodiment of the present disclosure. In one embodiment, the original data may include exponential function information. The original data may include values ​​of a plurality of quantized exponential functions. The values ​​of the quantized exponential functions may correspond to quantized input values. For example, the value of the exponential function corresponding to a quantized input value 0 may be 0001, and the value of the exponential function corresponding to a quantized input value 1 may be 0002.

[0124] Referring to FIG. 5A, an input value according to an embodiment of the present disclosure may be quantized into 256 (= 28 or 8 bit) quantization values, and the value of an exponential function may be quantized into 65,536 (= 216 or 16 bit) quantization values. The value of the exponential function may be expressed in hexadecimal format. The original data may be expressed such that the input value increases by 1 as it proceeds to the right in the same row, and the input value increases by 16 as it proceeds downward in the same column.

[0125] The exponential function value of the original data may have an increasing number of significant figures as the input value increases. For example, the exponential function value corresponding to the quantized input value "254" may be "F07B", with 16 significant figures. The exponential function value corresponding to the input value "2" may be "0002", with 2 significant figures.

[0126] In one embodiment, the input values ​​of the original data may be classified into one of multiple groups. For example, group #0 may include values ​​of the exponential function corresponding to input values ​​0 to 31, and group #7 may include values ​​of the exponential function corresponding to input values ​​234 to 255.

[0127] In one embodiment, the number of groups may be determined based on the group index. For example, the group index may represent a log2 scale value of the number of groups. For example, if the group index is 3, the number of groups may be 8. In one embodiment, multiple groups may include the same number of exponential function values. For example, if the number of groups is 8, each of the multiple groups may include 32 exponential function values.

[0128] According to one embodiment, a group of exponential function values ​​may be determined based on a group index and an input value. According to one embodiment, a group of exponential function values ​​may be determined by extracting as many upper bits of an input value as the number of group indices. For example, when the group index is 3, the value indicated by the upper 3 bits of the input value may be determined as the group number. For example, the input value "1" may be expressed as "00000001" in 8 bits, and since the upper 3 bits are "000", the group number (#N) may be 0. The input value "255" may be expressed as "11111111" in 8 bits, and since the upper 3 bits are "111", the group number (#N) may be 7. Here, the upper bit of the input value corresponding to the index of the group may be referred to as a group bit.

[0129] In one embodiment, the original data may be scaled. The scale factor may indicate information regarding the scaling of the original data. For example, if the scale factor is 1, the value of the exponential function may be scaled by a factor of 21. The scale factor may indicate scaling information using at least one of a logarithmic scale, an exponential scale, or a multiple scale.

[0130] Referring to Fig. 5b, the scaled data may be a result of the original data being scaled based on a scale factor. The values ​​of the exponential function included in group #N may be scaled based on the scale factor corresponding to group #N. For example, the exponential functions included in group #0 may represent a value increased by 27 based on the scale factor. The value of the exponential function corresponding to the input value "2" before scaling may be "0002" with 2 significant figures, but the value of the exponential function corresponding to the input value "2" after scaling may be "00D8" with 8 significant figures. Since the original data is scaled and stored, the significant figures for the values ​​of the exponential function may increase, and the value of the softmax function determined based on the value of the exponential function with the increased significant figures may have a reduced error.

[0131] In one embodiment, the scale factor may be determined for each group. For example, group #0 may have a scale factor of 7, group #1 may have a scale factor of 6, ..., and group #7 may have a scale factor of 0. In one embodiment, the scale factor of a group may be determined based on a shift unit. For example, when the shift unit of a group is 1, the difference in scale factors between the groups may be 1, as shown in FIG. 5b. For example, when the shift unit of a group is 3, the difference in scale factors between the groups may be 3. For example, the scale factors of the groups may be determined as 0, 3, 6, ..., etc.

[0132] In one embodiment, the lookup table module (230) may store scaled data. The scaled data may be obtained externally to the accelerator (110). For example, the scaled data may be data stored in a memory external to the processor (100).

[0133] FIG. 5b illustrates an example in which an input value is expressed in 8 bits, an exponential function value is expressed in 16 bits, a group index is 3, a shift unit is 1, and original data is scaled to increase by 2N according to a scale factor, according to one embodiment of the present disclosure, but is not limited thereto and may be replaced with another representation method.

[0134] FIG. 6 is a block diagram illustrating the operation of an exponential function processing module according to one embodiment of the present disclosure.

[0135] Referring to FIG. 6, an exponential function processing module (120) according to one embodiment of the present disclosure may determine a value of an exponential function for an input value. In one embodiment, the exponential function processing module (120) may include a lookup table module (230) and an interpolation module (240). In one embodiment, the input value may be a 16-bit number expressed in hexadecimal, such as "0xDFB4". In one embodiment, the input value may include a table index included in an upper bit area and an interpolation weight included in a lower bit area. For example, the table index may mean an upper bit value "0xDF" of the input value "0xDFB4", and the interpolation weight may mean a lower bit value "0xB4" of the input value "0xDFB4".

[0136] In one embodiment, the lookup table module (230) can output a value of an exponential function corresponding to an input value. The lookup table module (230) can store exponential function information. The lookup table module (230) can store a value of an exponential function corresponding to an index. For example, the lookup table module (230) can store a value of an exponential function of "0xXY00" for an index "0xXY". For example, the lookup table module (230) can store a first value, which is a value of an exponential function of "0x0000", for an index "0x00", and can store an Nth value, which is a value of an exponential function of "0xFF00", for an index "0xFF".

[0137] In one embodiment, the lookup table module (230) may output a value of an exponential function corresponding to an index determined based on at least a portion of the input value. In one embodiment, the lookup table module (230) may output a value of an exponential function corresponding to a first index value determined based on at least the upper bits of the input value. For example, if the input value is “0xDFB4”, the value “0xDF” corresponding to the upper 8 bits (e.g., two characters expressed in hexadecimal) may be determined as the first index value. For example, the hexadecimal value “0xDF” may be expressed in binary as “0b110111111111111”. The lookup table module (230) may output a value of an exponential function corresponding to the first index value. The value of the exponential function corresponding to the first index value may be referred to as a first reference value. For example, the lookup table module (230) may output the first reference value “0xB672”.

[0138] In one embodiment, the lookup table module (230) may output a value of an exponential function corresponding to a second index value determined based on a first index value. The second index may be a value that is relatively adjacent to the first index. For example, the second index may be a value that is increased by 1 from the first index. The lookup table module (230) may output a value of an exponential function corresponding to the second index value. The value of the exponential function corresponding to the second index value may be referred to as a second reference value. For example, the lookup table module (230) may output a second reference value "0xBC3E", which is a value of an exponential function corresponding to the second index "0xE0".

[0139] In one embodiment, the interpolation module (240) may determine the value of an exponential function for an input value based on the values ​​of one or more exponential functions output from the lookup table module (230). For example, the interpolation module (240) may determine the value of an exponential function corresponding to the input value based on a first reference value and a second reference value.

[0140] In one embodiment, the interpolation module (240) can determine the value of an exponential function for the input value based on the values ​​of one or more exponential functions output from the lookup table module (230) based on at least a portion of the input value. The interpolation module (240) can identify a predetermined low-order bit of the input value. In one embodiment, the low-order bit of the input value can be the remainder except for the high-order bit of the input value for determining the index of the lookup table module. For example, the low-order bit of the input value is the low-order 16 bits, and the value of the low-order bit of the input value can be 0xB4.

[0141] In one embodiment, the interpolation module (240) may perform interpolation on the values ​​of one or more exponential functions output from the lookup table module (230) based on the lower bits of the input value. For example, the interpolation module (240) may perform interpolation on the first reference value and the second reference value at a ratio of 0xB4 and 0x4C (e.g., 0x100-0xB4). The interpolation module (240) may determine "0xBA85", which is the result of performing the interpolation, as the value of the exponential function for the input value.

[0142] Referring to FIG. 6, an exponential function processing module (120) according to one embodiment of the present disclosure may determine a scale factor for an input value. The exponential function processing module (120) may include a group index module (610).

[0143] In one embodiment, the group index module (610) can determine a scale factor based on an input value. The group index module (610) can determine the scale factor based on the upper bits of the input value. The information input to the group index module (610) can be identical to the upper bits of the input value for determining the index of the lookup table module (230). For example, the group index module (610) can determine the scale factor based on the value "0xDF" of the upper 8 bits of the input value.

[0144] In one embodiment, the upper bits of the input value may be split into a first bit string and a second bit string. For example, the first bit string may refer to the upper bits of the input value as many as the group index, and the second bit string may refer to the remaining bits excluding the first bit string. For example, if the value of the group index is 3, the first bit string may be "0b110", which is the three upper bits of the input value, and the second bit string may be the remainder, "0b1111111111111". In one embodiment, the bit string may refer to a set of bits expressed in binary.

[0145] In one embodiment, the group index module (610) may determine a scale factor based on at least one of a shift unit of the group and a group index. The group index may be an index relating to the number of groups of exponential function information stored in the lookup table module (230).

[0146] In one embodiment, the group index module (610) can determine the scale factor based on the upper bits of the input value as many as the value of the group index. The group index module (610) can determine the scale factor based on the first bit string. In one embodiment, the group index module (610) can determine the scale factor by performing a NOT GATE operation on the upper bits of the input value as many as the value of the group index. For example, when the value of the group index is 3, the result of the NOT GATE operation on "0b110", which is the value of the three upper bits of the input value, may be determined as 1 (="0b001") as the value of the scale factor.

[0147] In one embodiment, the group index module (610) can determine the difference in scale factors between groups based on the shift unit of the groups. The group index module (610) can determine the scale factor by multiplying the result of performing the NOT GATE operation on the upper bits of the input value by the number of bits representing the value of the group index by the shift unit of the group. For example, if the value of the group index is 3 and the shift unit of the group is 3, the value of the scale factor can be determined as 3, which is the result of multiplying 1 (="0b001") of the NOT GATE operation on "0b110", which is the value of the three upper bits of the input value, by the shift unit of the group.

[0148] In an embodiment, the group index module (610) may determine the order within the group of the input values. The group index module (610) may determine the order within the group based on the remaining values ​​excluding the value of the group index from the upper bits of the input values. In one embodiment, the group index module (610) may determine the scale factor based on the second bit string. For example, the remaining value excluding the upper three bits (e.g., excluding the bit representing the group index in the input value "0b11011111111111111" of the group index module (610)) may be expressed as "0b1111111111111". In this example, the order within the group of the input values ​​may be determined as 8191 (="0b1111111111111"). For example, this may mean the last order within the group. In addition, this may mean that the next value of the input value is from a different group and that the scale factor may be changed. The group index module (610) may determine that the order within the group is the last if all the remaining values ​​except for the value of the group index among the predetermined upper bits of the input value are 1. In one embodiment, the group index module (610) may be implemented to perform a bitwise AND GATE operation.

[0149] In one embodiment, the interpolation module (240) can adjust the second reference value based on the order of the input values ​​within the group. The interpolation module (240) can increase the second reference value when the order of the input values ​​within the group is last. For example, the interpolation module (240) can perform a left shift operation on the second reference value when the order of the input values ​​within the group is last. For example, the interpolation module (240) can determine the value of the exponential function for the input value based on the first reference value and the value "0xBC3E", which is a left-shifted value of the second reference value "0x5E1F".

[0150] In one embodiment, the interpolation module (240) may perform a left shift operation based on the shift unit of the group. For example, if the shift unit of the group is 3, the interpolation module (240) may perform a left shift operation on the second reference value by 3. For example, the interpolation module (240) may increase the second reference value by a factor of 23.

[0151] In one embodiment, the group index module (610) may be implemented separately as a first group index module that determines a scale factor based on a group index and a second group index module that determines an order within a group. According to one embodiment, the first group index module may output a scale factor by taking as input at least one value from among the entire input value, a value input to the lookup table module (230), and the upper bits of the input value equal to the number of bits used to express the value of the group index. According to one embodiment, the second group index module may determine an order within a group by taking as input at least one value from among the entire input value, a value input to the lookup table module (230), and the remaining values ​​excluding the value input to the first group index module among the values ​​input to the lookup table module (230). In one embodiment, at least one of the first group index module and the second index module may be included in the interpolation module (240).

[0152] As the range of expression of the power value (e.g., number of bits) increases, the memory capacity of the lookup table module (230) of the accelerator (110) for storing the values ​​of the exponential function corresponding to all input values ​​increases exponentially. If the lookup table module (230) stores only the values ​​of the exponential function corresponding to a part of the input values, the accuracy may decrease. The exponential function processing module (120) according to one embodiment of the present disclosure can increase the accuracy of the values ​​of the exponential function for the input values ​​while reducing the memory capacity of the lookup table module (230) used in the accelerator (110) by using the exponential function values ​​scaled based on the scale factor.

[0153] FIG. 7A is a flowchart illustrating a method by which a processor including an accelerator according to one embodiment of the present disclosure determines a value of a softmax function.

[0154] In operation 710a, the accelerator (110) according to one embodiment can determine the sum of the exponential function of the input data. The accelerator (110) can determine the sum of the exponential function by accumulating and adding the values ​​of the exponential function for the input values ​​determined through the exponential function processing module (120).

[0155] In one embodiment, the accelerator (110) may determine the sum of the exponential functions based on a base scale factor. The sum of the exponential functions determined based on the base scale factor may be a scaled version of the actual sum of the values ​​of the exponential functions of the input values.

[0156] In one embodiment, the accelerator (110) may store the sum of the exponential functions in memory. The memory may include a register or SRAM. The accelerator (110) may determine the shift factor based on the capacity of the memory and the upper bound of the sum of the exponential functions. In one embodiment, the capacity of the memory may be a predetermined value.

[0157] In one embodiment, the upper limit of the sum of the exponential function may be determined based on the number of input values ​​included in the input data. The maximum value that the value of the exponential function can express may be determined based on the range in which the value of the exponential function can be expressed (e.g., 16 bits) and the zero point. The upper limit of the sum of the exponential function may be determined based on the product of the maximum value that can be expressed by the exponential function and the number of input values. When acquiring the input data, the accelerator (110) may determine the upper limit of the sum of the exponential function based on the range in which the value of the exponential function can be expressed, the number of input values, and the value of the zero point.

[0158] In one embodiment of the present disclosure, the accelerator (110) can adjust the scale factor and the base scale factor based on the memory capacity and the upper limit value of the sum of the exponential function. The accelerator (110) can determine the shift coefficient based on the difference between the capacity of the memory in bit units and the logarithmic scale value of the upper limit value of the sum of the exponential function. In one embodiment, the capacity of the memory may be greater than the number of bits for expressing the upper limit value of the sum of the exponential function. For example, the memory may express 19 bits and may be 18 bits for expressing the upper limit value of the sum of the exponential function. The accelerator (110) can determine the value of the shift coefficient as 1, which is the difference between the capacity of the memory and the number of bits for expressing the upper limit value of the sum of the exponential function.

[0159] In one embodiment, the memory capacity may be smaller than the number of bits required to represent the upper bound of the sum of the exponential function. For example, the memory may represent 18 bits, and the upper bound of the sum of the exponential function may be 19 bits. The accelerator (110) may determine the value of the shift coefficient as -1, which is the difference between the memory capacity and the number of bits required to represent the upper bound of the sum of the exponential function.

[0160] The accelerator (110) can adjust the scale factor and the base scale factor based on the shift factor. In one embodiment, the accelerator (110) can add the shift factor to each of the scale factor and the base scale factor. When the capacity of the memory of the accelerator (110) according to one embodiment of the present disclosure is greater than the number of bits for expressing the maximum value of the sum of the exponential function, the accelerator (110) can increase the scale factor and the base scale factor by the shift factor. By increasing the scale factor and the base scale factor, the accelerator (110) can improve the accuracy of the sum of the exponential function.

[0161] When the capacity of the memory of the accelerator (110) according to one embodiment of the present disclosure is smaller than the number of bits for expressing the maximum value of the sum of the exponential functions, the accelerator (110) can reduce the scale factor and the base scale factor by the absolute value of the shift coefficient. By reducing the scale factor and the base scale factor, the accelerator (110) can prevent overflow of the memory that stores the sum of the exponential functions.

[0162] In operation 720a, the accelerator (110) according to one embodiment can determine the value of the softmax function based on the value of the exponential function, the scale factor, and the sum of the exponential functions. The accelerator (110) can determine the value of the softmax function by dividing the result of scaling the value of the exponential function using the scale factor by the sum of the exponential functions.

[0163] In one embodiment, the accelerator (110) may determine the value of the softmax function based on the base scale factor. The accelerator (110) may determine the value of the softmax function by dividing the result of scaling the value of the exponential function using the scale factor and the base scale factor by the sum of the exponential functions.

[0164] The operation of the processor (100) including an accelerator (110) for processing a softmax function may not include at least one operation among operations 710a to 720a, and may omit or include some operations.

[0165] FIG. 7b is a flowchart illustrating a method by which a processor including an accelerator according to one embodiment of the present disclosure determines a sum of exponential functions.

[0166] A method for determining the sum of exponential functions according to one embodiment of the present disclosure can be performed by the calculation module (250) of FIG. 2. The calculation module (250) can determine the sum of exponential functions based on the values ​​of the exponential functions.

[0167] In operation 710b, the calculation module (250) can determine the value of the sum (SUM) of the exponential function and the base scale factor (BASE_EXPONENT). The calculation module (250) can initialize the sum of the exponential function to 0. The calculation module (250) can determine the base scale factor based on the configuration data. For example, the calculation module (250) can determine a predetermined value (RSHAMT_MAX) as the base scale factor.

[0168] In operation 720b, the calculation module (250) can determine whether there are additional values ​​of the exponential function. If there are additional values ​​of the exponential function (“yes” in operation 720b), the calculation module (250) can update the sum total of the exponential function SUM based on the values ​​of the exponential function. If there are no additional values ​​of the exponential function (“no” in operation 720b), the method can proceed to operation 730b.

[0169] In operation 730b, the calculation module (250) can obtain the value (VALUE) and scale factor (EXPONENT) of the exponential function. According to one embodiment, the calculation module (250) can obtain the value (VALUE) and scale factor (EXPONENT) of the exponential function corresponding to the input value from the exponential function processing module (120).

[0170] At operation 740b, the calculation module (250) may compare the scale factor (EXPONENT) with the base scale factor (BASE_EXPONENT). If the base scale factor (BASE_EXPONENT) is greater than the scale factor (EXPONENT) (“yes” in operation 740b), the process may proceed to operation 750b. If the base scale factor (BASE_EXPONENT) is less than or equal to the scale factor (EXPONENT) (“no” in operation 740b), the process may proceed to operation 760b.

[0171] In operation 750b, the operation module (250) can scale the sum (SUM) of the exponential function based on the scale factor (EXPONENT) and the base scale factor (BASE_EXPONENT). The operation module (250) can perform a right shift operation (SHRR) on the sum (SUM) of the exponential function based on the difference between the base scale factor (BASE_EXPONENT) and the scale factor (EXPONENT). The operation module (250) can update the base scale factor value (BASE_EXPONENT) of the input value to the value of the scale factor (EXPONENT).

[0172] In operation 760b, the operation module (250) can scale the value of the exponential function based on the scale factor (EXPONENT) and the base scale factor (BASE_EXPONENT). The operation module (250) can perform a right shift operation (SHRR) on the value (VALUE) of the exponential function based on the difference between the scale factor (EXPONENT) and the base scale factor (BASE_EXPONENT). The operation module (250) can update the base scale factor value (BASE_EXPONENT) of the input value to the value of the scale factor (EXPONENT).

[0173] At operation 770b, the calculation module (250) can update the sum of the exponential functions (SUM) by adding the sum of the exponential functions (SUM) and the value (VALUE) of the exponential functions. In one embodiment, the calculation module (250) can additionally add a constant (e.g., a zero point value) to the sum of the exponential functions (SUM). The calculation module (250) can proceed to operation 720b to determine the sum of the exponential functions (SUM) by accumulating and adding all the values ​​of the exponential functions.

[0174] Although FIG. 7b illustrates an example in which a base scale factor is set, the present invention is not limited thereto, and the operation module (250) according to one embodiment of the present disclosure may also operate in cases in which a base scale factor is not set. For example, in cases in which a base scale factor is not set, operations 740b, 750b, and 760b may be omitted and operated.

[0175] The operation of the calculation module (250) for determining the sum of the exponential function may not include at least one operation from operation 710b to operation 770b, and may omit or include some of the operations.

[0176] FIG. 8A is a flowchart illustrating a method by which a processor including an accelerator according to one embodiment of the present disclosure determines a value of a softmax function using parameters.

[0177] In operation 810a, the accelerator (110) according to one embodiment may determine one or more parameters based on the sum of the exponential function. The one or more parameters may include parameters for replacing the sum of the exponential function. The one or more parameters may include parameters for approximating the division of the sum of the exponential function to determine the value of the softmax function using shift operations and multiplication operations.

[0178] In one embodiment, the one or more parameters may include a first parameter and a second parameter. The first parameter and the second parameter may be determined such that the maximum quantized value of the value of the exponential function divided by the sum of the exponential functions is equal to the value obtained by performing a right shift operation on the first parameter by the second parameter. In one embodiment, the accelerator (110) may determine the one or more parameters using the method described in FIG. 8B.

[0179] In one embodiment, at least one of the first parameter or the second parameter may be determined based on a base scale factor. For example, the first parameter and the second parameter may be determined such that a value obtained by performing a shift operation by the base scale factor on the maximum quantized value of the exponential function divided by the sum of the exponential functions is equal to a value obtained by performing a right shift operation on the first parameter by the second parameter.

[0180] In operation 820a, an accelerator (110) according to one embodiment can determine a value of a softmax function based on a value of an exponential function, a scale factor, and one or more parameters. The accelerator (110) can determine the value of the softmax function by performing at least one of a multiplication, addition, or shift operation on a value scaled based on the value of the exponential function and the scale factor, using one or more parameters.

[0181] In one embodiment, if the parameters are not determined using the base scaling factor in operation 810a, the accelerator (110) may determine the value of the softmax function based on the base scale factor, the value of the exponential function, and the value scaled based on the scale factor.

[0182] The operation of a processor (100) including an accelerator (110) for processing a softmax function may not include at least one of operations 810a to 820a, and may omit or include some operations.

[0183] FIG. 8b is a flowchart illustrating the operation of a parameter module of an accelerator according to one embodiment of the present disclosure.

[0184] A method for determining a parameter according to one embodiment of the present disclosure may be performed by the parameter module (260) of FIG. 2. The parameter module (260) may determine the parameter based on a maximum quantized value (QMAX) and a sum of quantized exponential functions (SUM). The parameter may include a first parameter (MULT) and a second parameter (RSHAMT). The parameter may be a value for approximating a value obtained by dividing the maximum quantized value by the sum of quantized exponential functions. In one embodiment, the value obtained by dividing the maximum quantized value by the sum of quantized exponential functions may be approximated by a value obtained by performing a right shift operation on the first parameter by the amount of the second parameter.

[0185] At operation 810b, the parameter module (260) may determine the values ​​of variables for determining parameters. The variable DIVIDEND may be initialized to the maximum quantization value. The variable DIVISOR may be initialized to the sum of the exponential functions. The variable RSHMAT may be initialized to 0. The parameter module (260) may proceed to operation 820b.

[0186] At operation 820b, the parameter module (260) may compare the sizes of the variables DIVISOR and DIVIDEND. For example, the parameter module (260) may determine whether the variables DIVISOR and DIVIDEND are greater than each other. If the variable DIVISOR is greater than the variable DIVIDEND, the parameter module (260) may proceed to operation 830b. If the variable DIVISOR is less than or equal to the variable DIVIDEND, the parameter module (260) may proceed to operation 840b.

[0187] At operation 830b, the parameter module (260) may update the value of the variable DIVIDEND and the value of the variable RSHMAT. In one embodiment, the parameter module (260) may perform a left shift operation on the value of the variable DIVIDEND, for example, performing a left shift operation by 1 on the value of the variable DIVIDEND. For example, the parameter module (260) may update the value of the variable DIVIDEND to a value that is increased by two times. In one embodiment, the parameter module (260) may update the value of the variable RSHMAT to a value that is increased by 1. For example, if the value of the variable RSHMAT is 0, the parameter module (260) may change the value of the variable RSHMAT to 1. The parameter module (260) may proceed to operation 832b.

[0188] At operation 832b, the parameter module (260) may compare the sizes of the variables DIVISOR and DIVIDEND. For example, the parameter module (260) may determine whether the variables DIVISOR and DIVIDEND are greater than each other. If the variable DIVISOR is greater than the variable DIVIDEND, the parameter module (260) may proceed to operation 830b. If the variable DIVISOR is less than or equal to the variable DIVIDEND, the parameter module (260) may proceed to operation 850b.

[0189] At operation 840b, the parameter module (260) may update the value of the variable DIVISOR and the value of the variable RSHMAT. In one embodiment, the parameter module (260) may perform a left shift operation on the value of the variable DIVISOR, for example, performing a left shift operation on the value of the variable DIVISOR by 1. For example, the parameter module (260) may update the value of the variable DIVISOR to a value that is doubled. In one embodiment, the parameter module (260) may update the value of the variable RSHMAT to a value that is decreased by 1. For example, if the value of the variable RSHMAT is 0, the parameter module (260) may change the value of the variable RSHMAT to -1. The parameter module (260) may proceed to operation 842b.

[0190] In operation 842b, the parameter module (260) can compare the sizes of the variables DIVISOR and DIVIDEND. For example, the parameter module (260) can determine whether the variables DIVISOR and DIVIDEND are less than or equal to each other. If the variable DIVISOR is less than or equal to the variable DIVIDEND, the parameter module (260) can proceed to operation 840b. If the variable DIVISOR is greater than the variable DIVIDEND, the parameter module (260) can proceed to operation 844b.

[0191] At operation 844b, the parameter module (260) may update the value of the variable DIVISOR and the value of the variable RSHMAT. In one embodiment, the parameter module (260) may perform a right shift operation on the value of the variable DIVISOR, for example, may perform a right shift operation on the value of the variable DIVISOR by 1. For example, the parameter module (260) may update the value of the variable DIVISOR to a value that is reduced by a factor of two (e.g., divided by half). In one embodiment, the parameter module (260) may update the value of the variable RSHMAT to a value that is increased by 1. For example, if the value of the variable RSHMAT is -1, the parameter module (260) may change the value of the variable RSHMAT to 0. The parameter module (260) may proceed to operation 850b.

[0192] At operation 850b, the parameter module (260) can determine values ​​of variables for determining parameters. The parameter module (260) can update the value of the variable DIVIDEND to a difference value between the value of the variable DIVIDEND and the value of the variable DIVISOR. The parameter module (260) can determine the variable MULT to 1. The parameter module (260) can determine the variable MULT_BITS to 1. In one embodiment, the parameter module (260) can determine the variables MULT and MULT_BITS before operation 850b. For example, the parameter module (260) can determine the variables MULT and MULT_BITS at operation 810b. The parameter module (260) can proceed to operation 860b.

[0193] In operation 860b, the parameter module (260) may compare the sizes of the variables DIVISOR and DIVIDEND. For example, the parameter module (260) may determine whether the variables DIVISOR and DIVIDEND are greater than each other. If the variable DIVISOR is greater than the variable DIVIDEND, the parameter module (260) may proceed to operation 870b. If the variable DIVISOR is less than or equal to the variable DIVIDEND, the parameter module (260) may proceed to operation 875b.

[0194] In operation 870b, the parameter module (260) can update the value of the variable DIVIDEND, the value of the variable MULT, the value of the variable MULT_BITS, and the value of the variable RSHMAT. In one embodiment, the parameter module (260) can perform a left shift operation on the value of the variable DIVIDEND, for example, by performing a left shift operation by 1 on the value of the variable DIVIDEND. For example, the parameter module (260) can update the value of the variable DIVIDEND to a value that has been increased by two. In one embodiment, the parameter module (260) can perform a left shift operation by 1 on the value of the variable MULT. In one embodiment, the parameter module (260) can increment the value of the variable MULT_BITS, for example, by updating the value of the variable MULT_BITS to a value that has been increased by 1. For example, if the value of the variable MULT_BITS is 1, the parameter module (260) can change the value of the variable MULT_BITS to 2. In one embodiment, the parameter module (260) may update the value of the variable RSHMAT to a value increased by 1. For example, if the value of the variable RSHMAT is 1, the parameter module (260) may change the value of the variable RSHMAT to 2. The parameter module (260) may proceed to operation 880b.

[0195] At operation 875b, the parameter module (260) may update the value of the variable DIVIDEND to the difference between the value of the variable DIVIDEND and the value of the variable DIVISOR. The parameter module (260) may update the variable MULT to a value increased by 1. The parameter module (260) may proceed to operation 880b.

[0196] At operation 880b, the parameter module (260) can determine whether the variable BITS_MAX is greater than the variable MULT_BITS. If the variable BITS_MAX is greater than the variable MULT_BITS, the parameter module (260) can proceed to operation 860b. If the variable BITS_MAX is less than or equal to the variable MULT_BITS, the parameter module (260) can proceed to operation 890b.

[0197] In operation 890b, the parameter module (260) can pass the variable MULT and the variable RSHAMT to the post-processing module (270).

[0198] FIG. 8c is a flowchart illustrating a method by which a processor including an accelerator according to one embodiment of the present disclosure determines a value of a softmax function.

[0199] At operation 810c, the softmax function processing module (130) can determine whether the sum of the exponential functions is equal to 0. If the sum of the exponential functions is 0 (meaning “yes” in operation 810c), the process can proceed to operation 820c, and if the sum of the exponential functions is not 0 (meaning “no” in operation 810c), the process can proceed to operation 830c.

[0200] In operation 820c, the softmax function processing module (130) can modify the sum of the exponential function. The softmax function processing module (130) can determine the sum of the exponential function as the number of input values ​​included in the input data. The softmax function processing module (130) can determine the value of the zero flag. The zero flag can be a flag indicating whether the sum of the exponential function determined by the operation module (250) is 0. For example, when the value of the zero flag is 1, it can indicate that the sum of the exponential function by the operation module (250) is 0.

[0201] In operation 830c, the softmax function processing module (130) may determine one or more parameters based on the sum of the exponential functions. In one embodiment, operation 830c may correspond to operation 810a of FIG. 8a.

[0202] At operation 840c, the softmax function processing module (130) can determine whether the sum of the exponential functions determined by the operation module (250) is 0. For example, the softmax function processing module (130) can identify the value of the zero flag. If the sum of the exponential functions determined by the operation module (250) is 0, the process can proceed to operation 850c, and if not, the process can proceed to operation 860c.

[0203] In operation 850c, the softmax function processing module (130) may determine the value of the softmax function by setting the value of the exponential function to 1. The softmax function processing module (130) may set the value of the exponential function to 1 and determine the value of the softmax function based on one or more parameters. If the value of the zero flag is 1, one or more parameters may be determined by taking the sum of the exponential function as the number of input values ​​included in the input data. For example, if the number of input data is 32, the parameter may be determined by taking the sum SUM as 1. The values ​​of the softmax for the input values ​​may all be determined to be the same. For example, if the input values ​​are set to 1, the values ​​of the softmax function for the input values ​​may be determined as 1 / 32. If the sum of the exponential function is 0, it may mean that the values ​​of the exponential function are all small. Therefore, the softmax function processing module (130) may obtain the values ​​of the softmax function with a small error even if the values ​​of the softmax are determined to be the same.

[0204] In operation 860c, the value of the softmax function may be determined based on the value of the exponential function, the scale factor, and one or more parameters. In one embodiment, operation 860c may correspond to operation 820a of FIG. 8a.

[0205] The operation of the processor (100) including an accelerator (110) for processing a softmax function may not include one or more of operations 810c to 860c, and may omit or include some operations.

[0206] FIG. 9 is a flowchart illustrating an operation of a processor including an accelerator according to one embodiment of the present disclosure to initialize settings.

[0207] In operation 910a, a processor (100) according to one embodiment may obtain configuration data for input data. The configuration data may be determined differently depending on the range and maximum quantization value of the input data.

[0208] In one embodiment, the configuration data may include configuration data regarding at least one of exponential function information, a group index, a shift unit of the group, parameter expression bits, or a zero point value. In one embodiment, the configuration data may be obtained from a memory external to the processor (100).

[0209] In operation 920a, the processor (100) according to one embodiment may initialize the settings of the accelerator (110) based on the configuration data. In one embodiment, the processor (100) may initialize the settings of the accelerator (110) based on the configuration data before performing the operation of determining the value of the softmax function for the input data.

[0210] In one embodiment, the processor (100) may set a group index based on the configuration data. The processor (100) may store the group index in the lookup table module (230) of the accelerator (110). The group index may be a value related to the number of groups into which exponential function values ​​for the input data are divided. The accelerator (110) may determine at least one of a first bit string or a scale factor based on the set group index.

[0211] In one embodiment, the processor (100) may set a shift unit of a group based on configuration data. The processor (100) may store the shift unit of the group in a lookup table module (230) of the accelerator (110). The accelerator (110) may perform at least one of a shift operation for a reference value or an operation of determining a scale factor based on the set shift unit.

[0212] In one embodiment, the processor (100) may set parameter expression bits based on the configuration data. The processor (100) may store the parameter expression bits in the parameter module (260) of the accelerator (110). The parameter expression bits may mean the number of bits for representing one or more parameters determined by the parameter module (260). The accelerator (110) may determine one or more parameters based on the expression bits of the set parameters.

[0213] In one embodiment, the processor (100) may set a zero point value based on the configuration data. The accelerator (110) may determine the sum of the exponential function based on the set zero point value.

[0214] In one embodiment, the processor (100) may control the lookup table module to store exponential function information based on the configuration data. The exponential function information may be determined differently for each of the maximum quantization value (or quantization interval) of the input data, the maximum quantization value (or quantization interval) of the value of the exponential function, or the zero point value.

[0215] The operation of the processor (100) including an accelerator (110) for processing a softmax function may not include one or more of operations 910 to 920, and may omit or include some operations.

[0216] FIG. 10 is a block diagram illustrating the operation of a processor including an accelerator according to one embodiment of the present disclosure.

[0217] Referring to FIG. 10, the accelerator (110) can generate output data based on input data. The input data can include a plurality of quantized input values. The output data can include values ​​of a plurality of softmax functions corresponding to the input values ​​of the input data. The values ​​of the softmax function can be quantized values.

[0218] The input module (280) can obtain input data. The input module (280) can sequentially obtain input values ​​of the input data. The input module (280) can include a plurality of buffers.

[0219] The exponential function processing module (120) can obtain an input value from the input module (280). The exponential function processing module (120) can determine the value and scale factor of the exponential function corresponding to the input value. The exponential function processing module (120) can sequentially obtain the input values ​​of the input data and determine the value and scale factor of the exponential function corresponding to the input value. The exponential function processing module (120) can transfer the value and scale factor of the exponential function to the softmax function processing module (130).

[0220] The lookup table module (230) can output one or more reference values ​​corresponding to an input value. The lookup table module (230) can store a plurality of values ​​and output a stored value corresponding to an index among the plurality of values. The lookup table module (230) can store exponential function information obtained from the outside of the accelerator (110) or from the memory (285). The exponential function information can include a plurality of exponential function values.

[0221] The lookup table module (230) can output a first reference value, which is a value of an exponential function corresponding to a first index (e.g., a table index) determined by the upper bit of an input value. The lookup table module (230) can transmit the first reference value to the interpolation module (240). The lookup table module (230) can output a second reference value, which is a value of an exponential function corresponding to a second index value that is one greater than the first index value. The lookup table module (230) can transmit the second reference value to the interpolation module (240).

[0222] The interpolation module (240) can determine the value of an exponential function corresponding to an input value. The interpolation module (240) can determine the value of the exponential function corresponding to the input value based on a first reference value and a second reference value. The interpolation module (240) can perform interpolation on the first reference value and the second reference value based on an interpolation weight determined by a lower bit of the input value to determine the value of the exponential function corresponding to the input value. In one embodiment, the interpolation module (240) can modify the second reference value based on an order within a group of input values, and determine the value of the exponential function based on the modified second reference value.

[0223] The group index module (610) can determine a scale factor corresponding to an input value. The group index module (610) can determine the scale factor based on a table index. The table index can include a first bit string and a second bit string. The table index can be divided into the first bit string and the second bit string based on the group index. The group index module (610) can determine the scale factor based on the first bit string. The group index module (610) can determine a result value of a bitwise NOT operation on the first bit string as the scale factor. The group index module (610) can determine an order within a group of input values ​​based on the second bit string. In one embodiment, the group index module (610) can determine whether the input value is the last order within the group based on a result value of a bitwise AND operation on the second bit string.

[0224] The softmax function processing module (130) can determine the value of the softmax function for the input value based on the value of the exponential function and the scale factor. The softmax function processing module (130) can determine the sum of the exponential function based on the value of the exponential function and the scale factor. The softmax function processing module (130) can determine the value of the softmax function based on the value of the exponential function, the scale factor, and the sum of the exponential function. The softmax function processing module (130) can transmit the value of the softmax function to the output module (290).

[0225] In one embodiment, the softmax function processing module (130) can determine the value of the softmax function by dividing the value of the exponential function scaled using a scale factor by the sum of the exponential functions.

[0226] In one embodiment, the softmax function processing module (130) may determine one or more parameters based on the sum of the exponential function. The softmax function processing module (130) may determine the value of the softmax function by performing an operation using one or more parameters on a value of the exponential function scaled using a scale factor.

[0227] The calculation module (250) can determine the sum of the exponential function based on the value of the exponential function and the scale factor. The calculation module (250) can determine the sum of the exponential function by accumulating and adding the values ​​scaled using the scale factor to the values ​​of the exponential function.

[0228] The calculation module (250) can determine the sum of the exponential functions based on the base scale factor. The calculation module (250) can scale at least one of the sum of the exponential functions or the value of the exponential function based on the result of comparing the base scale factor and the scale factor. For example, if the base scale factor is greater than the scale factor, the sum of the exponential functions can be scaled by the difference between the base scale factor and the scale factor. For example, if the base scale factor is smaller than the scale factor, the value of the exponential function can be scaled by the difference between the scale factor and the base scale factor. The calculation module (250) can update the sum of the exponential functions by adding the sum of the exponential functions and the value of the exponential function. The calculation module (250) can determine the sum of the exponential functions by accumulating and adding the values ​​of the exponential functions. In one embodiment, the calculation module (250) can transmit the sum of the exponential functions and the base scale factor to the parameter module (260).

[0229] The parameter module (260) may determine one or more parameters based on the sum of the exponential functions. The parameter module (260) may determine one or more parameters for approximating the reciprocal of the sum of the exponential functions. In one embodiment, the parameter module (260) may determine one or more parameters by considering the result scaled by the base scale factor. The parameter module (260) may transmit one or more parameters determined by considering the base scale factor to the post-processing module (270).

[0230] In one embodiment, the parameter module (260) may determine one or more parameters without considering the result being scaled by the base scale factor and pass the base scale factor and the one or more parameters to the post-processing module (270).

[0231] The post-processing module (270) can determine the value of the softmax function for the input values ​​based on the value of the exponential function, the scale factor, and one or more parameters. In one embodiment, the exponential function processing module (120) can re-determine the value and the scale factor of the exponential function and pass them to the post-processing module (270). The exponential function processing module (120) can obtain input values ​​again from the input module (280) without storing the value and the scale factor of the exponential function passed to the calculation module (250), determine the value and the scale factor of the exponential function, and pass them to the post-processing module (270). In one embodiment, the exponential function processing module (120) can store the values ​​passed to the calculation module (250) in memory and pass the stored values ​​to the post-processing module (270).

[0232] The post-processing module (270) can determine the value of the softmax function by performing an operation using one or more parameters on the value of the exponential function scaled using a scale factor. The post-processing module (270) can transmit the value of the softmax function to the output module (290). The post-processing module (270) can perform scaling on the value of the exponential function using a base scale factor. For example, the value of the exponential function can be scaled using both a scale factor and a base scale factor.

[0233] The output module (290) can transmit output data to the outside of the accelerator (110) or to the memory (285). The output module (290) can sequentially transmit output values ​​of the output data to the outside of the accelerator (110) or to the memory (285). The output data can include multiple output values. The output value can be the value of a softmax function for the input value. The output module (290) can include multiple buffers.

[0234] The multiple modules included in the accelerator (110) can independently perform operations. Processing of multiple input values ​​of input data through the internal modules of the accelerator (110) can be performed in parallel. For example, the operation of the exponential function processing module (120) determining the value of the exponential function for the first input value can be performed simultaneously with the operation of the calculation module (250) updating the sum of the exponential function based on the value of the exponential function for the second input value.

[0235] FIG. 11 is a flowchart of a processor operation method according to input data according to one embodiment of the present disclosure.

[0236] In operation 1110, the processor (100) may obtain input data including a plurality of input values. The processor (100) may sequentially obtain the plurality of input values ​​of the input data.

[0237] In operation 1120, the processor (100) may determine the value of an exponential function corresponding to an input value. The processor (100) may determine the value of the exponential function corresponding to the input value based on the value of the exponential function stored in the lookup table module (230). The processor (100) may determine the value of the exponential function corresponding to the input value by performing interpolation on the values ​​of the exponential function stored in the lookup table module (230).

[0238] In operation 1130, the processor (100) may determine a scale factor corresponding to the input value. The processor (100) may determine the scale factor based on a predetermined upper bit of the input value. Operations 1120 and 1130 may be performed independently. Operations 1120 and 1130 may be performed simultaneously, or operation 1130 may be performed with priority over operation 1120.

[0239] In operation 1140, the processor (100) may determine a sum of exponential functions based on the value of the exponential function and the scale factor. The processor (100) may scale the value of the exponential function or the sum of the exponential functions using the scale factor. The processor (100) may update the sum of the exponential functions by adding the value of the exponential function to the sum of the exponential functions obtained as a result of the scaling. The processor (100) may determine the sum of the exponential functions by accumulating and adding exponential functions for a plurality of input values. In one embodiment, the processor (100) may determine the sum of the exponential functions based on the result of scaling the value of the exponential function or the sum of the exponential functions using the base scale factor.

[0240] In operation 1150, the processor (100) may determine one or more parameters based on the sum of the exponential function. The one or more parameters may be parameters that are approximated to perform a faster operation than a division operation on the sum of the exponential function.

[0241] In operation 1160, the processor (100) may determine a value of a softmax function based on a value of an exponential function, a scale factor, and one or more parameters. The processor (100) may determine a value of an exponential function and a scale factor separately from the value of an exponential function and a scale factor determined in operations 1120 and 1130. The processor (100) may determine a value of a softmax function by applying a parameter to a result of scaling the value of an exponential function using the scale factor.

[0242] In operation 1170, the processor (100) may output the value of the softmax function as an output value. The processor (100) may transfer the value of the softmax function to memory or to another processor or accelerator.

[0243] The operation of the processor (100) for processing the softmax function may not include one or more of the operations 1110 to 1170, and may omit or include some of the operations.

[0244] FIG. 12A is a diagram illustrating a process in which a processor performs artificial intelligence operations according to one embodiment of the present disclosure.

[0245] Referring to FIG. 12a, the artificial intelligence operations according to one embodiment may include a convolution operation, a Rectified Linear Unit (ReLU) operation, a pooling operation, a Fully Connected (FC) operation, and a Softmax operation. An artificial intelligence model having a structure like that of FIG. 12a may be referred to as a Convolutional Neural Network (CNN) model. Convolutional neural networks can be primarily utilized in image processing fields such as object recognition.

[0246] The processor (100) can perform at least one of the convolutional neural network operations using the accelerator (110). For example, the processor (100) can perform a convolution operation, a pooling operation, an FC operation, and a softmax operation using the accelerator (110). The convolution operation, the pooling operation, and the FC operation can be implemented only with addition operations, multiplication operations, and shift operations. However, in order to directly perform the softmax operation, an exponential function operation and a division operation may be required. However, the processor (100) can process the softmax function using the exponential function processing module (120) and the softmax function processing module (130) of the accelerator (110). Therefore, the electronic device can process both other operations of the artificial intelligence operation and the softmax operation using the accelerator (110) of the processor (100).

[0247] FIG. 12b is a diagram for explaining a process in which a processor performs artificial intelligence operations according to one embodiment of the present disclosure.

[0248] Referring to Fig. 12b, an LSTM model operation and a softmax operation according to one embodiment may be included. An artificial intelligence model with a structure like Fig. 12b can be primarily utilized in the field of natural language processing. For natural language processing, an electronic device can provide weights to data (e.g., an embedding corresponding to an input word as shown in Fig. 12b) using a softmax function.

[0249] As shown in FIG. 12b, a processor (100) according to one embodiment can perform a softmax operation using an accelerator (110). An electronic device can process a softmax operation required for natural language processing using the accelerator (110) of the processor (100). Similarly, according to one embodiment, the processor (100) can be used to process a softmax operation for generative artificial intelligence (Generative AI) or a large language model (LLM).

[0250] FIG. 13 is a block diagram illustrating an electronic device according to one embodiment of the present disclosure.

[0251] Referring to FIG. 13, an electronic device (1300) may be configured to process input information or may be a device that processes and performs a function corresponding to the input. For example, the electronic device (1300) may include a video display device, a mobile terminal, and a home appliance. For example, the electronic device (1300) may be at least one of a TV, a smart phone, a tablet, an air conditioner, a washing machine, and a vacuum cleaner. According to an embodiment, the electronic device (1300) may include a first processor (1310), a second processor (1320), and a memory (1330). In an embodiment, the electronic device (1300) may further include a communication module and a display. The first processor (1310) may include either the processor (100) of FIG. 1 or the processor (100) of FIG. 2, and may be a processor (100) with some components added or deleted.

[0252] In one embodiment, the second processor (1320) may control the overall operation of the electronic device (1300). The second processor (1320) may load at least one instruction stored in the memory (1330) to control the operation of the electronic device (1300). The second processor (1320) may perform various data processing or calculations to control the communication module and the display. The second processor (1320) may include at least one of a central processing unit (CPU), an application processor (AP), and an image signal processor (ISP).

[0253] In one embodiment, the first processor (1310) may be configured to process or be a processor for processing artificial intelligence operations. For example, the processing speed for artificial intelligence operations of the first processor (1310) may be faster than the processing speed for artificial intelligence operations of the second processor (1320). The first processor (1310) may include at least one of a graphics processing unit (GPU) or a neural processing unit (NPU).

[0254] In one embodiment, the first processor (1310) can train a neural network. The first processor (1310) can perform operations to train the neural network based on input information. The first processor (1310) can perform iterative training in a manner similar to machine learning (ML).

[0255] In one embodiment, the first processor (1310) may analyze input data using a trained neural network. The first processor (1310) may then allow the second processor (1320) to utilize the results of analyzing the input data. The second processor (1320) may then implement artificial intelligence (AI) using the results of the analysis by the first processor (1310).

[0256] In one embodiment, the communication module may enable the electronic device (1300) to perform a communication connection with an external electronic device or server. The communication module may include at least one communication processor (CP) to support wireless communication. The communication module may establish a wired or wireless communication channel between the electronic device (1300) and at least one of the external electronic device or server. The communication module may transmit a signal notifying the second processor (1320) of the establishment of the wired or wireless communication channel. The communication module may receive a signal from the second processor (1320) allowing the establishment of the wired or wireless communication channel. The second processor (1320) may determine whether to allow the establishment of the wired or wireless communication channel based on the results analyzed by the first processor (1310).

[0257] In one embodiment, the communication module may support the electronic device (1300) to transmit and receive signals or data from an external electronic device or server using an established wired or wireless communication channel. For example, if the electronic device (1300) is a video display device, the communication module may establish a wired or wireless communication channel with the external electronic device or server to receive video data from the external electronic device or server. For example, if the electronic device (1300) is a mobile terminal, the communication module may establish a wireless communication channel between the electronic device (1300) and the external electronic device, and may connect a call or send and receive messages using the established wireless communication channel. For example, if the electronic device (1300) is a home appliance, the communication module may connect to the external electronic device via short-range wireless communication such as Bluetooth Low Energy (BLE) communication, and may connect to the server via long-range wireless communication such as Wi-Fi communication. The communication module can receive information for artificial intelligence operations from at least one of an external electronic device or a server, or transmit results for artificial intelligence operations.

[0258] In one embodiment, the memory (1330) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., secure digital (SD) memory, eXtreme digital (XD) memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. The memory (1330) may receive a command or data from the second processor (1320). The memory (1330) may store a command or data received from the second processor (1320). The command or data stored in the memory (1330) may include a program for the second processor (1320) to process the input data and control the operation of the electronic device (1300). The second processor (1320) may load the command or data stored in the memory (1330). The second processor (1320) may analyze or process the command or data loaded from the memory (1330) using the first processor (1310).

[0259] In one embodiment, the display can provide visual information to the outside of the electronic device (1300). For example, the display can display a screen regarding information regarding the status of the electronic device (1300). For example, the display can display a screen regarding an operation or function performed by the electronic device (1300). The second processor (1320) can transmit image data related to the result of the artificial intelligence operation of the first processor (1310) to the display. For example, the display can display image data including object information in an image generated by the first processor (1310). The display can include a display panel for displaying a screen and a display driver integrated circuit (DDI) for driving the display panel. For example, the display can be at least one of an organic light emitting display (OLED), a quantum dot display (QD display), and a micro light emitting diode (Micro LED) display.

[0260] FIG. 14 is an exemplary drawing for explaining an electronic device including a processor according to one embodiment of the present invention.

[0261] Referring to FIG. 14, an electronic device including a processor according to an embodiment of the present disclosure may include a plurality of electronic devices (1400), including but not limited to a TV (1410), a head mounted display (HMD) (1420), a mobile device (1430) (e.g., a smartphone), a wearable device (1440), a home appliance (1450), and a server (1460). The plurality of electronic devices (1400) may include a processor including an accelerator (110) for processing a softmax function.

[0262] A plurality of electronic devices (1400) including a processor according to one embodiment of the disclosure may perform on-device image processing. For example, the plurality of electronic devices (1400) may perform image processing to recognize objects included in an image using artificial intelligence operations performed by the accelerator (110). In one embodiment, the processor (100) may obtain information about an image, perform artificial intelligence image processing using the accelerator (110), and provide the resulting information to another processor, display, or memory.

[0263] A plurality of electronic devices (1400) including a processor according to one embodiment of the present disclosure can perform on-device voice recognition. For example, the plurality of electronic devices (1400) can perform an operation of identifying a user's voice and a user's command using an artificial intelligence operation performed by the accelerator (110). In one embodiment, the processor (100) can obtain voice information, perform artificial intelligence voice recognition using the accelerator (110), and provide the resulting information to another processor, a display, a speaker, or a memory.

[0264] A plurality of electronic devices (1400) including a processor (100) according to one embodiment of the present disclosure can perform on-device natural language processing (e.g., translation). For example, the plurality of electronic devices (1400) can perform an operation of identifying the meaning and context of text or generating text using an artificial intelligence operation performed by the accelerator (110). In one embodiment, the processor (100) can obtain text information, perform artificial intelligence natural language processing using the accelerator (110), and provide the resulting information to another processor, speaker, display, or memory.

[0265] According to one embodiment, the server (1460) may use the processor (100) to process artificial intelligence operations on behalf of another electronic device, or perform procedures requiring large-scale operations, such as large-scale language model processing.

[0266] By using an accelerator (110) according to one embodiment of the present disclosure, a softmax operation can be processed quickly and accurately. Therefore, the inference speed of the entire neural network can be increased. (For example, latency can be reduced.) In addition, an accelerator (110) for processing a softmax function can be produced by adding a simple configuration of a processor (100) including a MAC (Multiplication-Accumulation) accelerator (110). Therefore, a separate processor related to the softmax function is not required, and the softmax function and MAC operation can be processed using only the processor (100). In this case, bandwidth and power consumption can be reduced due to a reduction in data movement between the accelerator (110) and the separate processor.

[0267] In one embodiment of the present disclosure, a method is provided, which is performed by a processor configured to implement an accelerator for processing a softmax function. The method may include an operation in which an exponential function processing module obtains input data including a plurality of input values. The method may include an operation in which the exponential function processing module obtains a value of an exponential function corresponding to an input value included in the input data using a lookup table module. The method may include an operation in which the exponential function processing module determines a scale factor corresponding to the input value. The method may include an operation in which the softmax function processing module determines a value of the softmax function corresponding to the input value based on the value of the exponential function and the scale factor. The lookup table module may store values ​​relating to a plurality of exponential functions. The values ​​relating to the plurality of exponential functions may be classified into a plurality of groups according to corresponding scale factors.

[0268] In one embodiment of the present disclosure, the operation of obtaining a value of an exponential function may include an operation of the exponential function processing module obtaining a first reference value and a second reference value corresponding to an input value from among a plurality of values ​​of exponential functions stored in the lookup table module. The operation of obtaining a value of an exponential function may include an operation of the exponential function processing module obtaining a value of an exponential function based on the first reference value and the second reference value.

[0269] In one embodiment of the present disclosure, the operation of obtaining the first reference value and the second reference value may include an operation of the exponential function processing module determining a first index value and a second index value corresponding to an input value. The operation of obtaining the first reference value and the second reference value may include an operation of the exponential function processing module obtaining the first reference value based on a first output value of the lookup table module corresponding to the first index value. The operation of obtaining the first reference value and the second reference value may include an operation of the exponential function processing module obtaining the second reference value based on a second output value of the lookup table module corresponding to the second index value.

[0270] In one embodiment of the present disclosure, the operation of determining the first index value and the second index value may include an operation of the exponential function processing module determining, from among the index values ​​of the lookup table module, a first index value that is identical to at least a portion of the input value. The operation of determining the first index value and the second index value may include an operation of the exponential function processing module determining the second index value based on its proximity to the first index value.

[0271] In one embodiment of the present disclosure, the method may include an operation in which the exponential function processing module identifies group bits corresponding to input data. The method may include an operation in which the exponential function processing module obtains a first bit string and a second bit string from the input value based on the group bits. A scale factor may be determined based on the first bit string.

[0272] In one embodiment of the present disclosure, the operation of determining the value of the exponential function may include an operation of determining whether the exponential function processing module adjusts the second reference value based on the second bit string. The operation of determining the value of the exponential function may include an operation of adjusting the second reference value based on the exponential function processing module's determination to adjust the second reference value, and determining the value of the exponential function based on the first reference value and the adjusted second reference value.

[0273] In one embodiment of the present disclosure, the value of the exponential function can be determined by performing interpolation on a first reference value and an adjusted second reference value based on a second bit string.

[0274] In one embodiment of the present disclosure, the operation of determining the value of the softmax function may include an operation of the softmax function processing module determining a sum of exponential functions of input data based on the value of the exponential function and a scale factor. The operation of determining the value of the softmax function may include an operation of the softmax function processing module determining a value of the softmax function corresponding to the input value based on the value of the exponential function, the scale factor, and the sum of the exponential functions.

[0275] In one embodiment of the present disclosure, the operation of determining the sum of the exponential function may include an operation of the softmax function processing module scaling the values ​​of the exponential function using a scale factor and a base scale factor. The operation of determining the sum of the exponential function may include an operation of the softmax function processing module determining the sum of the exponential function based on the values ​​of the scaled exponential function.

[0276] In one embodiment of the present disclosure, the method may include an operation in which a softmax function processing module determines one or more parameters based on a sum of an exponential function. The value of the softmax function may be determined by scaling the value of the exponential function based on a scale factor and one or more parameters.

[0277] In one embodiment of the present disclosure, the method may include an operation in which the softmax function processing module determines a value of a flag indicating whether the sum of the exponential function is 0. Based on the flag indicating that the sum of the exponential function is 0, one or more parameters may be determined by setting the sum of the exponential function to the number of multiple input values ​​included in the input data. The value of the softmax function may be determined by setting the value of the exponential function to 1.

[0278] In one embodiment of the present disclosure, a processor for processing a softmax function is provided. The processor may include an accelerator. The accelerator may include an exponential function processing module and a softmax function processing module. The exponential function processing module may obtain input data including a plurality of input values. The exponential function processing module may obtain a value of an exponential function corresponding to an input value included in the input data using a lookup table module. The exponential function processing module may determine a scale factor corresponding to the input value. The softmax function processing module may determine a value of the softmax function corresponding to the input value based on the value of the exponential function and the scale factor. The lookup table module may store values ​​for a plurality of exponential functions. The values ​​for the plurality of exponential functions may be classified into a plurality of groups according to corresponding scale factors.

[0279] In one embodiment of the present disclosure, the exponential function processing module can obtain a first reference value and a second reference value corresponding to an input value from among the values ​​of a plurality of exponential functions stored in the lookup table module. The exponential function processing module can obtain the value of the exponential function based on the first reference value and the second reference value.

[0280] In one embodiment of the present disclosure, the exponential function processing module can determine a first index value and a second index value corresponding to an input value. The exponential function processing module can obtain a first reference value based on a first output value of a lookup table module corresponding to the first index value. The exponential function processing module can obtain a second reference value based on a second output value of the lookup table module corresponding to the second index value.

[0281] In one embodiment of the present disclosure, the exponential function processing module may determine a first index value among the indices of the lookup table module based on a first index value that is identical to at least a portion of the input values. The exponential function processing module may determine a second index value based on a proximity to the first index value.

[0282] In one embodiment of the present disclosure, the exponential function processing module can identify a group of bits corresponding to input data. The exponential function processing module can obtain a first bit string and a second bit string from the input value based on the group of bits. The exponential function processing module can determine a scale factor based on the first bit string.

[0283] In one embodiment of the present disclosure, the exponential function processing module may determine whether to adjust the second reference value based on the second bit string. The exponential function processing module may adjust the second reference value based on the determination to adjust the second reference value. The exponential function processing module may determine the value of the exponential function based on the first reference value and the adjusted second reference value.

[0284] In one embodiment of the present disclosure, the value of the exponential function can be determined by performing interpolation on a first reference value and an adjusted second reference value based on a second bit string.

[0285] In one embodiment of the present disclosure, the softmax function processing module can determine the sum of the exponential functions of the input data based on the value of the exponential function and the scale factor. The softmax function processing module can determine the value of the softmax function corresponding to the input value based on the value of the exponential function, the scale factor, and the sum of the exponential functions.

[0286] In one embodiment of the present disclosure, an electronic device is provided. The electronic device may include a first processor, a second processor, and a memory for processing a softmax function. The second processor may control the first processor to process an artificial intelligence operation including a softmax function by executing instructions contained in the memory. The first processor may include an accelerator. The accelerator may implement an exponential function processing module and a softmax function processing module. The exponential function processing module may obtain input data including a plurality of input values. The exponential function processing module may obtain a value of an exponential function corresponding to an input value included in the input data using a lookup table module. The exponential function processing module may determine a scale factor corresponding to the input value. The softmax function processing module may determine a value of the softmax function corresponding to the input value based on the value of the exponential function and the scale factor. The lookup table module may store values ​​for a plurality of exponential functions. The values ​​for the plurality of exponential functions may be classified into a plurality of groups according to corresponding scale factors.

[0287] The processor, the method of operating the processor, and the electronic device including the processor according to the present disclosure can reduce the size and complexity of the processor or shorten the time required for calculation by using an accelerator that processes a nonlinear activation function. In addition, the processor, the method of operating the processor, and the electronic device including the processor according to the present disclosure can implement a dynamic range using a simpler method compared to the floating point method.

[0288] According to one embodiment of the present disclosure, the computer-executable instructions, such as program modules executed by a computer, may also be implemented in the form of a recording medium. Computer-readable media may be any available media that can be accessed by a computer, and include both volatile and nonvolatile media, removable and non-removable media. Computer-readable media may include computer storage media and communication media. Computer storage media include both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Communication media may typically include computer-readable instructions, data structures, or other data in a modulated data signal, such as program modules.

[0289] A computer-readable storage medium according to one embodiment of the present disclosure may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is stored semi-permanently in the storage medium and cases where data is stored temporarily. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0290] A method according to one embodiment of the present disclosure may be provided as a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0291] The above description of the present disclosure is provided for illustrative purposes only, and those skilled in the art will readily appreciate that modifications to other specific forms can be made without altering the technical spirit or essential features of the present disclosure. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, components described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined manner.

[0292] The scope of the present disclosure is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present disclosure.

Claims

1. A method performed by a processor configured to implement an accelerator for processing a softmax function, An operation (310) in which an exponential function processing module obtains input data including multiple input values; An operation (320) in which the above exponential function processing module obtains the value of an exponential function corresponding to an input value included in the input data using a look-up table module; The operation (330) of the above exponential function processing module determining a scale factor corresponding to the input value; and The softmax function processing module includes an operation (340) for determining the value of the softmax function corresponding to the input value based on the value of the exponential function and the scale factor, A method, characterized in that the lookup table module stores values ​​for a plurality of exponential functions, and the values ​​for the plurality of exponential functions are classified into a plurality of groups according to corresponding scale factors.

2. In paragraph 1, The operation of obtaining the value of the above exponential function is: An operation in which the above exponential function processing module obtains a first reference value and a second reference value corresponding to the input value from among the values ​​of a plurality of exponential functions stored in the lookup table module; and A method, wherein the exponential function processing module comprises an operation of obtaining a value of the exponential function based on the first reference value and the second reference value.

3. In paragraph 2, The operation of obtaining the first reference value and the second reference value comprises: An operation in which the above exponential function processing module determines a first index value and a second index value corresponding to the input value; An operation in which the above exponential function processing module obtains the first reference value based on the first output value of the lookup table module corresponding to the first index value; and A method, wherein the exponential function processing module comprises an operation of obtaining the second reference value based on a second output value of the lookup table module corresponding to the second index value.

4. In paragraph 3, The operation of determining the first index value and the second index value comprises: An operation in which the above exponential function processing module determines the first index value, which is identical to at least a portion of the input values, among the index values ​​of the lookup table module; and A method, wherein the exponential function processing module comprises an operation of determining the second index value based on proximity to the first index value.

5. In any one of paragraphs 1 to 4, The above exponential function processing module identifies group bits corresponding to the input data; and The above exponential function processing module further includes an operation of obtaining a first bitstring and a second bitstring from the input value based on the group bits; A method, characterized in that the exponential function processing module determines the scale factor based on the first bit string obtained from the input value.

6. In paragraph 5, The operation of determining the value of the above exponential function is: An operation of determining whether the above exponential function processing module adjusts the second reference value based on the second bit string; and A method comprising: adjusting the second reference value based on the determination by the exponential function processing module to adjust the second reference value; and determining a value of the exponential function based on the first reference value and the adjusted second reference value.

7. In paragraph 6, A method wherein the value of the exponential function is determined by performing interpolation on the first reference value and the adjusted second reference value based on the second bit string.

8. In any one of paragraphs 1 to 7, The action of determining the value of the above softmax function is: An operation in which the softmax function processing module determines the sum of the exponential function of the input data based on the value of the exponential function and the scale factor; A method, wherein the softmax function processing module includes an operation of determining a value of the softmax function corresponding to the input value based on a value of the exponential function, the scale factor, and a sum of the exponential functions.

9. In paragraph 8, The operation of determining the sum of the above exponential functions is: The above softmax function processing module scales the value of the exponential function using the scale factor and the base scale factor; and A method, wherein the softmax function processing module includes an operation of determining a sum of the exponential function based on a value of the scaled exponential function.

10. In any one of paragraphs 8 to 9, The above softmax function processing module includes an operation of determining one or more parameters based on the sum of the exponential functions, A method wherein the value of the softmax function is determined by scaling the value of the exponential function based on the scale factor and one or more parameters.

11. In paragraph 10, The above softmax function processing module includes an operation for determining the value of a flag indicating whether the sum of the above exponential function is 0, A method wherein the one or more parameters are determined by setting the sum of the exponential function to the number of the plurality of input values ​​included in the input data, based on the flag indicating that the sum of the exponential function is 0, and the value of the softmax function is determined by setting the value of the exponential function to 1.

12. In a processor for processing a softmax function, Contains an accelerator (110); The above accelerator includes an exponential function processing module (120) and a softmax function processing module (130). The above exponential function processing module (120) obtains input data including a plurality of input values, Using the lookup table module, obtain the value of the exponential function corresponding to the input value included in the input data, Determine the scale factor corresponding to the above input value, The above softmax function processing module (130) determines the value of the softmax function corresponding to the input value based on the value of the exponential function and the scale factor, A processor characterized in that the lookup table module stores values ​​for a plurality of exponential functions, and the values ​​for the plurality of exponential functions are classified into a plurality of groups according to corresponding scale factors.

13. In paragraph 12, The above exponential function processing module (120) obtains a first reference value and a second reference value corresponding to the input value from among the values ​​of a plurality of exponential functions stored in the lookup table module, A processor that obtains a value of the exponential function based on the first reference value and the second reference value.

14. In paragraph 13, The above exponential function processing module (120) determines a first index value and a second index value corresponding to the input value, Obtaining the first reference value based on the first output value of the lookup table module corresponding to the first index value, A processor that obtains the second reference value based on the second output value of the lookup table module corresponding to the second index value.

15. In an electronic device (1300), A first processor (1310) for processing a softmax function; a second processor (1320); and Contains memory (1330), The second processor (1320) controls the first processor (1310) to process an operation related to artificial intelligence including a softmax function by executing instructions included in the memory (1330). The above first processor (1310) includes an accelerator (110), The above accelerator implements an exponential function processing module (120) and a softmax function processing module (130), The above exponential function processing module (120) obtains input data including a plurality of input values, Using the lookup table module, obtain the value of the exponential function corresponding to the input value included in the input data, Determine the scale factor corresponding to the above input value, The above softmax function processing module (130) determines the value of the softmax function corresponding to the input value based on the value of the exponential function and the scale factor, An electronic device, characterized in that the lookup table module stores values ​​for a plurality of exponential functions, and the values ​​for the plurality of exponential functions are classified into a plurality of groups according to corresponding scale factors.

Images